Changing Student Perception through AI-Based Teaching and Learning in a Business Analytics Course
A 2023 summer course titled “Teaching with Cases” at Harvard Business School, organized by the Harvard Business Publishing, changed my perception of the use and application of AI in the classroom. It was fascinating how I watched a YouTube video on the use of AI in case teaching by a Harvard professor (as part of pre-readings for the 2023 Summer course) and then used AI to discuss the case study assigned to me for the course. That was my first-ever experience of handling AI in the context of a classroom where I was a student, learning how to use AI in teaching through a case study. While the short course at Harvard was illuminating, my most important takeaway from the experience was the confidence and eagerness it instilled in me to apply AI toward my own teaching at Penn State University in Fall 2023.
I was scheduled to teach a business analytics course for the first time in a hybrid mode across few campuses of the Penn State University with a handful of students attending my classes in person and the rest of the students being present virtually synchronously throughout the duration of the course. While the modality of the course (hybrid) was challenging in and of itself, I was inspired to teach using AI for the first time in my teaching career because of my experience of using AI as a student at the 2023 Summer course. I had no idea about the prior experience of my students regarding AI. They were all soon- to-be graduates and were taking Business Analytics as a required course to graduate. When asked about what they thought of AI in their profession, some of them mentioned that they thought AI was scary because AI was not only here to stay but it was bound to take many jobs away, rendering the educated workforce redundant. Instead of trying to convince the students to believe otherwise, I felt it was the best opportunity for me to help them work with AI to understand its role in their profession. The students mentioned that they did not work with AI in any way whatsoever at Penn State University due to the general lack of understanding of AI and potential threats of plagiarism around its use in the classroom. The most non-controversial approach for faculty members has been a complete ban of AI use in their courses. While faculty concerns around the use of AI in classroom are legitimate, I felt that not allowing the use of AI in the class could pose even greater challenges for the career readiness of graduates. For one, the ability to use generative AI in business is one of the most crucial skills that will be vetted by business employers. Lack of experience in using AI may mean graduates may not be adequately career-ready in the years to come. Hence the consequence of not using AI in learning and teaching is not an option for higher education faculty anymore. What remains to be seen and experienced are ways to make AI application in classroom transformative from a student’s learning experience.
AI and what we know about it
ChatGPT (https://chaptgpt.ai), Bing (https://bing.ai) and Perplexity (htttps://perplexity.ai) are each types of Large Language Models (LLMs) that are best used via prompt engineering for their application in the classroom for the courses I teach in business, economics and finance. “Prompt engineering” is nothing but “the ability to talk to AI software.” There are various kinds of prompt engineering methods that can be used in the classroom depending on the nature of the study or problem that students will solve. A helpful prompt library for educators from AI for Education can be accessed through this link. There are “mega prompts” which include various actions instructed to the AI based on a project that needs to be completed. These instructions are a series of specific steps that need to be followed in order to achieve the outcome. A good example fro Harvard Business School site of how teaching with AI can be used in a case study has been recorded in detail for anyone interested to implement it in class. It can be found at this link
AI skills as indicators of employability in the future
A study by Boston Consulting Group (BCG) in association with industry consultants and academics examined the implications of AI-based work on knowledge-based, realistic, and complex tasks. A total of around 750 consultants were involved in the study where three scenarios – no AI access, GPT-4 AI access, or GPT-4 AI access with a prompt engineering overview – were considered.
The results of the consultants using AI were significantly more productive. They completed 12.2% more tasks on average than those who did not use AI, completed tasks 25.1% more quickly, and produced results which were more than 40% higher in quality compared to the non-AI-using control group.
The capabilities of AI created a “jagged technological frontier” where some tasks are easily done by AI, while others, though seemingly similar in difficulty level, are outside the current capability of AI. For tasks selected to be outside the AI-is-useful frontier, consultants using AI were 19 percentage points less likely to produce correct solutions compared to those without AI. As the landscape of generative AI’s capabilities improve over time, we will witness a variety of work which will be performed by AI through prompt engineering, making students or the future workers more productive. Hence it is strongly recommended to expose students to AI in ways that keep them engaged while exploring the AI landscape. We as faculty need to create assessments which allow students to work collaboratively with AI and not misuse AI for plagiarism purposes. One of the ways of achieving that could be to allow students design more original or analytical work and check their approach against those of AI systems, and compare their performance against AI. Such approach taken by students in my business analytics course helped them to realize that AI may not be doing all the work well or as well as they perceive it to do. Examples of such tasks were an in-class assignment where students were asked to plan their own holiday with limited resources such as income and availability of goods and services. They then verified their choices and decisions by prompting generative AI to lay down some results using the same scenarios or constraints they have experienced in their decision making. Students surprisingly found that they used a variety of data and came up with same or better solutions for themselves than the Generative AI in most situations. This approach of using AI as a fact-checker or a consultant to their own project enabled them to realize that AI may not be the most efficient and most knowledgeable in this case, just as BCG study figured out that the AI frontier is a jagged one with productivity gains and losses associated with its use in different contexts.
Useful References
Course: “Teaching with Cases”, Aug 11-12, 2023, Harvard Business School
Prof. Laura Cruz, Research Support, Schreyer Institute for Teaching Excellence
TWT – Teaching with Technology Presentation on AI for faculty by Zach Lonsinger, Jessie Driver and Sara Davis, Penn State University
Dell'Acqua, F., McFowland, E., Mollick, E. R., Lifshitz-Assaf, H., Kellogg, K., Rajendran, S., ... & Lakhani, K. R. (2023). Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality. Harvard Business School Technology & Operations Mgt. Unit Working Paper, (24-013).
