Enhancing Student Engagement With Custom GPTs

• GPTs tailored to specific applications can provide students with personalized, instant feedback that works to enhance their critical thinking and communication skills.
• At the Indian School of Business, custom GPTs encourage students to write more thoughtful responses to assignments, while reducing opportunities for academic dishonesty.
• AI offers educators a dynamic solution that boosts student engagement and introspection, making learning more interactive and effective.

 
It has been nearly 15 years since Srikant Datar, David A. Garvin, and Patrick Cullen released their book Rethinking the MBA, in which they call for business schools to place greater emphasis on complex skills such as critical thinking and communication. But what are the best ways to cultivate those skills in future leaders?

Researchers such as Daniel Kahneman and Gary Klein in their 2009 paper have pointed to the need for timely and unambiguous feedback to master such complex skills. Such feedback is best delivered via personal tutoring. In his 1984 paper, Benjamin S. Bloom shows that personal tutoring can result in as much as a two-sigma improvement in student performance. Bloom noted that, when tutored, an average student performs at the 96th percentile of those taught by traditional methods.

But as effective as we know personal tutoring can be, it can be challenging for higher education institutions to deliver. The main issue is the lack of scalability due to its cost and a shortage of qualified faculty.

As early as almost 100 years ago, cognitive psychologist Sidney L. Pressey of Ohio State University proposed technology to solve this problem. Pressey developed a “teaching machine” that gave feedback on student responses to multiple-choice questions and even delivered a candy after a predetermined number of correct responses!

Yet, in all this time, the promise of that technology has not been realized—until now.

Designing an AI Intervention

My co-instructor, Harish Raichandani, and I teach Corporate Governance and Business Ethics, a course in an MBA program for working executives at the Indian School of Business (ISB) in Hyderabad. When we started teaching this course in August 2022, GPT technology was unknown. We gave traditional assignments that required students to submit write-ups. These took us more than month to grade, even with the help of teaching assistants.

In 2023, with the rise of ChatGPT, we soon realized that students could cut and paste responses that they generated with the AI platform, and Turnitin would fail to detect plagiarism. So, we switched to asking groups of students to record oral responses to questions given on the spot (noting that their grades would be affected if only a few students did most of the talking). But while effective, this method resulted in an even longer grading process.

It was in 2024 that I read about custom GPTs, a feature that allows users to tailor versions of ChatGPT to suit their specific purposes. I visualized the possibility of creating a tool that could provide students with personalized guidance and instant feedback on critical writing assignments.

Because ChatGPT does not record student responses, I wrote a web application for the purpose. The application communicated with the ChatGPT platform on the back end using the OpenAI application programming interface. Because I was unfamiliar with today’s tools, such as the Flask package in Python, I used large language models such as Anthropic’s Claude 2 for help creating the app. Developing, testing, and debugging the GPTs and the app took me about a week.

As we experimented with custom GPTs, Harish and I saw that we could achieve what we wanted through simple prompts. After some further tweaking and testing, we recently introduced our custom intervention to the 73 students enrolled in our course.

Keeping Our Prompts Simple

We created a custom GPT for each assignment, giving it the following instruction: “You are an expert viva examiner examining executive MBA students in a course on business ethics and corporate governance.” We then asked it to generate questions for each of our assessments.

Students accessed each assignment through the web app. After authenticating the student, the app presented the student with questions generated on the spot by the custom GPT and waited for the student’s response. Once the student answered, the app sent the response to the GPT and displayed the output.

Although we did not mention climate change or stakeholder capitalism in our instructions, the GPT came up with these topics for the questions independently.

When asking the GPT to generate the questions, we kept our instructions simple. For example, for Part 1 of an assignment on Milton Friedman’s 1970 article about the social responsibility of business, we gave the AI platform this brief instruction: “Give three multiple-choice questions that test whether the student has read the article.”

For Part 2 of the assignment, we told the custom GPT to “ask an open-ended question demonstrating a basic understanding of a core message from Friedman’s essay.” Based on this instruction, it produced questions such as “Explain the core message of Friedman’s argument regarding the social responsibility of businesses” and “Discuss what Friedman means by the ‘rules of the game’ in the context of corporate social responsibility.”

Finally, in Part 3, we instructed the custom GPT to ask a question that “tests students on whether they can apply the arguments to issues arising in contemporary times and carefully analyze Friedman’s opinion in that context.” In our prompt, we made it clear that “students should be able to articulate their opinion on the validity of Friedman’s arguments, consider counterarguments to their stated position, and draw a conclusion.”

In response, the GPT asked questions such as, “In the context of contemporary socioeconomic issues, such as climate change, how do you argue [whether] Friedman’s viewpoint holds or does not hold?” and “Given the rise of stakeholder capitalism, where businesses are increasingly expected to account for the interests of all stakeholders, not just shareholders, evaluate Friedman’s argument.”

Keep in mind, we did not mention climate change or stakeholder capitalism in our instructions. The GPT came up with these topics for the questions independently.

Using AI as Personal Tutor

Next, we instructed the AI to act as a tutor to help students refine their answers in parts 2 and 3 of the assignment. For this purpose, we entered the following prompt: “Once students have submitted their initial responses, you will give them an opportunity to add further to their response, hinting at specific areas they could address.”

At this stage, the AI let students know what they did well and what they could improve upon. This gave them opportunities to introspect, which is critical for learning complex skills. By inviting students to add to their responses in real time and offering suggestions for improvement, the AI made the assignment less stressful, more educational, and more engaging—indeed, a couple of students even completed the assignment twice.

We also asked the GPT to grade these assignments. To train the AI in the grading process, we wrote the following simple prompt, assigning point values for each of the three questions: “You will evaluate and award a maximum of 12 points at the end of a student's submission (Q1–3 points; Q2–4 points; Q3–5 points).” Even though we did not provide the GPT with an elaborate rubric, we found that its grading ability was good. The class average was about 10.5 points out of 12.

Students were free to approach us if they felt the grading was incorrect. A few students came to us, but it was mostly for grading errors in Part 1, where the GPT made obvious mistakes, such as saying, “The correct answer is ‘B’ and your response was ‘B.’” However, we did not receive a single request to regrade the open-ended questions in parts 2 and 3.

Protecting Academic Integrity

The suggested time to complete the assignment was 30 minutes, with a hard stop at the 60-minute mark (most students took about 40 minutes). Since the questions were generated on the spot by the GPT, students had little opportunity to look up answers on the web or in another AI tool.

The real-time tutoring that AI platforms such as ChatGPT can provide is a game changer in higher education.

Moreover, the application did not permit students to paste any text—students had to type their responses. We could have reduced the possibility for cheating even further by requiring students to agree to be recorded through their webcams. But we did not feel it necessary to use such an intrusive method in an executive MBA program.

As we deployed this assignment, we did keep issues of privacy in mind. Even though OpenAI promises that the data sent to its platform will not be used to train its models, we advised students not to reveal any identifying information in their interactions with the GPT.

Future Directions and Final Say

We plan to continue fine-tuning the prompts to the custom GPTs. Our goal is to ensure that, when students provide shallow responses to questions, the technology challenges them to go into more depth. Additionally, we are working on a more detailed grading rubric for the next version.

Now that we can turn to AI for help, we also intend to return to oral group assignments. From our experience, we have found that students come well-prepared to these recordings, because they do not want to let other group members down. Students also told us that they found group work useful in fostering positive classroom dynamics. For these reasons, we believe that combining oral recordings with the GPT’s tutoring will be a significant improvement over the first iteration of this pedagogical approach.

In Rethinking the MBA, Datar, Garvin, and Cullen point out that MBA education is at a crossroads. They find that MBA students have been engaging less and less with course material, and the time they spend preparing has decreased substantially over time. For instance, the authors offered a stark comparison: In 1975, one institution found that its students spent 45 to 50 hours per week attending and preparing for classes; by 2003, this number had decreased to 30.

Digital distractions have grown substantially since 2010, when their book was published. Moreover, in large classes, professors do not have the time to provide detailed feedback to every student on every assignment. Given these factors, it’s not surprising that students treat assignments as necessary evils rather than learning opportunities, and take shortcuts where they can.

But we now have a viable solution to these challenges. Because students’ every keystroke can be recorded and time-stamped, the technology reduces both motive and opportunity for cheating, two of the three sides of the fraud triangle. In addition, AI can quickly generate and grade assignments, saving instructors valuable time.

And because AI can offer immediate, tailored feedback, it offers true learning opportunities for students. The real-time tutoring that AI platforms such as ChatGPT can provide is a game changer in higher education. Pressey’s “teaching machine” is finally here.

I thank Seema Chowdhry, director of ISB’s Centre for Learning and Teaching Excellence, for her help in creating the GPT tool for our course and in preparing this article.