Abstract
Abstract
Purpose
Large language models (LLMs), such as OpenAI’s ChatGPT, have demonstrated tutoring benefits in small-scale pilot studies within focused areas of medical education. This study evaluated the large-scale implementation of AI-teaching assistants (AI-TAs) within a compulsory medical school course.
Method
A quasi-experimental observational study with mixed methods was conducted to assess the impact of AI-TAs in a compulsory first-year medical school course at the University of Toronto in 2024. The research team developed AI-TAs using OpenAI’s ChatGPT-4o and introduced them as a supplementary resource at the course’s midpoint. They analyzed exam performance among the students who used AI-TAs (n = 87) and students who did not (n = 206). Additionally, surveys (n = 18) and interviews (n = 10) explored student perceptions of AI-TAs’ effectiveness, usability, and impact on learning.
Results
Students who would later adopt AI-TAs had significantly lower pre-intervention exam scores than their peers [83.8% vs 88.1%; t(118.8) = −3.82, P
Conclusions
AI-TAs correlated with improved exam performance, fewer students in academic difficulty, and enhanced student engagement through a psychologically safe learning environment. These findings suggest that AI-TAs can serve as a scalable, cost-effective tool to support struggling students, while complementing traditional instruction.