Can Large Language Models Help with Meeting Accreditation Standards? An Exploration of the Opportunities and Limitations

The release of ChatGPT, Google Bard, and other artificial intelligence (AI) large language models (LLMs) have captured the world's attention for the potential they hold in facilitating work. While perhaps at the "peak of inflated expectations" of the Gartner Hype curve with regard to LLMs, there is little doubt that AI and LLMs will play an increasing role in our daily roles as physicians, educators, and program leaders. Among the opportunities is the use of LLMs to help with meeting program accreditation standards, from the ability to analyze and categorize large qualitative data sets (e.g., learner critiques, faculty descriptive assessments, survey results), to helping with writing reports, to a source for ideas for conducting program evaluation. In this workshop, presenters will explore how AI LLMs could be used to facilitate the work of clerkship directors, program directors, deans, and DIOs, through worked examples and real time analyses, while highlighting and discussing the limitations and concerns, including regulatory issues, with using LLMs for such purposes.