Overview
Creating an internal medicine residency evaluation tool can be a challenging task. Many programs struggle with resident assessment as it pertains to obtaining high quality information via evaluation tools to guide milestone reporting.   The internal medicine milestones serve as a tool for both formative (ongoing) and summative (periodic) feedback. Programs use the milestones to provide residents with specific, actionable feedback on their performance relative to the expected level of competence for their stage of training. Although not the intended use, many programs have resorted to using the milestones alone as their evaluation tool. Many evaluators report interpretation of questions and length of evaluations as barriers to providing quality assessments and feedback. Large language models (LLMs) are advanced artificial intelligence (AI) systems capable of recognizing, understanding, and generating text. When used properly, they can tackle complex tasks such as developing and streamlining assessment tools to improve data collection.  This workshop is designed for program directors, faculty, and medical educators who are looking to enhance their resident evaluation processes by integrating the internal medicine milestones with AI and LLMs. Participants will explore the fundamentals of the internal medicine milestones and how these competency-based benchmarks can be leveraged to create more comprehensive, aligned evaluations with a particular focus on streamlining these tools to optimize the information received from faculty members. Through interactive demonstrations and practical exercises, attendees will learn how to develop AI-enhanced evaluation tools to improve assessment processes, gather meaningful faculty feedback, and ultimately benefit resident professional growth.

Speakers

Allison Ferrara, MD Joshua Shultz, MD Andrea Berry, MPA

Content Track
Technology and Innovation

Audience

GME

Program Type
University-Based Programs, Community-Based Programs

Additional Information

Year Published: 2025 - AIMW 2025