Creating Meaningful Competency-Based Evaluations for Milestones 2.0

With the rollout of the ACGME milestones 2.0 comes the opportunity to create new evaluations and assessment methods. The milestones are designed to provide formative feedback and help residents understand how they are progressing developmentally through training. Residency programs frequently rely on evaluation data from rotations to inform the clinical competency committee assessment of performance on the sub-competencies. However, choosing the right questions for evaluations to decide subcompetency ratings is difficult. In addition, it can be challenging to get accurate data as some sub-competencies are not regularly observed on a given rotation. Finally, while competency-based medical education focuses on a resident's ability to progress at their own rate, traditional evaluation questions target behaviors seen at a particular stage of training. As such, those evaluations may not accurately assess residents who may be advancing more rapidly than their PGY year.

In this workshop, we will describe the techniques we implemented to create new evaluations that align with milestones 2.0 at two different residency programs. We will share our process to ensure that the most appropriate questions are asked for each rotation and our strategy to assess subcompetencies not observed during a specific rotation. We will also discuss some of the challenges of assessing resident clinical competence and brainstorm possible solutions together.