Evaluate Better: A Guide to Using Milestones – Webinar Recap

21 April, 2021

As we look to spring Clinical Competency Committee (CCC) meetings, it’s time to evaluate your own evaluation process. Milestones spring reporting is taking place now through June 25, 2021, and these competency-based markers can be a powerful assessment tool for both individual residents and fellows, as well as your program curriculum as a whole … if used appropriately.

 

As a relatively new assessment tool, ACGME Milestones have created a paradigm shift in how we evaluate, and you may find yourself questioning whether or not your program is maximizing their potential. How do Milestones fit into other assessment tools? What role does the CCC play in Milestones? And how can Milestones be leveraged to help strengthen individual skills and overall curriculum development?

 

We recently had the opportunity to host a webinar, “Evaluate Better: A Guide to Using Milestones”, presented by Kate McDaniel, PhD, MSW, to help Program Directors, CCC members, and GME program faculty better understand the role and purpose of Milestones so you can — as the name suggests — evaluate better. Dr. McDaniel is an Assistant Professor and the GME Educator in charge of Faculty Development at Duke University Hospital and Health System in the department of Neurosurgery, and spent five years as the GME Education Specialist at St. Louis University. Below are some key takeaways from Dr. McDaniel’s presentation. You can watch the full webinar here.

 

What is the purpose of ACGME Milestones within the context of GME assessment systems?

No single tool can determine readiness for independent practice. Milestones are not meant to be stand alone evaluation forms or direct assessment tools, but rather to provide a framework for residency programs to move trainees toward competent unsupervised practice. These Milestones create a shared language for residents and faculty, describing the learning trajectory for trainees within the core competencies.

 

Milestones were first introduced into the graduate medical education landscape in 2013. What is unique about them is that they are organized by behavioral anchors, which tie to subcompetencies within the six ACGME core competencies.

Using multiple assessment tools — formative and summative — gives a clearer picture of how your trainee is doing. From end of rotation evaluations, to direct observation and case logs, there are numerous assessment tools available, and all should be considered when determining Milestone markers.

 

Before Milestones, there was a wide variance on what deemed a trainee “competent.” Milestones help to somewhat standardize competency in a new way that’s based on actual behavior. Although a specific achievement level is not necessary in every milestone or subcompetency a level 4 competency is the target for graduation; level 5 is aspirational.

 

How the Clinical Competency Committee (CCC) informs Milestone ratings

All ACGME accredited programs are required to have a Clinical Competency Committee (CCC). Its members are appointed by the Program Director and must include at least three faculty members, who meet a minimum of two times per year to discuss resident performance. Specific details about CCC composition expectations are found in the Program Requirements (both in the Common Program Requirements for Residencies, Fellowships, One-Year Fellowships, and Post-Doctoral Education Programs, and also in the specialty- and subspecialty-specific Program Requirements).

The CCC plays a critical role in Milestone reporting. It is the funnel all resident assessments go through before determining a rating for each subcompetency within your specialty’s Milestones. Committee members review all data — rotational evaluations, peer evaluations, case logs, etc. — and use it to determine a rating for each subcompetency. These Milestone assessments are provided to the Program Director who then reviews and reports them to the ACGME.

 

Milestones are not a shortcut. Dr. McDaniel stressed in the webinar that it can be tempting to use the Milestone rubric in the place of other assessment tools, but that doing so diminishes the value of Milestones, which are meant to help distill a much broader picture.

 

How to leverage Milestones to assist struggling residents and perform curricular assessment

Milestone evaluation gives you an opportunity to look at both individual trainees as well as the curriculum as a whole.

You can identify struggling residents sooner, and help them create a clearer improvement plan. For example, if a resident was at a level 2, but they are expected to be at a level 3, you can create an actionable plan with steps to take to get there. This can be particularly helpful in subcompetencies that are less clinical; “harmonized” Milestones such as communication skills.

You may find, when assessing Milestones, that you have some gaps in your teaching, your assessments, or both. If you map how each subcompetency is taught and the tool(s) that are used to evaluate performance, you could discover that you don’t have adequate data to evaluate a Milestone and can adjust your curriculum or evaluations accordingly.

Improve your Milestone management with MedHub

Milestones can help paint a better picture of your trainees’ progress, but there’s a perception that it’s an administrative burden. MedHub can empower you to prepare practice-ready physicians by tracking milestone-based competencies, instituting higher quality evaluation practices, facilitating better procedure feedback and allow for reviewing a complete learner portfolio.

 

Learn more about MedHub’s Performance Assessment.