New Resident Assessment Standards Slow to Catch On

Problem-based learning: that residents learn from experience and errors, show evidence of improved patient care, incorporate information technology into their practice, and learn to adapt to changing scenarios.

In addition, Dr. Jewett’s surveys found:

• At least 66% of faculty and 85% of residents did not feel comfortable explaining, implementing, or measuring the system-based practice competency; and
• At least 77% of residents and 50% of faculty did not feel comfortable measuring the remaining general competencies.

We have identified a significant knowledge gap in the understanding of these competencies as well as several challenges that remain in the implementation of validated measurement tools, Dr. Jewett said.

Why Change Assessment?

The issue grew out of attempts to standardize graduate medical education, he said-attempts that date to the earliest years of the Triological Society, with discussions on how to assess medical education found in minutes of meetings occurring as early as 1912.

The Accreditation Council of Graduate Medical Education (ACGME) outcomes project aims at assessing learning objectives based on competencies, developing valid and reliable methods of assessment, and developing outcomes data that facilitate continuous improvement.

Dr. Jewett said there are differences between the current accreditation process and what was sought previously. The previous model for accreditation focused on the potential of the program and the assessment of the structure and processes of the program, he explained. Typically, administrators were asked: Does the graduate medical education program comply with requirements? Has the program established objectives and a curriculum? Does it evaluate residents as well as the program itself?

New Focus for Accreditation

Today there is a new model for accreditation. That new model focuses on the outcomes of the program-what has been accomplished by the resident training program-with an emphasis on methods used to assess those accomplishments, Dr. Jewett said.

Instead of wanting to know if a resident has attended the necessary classes, the new accreditation program wants institutions to figure out a way of telling if that resident actually learned anything by going to the class, he said.

The typical new questions that are being asked are: Is there evidence that the residents achieved the learning objectives? What kinds of measurement tools were used to make those assessments? How is the institution going to demonstrate continuous improvement in its educational processes? he said.

We have identified a significant knowledge gap in the understanding of these competencies as well as several challenges that remain in the implementation of validated measurement tools. – -Brian Jewett, MD