“Our objective was to figure out how otolaryngology residency programs viewed USMLE Step 1 and Step 2 when assessing applicants’ ability and whether the importance of applicant criteria would change after the transition to pass/fail,” said Lydia Yang, BS, a medical student and researcher at the University of Alabama, Birmingham.
Explore This Issue
August 2023Researchers developed a survey of seven multiple choice questions and two questions in which responders were asked to put together a ranked list of applicant criteria. The surveys were sent to 125 otolaryngology residency programs; the response rate was 40% for the survey and 24% for the rankings.
One question was, “Do you believe the USMLE Step scores accurately predict a resident’s ability to perform clinically in otolaryngology?” Fifty-eight percent said no for the Step 1 exam, while 12% said yes. For Step 2, 18% said yes, while 40% said no. Yang noted that while the percentage was low for Step 1, “not many more said yes for Step 2 versus Step 1.”
On, “Do you believe USMLE Step scores adequately predict a resident’s ability to pass otolaryngology board exams?” 52% said yes and 22% said no for Step 1, while 32% said yes and 28% said no for Step 2.
A majority of programs (66%) said that students would not be better prepared clinically with the Step 1 exam scored as pass/fail. Fifty percent of the otolaryngology programs said a student’s ranking would be considered more with the Step 1 exam becoming pass/fail.
Researchers found that the highest-ranked item in terms of usefulness was letters of recommendation, both before and after the change to pass/fail, when evaluating residency applicants. The mean number of otolaryngology abstracts, presentations, and publications ranked very high both before and after the change, as did the mean number of otolaryngology research experiences.
The item that rose the most in the evaluation was the Step 2 score, from an average ranking of 10.7 before the Step 1 change to the higher ranking of 7.8 after the change, said Yang. “Otolaryngology residency programs don’t believe the transition to pass/fail will better prepare students for residency,” she said. “Step 2 ideally would test clinical knowledge, but as you can see, only a small percentage of programs believe that otolaryngology students would be better prepared for residency.
“However,” she added, “Step 2 scores will still become more important when evaluating applicants, likely because schools are still looking for objective measures to report now that Step 1 scores have gone away.”