The board provided several training sessions as well as a post-exam debriefing, which was quite helpful from a community standpoint. —Andrew J. Holcomb, MD
Explore This Issue
August 2021
The board’s second advantage concerned its exam schedule. Under pre-COVID-19 circumstances, the primary certification exam—generally taken by 300-plus candidates—is administered just a couple of days after the board’s much smaller neurotology exam, which had just 25 registered examinees in 2020. So, board leadership decided to separate the exam administrations by more than the usual few days. “We thought this would be a good opportunity to administer the virtual exam with the neurotology group in December 2020, and then schedule the primary exam for February 2021,” Dr. Nussenbaum said. “That way, we would be able to apply whatever we learned from the smaller group experience to the larger one. We would have time to make necessary changes and adjustments.”
Following the decision, approximately 5% of candidates who were eligible to take the February 2021 oral certification exam deferred, for reasons that might or might not have been related to the virtual format, noted Dr. Nussenbaum. In addition, given the unusual circumstances, the board granted all deferral requests with no questions asked and no time lost from candidates’ five-year board eligibility period. After deferrals, 308 candidates took the primary certification virtual exam.
Adapting the Format
As it turned out, the board’s strategy to administer the smaller neurotology virtual exam well in advance of the primary one was a crucial call. “Although our method worked well with the small group for the neurotology exam, we also learned that the process and the technology didn’t allow for easy scalability of the process to the numbers that we needed to administer the primary certification exam,” Dr. Nussbaum said. “That meant that we had to fit the exam format construction to the constraints of the technology. Fortunately, that two-month window allowed us to rethink the format structure and determine what changes needed to be made.”
Customarily, the in-person version of the ABO-HNS certification exam is approximately four hours long. During the process, the candidate visits five different stations, each of which presents three protocols during a 40-minute period. Concerned about the challenging logistics of so much movement of multiple candidates into multiple virtual rooms, as well as the likelihood of participant “Zoom fatigue,” the board scaled down the exam. Instead of five stations, there was to be just a single video conference room in which all the examiners were present. The three cases per station was reduced to two shorter-in-length cases. The newly designed virtual exam clocked in at 90 minutes.
Given the reduction in cases to analyze, great care was taken with the exam’s content selection, noted Dr. Nussenbaum. “We worked very closely with our psychometrician in a tremendous team effort to ensure that the redesign would still allow us to create a psychometrically valid exam, enabling us to make consequential pass/fail decisions on the candidates,” he explained.
Andrew J. Holcomb, MD, head and neck surgeon at Estabrook Cancer Center, Nebraska Methodist Hospital, and assistant professor in the department of surgery at Creighton University in Omaha, Neb., experienced the new virtual format firsthand when he took the exam in February 2021. Although Dr. Holcomb attests to major advantages offered by the revamped approach, he did view the shortened time per protocol as a disadvantage. “It challenged many candidates to adequately progress through the prompts in the allotted time,” he said. “There were prompts that would address only a portion of the clinical scenario, which was confusing to candidates both in terms of the approach and grading implications. That said, I did feel that the exam tested a broad range of clinical knowledge and judgement, and that the tone felt similar to what I would have imagined an in-person environment to provide.”
More feedback is forthcoming when the ABO-HNS virtual board exam is administered again in October 2021. Among the seated examinees will be Meha Goyal Fox, MD, who recently completed fellowship training in rhinology and skull base surgery in the department of otolaryngology–head and neck surgery at the University of Kansas Medical Center, Kansas City, Kan. Dr. Fox also has some concerns about the shortened format. “It leaves less time for examinees to explain their decisions, which could be detrimental,” she said. “The folks I’ve spoken to who took the exam in February said they felt a little rushed on the cases.”