Lesson learned? “An organized educational program that uses review questions to assess knowledge can be used to track acquisition of medical knowledge, identify gaps, and facilitate the development of remediation plans that result in improved medical knowledge,” Dr. Grillone said.
Explore This Issue
December 2014Ask the Right Questions
Before the start of each academic year, professors should research their residents’ knowledge base and adapt lectures accordingly, said Pamela A. Rowland, PhD, professor of surgery and associate chair of education at the University of North Carolina at Chapel Hill. “If most people know what the parathyroid is and how it functions, and one person doesn’t, assign readings to that person ahead of time. Otherwise, you’ll end up boring the other people in that room,” she said.
Dr. Rowland recommended sending students an e-mail with questions before each new topic is introduced. For students who indicate a lack of knowledge about the topic, she suggested sending them a list of recommended readings or videos to watch so that they can catch up with the rest of the class.
Evaluate Surgical and Clinical Skills
How do you identify which residents need to improve their surgical and clinical skills while there’s still time to train them? For Dr. Grillone and his colleagues at Boston University, the way to do this is through regular assessments. Approximately 48 hours before they participate in a surgery, residents complete a surgical competency assessment form that tests their knowledge of the case, including rationale for the procedure, surgical steps involved, and any potential risks for that patient. They must then e-mail the form back to the attending, who adds his or her comments. The day of the surgery, the resident brings a copy of the evaluation form, and, after the procedure, the attending completes the backside of the form, which uses a Likert scale to evaluate the resident’s performance. Questions focus on understanding of the rationale and indications for surgery, knowledge of how to do the surgical procedure, overall technical skill, use of time and motion, communication with the surgical team, and ability to code the case correctly.
Dr. Grillone uses a similar tool, the Clinical Competency Assessment Tool, or CCAT, to assess competency in an outpatient clinical setting. “We’re glad we implemented this system because when the ACGME Milestones were introduced last year, we realized this would be an important component of how we could make meaningful and valid assessments of resident competency, Dr. Grillone said.
Give Immediate Feedback
According to Nasir I. Bhatti, MD, associate professor of otolaryngology-head and neck surgery and anesthesiology critical care medicine and director of Adult Tracheostomy Airway Services at Johns Hopkins University School of Medicine in Baltimore, the key to better surgical evaluations is to give feedback as close to the end of the surgery as possible. In an arrangement similar to the one used at Boston University, residents at Johns Hopkins are evaluated by a scoring system called objective structured assessment of technical skill (Visit enttoday.org for an example of this form). Dr. Bhatti and his colleagues authored a study published in The Laryngoscope in 2012 that reported that evaluations completed within six days were more reliable and indicative of actual performance than those completed after six days (2012;122:2418-2421).
When more time passes between an actual performance and evaluation, a lot of recall bias comes in. An evaluation based on a fresh recollection is a true reflection of theperformance.
—Nasir I. Bhatti, MD
“When more time passes between an actual performance and evaluation, a lot of recall bias comes in,” he said. “If I had a good impression of you in a previous testing, I will score you higher. An evaluation based on a fresh recollection is a true reflection of the performance.”