Starting in 2015, undergraduates wanting to go to medical school will be taking a revamped version of the Medical College Admission Test (MCAT), with new sections focusing on social and behavioral sciences (see “Major Changes on the Horizon for the MCAT” in the August issue of ENT Today).
But that’s not the only planned change to the med school admissions process. The Association of American Medical Colleges (AAMC) is preparing a whole new slate of recommendations for improving the way med schools screen students. The changes will be designed to look at more than just classroom and test performance. They’re an effort to get at what kind of people the applicants are—and whether they have the qualities that will make them good, caring doctors.
The Multiple Mini-Interview
The new recommendations are a turn toward what the AAMC is calling “holistic admissions.” The goal is to assess each student’s “pre-professional attributes,” including communication skills, ability to work as part of a team, ethics sensibilities and other traits.
“Admissions requires a toolbox,” said Darrel Kirch, MD, president and CEO of AAMC. “Historically, we’ve very heavily relied on the MCAT because it was the standardized, accepted tool. We’re trying to improve the MCAT as a tool, but we’re also trying to improve other tools … so that the toolbox is a better toolbox.”
The changes will involve a new and, hopefully, improved application process. For example, one of the main ideas is to ask students to describe situations or experiences they’ve had that show they have the desired qualities. The AAMC, which is still fine-tuning the process, is also working with med school admissions offices to improve letters of recommendation so that they provide more relevant information.
Still another, and perhaps more significant, change will involve recommended improvements to the interviewing process, including adding more tailored questions and observations of role-playing scenarios and utilizing many interviewers rather than just one or two. “We’re seeing more and more innovation around interviewing to help it become more robust,” Dr. Kirch said.
One model for the interview process changes is the multiple mini-interview (MMI), in which med school applicants move from station to station undergoing one “mini-interview” after another. Each interview is designed to assess a particular trait and usually lasts about eight minutes, with a couple of minutes between stations. During these breaks, the applicant usually has a chance to review the next station’s question.
McMaster University School of Medicine near Toronto began using the MMI in 2005 after an evaluation period of about two years. The idea stems from the Objective Structured Clinical Exam, which measures clinical competence using a similar station-style format. Robert Whyte, MD, assistant dean of the undergraduate medical education program at McMaster’s Michael G. DeGroote School of Medicine, said admissions officials figured, “‘Well, if we can do that to see if people have acquired the skills to be doctors, then maybe we could use the same concept to see if people have the skills or attributes to enter medicine in the first place.’”
McMaster gets about 4,500 applicants to med school. They take an online version of a test that assesses the relevant student characteristics, and just under 600 of them are chosen to go through the MMI process.
“We all have bias,” Dr. Whyte said. “Our bias is minimized when we have more of us doing a sample of an applicant.” Success on the MMI has correlated with success in clerkship performance and on licensing examinations (Med Educ. 2007;41:378-384.). “The MMI predicts best for how residents do on the communication part and the more non-technical or more non-academic characteristics of physicians,” he said.
Fifteen of the 17 medical schools in Canada are now using the MMI. The MMI is not used nearly as much in the U.S., but it is catching on. The University of Cincinnati College of Medicine was the first U.S. institution to use this process.
Stephen Manuel, PhD, assistant dean of admissions at the University of Cincinnati College of Medicine, said he was skeptical at first, wondering whether it was really possible to assess a student in just eight minutes. When he saw the MMI in action, his doubts were erased, he said. “When you interview someone for a job, they don’t just interview with one person,” Dr. Manuel said. “You want people with different perspectives.”
The traditional interviewing process at medical schools is seriously flawed, he said, a statement that he noted is not only supported by the research literature but can also be seen anecdotally in Internet chat rooms in which students trade stories about their interviews. He described it this way: “Student No. 1: ‘Interview went great. He’s from New York, I’m from New York. We talked about the Mets.’ Student No. 2: ‘Interview went horrible.’ You will see such diversity in the students and how they were treated during the interview, how they felt about the interview and how they were assessed.”
Bias Is Inevitable
Barry Conchie, who leads the global leadership research and development program for Gallup, a company that helps corporations and organizations find executives, said that when screening is flawed, it can result in hiring exactly the wrong kind of person for the job. For instance, humility is often given as a top trait companies want in a leader, but that trait actually negatively correlates with the qualities of a top-performing leader.
Bias is inevitable, he said, which is why Gallup officials interview candidates by telephone, not knowing height, skin color or physical disabilities. He also said that it’s crucial for every interview to include multiple appraisers. In the MMI, while there are multiple stations, there is generally just one interviewer per station.
Multiple interviewers are needed even when the criteria being evaluated are precisely known beforehand, Conchie said. “The fact that it’s an individual lays it open to bias,” he said. “The issue about bias is that the individual’s unaware of it. I mean, that’s what bias means, right? Bias isn’t something that afflicts other people. We all have them. I have them … I would never trust my own judgment in a face-to-face interview on my own with a candidate, however explicit the criteria.”
Better Results for Patients
Robert Miller, MD, executive director of the American Board of Otolaryngology and physician editor of ENT Today, said better screening of candidates should be a top priority of the AAMC, especially to weed out those with the potential for professional misconduct. And he expressed support for Gallup’s approach. “We need a better way of screening for these issues,” he said.
Fred Telischi, MD, chairman of the department of otolaryngology at the University of Miami Miller School of Medicine, said that a better focus on the non-academic qualities of med school applicants could yield better results for patients in the long run.
“We know that people who have a better outlook, or a more positive outlook, will frequently do better,” he said. “There are certainly examples of physicians who see too many patients in a day and don’t spend enough time with them. So, consequently, patients don’t feel like they’ve been heard, and that affects their treatment. They won’t listen to the doctor, they’ll be less compliant with medication, they’ll be more likely to go see another doctor and use more health care resources.”