XR Mandatory Trainee Proficiency Before OR or Not? Which Would You Do?

In the current training environment and with so many great technological advancements, much discussion has been centered around the future of medical and surgical education. In the spirit of this discussion, we entertained the question of whether our trainees should be required to use extended reality (XR) technologies to achieve a certain level of proficiency before engaging with hands-on surgical experiences. These technologies include virtual reality (VR), augmented reality (AR), and mixed reality (MR). 

Our con debate is headlined by Anaïs Rameau, MD, MPhil, MS, who is an associate professor, National Institute of Health (NIH)-funded laryngologist, and director of new technologies and chief of dysphagia in the department of otolaryngology at Weill Cornell Medical College in New York. She has clinical expertise in voice, cough, airway, and swallowing dysfunction, and scientific expertise in wearable device technology, AI application to voice and video data, and low-cost tech for low-resource settings.

Our pro debate is led by Eric Gantwerker, MD, MSc, MS, who is a pediatric otolaryngologist at Northwell Health/Cohen Children’s Medical Center and an associate professor of otolaryngology at Zucker School of Medicine at Hofstra/Northwell in New Hyde Park, N.Y. He holds a Master of Medical Science in medical education with a special focus on educational technology, educational research, and game-based learning.

Dr. Rameau: Against XR Mandatory Trainee Proficiency Before OR

Though the promise of improving trainees’ performance in the operating room with extended reality (XR) is exciting, I see several arguments against mandatory use prior to surgical participation.

By far the most convincing argument is that the validity and effectiveness of these simulation technologies have not been demonstrated at scale. Current research is primarily at the pilot stage, with few prospective and external validations. A recent scoping review (Int Orthop. doi: 10.1007/s00264-022-05663-z) of XR in surgery broadly found that most studies focused on XR and surgical training. Among these studies, measures of validity of XR programs varied, including construct validity (Does the program differentiate performance level based on trainee experience?), face validity (Is the program realistic?), content validity (Is the program representative of the surgical task it aims to measure?), and concurrent validity (How well does the program compare to existing training programs?). Some studies focused on surgical outcomes after XR training, including surgery duration, complications, degree of hand/head movement, and reported self-confidence.

With such a diversity of measures of validity and effectiveness, it is hard to draw firm conclusions on the value of XR in surgical training, and there is no basis for making such training tools mandatory. This problem is further compounded by evidence of the superiority of traditional dry lab simulations to XR-based training. Within otolaryngology–head and neck surgery (OHNS), researchers conducted a scoping review of XR applications and found that most studies remained at the proof-of-concept level to demonstrate feasibility, most relied on retrospective surgical data to create XR environments and none measured surgical outcomes prospectively, and most studies rely on subjective assessment of the XR value and not on quantitative measures of clinical benefit (J Clin Med. doi: 10.3390/jcm13216295).

Although it remains to be studied, the cognitive load of XR surgical training could actually impede learning among trainees. In other words, the current evidence for XR applications in OHNS does substantiate a mandate to use it in training, and it remains to be proven that XR is superior to existing training methods, including well-established ones in the dry lab and the cadaver lab.

Other barriers may be surmountable with advances in technology, but they remain real at this time. These include the cost and complexity of developing a realistic XR surgical training program. An augmented reality training pilot program typically costs between $25,000 and $50,000 or more, depending on the instructional design, procedure complexity, and custom features (Roundtable Learning. https://tinyurl.com/4v23xna8). XR software costs do not include the cost related to expert surgeons’ time. To create a high-quality scenario, the participation of expert surgeons in content development and feedback is mandatory. Getting feedback from multiple expert surgeons would be ideal to reach a consensus on content and represent diverse surgical scenarios and interventions. Expert surgeons’ time is not only scarce but also extremely valuable.

Robust validation of the XR training program, as mentioned earlier, is essential and bears its own cost, especially if done rigorously. One should, therefore, not equate the current low cost of VR/AR headsets with the actual cost of an XR software training program. Given the lack of robust data on return on investment, it is unclear if such large software expenses are justified. Although some argue that XR training may increase access to surgical training, the reality is that the current cost of VR/AR headsets may be prohibitive in low-resource settings.

Furthermore, given the high cost of developing XR-based surgical training software, creators of the software may be more motivated to monetize their work than to make it open access. VR/AR headsets are regularly updated, software developed for new hardware may not be compatible with older models, and it may be a high financial burden for residency programs to purchase new headsets every few years. The XR training program cost will likely improve with time as generative AI continues to advance with its already-proven capabilities in video creation. This will likely lead to more cost-effective, high-quality software development that could be performed by less highly skilled engineers.

Finally, there are at least two technical arguments against mandatory XR training in OHNS that relate to clinical informatics. The first one is that software compatibility is currently an existing issue with the various XR platforms. Interoperability issues may limit the ability to operate surgical training software depending on the brand of AR/VR headsets a residency program owns. Interoperability is a common issue in the medical industry and a complex one to resolve.

The second issue has to do with the privacy and security of patients’ and trainees’ data. Many XR platforms are developed by companies that have a vested interest in obtaining more real-world data on patients’ surgical cases and presentations and trainees’ performance to advance their own research and development programs. Reviewing XR companies’ data flows and practices is a complex and often lengthy process. Many healthcare institutions have developed committees and protocols to review tech companies’ data-sharing practices and conformity with HIPAA and institutional policy before issuing business associate agreement contracts. The liability of these contracts may be too burdensome for small XR startups to sign, which could hamper their success and ultimately hurt the rich startup ecosystem needed to support substantial innovation in XR surgical training.

Though XR’s potential in surgical training is exciting, there is a clear dissonance between the hype and the reality. As I argued, there is a lack of evidence for the validity of current prototypes, costs are prohibitive for widespread adoption of XR mandatory training, and finally, interoperability and data security challenges are common problems that are complex to resolve.

Dr. Gantwerker: For XR Mandatory Trainee Proficiency Before OR