A plastic surgery research group in Japan has developed an augmented reality (AR) system that enables surgeons to create 3D simulations of the desired outcomes of facial reconstructive procedures and project them over a patient’s face during surgery. Details of their simulations were published in August in Plastic and Reconstructive Surgery—Global.
Koichi Ueda, MD, PhD, a professor of surgery in the department of plastic and reconstructive surgery at Osaka Medical College in Japan, and colleagues developed an AR system to evaluate improvements of the body surface. Through their initial experience with eight cases, the researchers concluded that the software could be a useful guide to planning, performing, and evaluating the results of facial reconstruction.
The team used a high-definition digital camera to capture 3D images of the facial surface and CT scans to obtain digital information on the underlying facial bones for each patient. These digital data were then manipulated to create 3D simulations of the ideal final results. For example, in a patient with a fractured cheekbone, the reconstruction was simulated by obtaining and reversing an image of the opposite, uninjured bone.
Using a pair of commercially available smart glasses, the surgeons were able to superimpose the 3D digital simulation image of the desired appearance over the patient’s face during surgery. The investigators used free, open source software to solve various technical problems, including manipulating and displaying the 3D simulations and registration with the surgical field.
The AR system helped in planning and confirming reconstruction of the underlying facial bones, for example, in a patient with a congenital bone development disorder and another patient with a complex facial fracture. In all cases, the 3D simulation of the body surface provided a visual reference of the final facial appearance.
The group plans further studies to confirm the benefits of using the AR system during plastic and reconstructive surgery and to refine the display method for comparing before and after images. Future innovations may include the ability to quantitatively evaluate the improvements in body surface and even to enable simple navigation of internal organs.
Drs. Ueda and Mitsuno believe AR technology could also become a useful tool for teaching surgical skills.