Holographic Assistance Using Spatial Computing (XR/VR/AR/MR) in Endoscopic Biliary Interventions

To further enhance the efficiency and safety of endoscopic biliary interventions for complex conditions such as hilar cholangiocarcinoma and cases involving surgically altered anatomies we developed a cloud-based service called Holoeyes MD. By leveraging Spatial Computing (XR/VR/AR/MR), this system has been approved as a regulated medical device and offers an innovative way to enhance the efficiency and safety of endoscopic biliary interventions, particularly for complex conditions such as hilar cholangiocarcinoma and patients with surgically altered anatomies.When operators wear video pass-through goggles, they can view endoscopic images and floating 3D organ models simultaneously. This setup allows clear visualization of tumors, biliary strictures, and extraluminal structures, facilitating optimal puncture and stent placement while reducing the risk of accidental injury. Moreover, preoperative simulation and postoperative review in XR space enable detailed replay of procedural maneuvers, improving treatment planning, communication among team members, and education for trainees.By combining spatial computing with holographic assistance, this approach offers a novel pathway to enhance both efficiency and safety in minimally invasive endoscopic procedures. Here, we present clinical data and real-world insights and discuss prospects and challenges for this groundbreaking technology.