On the Fly Robotic-Assisted Medical Instrument Planning and Execution Using Mixed Reality
<|reference_start|>On the Fly Robotic-Assisted Medical Instrument Planning and Execution Using Mixed Reality: Robotic-assisted medical systems (RAMS) have gained significant attention for their advantages in alleviating surgeons' fatigue and improving patients' outcomes. These systems comprise a range of human-computer interactions, including medical scene monitoring, anatomical target planning, and robot manipulation. However, despite its versatility and effectiveness, RAMS demands expertise in robotics, leading to a high learning cost for the operator. In this work, we introduce a novel framework using mixed reality technologies to ease the use of RAMS. The proposed framework achieves real-time planning and execution of medical instruments by providing 3D anatomical image overlay, human-robot collision detection, and robot programming interface. These features, integrated with an easy-to-use calibration method for head-mounted display, improve the effectiveness of human-robot interactions. To assess the feasibility of the framework, two medical applications are presented in this work: 1) coil placement during transcranial magnetic stimulation and 2) drill and injector device positioning during femoroplasty. Results from these use cases demonstrate its potential to extend to a wider range of medical scenarios.<|reference_end|>
@article{ai2024on,
    title={On the Fly Robotic-Assisted Medical Instrument Planning and Execution
  Using Mixed Reality},
    author={Letian Ai, Yihao Liu, Mehran Armand, Amir Kheradmand, Alejandro
  Martin-Gomez},
    journal={2024 IEEE ICRA, Yokohama, Japan, 2024, pp. 13192-13199},
    year={2024},
    doi={10.1109/ICRA57147.2024.10611515},
    archivePrefix={arXiv},
    eprint={2404.05887},
    primaryClass={cs.RO}
}