Immersive HRI
As robots become a more integral part of our everyday lives, interactions with robots will become more common. Through the use of Extended Reality (XR), we enable users to interact with robots in a more immersive way, allowing them to better understand the robot’s capabilities and limitations.
We explore the use of robots in XR as a safe, immersive, and helpful teaching tool to allow users of varying robotics experiences to interact with robots.
Immersive HRI: ERUPT
The Extended Reality Universal Planning Toolkit (ERUPT) is an open-source XR system for interactive robot motion planning, enabling users to configure environments, plan trajectories, and visualize robot behavior in immersive VR/AR.
Immersive HRI: Learning from Demonstration
The Robot Action Demonstration in Extended Reality (RADER) system provides a generic XR interface for learning from demonstration, enabling users to safely demonstrate robotic trajectories using hand tracking in augmented reality.
Publications
- Ngui, I. , McBeth, C. , Santos, A. , He, G. , Mimnaugh, K.J. , Motes, J.D. , Soares, L. , Morales, M. , & Amato, N.M. (2025). ERUPT: An Open Toolkit for Interfacing with Robot Motion Planners in Extended Reality. ArXiv Preprint. https://doi.org/https://doi.org/10.48550/arXiv.2510.02464
- Ngui, I. , McBeth, C. , He, G. , Santos, A.C. , Soares, L. , Morales, M. , & Amato, N.M. (2024). Extended Reality System for Robotic Learning from Human Demonstration. ArXiv Preprint. arXiv