Overview

The Extended Reality Universal Planning Toolkit (ERUPT) is an open-source XR system designed to support interactive robot motion planning. ERUPT enables users to configure environments, plan robot trajectories, and visualize robot behavior within immersive virtual or augmented reality environments.

By leveraging extended reality, ERUPT provides operators with enhanced spatial understanding and more natural interaction modalities compared to traditional screen-based tools.

Motivation

Traditional robot motion planning tools (e.g., MoveIt + RViz) rely on 2D visualizations displayed on a monitor. This can limit spatial awareness and make it difficult to intuitively understand robot behavior in complex environments.

ERUPT addresses this limitation by enabling:

  • Immersive 3D visualization of robot motion
  • Direct manipulation of environment objects
  • Real-time reconfiguration of planning scenes
  • Safe previewing of robot trajectories before execution

This immersive interface improves interpretability, usability, and safety in human-robot interaction workflows.

System Architecture

ERUPT integrates:

  • Unity + OpenXR for immersive visualization and interaction
  • ROS2 for backend robotic communication
  • MoveIt for motion planning
  • Unity ROS-TCP Connector for synchronization

The system maintains synchronization between the XR environment and the MoveIt planning scene, ensuring that object modifications and robot configurations remain consistent across platforms.

ERUPT System Diagram

Figure 1: System diagram showing the XR interface, ROS2 integration, and MoveIt planning pipeline.

XR Interface and Object Interaction

ERUPT’s XR interface allows users to directly interact with both the robot and the environment. Objects can be added, scaled, rotated, and translated using a wrist menu or direct manipulation in 3D space. This allows rapid evaluation of environment changes and natural planning interactions.

Wrist Menu Object Selection

Figure 2: (a) Wrist Menu for adding and editing objects, (b) Selecting and modifying objects in XR.

Robots can also be placed in the environment using QR markers for AR or positioned in VR spaces for immersive planning. Operators can adjust joint angles or move the end-effector to define start and goal configurations for motion planning.

QR Code Robot Placement

Figure 3: (a) Using a QR code for AR placement, (b) Placing a virtual robot in the XR environment.

Key Capabilities

  • Interactive environment reconfiguration in XR
  • Direct manipulation of robot joints and end-effector
  • Customizable motion planning requests
  • Trajectory preview in immersive 3D
  • Execution of validated trajectories on physical robots
  • Open-source architecture for extensibility

Demonstrations

ERUPT has been demonstrated in both simulated and physical robot settings, showcasing interactive planning and environmental adaptability.

Environment Reconfiguration

Users can rearrange objects in XR and replan trajectories in real time, simulating dynamic workspaces.

Env Reconfig 1 Env Reconfig 2 Env Reconfig 3 Env Reconfig 4 Env Reconfig 5 Env Reconfig 6

Figure 4: Time-lapse of rearranging a virtual environment and replanning robot trajectories. The user adjusts object positions and observes the robot’s updated path through multiple replanning steps.

Trajectory Previewing

Planned trajectories can be previewed in XR before executing on a physical robot. This allows the operator to verify path feasibility and safety, improving confidence and reducing risk.

Trajectory Preview 1 Trajectory Preview 2 Trajectory Preview 3 Trajectory Preview 4 Trajectory Preview 5 Trajectory Preview 6

Figure 5: Time-lapse of previewing a planned trajectory in XR. The operator inspects the path relative to virtual obstacles to ensure safety before executing the motion on a physical robot.

Publications

Updated: