DOI
10.5703/1288284317862
Description
This project presents a digital twin framework for controlling a hydraulic crane using AI, mixed reality, and real-time actuation. Leveraging a Jetson Nano and Xbox Kinect for perception and a Raspberry Pi with DRV103 for control, the system enables adaptive motion planning and obstacle-aware navigation. A Unity-based interface integrated with HoloLens2 allows operators to visualize and manipulate the crane through FABRIK inverse kinematics. Real-time environmental mapping from Kinect enables obstacle detection to update the motion path dynamically. Early experiments confirm the ability to compute joint angles in real-time, detect environmental objects, and demonstrate closed-loop control in a virtual-physical hybrid system.
Demonstration of a Digital Twin framework for a two-actuator hydraulic application
This project presents a digital twin framework for controlling a hydraulic crane using AI, mixed reality, and real-time actuation. Leveraging a Jetson Nano and Xbox Kinect for perception and a Raspberry Pi with DRV103 for control, the system enables adaptive motion planning and obstacle-aware navigation. A Unity-based interface integrated with HoloLens2 allows operators to visualize and manipulate the crane through FABRIK inverse kinematics. Real-time environmental mapping from Kinect enables obstacle detection to update the motion path dynamically. Early experiments confirm the ability to compute joint angles in real-time, detect environmental objects, and demonstrate closed-loop control in a virtual-physical hybrid system.