One of the greatest impacts augmented reality (AR) headsets can have is by increasing the intuitiveness of human-robot collaboration when humans and robots share the same workspace. We know that a robot can have multiple information sources to gain awareness about a factory floor and any ongoing manufacturing process. The information can be gained by onboard sensors and well as interfacing with cloud-systems managing the entire production environments. Humans are often expected to take on the role of supervisor/overseer and intervene as needed. An augmented reality headset can provide information for humans within the production environment by overlaying any relevant system data on real world objects. In addition to visualizing augmented information, the AR device can be used to intuitively control any robots within the scene.
At the University of Tartu we are focusing on utilizing Microsoft HoloLens2 for improving the usability for controlling robots within the operator’s working environment. With the help of the AR headset we can render any conventional robot controller (e.g., gamepad, keyboards, touchscreens) obsolete as the operator can use virtual markers in collaboration with gestures to designate goal and waypoints for the robot by pointing to real-world objects and locations.
In collaboration with MirrorLabs activity, the students and researchers are developing AR-based free-hands user-experience for mobile robots in shared dynamic space. Our demonstrator uses the open-source mobile robot Robotont (http://robotont.ut.ee/) but the solution is designed to work for any ROS Navigation compatible robot.
