Within this tutorial, we want to explain the basics of using the MirrorLabs projects available online. These projects aim to simplify the development of AR/VR (or generic) visualization of ROS-based installations and simulations. As such, the MirrorLabs projects can be seen as an extension of the ROS# project published by Dr. M.Bischoff (Siemens AG).
The MirrorLabs project includes correctly set up projects, a library of tested sample scenes, and a description of how to deploy your application. Hence, already with limited unity experience, anyone can create interesting, functional AR/VR applications.
Note: Within this tutorial, we will focus on the Unity side of things. The installation of the necessary ROS components will only be covered briefly.
Let’s get started!
Step 01: ROS setup
(Assuming you have a ROS distro is already installed, if not please follow the tutorials on ros.org)
1. Update your setup $ sudo apt-get update
2. Install the rosbridge_suit (adapt the ROS-distro in the command if necessary) $ sudo apt-get install ros-kinetic-rosbridge-server
3. Download the file_server packge published by Siemens a) Go to https://github.com/siemens/ros-sharp/tree/master/ROS download it as a zip file and
unpack it on your desktop b) Move the "file_server" directory to the /src directory of the catkin workspace (e.g.
/catkin_ws/src/file_server) c) Add the ros_sharp_communication.launch to the launch procedure Open the launch file of your desired simulation and add the following lines:
<include file="$(find file_server)/launch/ros_sharp_communication.launch" > <arg name="port"
value="9090"/></include>>
4. Rebuild your catkin workspace:
$ catkin_make
Step 02: Download the project
We first need to download the projects from the online repository (https://github.com/fate4gle/MirrorLabs.git , subject to change in the future) and save them somewhere locally. We highly recommend the usage of an SSD!
1. Download the repository
a. Either use a Github client or click on the download button (“Code”)
b. Extract the downloaded file to a location which suits your preference
c. Note the path of your project folder somewhere.
In the past we recommended Unity 2019.3.10f, however, since then the long-term supported version of Unity 2019 changed to Unity 2019.4.XYf we will use this version from now. If this is not present on your workstation, please follow the installation guide (https://mirrorlabs-ide.wixsite.com/mirrorlabs/post/tutorial-getting-started-with-ar) and select the available 2019.4.XYf (LTS) version.
Step 03: Adding projects to Unity Hub
Next up we need to let Unity know about our downloaded projects. (Once this is done successfully, we will see our projects listed within Unity Hub.)
1. Start Unity Hub and click on “ADD”
2. In the pop-up window, select one of the downloaded projects. (In this case, we will proceed with the HoloLens version (“MIrrorLabs_HL2”).
3. The selected project will appear in the list of projects! If in the “Unity Version” column “Select a version” is written in light grey: please click on it and select the install that is available on your workstation. If a warning regarding Unity versioning is presented, please approve. As long as you are using the 2019.4 (LTS) build, the projects will work just fine.
With all prerequisites downloaded and installed, it is time to explore MirrorLabs projects! Within this tutorial, we will work on the visualization of a UR10. For this we can use two methods, 1) using the included sample scene and 2) making the scene from scratch using the “scene configurator” extension. In this tutorial, we will only use the sample scene. However, in the next tutorial, we will have a closer look at the MirrorLabs extension for creating a new scene from scratch!
Step 04: Augmented Reality UR10 using the sample scenes
1. In Unity Hub, click on the “MirrorLabs_HL2” to start the pre-configured HoloLens 2 Project. (Note: this might take a moment to load for the first time. Time for a coffee.)
2. Once the project is fully loaded, in the project folder (where all our assets are stored), navigate to Assets -> MirrorLabs -> Scenes.
Here, all of the pre-made sample scenes are located. Each of these is functional and tested to work out-of-the-box. This is intended to be the starting point for any new robot specific project.
3. Now we double-click the ML_UniversalRobotic_ur10 scene to load it.
4. All that is left to do is configure the ROS connection! For this, in the hierarchy, click on RosConenctors and select the child-object ur10.
5. This RosConnector is responsible for the data transfer from and to ROS. To get our project ready to go, we only need to adapt the Ros Bridge Server URL (IP). For this, simply enter the remote IP of the ROS-System where the ros_sharp_communication is launched.
6. Press Play and watch what happens in the Unity Console tab. Either it will inform you about a successful connection or after 5 seconds warns about a time-out. If the later is the case, please verify you typed in the IP address correctly and hit play again.
7. Once the connection is established successfully, we only need to adjust the topics we want to subscribe to. Within each of the sample scenes, the RosConnector has the most commonly used subscribers and publishers attached. (E.g. JointStateSubscriber, JointStatePublisher, and OdometrySubscriber) For this example, we only want to visualize what a physical or simulated robot will do. This information is published in the /joint_states topic in our ROS configuration. (Make sure to activate the checkbox next to the script!)
8. Give the scene a final test by hitting the Play button again. The robot should mirror the movements in ROS with barely (if at all) noticeable delay!
Now all that is left to do is to enjoy the fruits of our work using the HoloLens! Simply follow the building/Deployment guide of the previous tutorial (https://mirrorlabs-ide.wixsite.com/mirrorlabs/post/tutorial-getting-started-with-ar) and enjoy.
Thanks for your attention! If you encounter difficulties, please do not hesitate to contact us!
Yorumlar