Abstract
Debugging and monitoring robotic applications is a very intricate and error-prone task. To this end, we propose a mixed-reality approach to facilitate this process along a concrete scenario. We connected the Microsoft HoloLens smart glass to the Robot Operating System (ROS), which is used to control robots, and visualize arbitrary flight data of a quadrocopter. Hereby, we display holograms correctly in the real world based on a conversion of the internal tracking coordinates into coordinates provided by a motion capturing system. Moreover, we describe the synchronization process of the internal tracking with the motion capturing. Altogether, the combination of the HoloLens and the external tracking system shows promising preliminary results. Moreover, our approach can be extended to directly manipulate source code through its mixed-reality visualization and offers new interaction methods to debug and develop robotic applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
HoloLens-Rosbridge. https://github.com/roastedpork/hololens-rosbridge. Accessed 13 Feb 2019
HoloLens Tracking-System. https://docs.microsoft.com/en-us/windows/mixed-reality/enthusiast-guide/tracking-system. Accessed 13 Feb 2019
Optitrack. http://optitrack.com. Accessed 15 Feb 2019
ROS Vizualization rviz. http://wiki.ros.org/rviz. Accessed 13 Feb 2019
ROSBridge. http://wiki.ros.org/rosbridge_suite. Accessed 13 Feb 2019
TCPROS. http://wiki.ros.org/ROS/TCPROS. Accessed 15 Feb 2019
Bamburry, D.: Drones: Designed for product delivery. Des. Manag. Rev. 26(1), 40–48 (2015)
Ben-Kiki, O., Evans, C., Ingerson, B.: Yaml ain’t markup language (yaml™) version 1.1. Technical report, p. 23 (2005). yaml.org
Breckel, A., Tichy, M.: Live programming with code portals. In: Workshop on Live Programming Systems - LIVE 2016 (2016)
Gageik, N., Müller, T., Montenegro, S.: Obstacle detection and collision avoidance using ultrasonic distance sensors for an autonomous quadrocopter. University of Wurzburg, Aerospace information Technologhy (Germany) Wurzburg, pp. 3–23 (2012)
Ghiringhelli, F., Guzzi, J., Di Caro, G.A., Caglioti, V., Gambardella, L.M., Giusti, A.: Interactive augmented reality for understanding and analyzing multi-robot systems. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), pp. 1195–1201. IEEE (2014)
Green, T.R.G., Petre, M., et al.: Usability analysis of visual programming environments: a ‘cognitive dimensions’ framework. J. Vis. Lang. Comput. 7(2), 131–174 (1996)
Hehn, M., D’Andrea, R.: Quadrocopter trajectory generation and control. In: IFAC World Congress, vol. 18, pp. 1485–1491 (2011)
Hoenig, W., Milanes, C., Scaria, L., Phan, T., Bolas, M., Ayanian, N.: Mixed reality for robotics. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5382–5387. IEEE (2015)
Leutert, F., Herrmann, C., Schilling, K.: A spatial augmented reality system for intuitive display of robotic data. In: Proceedings of the 8th ACM/IEEE International Conference on Human-Robot Interaction, pp. 179–180. IEEE Press (2013)
Magnenat, S., Ben-Ari, M., Klinger, S., Sumner, R.W.: Enhancing robot programming with visual feedback and augmented reality. In: Proceedings of the 2015 ACM Conference on Innovation and Technology in Computer Science Education, pp. 153–158. ACM (2015)
Milgram, P., Takemura, H., Utsumi, A., Kishino, F.: Augmented reality: a class of displays on the reality-virtuality continuum. In: Telemanipulator and Telepresence Technologies, vol. 2351, pp. 282–293. International Society for Optics and Photonics (1995)
Quigley, M., et al.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software, Kobe, Japan, vol. 3, p. 5 (2009)
Ralston, S.E.: Augmented vision for survey work and machine control, US Patent 6,094,625, 25 July 2000
Ritz, R., Müller, M.W., Hehn, M., D’Andrea, R.: Cooperative quadrocopter ball throwing and catching. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4972–4978. IEEE (2012)
Schobel, J., Pryss, R., Probst, T., Schlee, W., Schickler, M., Reichert, M.: Learnability of a configurator empowering end users to create mobile data collection instruments: usability study. JMIR mHealth uHealth 6(6) (2018)
Stolaroff, J.: The need for a life cycle assessment of drone-based commercial package delivery. Technical report, Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States) (2014)
Taylor II, R.M., Hudson, T.C., Seeger, A., Weber, H., Juliano, J., Helser, A.T.: VRPN: a device-independent, network-transparent VR peripheral system. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, pp. 55–61. ACM (2001)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Hoppenstedt, B. et al. (2019). Debugging Quadrocopter Trajectories in Mixed Reality. In: De Paolis, L., Bourdot, P. (eds) Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2019. Lecture Notes in Computer Science(), vol 11614. Springer, Cham. https://doi.org/10.1007/978-3-030-25999-0_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-25999-0_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-25998-3
Online ISBN: 978-3-030-25999-0
eBook Packages: Computer ScienceComputer Science (R0)