Sebastian Boring
Dominikus Baur
Andreas Butz
Sean Gustafson
Patrick Baudisch
ABSTRACT
In 1992, Tani et al. proposed remotely operating machines in a factory by manipulating a live video image on a computer screen. In this paper we revisit this metaphor and investigate its suitability for mobile use. We present Touch Projector, a system that enables users to interact with remote screens through a live video image on their mobile device. The handheld device tracks itself with respect to the surrounding displays. Touch on the video image is "projected" onto the target display in view, as if it had occurred there. This literal adaptation of Tani's idea, however, fails because handheld video does not offer enough stability and control to enable precise manipulation. We address this with a series of improvements, including zooming and freezing the video image. In a user study, participants selected targets and dragged targets between displays using the literal and three improved versions. We found that participants achieved highest performance with automatic zooming and temporary image freezing.
Supplemental Material
- Balakrishnan, R., and Baudisch, P. (2009). Special Issue on Ubiquitous Multi-Display Environments. HCI Journal 24, 1 & 2.Google Scholar
- Ballagas, R., Rohs, M., and Sheridan, J.G. (2005). Sweep and point & shoot: phonecam-based interactions for large public displays. Ext. Abstracts CHI 2005, 1200--1203. Google ScholarDigital Library
- Ballagas, R., Borchers, J., Rohs, M., and Sheridan, J.G. (2006). The Smart Phone: a ubiquitous input device. IEEE Pervasive Computing 5, 1, 70--77. Google ScholarDigital Library
- Baudisch, P., Cutrell, E., Robbins, D., Czerwinski, M., Tandler, P., Bederson, B., and Zierlinger, A. (2003). Drag-and-pop and drag-and-pick: Techniques for accessing remote screen content on touch- and pen-operated systems. Proc. Interact 2003, 57--64.Google Scholar
- Bier, E.A., Stone, M.C., Pier, K., Buxton, W., and DeRose, T.D. (1993). Toolglass and magic lenses: the see--through interface. Proc. SIGGRAPH 1993, 73--80. Google ScholarDigital Library
- Boring, S., Altendorfer, M., Broll, G., Hilliges, O., and Butz, A. (2007). Shoot & copy: phonecam-based information transfer from public displays onto mobile phones. Proc. Mobility 2007, 24--31. Google ScholarDigital Library
- Buxton, W., and Myers, B. (1986). A study in two-handed input. Proc. CHI 1986, 321--326. Google ScholarDigital Library
- Fitzmaurice, G.W. (1993). Situated information spaces and spatially aware palmtop computers. Communications of the ACM 36, 7, 39--49. Google ScholarDigital Library
- Forlines, C., Vogel, D., and Balakrishnan, R. (2006). HybridPointing: fluid switching between absolute and relative pointing with a direct input device. Proc. UIST 2006, 211--220. Google ScholarDigital Library
- Guimbretière, F., Martin, A., and Winograd, T. (2005) Benefits of merging command selection and direct manipulation. ACM Transactions on Computer-Human Interaction 12, 3, 460--476. Google ScholarDigital Library
- Hinckley, K., Ramos, G., Guimbretière, F., Baudisch, P., and Smith, M. (2004). Stitching: pen gestures that span multiple displays. Proc. AVI 2004, 23--31. Google ScholarDigital Library
- Johanson, B., Hutchins, G., Winograd, T., and Stone, M. (2002). PointRight: experience with flexible input redirection in interactive workspaces. Proc. UIST 2002, 227--234. Google ScholarDigital Library
- Kabbash, P., Buxton, W., Sellen, A. (1994). Two-handed input in a compound task. Proc. CHI 1994, 417--423. Google ScholarDigital Library
- Kato, H., and Billinghurst, M. (1999). Marker Tracking and HMD Calibration for a Video-Based Augmented Reality Conferencing System. Proc. IEEE and ACM International Workshop on Augmented Reality, 85--94. Google ScholarDigital Library
- Latulipe, C., Kaplan, C.S., Clarke, C.L.A. (2005). Bimanual and unimanual image alignment: an evaluation of mouse-based techniques. Proc. UIST 2005, 123--131. Google ScholarDigital Library
- Liao, C., Liu, Q., Kimber, D., Chiu, P., Foote, J., and Wilcox, L. (2003). Shared interactive video for teleconferencing. Proc. ACM Multimedia 2003, 546--554. Google ScholarDigital Library
- Myers, B., Bhatnagar, R., Nichols, J., Peck, C.H., Kong, D., Miller, R., and Long, A.C. (2002). Interacting at a distance: measuring the performance of laser pointers and other devices. Proc. CHI 2002, 33--40. Google ScholarDigital Library
- Nacenta, M.A., Sallam, S., Champoux, B., Subramanian, S., and Gutwin, C. (2006). Perspective cursor: perspective-based interaction for multi-display environments. Proc. CHI 2006, 289--298. Google ScholarDigital Library
- Nacenta, M.A., Gutwin, C., Aliakseyeu, D., and Subra-manian, S. (2009). There and back again: cross-display object movement in multi-display environments. HCI Journal 24, 1, 170--229.Google Scholar
- Pears, N., Jackson, D., and Olivier, P. (2009). Smart phone interaction with registered displays. IEEE Computer 8, 2, 14--21. Google ScholarDigital Library
- Pierce, J.S., Forsberg, A.S., Conway, M.J., Hong, S., Zeleznik, R.C., and Mine, M.R. (1997). Image plane interaction techniques in 3D immersive environments. Proc. Symposium on Interactive 3D Graphics, 39--43. Google ScholarDigital Library
- Poupyrev, I., Billinghurst, M., Weghorst, S., and Ichikawa, T. (1996). The go-go interaction technique: non-linear mapping for direct manipulation in VR. Proc. UIST 1996, 79--80. Google ScholarDigital Library
- Rekimoto, J. (1997). Pick-and-drop: a direct manipulation technique for multiple computer environments. Proc. UIST 1997, 31--39. Google ScholarDigital Library
- Robertson, G., Czerwinski, M., Baudisch, P., Meyers, B., Robbins, D., Smith, G., and Tan, D. (2005). The large-display user experience. IEEE Computer Graphics and Applications, 25, 4, 44--51. Google ScholarDigital Library
- Rohs, M., Schöning, J., Raubal, M., Essl, G., and Krüger, A. (2007). Map navigation with mobile devices: virtual versus physical movement with and without visual context. Proc. ICMI 2007, 146--153. Google ScholarDigital Library
- Sakamoto, D., Honda, K., Inami, M., and Igarashi, T. (2009). Sketch and run: a stroke-based interface for home robots. Proc. CHI 2009, 197--200. Google ScholarDigital Library
- Sears, A., and Shneiderman, B. (1991). High precision touchscreens: design strategies and comparisons with a mouse. International Journal of Man-Machine Studies 34, 4, 593--613. Google ScholarDigital Library
- Seifried, T., Haller, M., Scott, S.D., Perteneder, C., Rendl, C., Sakamoto, D., Inami, M. (2009). CRISTAL: design and implementation of a remote control system based on a multi-touch display. Proc. ITS 2009, 33--40. Google ScholarDigital Library
- Shneiderman, B. (1983). Direct manipulation: a step beyond programming languages. IEEE Computer 16, 8, 57--69. Google ScholarDigital Library
- Shoemaker, G., Tang, A., and Booth, K.S. (2007). Shadow reaching: a new perspective on interaction for large displays. Proc. UIST 2007, 53--56. Google ScholarDigital Library
- Stoakley, R., Conway, M.J., and Pausch, R. (1995). Virtual reality on a WIM: interactive worlds in miniature. Proc. CHI 1995, 265--272. Google ScholarDigital Library
- Tan, D.S., Meyers, B., and Czerwinski, M. (2004). WinCuts: manipulating arbitrary window regions for more effective use of screen space. Ext. Abstracts CHI 2004, 1525--1528. Google ScholarDigital Library
- Tani, M., Yamaashi, K., Tanikoshi, K., Futakawa, M., and Tanifuji, S. (1992). Object-oriented video: interaction with real-world objects through live video. Proc. CHI 1992, 593--598. Google ScholarDigital Library
- Tsang, M., Fitzmaurice, G.W., Kurtenbach, G., Khan, A., Buxton, B. (2002). Boom chameleon: simultaneous capture of 3D viewpoint, voice and gesture annotations on a spatially-aware display. Proc. UIST 2002, 111--120. Google ScholarDigital Library
- Vogel, D., and Baudisch, P. (2007). Shift: a technique for operating pen-based interfaces using touch. Proc. CHI 2007, 657--666. Google ScholarDigital Library
- Wilson, A., and Shafer, S. (2003). XWand: UI for intelligent spaces. Proc. CHI 2003, 545--552. Google ScholarDigital Library
- Yee, K.-P. (2003). Peephole displays: pen interaction on spatially aware handheld computers. Proc. CHI'03, 1--8. Google ScholarDigital Library
Index Terms
- Touch projector: mobile interaction through video
Recommendations
TDome: A Touch-Enabled 6DOF Interactive Device for Multi-Display Environments
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing SystemsThe rapid evolution of multi-display environments (MDEs) has created a vacuum in need of novel input devices to optimize interaction in MDEs. In this paper, we propose TDome, a novel touch-enabled 6DOF input and output device to facilitate interactions ...
Lucid touch: a see-through mobile device
UIST '07: Proceedings of the 20th annual ACM symposium on User interface software and technologyTouch is a compelling input modality for interactive devices; however, touch input on the small screen of a mobile device is problematic because a user's fingers occlude the graphical elements he wishes to work with. In this paper, we present LucidTouch,...
Pointable: an in-air pointing technique to manipulate out-of-reach targets on tabletops
ITS '11: Proceedings of the ACM International Conference on Interactive Tabletops and SurfacesSelecting and moving digital content on interactive tabletops often involves accessing the workspace beyond arm's reach. We present Pointable, an in-air, bimanual perspective-based interaction technique that augments touch input on a tabletop for ...
Comments