Martin Pielot
Bruno Cardoso
Nuria Oliver
Skip Abstract Section
Abstract
Many of today's mobile products and services engage their users proactively via push notifications. However, such notifications are not always delivered at the right moment, therefore not meeting products' and users' expectations. To address this challenge, we aim at developing an intelligent mobile system that automatically infers moments in which users are open to engage with suggested content. To inform the development of such a system, we carried out a field study with 337 mobile phone users. For 4 weeks, participants ran a study application on their primary phones. They were tasked to frequently report their current mood via a notification-administered experience-sampling questionnaire. In this study, however, we analyze whether they voluntarily engaged with content that we offered at the bottom of that questionnaire. In addition, the study app logged a wide range of data related to their phone use. Based on 120 Million phone-use events and 78,930 questionnaire notifications, we build a machine-learning model that before delivering a notification predicts whether a participant will click on the notification and subsequently engage with the offered content. When compared to a naïve baseline, which emulates current non-intelligent engagement strategies, our model achieves 66.6% higher success rate in its predictions. If the model also considers the user's past behavior, predictions improve 5-fold over the baseline. Based on these findings, we discuss the implications for building an intelligent service that identifies opportune moments for proactive user engagement, while, at the same time, reduces the number of undesirable interruptions.
- Saeed Abdullah, Elizabeth L. Murnane, Mark Matthews, Matthew Kay, Julie A. Kientz, Geri Gay, and Tanzeem Choudhury. 2016. Cognitive Rhythms: Unobtrusive and Continuous Sensing of Alertness Using a Mobile Phone. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’16). ACM, New York, NY, USA, 178--189. Google Scholar
Digital Library
- Piotr D. Adamczyk and Brian P. Bailey. 2004. If Not Now, when?: The Effects of Interruption at Different Moments Within Task Execution. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’04). ACM, New York, NY, USA, 271--278. Google ScholarDigital Library
- Ionut Andone, Konrad Blaszkiewicz, Mark Eibes, Boris Trendafilov, Christian Montag, and Alexander Markowetz. 2016. How Age and Gender Affect Smartphone Usage. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct (UbiComp ’16). ACM, New York, NY, USA, 9--12. Google ScholarDigital Library
- Daniel Avrahami and Scott E. Hudson. 2006. Responsiveness in Instant Messaging: Predictive Models Supporting Inter-personal Communication. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’06). ACM, New York, NY, USA, 731--740. Google ScholarDigital Library
- L. Breiman. 2001. Random forests. Machine Learning 45, 1 (2001), 5--32. Google ScholarDigital Library
- L. Breiman and J. Friedman. 1984. Classification and regression trees. Chapman and Hall/CRC, Monterey, USA.Google Scholar
- Yung-Ju Chang and John C. Tang. 2015. Investigating Mobile Users’ Ringer Mode Usage and Attentiveness and Responsiveness to Communication. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ’15). ACM, New York, NY, USA, 6--15. Google ScholarDigital Library
- Peter McFaul Chapman. 1997. Models of engagement: Intrinsically motivated interaction with multimedia learning software. Ph.D. Dissertation. University of Waterloo.Google Scholar
- T. Chen and C. Guestrin. 2016. XGBoost: a scalable tree boosting system. In Proc. of the ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining (KDD). 785--794. Google ScholarDigital Library
- Minsoo Choy, Daehoon Kim, Jae-Gil Lee, Heeyoung Kim, and Hiroshi Motoda. 2016. Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’16). ACM, New York, NY, USA, 1004--1015. Google ScholarDigital Library
- Anind K Dey, Katarzyna Wac, Denzil Ferreira, Kevin Tassini, Jin-Hyuk Hong, and Julian Ramos. 2011. Getting closer: an empirical investigation of the proximity of user to their smart phones. In Proc. UbiComp ’11. ACM. http://www.mediateam.oulu.fi/publications/pdf/1432.pdf Google ScholarDigital Library
- Richard Farmer and Norman D. Sundberg. 1986. Boredom Proneness -- The development and correlates of a new scale. Journal of Personality Assessment 50 (1986), 4--17. Issue 1.Google ScholarCross Ref
- Joel E. Fischer, Chris Greenhalgh, and Steve Benford. 2011. Investigating Episodes of Mobile Phone Activity As Indicators of Opportune Moments to Deliver Notifications. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI ’11). ACM, New York, NY, USA, 181--190. Google ScholarDigital Library
- Joel E. Fischer, Nick Yee, Victoria Bellotti, Nathan Good, Steve Benford, and Chris Greenhalgh. 2010. Effects of Content and Time of Delivery on Receptivity to Mobile Interruptions. In Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI ’10). ACM, New York, NY, USA, 103--112. Google ScholarDigital Library
- James Fogarty, Scott E. Hudson, Christopher G. Atkeson, Daniel Avrahami, Jodi Forlizzi, Sara Kiesler, Johnny C. Lee, and Jie Yang. 2005. Predicting Human Interruptibility with Sensors. ACM Trans. Comput.-Hum. Interact. 12, 1 (March 2005), 119--146. Google ScholarDigital Library
- T. Hastie, R. Tibshirani, and J. Friedman. 2009. The elements of statistical learning (2nd ed.). Springer, Berlin, Germany.Google Scholar
- Alexis Hiniker, Shwetak N. Patel, Tadayoshi Kohno, and Julie A. Kientz. 2016. Why Would You Do That? Predicting the Uses and Gratifications Behind Smartphone-usage Behaviors. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’16). ACM, New York, NY, USA, 634--645. Google ScholarDigital Library
- Joyce Ho and Stephen S. Intille. 2005. Using Context-aware Computing to Reduce the Perceived Burden of Interruptions from Mobile Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’05). ACM, New York, NY, USA, 909--918. Google ScholarDigital Library
- Eric Horvitz, Carl Kadie, Tim Paek, and David Hovel. 2003. Models of Attention in Computing and Communication: From Principles to Applications. Commun. ACM 46, 3 (March 2003), 52--59. Google ScholarDigital Library
- Eric Horvitz, Paul Koch, Raman Sarin, Johnson Apacible, and Muru Subramani1. 2005. Bayesphone: Precomputation of Context-Sensitive Policies for Inquiry and Action in Mobile Devices. In Proc UM ’05. Google ScholarDigital Library
- Shamsi T. Iqbal and Brian P. Bailey. 2008. Effects of Intelligent Notification Management on Users and Their Tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’08). ACM, New York, NY, USA, 93--102. Google ScholarDigital Library
- Shamsi T. Iqbal and Brian P. Bailey. 2010. Oasis: A Framework for Linking Notification Delivery to the Perceptual Structure of Goal-directed Tasks. ACM Trans. Comput.-Hum. Interact. 17, 4, Article 15 (Dec. 2010), 28 pages. Google ScholarDigital Library
- Yasumasa Kobayashi, Takahiro Tanaka, Kazuaki Aoki, and Kinya Fujita. 2015. Automatic Delivery Timing Control of Incoming Email Based on User Interruptibility. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’15). ACM, New York, NY, USA, 1779--1784. Google ScholarDigital Library
- Kostadin Kushlev, Bruno Cardoso, and Martin Pielot. 2017. Too Tense for Candy Crush: Affect Influences User Engagement With Proactively Suggested Content. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ‘17). ACM, New York, NY, USA. Google ScholarDigital Library
- Kostadin Kushlev, Jason Proulx, and Elizabeth W. Dunn. 2016. “Silence Your Phones”: Smartphone Notifications Increase Inattention and Hyperactivity Symptoms. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ’16). ACM, New York, NY, USA, 1011--1020. Google ScholarDigital Library
- R. Larson and M. Csikszentmihalyi. 1983. The experience sampling method. New Directions for Methodology of Social and Behavioral Science 15 (1983), 41?--56.Google Scholar
- Hugo Lopez-Tovar, Andreas Charalambous, and John Dowell. 2015. Managing Smartphone Interruptions Through Adaptive Modes and Modulation of Notifications. In Proceedings of the 20th International Conference on Intelligent User Interfaces (IUI ’15). ACM, New York, NY, USA, 296--299. Google ScholarDigital Library
- Gloria Mark, Daniela Gudith, and Ulrich Klocke. 2008. The Cost of Interrupted Work: More Speed and Stress. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’08). ACM, New York, NY, USA, 107--110. Google ScholarDigital Library
- Gloria Mark, Shamsi Iqbal, Mary Czerwinski, and Paul Johns. 2015. Focused, Aroused, but So Distractible: Temporal Perspectives on Multitasking and Communications. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work 8 Social Computing (CSCW ’15). ACM, New York, NY, USA, 903--916. Google ScholarDigital Library
- Gloria Mark, Shamsi T. Iqbal, Mary Czerwinski, and Paul Johns. 2014. Bored Mondays and Focused Afternoons: The Rhythm of Attention and Online Activity in the Workplace. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI ’14). ACM, New York, NY, USA, 3025--3034. Google ScholarDigital Library
- Afra Mashhadi, Akhil Mathur, and Fahim Kawsar. 2014. The Myth of Subtle Notifications. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication (UbiComp ’14 Adjunct). ACM, New York, NY, USA, 111--114. Google ScholarDigital Library
- Akhil Mathur, Nicholas D. Lane, and Fahim Kawsar. 2016. Engagement-aware Computing: Modelling User Engagement from Mobile Contexts. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’16). ACM, New York, NY, USA, 622--633. Google ScholarDigital Library
- Abhinav Mehrotra, Mirco Musolesi, Robert Hendley, and Veljko Pejovic. 2015. Designing Content-driven Intelligent Notification Mechanisms for Mobile Applications. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’15). ACM, New York, NY, USA, 813--824. Google ScholarDigital Library
- Abhinav Mehrotra, Veljko Pejovic, Jo Vermeulen, Robert Hendley, and Mirco Musolesi. 2016. My Phone and Me: Understanding People’s Receptivity to Mobile Notifications. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ’16). ACM, New York, NY, USA, 1021--1032. Google ScholarDigital Library
- T. M. Mitchell. 1997. Machine Learning. McGraw-Hill, New York, USA. Google ScholarDigital Library
- Elizabeth L. Murnane, Saeed Abdullah, Mark Matthews, Matthew Kay, Julie A. Kientz, Tanzeem Choudhury, Geri Gay, and Dan Cosley. 2016. Mobile Manifestations of Alertness: Connecting Biological Rhythms with Patterns of Smartphone App Use. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ’16). ACM, New York, NY, USA, 465--477. Google ScholarDigital Library
- Heather L O’Brien and Elaine G Toms. 2008. What is user engagement? A conceptual framework for defining user engagement with technology. Journal of the American Society for Information Science and Technology 59, 6 (2008), 938--955. Google ScholarDigital Library
- Tadashi Okoshi, Hiroki Nozaki, Jin Nakazawa, Hideyuki Tokuda, Julian Ramos, and Anind K. Dey. 2016. Towards attention-aware adaptive notification on smart phones. Pervasive and Mobile Computing (2016). Google ScholarDigital Library
- Tadashi Okoshi, Julian Ramos, Hiroki Nozaki, Jin Nakazawa, Anind K. Dey, and Hideyuki Tokuda. 2015. Reducing Users’ Perceived Mental Effort Due to Interruptive Notifications in Multi-device Mobile Environments. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’15). ACM, New York, NY, USA, 475--486. Google ScholarDigital Library
- Tadashi Okoshi, Kota Tsubouchi, Masaya Taji, Takanori Ichikawa, and Hideyuki Tokuda. 2017. Attention and Engagement-Awareness in the Wild: A Large-Scale Study with Adaptive Notifications. In IEEE International Conference on Pervasive Computing and Communications.Google Scholar
- F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: machine learning in python. Journal of Machine Learning Research 12 (2011), 2825--2830. Google ScholarDigital Library
- Veljko Pejovic and Mirco Musolesi. 2014. InterruptMe: Designing Intelligent Prompting Mechanisms for Pervasive Applications. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’14). ACM, New York, NY, USA, 897--908. Google ScholarDigital Library
- Veljko Pejovic, Mirco Musolesi, and Abhinav Mehrotra. 2015. Investigating The Role of Task Engagement in Mobile Interruptibility. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI ’15). ACM, New York, NY, USA, 1100--1105. Google ScholarDigital Library
- Martin Pielot. 2014. Large-scale Evaluation of Call-availability Prediction. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’14). ACM, New York, NY, USA, 933--937. Google ScholarDigital Library
- Martin Pielot, Karen Church, and Rodrigo de Oliveira. 2014. An In-situ Study of Mobile Phone Notifications. In Proceedings of the 16th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI ’14). ACM, New York, NY, USA, 233--242. Google ScholarDigital Library
- Martin Pielot, Rodrigo de Oliveira, Haewoon Kwak, and Nuria Oliver. 2014. Didn’t You See My Message?: Predicting Attentiveness to Mobile Instant Messages. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI ’14). ACM, New York, NY, USA, 3319--3328. Google ScholarDigital Library
- Martin Pielot, Tilman Dingler, Jose San Pedro, and Nuria Oliver. 2015. When Attention is Not Scarce - Detecting Boredom from Mobile Phone Usage. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’15). ACM, New York, NY, USA, 825--836. Google ScholarDigital Library
- Martin Pielot and Luz Rello. 2017. Productive, Anxious, Lonely ? 24 Hours Without Push Notifications. In MobileHCI ’17: Proceedings of the 18th International Conference on Human-computer Interaction with Mobile Devices and Services. Google ScholarDigital Library
- Benjamin Poppinga, Wilko Heuten, and Susanne Boll. 2014. Sensor-Based Identification of Opportune Moments for Triggering Notifications. IEEE Pervasive Computing 13, 1 (Jan. 2014), 22--29. Google ScholarDigital Library
- Stephanie Rosenthal, Anind K. Dey, and Manuela Veloso. 2011. Using decision-theoretic experience sampling to build personalized mobile phone interruption models. In Proc. Pervasive ’11. Springer-Verlag, 18. http://dl.acm.org/citation.cfm?id=2021975.2021991 Google ScholarDigital Library
- Alireza Sahami Shirazi, Niels Henze, Tilman Dingler, Martin Pielot, Dominik Weber, and Albrecht Schmidt. 2014. Large-scale Assessment of Mobile Notifications. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’14). ACM, New York, NY, USA, 3055--3064. Google ScholarDigital Library
- Hillol Sarker, Moushumi Sharmin, Amin Ahsan Ali, Md. Mahbubur Rahman, Rummana Bari, Syed Monowar Hossain, and Santosh Kumar. 2014. Assessing the Availability of Users to Engage in Just-in-time Intervention in the Natural Environment. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’14). ACM, New York, NY, USA, 909--920. Google ScholarDigital Library
- Florian Schulze and Georg Groh. 2016. Conversational Context Helps Improve Mobile Notification Management. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ’16). ACM, New York, NY, USA, 518--528. Google ScholarDigital Library
- Liam D. Turner, Stuart M. Allen, and Roger M. Whitaker. 2015. Interruptibility Prediction for Ubiquitous Systems: Conventions and New Directions from a Growing Field. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’15). ACM, New York, NY, USA, 801--812. Google ScholarDigital Library
- Liam D. Turner, Stuart M. Allen, and Roger M. Whitaker. 2017. Reachable but not Receptive: Enhancing Smartphone Interruptibility Prediction by Modelling the Extent of User Engagement with Notifications. Pervasive and Mobile Computing (2017). accepted Jan 31, 2017. Google ScholarDigital Library
- Tilo Westermann, Sebastian Möller, and Ina Wechsung. 2015. Assessing the Relationship between Technical Affinity, Stress and Notifications on Smartphones. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI ’15). ACM, New York, NY, USA, 652--659. Google ScholarDigital Library
- Tilo Westermann, Ina Wechsung, and Sebastian Möller. 2016. Smartphone Notifications in Context: A Case Study on Receptivity by the Example of an Advertising Service. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’16). ACM, New York, NY, USA, 2355--2361. Google ScholarDigital Library
- Fred Zijlstra, Robert Roe, Anna B. Leonora, and Irene Krediet. 1999. Temporal factors in mental work: Effects of interrupted activities. Journal of Occupational and Organizational Psychology 72 (June 1999), 163--185. Issue 2.Google ScholarCross Ref
Index Terms
- Beyond Interruptibility: Predicting Opportune Moments to Engage Mobile Phone Users
Recommendations
Too tense for candy crush: affect influences user engagement with proactively suggested content
MobileHCI '17: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and ServicesPush notifications are increasingly being used to engage users with app content. In the present research, we propose that users' current affect (i.e., how they are feeling) should be a critical---yet underexplored---factor in user engagement. ...
Interpersonal interruptibility: a framework and research program
CHI EA '08: CHI '08 Extended Abstracts on Human Factors in Computing SystemsTo date, research exploring interpersonal technology-mediated interruptions has focused on understanding how knowledge of an "interruptee's-local-context" can be utilized to reduce unwanted intrusions. However, the value of everyday interruptions are ...
The Influence Factors of Mobile Devices Adoption Among Older People
This article describes how the use of mobile devices has huge potential in assisting mobile users to perform everyday tasks. However, there is a small percentage of older people that uses mobile devices. This shows that, there are several factors that ...
Comments