Daniel McDuff
Mary Czerwinski
ABSTRACT
We present AffectAura, an emotional prosthetic that allows users to reflect on their emotional states over long periods of time. We designed a multimodal sensor set-up for continuous logging of audio, visual, physiological and contextual data, a classification scheme for predicting user affective state and an interface for user reflection. The system continuously predicts a user's valence, arousal and engage-ment, and correlates this with information on events, communications and data interactions. We evaluate the interface through a user study consisting of six users and over 240 hours of data, and demonstrate the utility of such a reflection tool. We show that users could reason forward and backward in time about their emotional experiences using the interface, and found this useful.
- Aigner, W., Miksch, S., Schumann, H. and Tominski,C. Visualization of Time-Oriented Data. Springer, 2011. Google Scholar
Digital Library
- Boehner, K., DePaula, R., Dourish, P. and Sengers, P. Affect: from information to interaction. In Proc. CC'05, ACM (2005), 59--68. Google ScholarDigital Library
- Boucsein, W. Electrodermal Activity. Springer-Verlag, New York, 1992.Google Scholar
- Bradley, M.M. and Lang, P.J. Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Ther Exp Psy, 25, 1 (1994), 49--59.Google ScholarCross Ref
- Brush, AJ, Meyers, B.R., Tan, D.S. and Czerwinski, M. Understanding memory triggers for task tracking. In Proc. CHI'07, ACM (2007), 947--950. Google ScholarDigital Library
- Cahill, L. and McGaugh, J.L. A novel demonstration of enhanced memory associated with emotional arousal. Conscious Cogn, 4 (1995), 410--421.Google ScholarCross Ref
- Cootes, T.F., Edwards, G.J. and Taylor, C.J. Active appearance models. IEEE Pattern Anal (2001), 681--685. Google ScholarDigital Library
- Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J. et al. Active shape models-their training and application. Comput Viz Image Und (1995), 38--59. Google ScholarDigital Library
- Daily, S. B. and Picard, R. W. Girls Involved in Real Life Sharing: Utilizing Technology to Support the Emotional Development of Teenaged Girls, J School Counseling, 5,1(2007), 20.Google Scholar
- De la Torre, F. and Agell, C. Multimodal Diaries. In Proc. ICME '07 (2007), IEEE, 839--842.Google Scholar
- Dietz, R. and Lang, A. Affective agents: Effects of agent affect on arousal, attention, liking and learning. In Proc. COGNITIVE'11, (1999).Google Scholar
- Donath, J., Dragulescu, A., Zinman, A., and Viegas, F. Data portraits. In Art Gallery ACM SIGGRAPH (2010). Google ScholarDigital Library
- Ebbinghaus, H. (1885). Memory: A Contribution to Experimental Psychology. Tran.Ruger & Bussenius(1913).Google Scholar
- El Kaliouby, R., Teeters, A. and Picard, R.W. Invited Talk: An Exploratory Social-Emotional Prosthetic for Autism Spectrum Disorders. In BSN'06, (2006), IEEE. Google ScholarDigital Library
- Fernandez, R. A computational model for the automatic recognition of affect in speech. Massachusetts Institute of Technology, 2004.Google Scholar
- Goldberger, J. and Roweis, S. and Hinton, G. and Salakhutdinov, R. Neighbourhood Components Analysis. In Proc. NIPS'04, MIT (2004).Google Scholar
- Healy, J. and Picard, R. W. StartleCam: A cybernetic Wearable Camera, In Proc. 2nd International Symposium on Wearable Computers, 1998. Google ScholarDigital Library
- Higgins, E.T. Beyond pleasure and pain. Am Psychol (1997), 1280.Google Scholar
- Fogarty, J., Ko, A.J., Aung, H.H., Golden, E., Tang, K.P. and Hudson, S.E. Examining task engagement in sensor-based statistical models of human interruptibility. Proc. CHI', ACM (2005), 331--340. Google ScholarDigital Library
- Jax, P. and Vary, P. On artificial bandwidth extension of telephone speech. Signal Processing (2003), 1707--1719. Google ScholarDigital Library
- Kapoor, A. and Picard, R.W. Multimodal affect recognition in learning environments. In Proc. MULTIMEDIA'05 ( 2005), ACM, 677--682. Google ScholarDigital Library
- Larsen, R.J. and Diener, E. Promises and problems with the circumplex model of emotion. Pers Soc Psychol Rev (1992), 25--59.Google Scholar
- Liu, K.K.L. A personal, mobile system for understanding stress and interruptions. MIT, 2004.Google Scholar
- Loewenstein, G. and Lerner, J.S. The role of affect in decision making. In Handbook of affective science. Oxford University Press, Oxford, 2003.Google Scholar
- Nicolaou, M.A., Gunes, H. and Pantic, M. Continuous Prediction of Spontaneous Affect from Multiple Cues and Modalities in Valence-Arousal Space. IEEE T Affect Comput (2011). Google ScholarDigital Library
- Poh, M.Z., Swenson, N.C. and Picard, R.W. A wearable sensor for unobtrusive, long-term assessment of electrodermal activity. IEEE T Bio-med Eng (2010).Google Scholar
- Shotton, J., Fitzgibbon, A., et al. Real-time human pose recognition in parts from single depth images. In Proc. CVPR'11, IEEE (2011), 1297--1304. Google ScholarDigital Library
- Singer, E. The Measured Life. Technology Review (July/August 2011).Google Scholar
- Ståhl, A., Höök, K., Svensson, M., Taylora, A. and Combetto, M. Experiencing the affective diary. Pers Ubiquit Comput (2009), 365--378. Google ScholarDigital Library
- Suls, J., Green, P. and Hillis, S. Emotional reactivity to everyday problems, affective inertia, and neuroticism. Pers Soc Psychol B (1998), 127.Google Scholar
- Viégas, F., Golder, S. and Donath J. Visualizing email content: portraying relationships from conversational histories. In Proc. CHI'06, ACM (2006), 979--988. Google ScholarDigital Library
- Walker, W.R. and Skowronski, J.J. and Thompson, C.P. Life is pleasant-and memory helps to keep it that way! Rev Gen Psychol (2003), 203.Google Scholar
Index Terms
- AffectAura: an intelligent system for emotional memory
Recommendations
The communicative role of non-face emojis
Emojis have evolved from imitations of facial expressions meant to communicate affect into pictures of objects, food, and places that are not directly linked to affect. While emojis that resemble facial expressions are well-researched, emojis that ...
Are emoji valid indicators of in-the-moment mood?
AbstractDespite widespread assumptions that emoji represent emotions, research findings show that we do not process emoji in a way which we would expect for emotional stimuli. As such, we might be better placed to consider them more in line with mood ...
Highlights- The Emoji PANAS scale can reduce linguistic confounds when reporting current affective state
- Emoji are unlikely to be a valid means for reporting mood for those high in emotionality
- Emoji may be better consider mood indicators ...
Affective guidance of intelligent agents: How emotion controls cognition
How do emotions and moods color cognition? In this article, we examine how such reactions influence both judgments and cognitive performance. We argue that many affective influences are due, not to affective reactions themselves, but to the information ...
Comments