Daniel McDuff
ABSTRACT
Facial expressions play a significant role in everyday interactions. A majority of the research on facial expressions of emotion has focused on a small set of "basic" states. However, in real-life the expression of emotions is highly context dependent and prototypic expressions of "basic" emotions may not always be present. In this paper we attempt to discover expressions associated with alternate states of informed, inspired, persuaded, sentimental and amused based on a very large dataset of observed facial responses. We used a curated set of 395 everyday videos that were found to reliably elicit the states and recorded 49,869 facial responses as viewers watched the videos in their homes. Using automated facial coding we quantified the presence of 18 facial actions in each of the 23.4 million frames. Lip corner pulls, lip sucks and inner brow raises were prominent in sentimental responses. Outer brow raises and eye widening were prominent in persuaded and informed responses. More brow furrowing distinguished informed from persuaded responses potentially indicating higher cognition.
- J. F. Cohn, Z. Ambadar, and P. Ekman. Observer-based measurement of facial expression with the facial action coding system. The handbook of emotion elicitation and assessment, pages 203–221, 2007.Google Scholar
- C. Darwin. The expression of emotions in man and animals. Oxford University, New York, NY, 3rd edition, 1872.Google ScholarCross Ref
- P. Ekman and W. Friesen. Facial action coding system. 1977.Google Scholar
- P. Ekman, E. R. Sorenson, and W. V. Friesen. Pan-cultural elements in facial displays of emotion. Science, 164(3875):86–88, 1969.Google ScholarCross Ref
- E. C. Evans. Physiognomies in the ancient world. Transactions of the American Philosophical Society, 59:1–101, 1969.Google ScholarCross Ref
- W. V. Friesen and P. Ekman. EMFACS-7: Emotional facial action coding system. Unpublished manuscript, University of California at San Francisco, 2(36):1, 1983.Google Scholar
- H. Gunes and H. Hung. Is automatic facial expression recognition of emotions coming to a dead end? the rise of the new kids on the block. Image and Vision Computing, 2016.Google ScholarDigital Library
- C. E. Izard. The Maximally Discriminative Facial Movement Coding System (MAX). Instructional Resources Center, University of Delaware, 1979.Google Scholar
- C. E. Izard. Innate and universal facial expressions: Evidence from developmental and cross-cultural research. Psychological Bulletin, 115(2):288–299, 1994.Google ScholarCross Ref
- C. E. Izard, L. M. Dougherty, and E. A. Hembree. A system for identifying affect expressions by holistic judgments (AFFEX). Instructional Resources Center, University of Delaware, 1983.Google Scholar
- D. Keltner. Signs of appeasement: Evidence for the distinct displays of embarrassment, amusement, and shame. Journal of Personality and Social Psychology, 68(3):441, 1995.Google ScholarCross Ref
- P. Lewinski, M. L. Fransen, and E. S. Tan. Predicting advertising effectiveness by facial expressions in response to amusing persuasive stimuli. Journal of Neuroscience, Psychology, and Economics, 7(1):1, 2014.Google Scholar
- D. Matsumoto. Cultural similarities and differences in display rules. Motivation and Emotion, 14(3):195–214, 1990.Google ScholarCross Ref
- D. Matsumoto, H. S. Hwang, and H. Yamada. Cultural differences in the relative contributions of face and context to judgments of emotions. Journal of Cross-Cultural Psychology, page 0022022110387426, 2010.Google Scholar
- D. McDuff. Crowdsourcing affective responses for predicting media effectiveness. PhD thesis, Massachusetts Institute of Technology, 2014.Google Scholar
- D. McDuff, R. El Kaliouby, J. Cohn, and R. Picard. Predicting ad liking and purchase intent: Large-scale analysis of facial responses to ads. Affective Computing, IEEE Transactions on, 6(3):223–235, July 2015.Google Scholar
- D. McDuff, R. El Kaliouby, E. Kodra, and R. Picard. Measuring voter’s candidate preference based on affective responses to election debates. In Affective Computing and Intelligent Interaction (ACII), 2013 Humaine Association Conference on, pages 369–374. IEEE, 2013. Google ScholarDigital Library
- D. McDuff, R. El Kaliouby, and R. Picard. Crowdsourcing facial responses to online videos. IEEE Transactions on Affective Computing, 3(4):456–468, 2012. Google ScholarDigital Library
- D. McDuff, A. Mahmoud, M. Mavadati, M. Amr, J. Turcot, and R. e. Kaliouby. Affdex sdk: A cross-platform real-time multi-face expression recognition toolkit. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, pages 3723–3726. ACM, 2016. Google ScholarDigital Library
- H. Oster. Facial expression and affect development. In M. Lewis and L. A. Rosenblum, editors, The development of affect. Plenum Press, New York, NY, 1978.Google ScholarCross Ref
- J. A. Russell. Is there universal recognition of emotion from facial expressions? A review of the cross-cultural studies. Psychological Bulletin, 115(1):102, 1994.Google ScholarCross Ref
- T. Senechal, D. McDuff, and R. Kaliouby. Facial action unit detection using active learning and an efficient non-linear kernel approximation. In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 10–18, 2015. Google ScholarDigital Library
- T. Teixeira, M. Wedel, and R. Pieters. Emotion-induced engagement in internet video ads. Journal of Marketing Research, 2010.Google Scholar
- F. Torre and J. Cohn. Facial expression analysis. Visual Analysis of Humans, pages 377–409, 2011.Google ScholarCross Ref
- P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. In Computer Vision and Pattern Recognition, 2001. Proceedings of the IEEE Computer Society Conference on, volume 1, pages I–511. IEEE, 2001.Google Scholar
- X. Xiong and F. De la Torre. Supervised descent method and its applications to face alignment. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pages 532–539. IEEE, 2013. Google ScholarDigital Library
Index Terms
- Discovering facial expressions for states of amused, persuaded, informed, sentimental and inspired
Recommendations
Determining facial expressions in real time
ICCV '95: Proceedings of the Fifth International Conference on Computer VisionWe suggest an approach to describing and tracking the deformation of facial features. We concentrate on the mouth since its shape is important in detecting emotion. However, we believe that our system could be extended to deal with other facial ...
Virtual proximity and facial expressions of computer agents regulate human emotions and attention
CASA' 2010 Special IssueEmotion- and attention-related subjective and physiological responses to virtual proximity and facial expressions of embodied computer agents (ECA) were studied. Thirty participants viewed female and male characters with a neutral, unpleasant, or ...
Cross-cultural design of facial expressions for humanoids: is there cultural difference between Japan and Denmark?
MMAsia '20: Proceedings of the 2nd ACM International Conference on Multimedia in AsiaIn this research, the authors succeeded in creating facial expressions made with the minimum necessary elements for recognizing a face. The elements are two eyes and a mouth made using precise circles, which are transformed to make facial expressions ...
Comments