Abstract
Happy, surprised, disgusted, angry, sad, fearful, and neutral faces were presented extrafoveally, with fixations on faces allowed or not. The faces were preceded by a cue word that designated the face to be saccaded in a two-alternative forced-choice discrimination task (2AFC; Experiments 1 and 2), or were followed by a probe word for recognition (Experiment 3). Eye tracking was used to decompose the recognition process into stages. Relative to the other expressions, happy faces (1) were identified faster (as early as 160 msec from stimulus onset) in extrafoveal vision, as revealed by shorter saccade latencies in the 2AFC task; (2) required less encoding effort, as indexed by shorter first fixations and dwell times; and (3) required less decision-making effort, as indicated by fewer refixations on the face after the recognition probe was presented. This reveals a happy-face identification advantage both prior to and during overt attentional processing. The results are discussed in relation to prior neurophysiological findings on latencies in facial expression recognition.
Article PDF
Similar content being viewed by others
References
Batty, M., & Taylor, M. J. (2003). Early processing of the six basic facial emotional expressions. Cognitive Brain Research, 17, 613–620.
Bentin, S., Allison, T., Puce, A., Perez, E., & McCarthy, G. (1996). Electrophysiological studies of face perception in humans. Journal of Cognitive Neuroscience, 8, 551–565.
Bond, N. W., & Siddle, D. (1996). The preparedness account of social phobia: Some data and alternative explanations. In R. M. Rapee (Ed.), Current controversies in the anxiety disorders (pp. 291–316). New York: Guilford.
Caharel, S., Courtay, N., Bernard, C., Lalonde, R., & Rebaï, M. (2005). Familiarity and emotional expression influence an early stage of face processing: An electrophysiological study. Brain & Cognition, 59, 96–100.
Calder, A. J., & Young, A. W. (2005). Understanding the recognition of facial identity and facial expression. Nature Reviews Neuroscience, 6, 641–651.
Calvo, M. G., & Lundqvist, D. (2008). Facial expressions of emotion (KDEF): Identification under different display-duration conditions. Behavior Research Methods, 40, 109–115.
Calvo, M. G., & Marrero, H. (2009). Visual search of emotional faces: The role of affective content and featural distinctiveness. Cognition & Emotion, 23, 782–806.
Calvo, M. G., & Nummenmaa, L. (2007). Processing of unattended emotional visual scenes. Journal of Experimental Psychology: General, 136, 347–369.
Calvo, M. G., & Nummenmaa, L. (2008). Detection of emotional faces: Salient physical features guide effective visual search. Journal of Experimental Psychology: General, 137, 471–494.
Calvo, M. G., Nummenmaa, L., & Avero, P. (2008). Visual search of emotional faces: Eye-movement assessment of component processes. Experimental Psychology, 55, 359–370.
Calvo, M. G., Nummenmaa, L., & Avero, P. (2009). Recognition advantage of happy faces in extrafoveal vision: Featural and affective processing. Manuscript submitted for publication.
Eimer, M., & Holmes, A. (2007). Event-related brain potential correlates of emotional face processing. Neuropsychologia, 45, 15–31.
Eimer, M., Holmes, A., & McGlone, F. P. (2003). The role of spatial attention in the processing of facial expression: An ERP study of rapid brain responses to six basic emotions. Cognitive, Affective, & Behavioral Neuroscience, 3, 97–110.
Ekman, P., & Friesen, W. V. (1976). Pictures of facial affect. Palo Alto, CA: Consulting Psychologists Press.
Esteves, F., & Öhman, A. (1993). Masking the face: Recognition of emotional facial expressions as a function of the parameters of backward masking. Scandinavian Journal of Psychology, 34, 1–18.
Fischer, B., & Weber, H. (1993). Express saccades and visual attention. Behavioral & Brain Sciences, 16, 553–610.
Frischen, A., Eastwood, J. D., & Smilek, D. (2008). Visual search for faces with emotional expressions. Psychological Bulletin, 134, 662–676.
Grill-Spector, K., & Kanwisher, N. (2005). Visual recognition: As soon as you know it is there, you know what it is. Psychological Science, 16, 152–160.
Horstmann, G., & Bauland, A. (2006). Search asymmetries with real faces: Testing the anger superiority effect. Emotion, 6, 193–207.
Itti, L., & Koch, C. (2001). Computational modelling of visual attention. Nature Reviews Neuroscience, 2, 194–203.
Juth, P., Lundqvist, D., Karlsson, A., & Öhman, A. (2005). Looking for foes and friends: Perceptual and emotional factors when finding a face in the crowd. Emotion, 5, 379–395.
Kirchner, H., & Thorpe, S. J. (2006). Ultra-rapid object detection with saccadic eye movements: Visual processing speed revisited. Vision Research, 46, 1762–1776.
Kirita, T., & Endo, M. (1995). Happy face advantage in recognizing facial expressions. Acta Psychologica, 89, 149–163.
Leppänen, J. M., & Hietanen, J. K. (2004). Positive facial expressions are recognized faster than negative facial expressions, but why? Psychological Research, 69, 22–29.
Leppänen, J. M., & Hietanen, J. K. (2007). Is there more in a happy face than just a big smile? Visual Cognition, 15, 468–490.
Leppänen, J. M., Kauppinen, P., Peltola, M. J., & Hietanen, J. K. (2007). Differential electrocortical responses to increasing intensities of fearful and happy emotional expressions. Brain Research, 1166, 103–109.
Leppänen, J. M., Moulson, M. C., Vogel-Farley, V. K., & Nelson, C. A. (2007). An ERP study on emotional face processing in the adult and infant brain. Child Development, 78, 232–245.
Leppänen, J. M., Tenhunen, M., & Hietanen, J. K. (2003). Faster choice-reaction times to positive than to negative facial expressions: The role of cognitive and motor processes. Journal of Psychophysiology, 17, 113–123.
Lundqvist, D., Flykt, A., & Öhman, A. (1998). The Karolinska Directed Emotional Faces-KDEF [CD-ROM]. Stockholm: Karolinska Institutet, Department of Clinical Neuroscience, Psychology section.
Maxwell, J. S., & Davidson, R. J. (2004). Unequally masked: Indexing differences in the perceptual salience of “unseen” facial expressions. Cognition & Emotion, 18, 1009–1026.
Meeren, H. K. M., Hadjikhani, N., Ahlfors, S. P., Hämäläinen, M. S., & de Gelder, B. (2008). Early category-specific cortical activation revealed by visual stimulus inversion. PLoS ONE, 3, e3503.
Mendolia, M. (2007). Explicit use of categorical and dimensional strategies to decode facial expressions of emotion. Journal of Nonverbal Behavior, 31, 57–75.
Palermo, R., & Coltheart, M. (2004). Photographs of facial expression: Accuracy, response times, and ratings of intensity. Behavior Research Methods, 36, 634–638.
Palermo, R., & Rhodes, G. (2007). Are you always on my mind? A review of how face perception and attention interact. Neuropsychologia, 45, 75–92.
Rayner, K. (2009). Eye movements and attention in reading, scene perception, and visual search. Quarterly Journal of Experimental Psychology, 62, 1457–1506.
Reynolds, M. G. C., Eastwood, J. D., Partanen, M. A., Frischen, A., & Smilek, D. (2009). Monitoring eye movements while searching for affective faces. Visual Cognition, 17, 318–333.
Schiller, P. H., & Kendall, J. (2004). Temporal factors in target selection with saccadic eye movements. Experimental Brain Research, 154, 154–159.
Schupp, H. T., Öhman, A., Junghöfer, M., Stockburger, J., Hamm, A. O., & Weike, A. I. (2004). The facilitated processing of threatening faces: An ERP analysis. Emotion, 4, 189–200.
Storbeck, J., Robinson, M. D., & McCourt, M. E. (2006). Semantic processing precedes affect retrieval: The neurological case for cognitive primacy in visual processing. Review of General Psychology, 10, 41–55.
Underwood, G., & Foulsham, T. (2006). Visual saliency and semantic incongruency influence eye movements when inspecting pictures. Quarterly Journal of Experimental Psychology, 59, 1931–1949.
Vuilleumier, P., & Pourtois, G. (2007). Distributed and interactive brain mechanisms during emotion face perception: Evidence from functional neuroimaging. Neuropsychologia, 45, 174–194.
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was supported by Grant PSI2009-07245 from the Spanish Ministry of Science and Innovation to M.G.C. and by the Academy of Finland Grant 119088 to L.N.
Rights and permissions
About this article
Cite this article
Calvo, M.G., Nummenmaa, L. Eye-movement assessment of the time course in facial expression recognition: Neurophysiological implications. Cognitive, Affective, & Behavioral Neuroscience 9, 398–411 (2009). https://doi.org/10.3758/CABN.9.4.398
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/CABN.9.4.398