When eye creates the contact! ERP evidence for early dissociation between direct and averted gaze motion processing

https://doi.org/10.1016/j.neuropsychologia.2007.05.017Get rights and content

Abstract

Direct gaze or eye contact is a strong social signal, which triggers distinct processes as compared to other gaze directions. Thus, direct gaze could be early dissociated from averted gaze during the processing of gaze direction. In order to examine the processing of gaze contact relative to averted gaze, we studied evoked potentials in human adults in response to the apparent motion of gaze. Deviated and frontal faces were presented with a fixed gaze direction, followed by an apparent movement of the eyes either toward the subject or away from him/her. The results showed that the perception of direct relative to averted gaze evoked a greater, later and longer lasting N170, suggesting that gaze contact recruited more resources than averted gaze in the early stage of gaze direction processing. Furthermore, direct and averted motion of gaze elicited distinct ERP components between 160 and 210 ms, initiated over centro-parietal electrodes. Source reconstruction revealed the involvement of the Theory-of-Mind network, including the regions of the superior temporal sulcus, the medial prefrontal and the orbitofrontal cortices, in this early dissociation. In addition, the perception of gaze contact relative to averted gaze yielded increased fronto-central P3a and parieto-occipital P3b. All the results were significant whatever the head orientation. Our findings show that gaze contact, as compared to other gaze directions, is an essential social cue which recruits early specific processes. The dissociation between direct and averted gaze processing occurs as soon as 160 ms, involving the social brain network.

Introduction

Gaze is a crucial cue to decode for adapting one's interpersonal behaviour. Indeed, people's eyes and gaze direction communicate their emotion and their focus of interest in the environment. Thus, decoding others’ gaze direction is an essential component and the precursor of the “Theory of Mind” (ToM), namely, the human ability to attribute mental states to other people (Baron-Cohen, 1995). Two main aspects of the perception of gaze direction have been distinguished in the literature: the perception of gaze directed toward the observer (direct gaze or gaze contact) and the perception of gaze directed away from the observer, toward the surrounding space (averted gaze). These two aspects have been shown to involve distinct cognitive processes.

What happens when our interlocutor directs his/her attention toward us? The first function of direct gaze seems to be to inform about the intention of the gazer toward the perceiver. Whereas many species present an aversive response to gaze contact, human primates use this cue together with a large panel of behaviours for communicating and synchronising their interaction with others (Kleinke, 1986, Patterson, 1982). The majority of the pathologies characterized by a social withdrawal (e.g. social phobia, schizophrenia with negative symptomatology, autism) include among their symptoms the avoidance of gaze contact (Greist, 1995; Horley, Williams, Gonsalvez, & Gordon, 2003). These data underline the importance of decoding gaze contact efficiently for well-adapted social interactions and suggests that direct gaze may be early dissociated from averted gaze during the processing of gaze direction.

Behavioural studies are rich in evidence for the dissociation between direct and averted gaze processing. As for gaze contact, it has been shown that we are highly efficient at detecting direct relative to averted gaze (Conty, Tijus, Hugueville, Coelho, & George, 2006; Senju, Hasegawa, & Tojo, 2005; Von Grünau & Anston, 1995), suggesting that we are biased to detect eye contact. Moreover, perceiving gaze contact can capture visuospatial attention, resulting in delayed orienting of attention toward peripheral cues (Senju & Hasegawa, 2005). Such effect has been attributed to deeper face processing during the gaze contact condition (see Hood, Macrae, Cole-Davies, & Dias, 2003; Vuilleumier, George, Lister, Armony, & Driver, 2005). Likewise, the perception of averted gaze seems to trigger specific processes on its own. It is known to induce an automatic shift of spatial attention toward the gazed-at direction (e.g. Driver, Davis, & Ricciardelli, 1999; Friesen & Kingstone, 2003), which can be observed as soon as at 3 months of age (Hood, Willen, & Driver, 1998). Responding to averted gaze leads very early in development to joint attention, the ability to direct our attention at the same object as our interlocutor (Reddy, 2003). Altogether, these data suggest that the perception of averted gaze triggers processes mainly related to the orientation of spatial attention and joint attention. By contrast, direct gaze triggers preferential detection processes and enhanced allocation of resources to the seen face, affecting cognitive performance. The clear dissociation between the processes elicited by direct versus averted gaze perception raises the question as to when and where responses to direct and averted gaze are dissociated in the brain. Here we used functional brain imaging based on electroencephalography (EEG) and source reconstruction to investigate this question.

While movement appears as an essential component of gaze, a number of brain imaging studies have examined the perception of gaze direction using static face stimuli. They have shown clear dissociation between direct and averted gaze processing which converged closely with the conclusions of behavioural studies. Direct gaze induces increased activation in bilateral fusiform gyrus, a region known to be involved in face encoding (George, Driver, & Dolan, 2001, see also Calder et al., 2002, George et al., 2001; Kampe, Frith, Dolan, & Frith, 2001). Moreover, it activates the amygdala, a structure involved in emotional processing (George et al., 2001, Kawashima et al., 1999; Wicker, Perrett, Baron-Cohen, & Decety, 2003), supporting the idea that direct gaze is emotionally significant even when the gazing face does not express any particular emotion. By contrast, Hoffman and Haxby (2000, Experiment 2) found that the superior temporal sulcus (STS), involved in biological motion processing and social attention processes, and the intraparietal sulcus (IPS), involved in spatial attention processes, showed greater activation for passive viewing of faces with averted compared to direct gaze (see also Puce, Allison, Asgari, Gore, & McCarthy, 1998 using moving stimuli). Furthermore, in a positron emission tomography (PET) study, Calder et al. (2002) showed that the medial prefrontal cortex (MPF), known to be involved in ToM tasks, is engaged in the processing of both direct and averted gaze, but would be primarily involved when viewing averted rather than direct gaze.

While the neural bases of direct and averted gaze processing have started to be uncovered, little is known about the temporal dynamics of these brain responses. Using EEG in human infant, Farroni, Csibra, Simion, and Johnson (2002) studied the N170, an early face-sensitive event-related potential (ERP) elicited on occipito-temporal electrodes and known to reflect face encoding (e.g. Bentin, Allison, Puce, Perez, & McCarthy, 1996; Jemel, Pisani, Calabria, Crommelinck, & Bruyer, 2003). These authors showed enhanced “infant N170” (around 240 ms) in 4-month-old babies looking at static faces establishing gaze contact. This reinforces the idea that the perception of direct gaze induces deeper face processing. Importantly, these results also suggest that the perception of direct relative to averted gaze may elicit early increased responses in the brain. However, the studies ran in EEG and magnetoencephalography (MEG) in human adult have failed to confirm this hypothesis so far.

Using static face stimuli, Watanabe, Miki, and Kakigi (2002) found a very limited effect of gaze with greater N190 to right averted (relative to straight) gaze on the right T6′ electrode (see also Taylor, Itier, Allison, & Edmonds, 2001). Moreover, several recent EEG and MEG studies have focused on the perception of moving gaze. The first investigators were Puce, Smith, and Allison (2000). When recording ERPs to eye and mouth movements in human adults, these authors found evoked N170 and P350. The N170 was found to be modulated by the direction of gaze movement as it was greater and earlier for gaze directing away from the viewer compared to gaze directing toward him/her. Moreover, using MEG, Watanabe, Kakigi and Puce (2001) did not find any significant effect of gaze direction on the M170. These results stand in contrast with the findings of greater face-encoding related activity in response to direct relative to averted gaze in babies (Farroni et al., 2002). It is possible that the N170 elicited by eye motion originates from different sources and reflects different encoding processes as compared to the N170 elicited by static faces. However, discrepancies in gaze effects have been obtained within, respectively, static as well as dynamic protocols. Alternatively, these discrepancies may originate from a baseline bias in most dynamic protocols. Indeed, a common feature of the EEG and MEG studies on the perception of gaze motion so far has been to use direct gaze as a starting point condition with the eyes then moving sideways (averted motion condition) and returning afterwards to central position (direct motion condition). Such protocol could have biased the data toward eliciting greater brain responses to averted gaze than to direct gaze motion, for direct gaze served as a baseline. In a recent study, Pelphrey et al. (2004) did control for such bias: they used an animated male character who shifted furtively his gaze from a determinate common position, either toward the viewer or away from him/her, with the same movement quantity. Using fMRI, the authors found that direct gaze motion evoked greater STS activity than averted gaze motion did. Such result predicts that the processes related to the encoding of both static and moving gaze may be enhanced by gaze contact. Furthermore, in an EEG study, Senju et al., 2005a, Senju et al., 2005b used an odd-ball paradigm with a frequent face stimuli glancing downward, and rare stimuli glancing either toward or away from the viewer. With such change in eye direction from a common starting gaze condition, they showed an enhanced N200 for faces establishing gaze contact in typically developed children. Thus, using a common baseline for direct and averted gaze condition seems to favour the observation of the processes triggered during the perception of direct gaze motion (see also Calder et al., 2002; Kampe, Frith, & Frith, 2003).

To the best of our knowledge, no EEG study using symmetric condition between direct and averted motion of gaze, with a common baseline for both gaze conditions, has been run in human adults. Our hypothesis was that under this condition, early increased brain responses elicited by direct relative to averted motion of gaze may emerge in ERPs. Thus, face photographs were presented with a fixed gaze direction, then followed by an apparent movement of the eyes either toward the subject or away from him/her with the same movement quantity. Another important issue concerns the interaction between cues to the direction of attention extracted from gaze and head (e.g. Todorovic, 2006). The previous studies on the perception of gaze movement used faces seen under one orientation only, either frontal (Pelphrey, Singerman, Allison, & McCarthy, 2003; Puce et al., 2000, Puce et al., 2003, Watanabe et al., 2001) or deviated (Pelphrey et al., 2004; Senju, Tojo, et al., 2005). However, there is evidence that incongruent configuration between the gaze direction and the head orientation elicits enhanced brain activity (McCarthy, Puce, Belger, & Allison, 1999; Puce et al., 2000). Thus, it is not clear whether the enhancement of N200 for direct gaze under deviated head view in Senju, Tojo, et al. (2005) study and that of N170 for averted gaze under frontal head view in Puce et al., 2000, Puce et al., 2003 was not just reflecting the incongruent aspect of the eye direction relative to the head orientation in both protocols. Thus, we used both frontal and deviated head orientations to test for any effect of gaze independent of a specific face view.

We found an occipito-temporal N170 that was modulated by the direction of gaze motion. Namely, this ERP component was greater, later and longer lasting for direct as compared to averted motions of gaze. These effects were found whatever the head orientation. Moreover, between 160 and 210 ms, the direction of gaze motion modulated the brain response over central scalp regions. This modulation overlapped partly with the N170 activity. In order to clarify this phenomenon and identify the brain regions involved, we used source reconstruction. This analysis suggested that the differential processing of direct and averted movement of gaze involved a structured brain network from the prefrontal cortex to the temporal regions, as soon as between 150 and 220 ms. In addition, we report P300 components (fronto-central P3a and parieto-occipital P3b) that were greater for direct than averted gaze motion.

Section snippets

Participants

Seventeen healthy volunteers participated in the experiment (nine males/eight females). Due to fuzziness and/or bad signal-to-noise ratio, three subjects (two males and one female) were excluded from the final sample. Thus, 14 participants (7 males/7 females) were included in this study (mean age = 24 ± 1 years). All had normal or corrected-to-normal vision, were naive to the aim of the experiment and were right-handed according to an abbreviated version of the handedness inventory of Dellatolas et

Behavioural results

The analysis on RTs showed a significant main effect of gaze direction (F(1,13) = 11.8, p < .01). The RTs were shorter for the direct compared to the averted gaze motion (see Table 1). An interaction between gaze direction and head orientation was observed (F(1,13) = 7.6, p < .02), revealing that the gaze effect was more marked under frontal head view (mean difference = 41 ± 11 ms, F(1,13) = 13.1, p < .01) compared to deviated head view (mean difference = 18 ± 7 ms, F(1,13) = 5.9, p < .04). Note, however, that the gaze

Discussion

Our objective was to examine the temporal dynamics of the perception of direct and averted gaze motion. Using EEG, we showed that the perception of a gaze directing toward the subject, as compared to the perception of a gaze directing away from the viewer, yielded greater, later and longer lasting occipito-temporal N170. Moreover, the direction of gaze motion influenced the mean amplitude of ERP responses between 160 and 210 ms starting over centro-parietal electrodes and extending into

Conclusion

In conclusion, all our results converge to show that gaze contact is a rich source of information which recruits more processing resources than other gaze directions do. Such resource mobilisation is associated with early dissociation of brain responses during direct and averted gaze perception. This dissociation occurs as soon as 150 ms within the ToM brain network. It would subtend the efficient detection of gaze contact and the resulting adaptation of one's social behaviour.

Acknowledgments

This work was supported by an ACI “Systèmes Complexes en SHS” (project no. SCSHS-2004-05) from the French Ministère de la Recherche. We thank Anne-Lise Paradis for her contribution to the experimental paradigm, Sylvain Baillet for his support with Brainstorm, and Line Garnero for helpful discussion on the manuscript.

References (65)

  • B. Jemel et al.

    Is the N170 for faces cognitively penetrable? Evidence from repetition priming of Mooney faces of familiar and unfamiliar persons.

    Brain Research. Cognitive Brain Research

    (2003)
  • C.M. Michel et al.

    EEG source imaging

    Clinical Neurophysiology

    (2004)
  • K.A. Pelphrey et al.

    Brain activation evoked by perception of gaze shifts: the influence of context

    Neuropsychologia

    (2003)
  • D.I. Perrett et al.

    Evidence accumulation in cell populations responsive to faces: an account of generalisation of recognition without mental transformations

    Cognition

    (1998)
  • A. Puce et al.

    The human temporal lobe integrates facial form and motion: evidence from fMRI and ERP studies

    Neuroimage

    (2003)
  • V. Reddy

    On being the object of attention: implications for self-other consciousness

    Trends in Cognitive Science

    (2003)
  • A. Senju et al.

    Deviant gaze processing in children with autism: an ERP study

    Neuropsychologia

    (2005)
  • M.J. Taylor et al.

    Direction of gaze effects on early face processing: eyes-only versus full faces

    Brain Research. Cognitive Brain Research

    (2001)
  • D. Todorovic

    Geometrical basis of perception of gaze direction

    Vision Research

    (2006)
  • S. Watanabe et al.

    Occipitotemporal activity by viewing eye movements: a magnetoencephalographic study

    Neuroimage

    (2001)
  • S. Watanabe et al.

    Gaze direction affects face perception in humans

    Neuroscience Letters

    (2002)
  • B. Wicker et al.

    Brain regions involved in the perception of gaze: a PET study

    Neuroimage

    (1998)
  • B. Wicker et al.

    Being the target of another's emotion: a PET study

    Neuropsychologia

    (2003)
  • S. Baillet et al.

    Electromagnetic brain mapping

    IEEE Signal Processing Magazine

    (2001)
  • T. Ban et al.

    Cortico-cortical projections from the prefrontal cortex to the superior temporal sulcal area (STs) in the monkey studied by means of HRP method

    Archives Italiennes de Biologie

    (1991)
  • S. Baron-Cohen

    Mindblindness: an essay on autism and theory of mind

    (1995)
  • S. Bentin et al.

    Electrophysiological studies of face perception in humans

    Journal of Cognitive Neuroscience

    (1996)
  • S. Campanella et al.

    Categorical perception of happiness and fear facial expressions: an ERP study

    Journal of Cognitive Neuroscience

    (2002)
  • L. Conty et al.

    Searching for asymmetries in the detection of gaze contact versus averted gaze under different head views: a behavioural study

    Spatial Vision

    (2006)
  • F. Darvas et al.

    Generic head models for atlas-based EEG source analysis

    Human Brain Mapping

    (2006)
  • G. Dellatolas et al.

    Mesure de la préférence manuelle par autoquestionnaire dans la population française adulte

    Revue de Psychologie Appliquée

    (1988)
  • J. Driver et al.

    Perception triggers reflexive visuospatial orienting

    Visual Cognition

    (1999)
  • Cited by (0)

    View full text