Duration matters: Dissociating neural correlates of detection and evaluation of social gaze
Introduction
Everyday experience as well as extensive research in psychology and social cognitive neuroscience confirm the pivotal role of human gaze in social interactions and its impact on cognitive, affective and motivational processes (Argyle and Cook, 1976, Gueguen and Jacob, 2002, Hood and Macrae, 2007, Vuilleumier and Pourtois, 2007). “Social gaze” might be metaphorically considered as glue in interpersonal communication as it allows not only for the coordination of attention and activities (Argyle and Cook, 1976) but also influences our social perception and the evaluation of others (Argyle et al., 1974, Kleinke, 1986, Mason et al., 2005, Mirenda et al., 1983). Moreover, understanding the ostensive function of eye gaze is closely linked to the ability to infer mental states of others (Baron-Cohen, 1995, Eskritt and Lee, 2007), a cognitive process that is usually referred to as “Theory of Mind” (Premack and Woodruff, 1978) or “mentalizing” (Frith, 2001). On the one hand, during earlier years of human ontogeny, recognizing that people can perceive an event differently depending on their gaze directions may aid in overcoming the initial egocentric perspective and in understanding of mind in general (Gopnik et al., 1994). On the other hand, more sophisticated knowledge about the relations and causes of mental states could help interpret eye gaze information at later ages (Eskritt and Lee, 2007). Thus, both the capacity to understand the eye gaze and to infer mental states influence each other's developments resulting in an automatic association between a particular gaze behavior and the intention “behind” it.
Particularly in the context of direct gaze, i.e. when a person is looking straight into the eyes of another person, mentalizing processes are likely to come into play. As a nonverbal signal of notable salience (Gibson and Pick, 1963, von Grunau and Anston, 1995), direct gaze can be used to initiate communication by indicating “self-reference” and by conveying interpersonal attitudes (Kampe et al., 2003, Kleinke, 1986, Mirenda et al., 1983, Valentine and Ehrlichman, 1979, Wicker et al., 1998). Here, self-reference connotes that a person (the sender) transfers an intention to communicate by looking directly at someone (a receiver). The receiver, in turn, must be able to understand this intention and to recognize that he/she is addressed, thus experiencing that the direct gaze refers to him/her. In this case, direct gaze would be “more than a trivial sensory experience” thereby triggering further reasoning about the meaning of and the motives for the other's messages as well as about the appropriateness of one's own response (Wicker et al., 1998).
Converging neuroimaging evidence has demonstrated that processing of gaze as a component of biological motion is consistently associated with activation in the posterior superior temporal sulcus (pSTS) (Akiyama et al., 2006, Hoffman and Haxby, 2000, Hooker et al., 2003, Pelphrey et al., 2004b, Puce et al., 1998, Wicker et al., 1998). Moreover, direct gaze in particular has been shown to recruit additional neural regions associated with complex social cognitive processing including the amygdala, the fusiform gyrus and the prefrontal cortex (Conty et al., 2007, George et al., 2001, Kawashima et al., 1999, Wicker et al., 2003). For instance, Kampe et al. (2003) showed that direct gaze (compared to averted gaze) elicited activation in the paracingulate part of the medial prefrontal cortex (MPFC). Interestingly, neuroimaging studies consistently suggest a special role of the MPFC in social cognition (Amodio and Frith, 2006). Within this region, functional divisions were defined differentiating between the orbitofrontal cortex (OFC) that is related to monitoring the reward value of stimuli and possible outcomes (Amodio and Frith, 2006, Kringelbach, 2005) and the above mentioned paracingulate cortex (PCC) commonly involved in mentalizing and self-referential processing (Gallagher et al., 2002, Gobbini et al., 2007, Mitchell et al., 2005, Ochsner et al., 2005, Vogeley et al., 2001, Walter et al., 2004). Thus, the result reported by Kampe et al. (2003) confirms the proposed link between direct gaze and understanding self-relevant intentions of others. Furthermore, differential involvement of the MPFC during processing of direct gaze has also been observed in combination with communicative facial expressions (Schilbach et al., 2006). However, the majority of previous neuroimaging studies on direct gaze, independent of facial expression, were unable to demonstrate the specific role of the MPFC during the processing of gaze-related information (George et al., 2001, Hoffman and Haxby, 2000, Wicker et al., 1998). For instance, a functional magnetic resonance imaging (fMRI) study by Calder et al. (2002) even reported that the MPFC was primarily engaged in the processing of averted, rather than direct gaze.
The lack of unequivocal evidence with respect to MPFC recruitment during the processing of context-free and neutral direct gaze could be attributed to the confinement to rather simplistic, categorical experimental designs using static stimulus material and considering only gaze direction. However, in everyday human interactions, the interpretation of gaze behavior appears to depend crucially on subtle, dynamic parameters, among which gaze duration represents a prominent example (Argyle et al., 1974, Brooks et al., 1986, Droney and Brooks, 1993, Gueguen and Jacob, 2002, Knackstedt and Kleinke, 1991, Montgomery et al., 1998). Previous behavioral studies have demonstrated that the longer a person looked straight into the observer's eyes, the more favorably this person was judged with regard to likeability, potency and self-esteem (Argyle et al., 1974, Brooks et al., 1986, Droney and Brooks, 1993, Knackstedt and Kleinke, 1991). Along the same line, Gueguen and Jacob (2002) were able to show that, when direct gaze was maintained, pedestrians were more likely to participate in an intercept survey. Argyle and Cook (1976) explain these results by postulating that in a social interaction prolonged gaze can convey approach signals including the need for feedback or affiliative needs. The receiver, in turn, may experience these signals as rewarding and could reciprocate them by evaluating the gazing person more positively and by adapting one's own behavior. Thereby, these approach forces “behind” the direct gaze motivate and guide interpersonal behavior and influence the perception and evaluation of others. However, prolonged direct gaze does not solely convey rewarding aspects like social attention and interest (Mason et al., 2005), but may also comprise avoidance components including the demonstration of dominance and the fear of revealing inner states (Argyle and Dean, 1965). As a consequence of this ambiguity, changing gaze signals require continuous decoding for the purpose of effective outcome monitoring.
Furthermore, it could be demonstrated that gaze duration as compared to gaze direction represents a more complex source of social information because it requires more sophisticated mentalizing abilities in order to perform an adequate interpretation (Eskritt and Lee, 2007). In tasks requiring inferences about other's desires from the simple detection of eye direction, 4-year olds were already successful (Baron-Cohen, 1995). However, advanced levels of understanding gaze cues based on their relative duration were not yet present at this age (Montgomery et al., 1998). In contrast, the performance of 5- and 6-year olds was comparable to that of adults, indicating that during later developmental stages relative gaze duration towards different test objects can be efficiently used to infer others' preferences (Einav and Hood, 2006, Montgomery et al., 1998).
In spite of the behaviorally well established relevance of gaze duration for impression formation and motivation in social contexts, to the best of our knowledge, no fMRI study has yet focused on the neural correlates of this parameter. Thus, the aim of the present study was to investigate neural correlates of both the direction and the duration of the interpersonal gaze. We expect that the greater amount of social information inherent in varying the duration of the direct gaze as compared to simply contrasting the direct and the averted gaze direction would enclose the involvement of more complex social cognitive processes. Thus, we performed the present fMRI study to analyze whether systematically varying the duration of direct gaze significantly modulates its neural correlates. In the context of a person judgment task, virtual characters that initially displayed averted gaze and subsequently shifted their gaze towards the participant were presented. After a systematically varied amount of time (i.e., 1 s, 2.5 s or 4 s), gaze was shifted back to the initial, averted position. Additionally, a control condition without a gaze shift towards the participant, i.e. consisting solely of averted gaze, was presented. Given that the communicative intention conveyed by direct gaze has been conjectured to automatically and implicitly trigger the mentalizing mechanism (Kampe et al., 2003), no explicit instruction to infer mental states was given but participants were asked to rate the likeability of the stimulus faces after each trial. Predicted on the link between duration-sensitive gaze processing and understanding interpersonal intentions we hypothesize a direct gaze duration (DGd) related involvement of the PCC. Furthermore, we seek to confirm the proposed reward value of increasing DGd by replicating a more positive person evaluation on the behavioral level and by demonstrating differential responses in the OFC as a function of prolonged direct gaze. On the other hand, for the simple comparison between directed and averted gaze directions we expect the involvement of temporal regions including the STS as reported in previous neuroimaging studies in this field. In sum, as the continuous monitoring of DGd provides access to an additional source of social information, changes in the DGd should have a discriminative value for the recruitment of the MPFC including the OFC and the PCC.
Section snippets
Subjects
22 participants [12 male, 10 female; mean age 27.2 ± 3.9 (SD) years] with no history of neurological or psychiatric illness gave informed consent and participated in the fMRI study. All participants were naïve with respect to the experimental task and the purpose of the study. Right-handedness was confirmed by the Edinburgh Inventory for Handedness (Oldfield, 1971). The study was approved by the local ethics committee of the Medical Faculty of the University of Cologne.
Stimulus material and experimental design
Due to the advantage of
Behavioral results
The effect of DGd on likeability ratings was tested by a one-way repeated measures analysis of variance (ANOVA). The trials with purely averted gaze were excluded from this analysis as their primary purpose was to provide a control condition for the fMRI paradigm. Nevertheless, pair-wise comparisons revealed that likeability ratings for faces showing averted gaze were significantly lower than those for all direct gaze conditions (AG vs. DG1: pc = .004; AG vs. DG2.5: pc = .003; AG vs. DG4: pc = .002,
Discussion
The main result of this study shows that processing gaze direction and gaze duration recruit distinct neural systems. Whereas the comparison of direct gaze with averted gaze, irrespective of gaze duration, yielded prominent activations in bilateral occipito-temporal regions including the pSTS as a correlate of socially relevant gaze direction processing, the processing of gaze duration was associated with differential neural activity in the MPFC.
Conclusions
In summary, cognitive processing of social gaze appears to be dependent on specific gaze parameters that have to be considered in detail. On the one hand, the detection of socially relevant gaze direction conveying the impression of being looked at by a virtual character, irrespective of gaze duration, appears to be mediated by bilateral occipito-temporal regions including the multimodal sensory area pSTS. Indicating social perception via complex visual analysis, the recruitment of these
References (104)
- et al.
Beautiful faces have variable reward value: fMRI and behavioral evidence
Neuron
(2001) - et al.
A deficit in discriminating gaze direction in a case with right superior temporal gyrus lesion
Neuropsychologia
(2006) - et al.
Social perception from visual cues: role of the STS region
Trends Cogn. Sci.
(2000) - et al.
Cognitive and emotional influences in anterior cingulate cortex
Trends Cogn. Sci.
(2000) - et al.
Reading the mind from eye gaze
Neuropsychologia
(2002) - et al.
When eye creates the contact! ERP evidence for early dissociation between direct and averted gaze motion processing
Neuropsychologia
(2007) - et al.
Self-referential reflective activity and its relationship with rest: a PET study
Neuroimage
(2005) - et al.
A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data
Neuroimage
(2005) - et al.
Testing anatomically specified hypotheses in functional imaging using cytoarchitectonic maps
Neuroimage
(2006) - et al.
Assignment of functional activations to probabilistic cytoarchitectonic areas revisited
Neuroimage
(2007)
Anatomical mapping of functional activation in stereotactic coordinate space
Neuroimage
Detecting activations in PET and fMRI: levels of inference and power
Neuroimage
Mind blindness and the brain in autism
Neuron
Imaging the intentional stance in a competitive game
Neuroimage
Seen gaze-direction modulates fusiform activity and its coupling with other brain areas during face processing
Neuroimage
Brain areas active during visual perception of biological motion
Neuron
How do we modulate our emotions? Parametric fMRI reveals cortical midline structures as regions specifically involved in the processing of emotional valences
Brain Res. Cogn. Brain Res.
Brain networks for analyzing eye gaze
Brain Res. Cogn. Brain Res.
Watching social interactions produces dorsomedial prefrontal and medial parietal BOLD fMRI signal increases compared to a resting baseline
Neuroimage
Face perception is modulated by sexual preference
Curr. Biol.
The functional neuroanatomy of the human orbitofrontal cortex: evidence from neuroimaging and neuropsychology
Prog. Neurobiol.
An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets
Neuroimage
Cortical networks for working memory and executive functions sustain the conscious resting state in man
Brain Res. Bull.
Amygdala activation at 3 T in response to human and avatar facial expressions of emotions
J. Neurosci. Methods
Eye gaze and the GSR
J. Exp. Soc. Psychol.
Reward representations and reward-related learning in the human brain: insights from neuroimaging
Curr. Opin. Neurobiol.
Beauty in a smile: the role of medial orbitofrontal cortex in facial attractiveness
Neuropsychologia
The neural correlates of direct and reflected self-knowledge
Neuroimage
The assessment and analysis of handedness: the Edinburgh inventory
Neuropsychologia
Brain activation evoked by perception of gaze shifts: the influence of context
Neuropsychologia
Being with virtual others: neural correlates of social interaction
Neuropsychologia
Minds at rest? Social cognition as the default mode of cognizing and its putative relationship to the “default system” of the brain
Conscious. Cogn.
A comparison of methods for characterizing the event-related BOLD timeseries in rapid fMRI
Neuroimage
Look at me: five-month-old infants' sensitivity to very small deviations in eye-gaze during social interactions
Infant Behavi. Dev.
Mind reading: neural mechanisms of theory of mind and self-perspective
Neuroimage
Distributed and interactive brain mechanisms during emotion face perception: evidence from functional neuroimaging
Neuropsychologia
Occipitotemporal activity elicited by viewing eye movements: a magnetoencephalographic study
Neuroimage
Human MT/V5 activity on viewing eye gaze changes in others: a magnetoencephalographic study
Brain Res.
Meeting of minds: the medial frontal cortex and social cognition
Nat. Rev. Neurosci.
Gaze and Mutual Gaze
Eye-contact, distance and affiliation
Sociometry
The meaning of five patterns of gaze
Eur. J. Soc. Psychol.
Rigid body registration
Interpersonal distance in immersive virtual environments
Pers. Soc. Psychol. Bull.
Mindblindness: An Essay on Autism and Theory Of Mind
Facilities for the graphical computer simulation of head and body movements
Behav. Res. Meth. Instrum. Comput.
Person perception effects of computer-simulated male and female head movement/Personwahrnehmungseffekte von computersimulierten männlichen und weiblichen Kopfbewegungen
J. Nonverbal Behav.
Computer animated movement and person perception: methodological advances in nonverbal behavior research
J. Nonverbal Behav.
Transcript-based computer animation of movement: evaluating a new tool for nonverbal behavior research
Behav. Res. Meth. Instrum. Comput.
Immersive virtual environment technology as a methodological tool for social psychology
Psychol. Inq.
Cited by (95)
University Students’ Opinions on Using Intelligent Agents to Cope with Stress and Anxiety in Social Situations
2024, Computers in Human BehaviorNeural and Affective Responses to Prolonged Eye Contact with One's Own Adolescent Child and Unfamiliar Others
2022, NeuroImageCitation Excerpt :This is striking as the affiliative role of eye contact in interpersonal contact with others most likely comes into play as the duration of eye contact increases. For example, behavioral studies have shown that a prolonged presentation duration of dynamic face stimuli induces greater emotional engagement (Cowan et al., 2014; Regenbogen et al., 2012) and more favorable judgements from receiver to sender (Argyle et al., 1974; Brooks et al., 1986; Kuzmanovic et al., 2009; Montgomery et al., 1998). Prolonged exposure is also thought to more strongly engage higher-order cognitive processes related to the evaluation of eye contact, such as mentalizing in order to infer others’ thoughts and feelings (Eskritt & Lee, 2007; Kuzmanovic et al., 2009).
Perception of direct vs. averted gaze in portrait paintings: An fMRI and eye-tracking study
2018, Brain and CognitionThe semantics of gaze in person perception: a novel qualitative-quantitative approach
2024, Scientific ReportsSharing and Receiving Eye-Contact Predicts Mate Choice After a 5-Minute Conversation: Evidence from a Speed-Dating Study
2024, Archives of Sexual Behavior