Pointing with the eyes: The role of gaze in communicating danger

Abstract

Facial expression and direction of gaze are two important sources of social information, and what message each conveys may ultimately depend on how the respective information interacts in the eye of the perceiver. Direct gaze signals an interaction with the observer but averted gaze amounts to “pointing with the eyes”, and in combination with a fearful facial expression may signal the presence of environmental danger. We used fMRI to examine how gaze direction influences brain processing of facial expression of fear. The combination of fearful faces and averted gazes activated areas related to gaze shifting (STS, IPS) and fear-processing (amygdala, hypothalamus, pallidum). Additional modulation of activation was observed in motion detection areas, in premotor areas and in the somatosensory cortex, bilaterally.

Our results indicate that the direction of gaze prompts a process whereby the brain combines the meaning of the facial expression with the information provided by gaze direction, and in the process computes the behavioral implications for the observer.

Introduction

Facial expression and direction of gaze are significant components of the information provided by a face. In the course of development, detection of gaze orientation plays an important role. Human and nonhuman primates interact with their caregivers and learn from observing the direction of gaze what objects to avoid. In primates, fear can be communicated through the mechanism of joint attention: young rhesus monkeys, initially unafraid of snakes, show fear after witnessing the fearful reaction of their parents to a snake (toy or real), suggesting that they are able to couple their parents’ fearful expression and their direction of gaze to learn that snakes are dangerous (Mineka, Davidson, Cook, & Keir, 1984).

Until recently, studies of gaze processing have used neutral faces, and investigated the attentional cues provided by gaze direction. Direction of gaze influences the level of activity in areas involved in processing faces, including the fusiform gyrus, the superior temporal sulcus and the intraparietal sulcus (George et al., 2001, Hoffman and Haxby, 2000, Pelphrey et al., 2003). But gaze direction may play a more complex role when it belongs to a face expressing a specific emotion (Adams et al., 2003, Adams and Kleck, 2003, Klucharev and Sams, 2004, Mathews et al., 2003, Sato et al., 2004); but see (Hietanen & Leppanen, 2003). A fearful facial expression with an observer-averted gaze is automatically recognized (Anderson, Christoff, Panitz, De Rosa, & Gabrieli, 2003) as possibly indicating environmental threat (“there is a danger located where I am looking at”) and as requiring an adaptive response (“you need to avoid it”).

Several recent studies indicate gaze direction influences other brain areas besides the ones specifically associated with face recognition. To account for this, several groups have developed a distributed representation model of face perception (Bruce and Young, 1986, de Gelder et al., 2003, Haxby et al., 2000, Haxby et al., 1994, Haxby et al., 1996, Hoffman and Haxby, 2000), in which different areas of the brain respond to different attributes of a face, such as identity (fusiform gyrus, inferior occipital gyrus), gaze direction and recognition of action (superior temporal sulcus), and expression and/or emotion (orbitofrontal cortex, amygdala, anterior cingulate cortex, premotor cortex).

It is well known that the sight of a fearful face expression provides a very strong signal, however, a fearful face can be either empathy-evoking to the observer or threat-related, depending on its gaze direction. Facial expression of fear can be ambiguous because it may be unclear whether the emphasis is on communicating an experienced emotion to the observer and possibly provoking empathy, or on providing a danger signal to an observer with the goal of preparing him to act. To date there have only been very few studies that have manipulated separately the facial expression and the direction of gaze (Adams and Kleck, 2005, Ganel et al., 2005). Two previous studies used behavioral measures (Adams & Kleck, 2003) and event-related brain imaging (Adams et al., 2003) to investigate the combined effect of gaze and facial expression. The behavioral study revealed a shorter reaction time for fearful faces looking away compared with fearful faces looking at the observer. The brain imaging study revealed increased amygdala activation for stimuli consisting of angry faces with an averted gaze and fearful faces with a direct gaze, that were ambiguous in terms of their significance as threat to the observer. No other areas were reported.

It is reasonable to expect that different brain networks are involved as a function of the direction of gaze a facial expression. To address this issue, we manipulated direction of gaze in fearful faces. Our goal was to test a single very specific prediction related to the combined perception of fear and averted gaze and its impact on action readiness. Our hypothesis was that a fearful face expression with averted gaze signals a danger in the environment and may trigger activity in brain areas involved in characteristic adaptive action associated with fear such as preparing to flight. If fearful faces with averted gaze do indeed signal danger in the environment (threat-related), then they should prompt more premotor and motor activity than faces with directed gaze where the emphasis is more on communicating the emotion and triggering empathy in the observer (empathy-evoking). We tested the hypothesis that threat-related fearful face would modulate activation in areas involved in stimulus detection, in fear processing, and in preparation for action. We did not use neutral faces in this study, because in the context of an emotional expression study, neutral faces suffer from carry-over effects and acquire an unintended emotional significance. To specifically address our hypothesis related to fear processing we limited our paradigm to fearful expression of emotion.