Introduction
A salient stimulus in the environment will automatically attract one’s attention. Such an orienting reflex is quite adaptive, as the stimulus may signal a threat or a desired resource and require subsequent action. Likewise, viewing the direction of another’s eye gaze automatically directs attention toward the location of the gaze (Friesen & Kingstone,
1998; Friesen, Moore, & Kingstone,
2005). This orienting response is also evolutionarily adaptive, as the focus of another’s attention often indicates intentions to act upon the environment. Such intentions will likely influence our own actions, especially in the case of a threat or desired resource.
Experimentally, orienting attention in space has been studied extensively using a spatial cueing paradigm originally described by Posner (
1980). Traditional models of spatial attention distinguish between automatic and controlled orienting, with distinct methodological specifications to measure each one (Jonides,
1981). When measuring automatic orienting, a peripheral cue (e.g., an asterisk appearing in a possible target location) typically precedes a target stimulus to which the participant must respond. Generally, peripheral cues are nonpredictive, indicating the location of the target 50% of the time. When measuring controlled orienting, a centrally presented, symbolic cue (e.g., an arrow pointing toward a possible target location) typically precedes the target. To further engage controlled processes, central cues are often predictive, indicating the location of the target more than 50% of the time. In spite of these differences, gaze cues that were centrally presented (though nonpredictive) demonstrated similar behavioral effects to traditional automatic (peripheral) cues (Driver et al.,
1999; Friesen & Kingstone,
1998).
To examine the effects of social cues on orienting attention (henceforth
social orienting), Friesen & Kingstone (
1998) developed a variant of Posner’s spatial cueing paradigm using schematic eyes looking left or right as the cue preceding the target. Several investigators have since looked at the effects of gaze cues with spatial cueing paradigms, using both schematic faces (Friesen et al.,
2005; Ristic, Friesen, & Kingstone,
2002) and pictures of real faces (Driver et al
.,
1999; Langton,
2000). These studies suggest that directed eye gaze influences attentional orienting in a seemingly automatic manner. Similar to classic automatic cues, participants respond faster to validly cued (gazed at) targets than invalidly cued targets, an effect termed
facilitation, even when the gaze is not predictive of the target location. In addition, this effect has a relatively short time course, occurring when the time interval from the onset of the cue to the onset of the target (generally referred to as the stimulus onset asynchrony: SOA) is very short (105 ms). Further, facilitation holds even when the gaze cues are counter-predictive (Driver et al.,
1999; Friesen, Ristic, & Kingstone,
2004). That is, when participants know that eyes looking left indicate that the target will most likely appear on the right, they are still faster to respond to a target in the gazed at location. Thus, there is something special about eye gaze that automatically directs attention, even when contradicting cognitive goals.
It has been proposed that social orienting depends on a distinct neural system from that supporting automatic orienting in response to nonsocial information (Kingstone, Friesen, & Gazzaniga,
2000; Ristic et al.,
2002). One reason for this claim stems from a behavioral difference between social and peripheral cueing. When the SOA is longer than approximately 300 ms, participants are actually slower to respond to targets that are validly cued than invalidly cued when peripheral cues are used. This inhibitory effect, termed inhibition of return (IOR), is thought to reflect an evolutionary mechanism that aids in efficient visual search by preventing a return to previously searched locations (Klein,
2000). However, IOR is rarely found with gaze cues (Friesen & Kingstone,
1998; McKee, Christie, & Klein,
2007; but see Frischen & Tipper,
2004). Rather, as the SOA increases, facilitation decreases and eventually disappears without a reverse in the effect. Does this imply that social cues override the inhibitory mechanism present for peripheral cues? This is a tempting theory, as a stimulus worthy of another’s attention is most often worthy of our own. In addition, with the recent interest in identifying neural networks relevant to social behavior, it is important to investigate whether some attentional systems may be specialized for social information and subserving social interactions. From an evolutionary perspective, social interactions and the use of social cues have become increasingly sophisticated throughout evolution (Emery,
2000). Therefore, identifying neural networks that are relevant to orienting based on social cues may provide clues as to how the brain evolved to support this change.
To investigate the neural mechanisms underlying social orienting, Kingstone, Friesen, & Gazzaniga (
2000) administered a lateralized version of the gaze cueing paradigm to two split-brain patients. Previous testing with these patients revealed that one had face processing lateralized to the right hemisphere, while the other had face processing lateralized to the left hemisphere. Interestingly, they found that social orienting effects only occurred in the hemisphere specialized for processing faces. This laterality effect was not found in previous work using nonsocial cues (Kingstone, Grabowecky, Mangun, Valsangkar-Smyth, & Gazzaniga,
1997). Thus, the authors argue that social orienting relies on asymmetric cortical mechanisms, whereas orienting to nonsocial cues relies on subcortical mechanisms that are shared between the hemispheres.
Studies investigating the specific cortical networks involved in social orienting are limited. Neuroimaging and single-unit recording studies examining gaze
perception have found that the superior temporal sulcus (STS) plays a prominent role in processing gaze (Hoffman & Haxby,
2000; Hooker et al.,
2003; Perrett et al.,
1985). When Hoffman and Haxby compared blood-oxygenation-level-dependent (BOLD) activity when viewing gaze directed toward and away from the participant, they found stronger activation in the intraparietal sulcus (IPS) in the latter. The posterior parietal cortex (PPC), including the IPS, is part of a fronto-parietal network presumed to support spatial attention (Corbetta & Shulman,
2002; Mesulam,
1981,
1999). Parietal activity, along with STS activity, has also been found using fMRI in adults and children who viewed gaze shifts (Mosconi, Mack, McCarthy, & Pelphrey,
2005; Pelphrey, Singerman, Allison, & McCarthy,
2003). Thus, perceiving another’s averted gaze recruits part of the circuitry involved in attentional orienting.
To date, there are three neuroimaging studies that have compared brain activity involved in social orienting to nonsocial orienting. Kingstone, Tipper, Ristic and Ngan (
2004) measured BOLD activity while participants responded to a target that was cued by an ambiguous stimulus, which could be interpreted as a social figure (eyes with a hat) or as a nonsocial figure (a car). They found the right STS to be selectively active for the social cue condition in contrast to the nonsocial cue condition. More recently, Tipper, Handy, Giesbrecht and Kingstone (
2008) developed another ambiguous stimulus cue, which could be perceived as a profile of an eye or as an arrowhead. They found largely overlapping areas of BOLD activation for both percepts in fronto-parietal regions, with stronger activation for the social cue in select occipital and frontal regions. They also measured event-related potentials (ERPs) from the two possible percepts, finding evidence for a sensory gain effect when the cue was perceived as an eye profile. The authors argue that social and nonsocial orienting engage the same network, with stronger engagement of select regions for social orienting. In contrast to Kingstone et al. (
2004), they did not find selective activation in the STS for the eye profile percept. Thus, it is possible that the ambiguous stimulus used by Tipper et al. did not resemble an actual eye well enough to engage social networks, and simply required more symbol interpretation. In the third fMRI study, Hietanen et al. (
2006) compared schematic gaze and central arrow cues. They found that directional arrow cues selectively recruited fronto-parietal areas, while both arrow and gaze cues recruited occipito-temporal areas. In these overlapping regions, activation was stronger and the extent was larger for the arrow cues than the gaze cues, a finding that parallels comparisons of controlled and automatic orienting, respectively (Kim et al.,
1999; Rosen et al.,
1999). Thus, gaze cues may in fact rely on more automatic processes than arrow cues.
In the above imaging studies, gaze cues were compared to symbolic cues, which typically engage controlled orienting. Although it has been suggested that nonpredictive arrow cues can elicit automatic orienting (Tipples,
2002), they may not be purely automatic and likely involve controlled processes as well (Friesen et al.,
2004). The imaging results of Hietanen et al. (
2006) further suggest that gaze cues are more automatic than arrow cues. However, a direct comparison of the neural activity involved in gaze cueing and purely automatic nonsocial cueing has not been conducted, and that was the purpose of the present study.
The present experiment used fMRI to directly compare BOLD activity involved in social orienting to that involved in purely automatic nonsocial orienting. In order to maximally engage automatic processes, social and nonsocial cues were nonpredictive of the target location. Since the major behavioral difference between gaze cues and traditional automatic cues is the influence of the SOA on behavior (IOR found at long SOAs for nonsocial cues, but not for social cues), we implemented two SOAs to investigate the neural correlates of this effect. Subcortical regions, specifically the superior colliculus, have been shown to play a role in IOR (Dorris, Klein, Everling, & Munoz,
2002; Sapir, Soroker, Berger, & Henik,
1999). We hypothesized that the neuroimaging data will parallel the behavioral results, demonstrating subcortical activation in conditions that yield IOR. This hypothesis is consistent with the theory that social orienting relies on cortical mechanisms, while nonsocial orienting recruits subcortical mechanisms shared between the hemispheres.
Discussion
The goal of the present study was to compare the neural circuitry involved in social orienting to that involved in automatic nonsocial orienting. We used event-related fMRI to measure BOLD activity while participants performed a spatial cueing task consisting of social (eye gaze) and automatic nonsocial (peripheral) cues. Our results revealed that the neural networks involved in automatic orienting driven by social and nonsocial cues are largely similar, involving oculomotor and attentional regions. On the other hand, social cueing recruited extrastriate regions involved in social perception, while nonsocial cueing recruited subcortical regions during conditions that produced IOR.
The task implemented in the present study allowed us to examine how automatic orienting influences subsequent actions. Participants had the specific goal of responding to a target stimulus. Thus, they maintained an intention to act solely upon the location of the target. However, their actions were influenced by automatic orienting processes driven by the cues (both social and nonsocial), resulting in variations in response speed. This reflects the close relationship between automatic orienting and intentional action. On a daily basis, we automatically orient toward important stimuli, but what is the purpose of this process? We are not simply passive observers of the world. Rather, we are engaged, active participants, and as such, automatic processes are useful to guide intentional actions.
Our finding that social cueing and automatic nonsocial cueing recruited largely overlapping cortical networks of activity is consistent with the results of Tipper et al. (
2008). Although Tipper et al. used an ambiguous stimulus and we used two visually different stimuli, the social and nonsocial conditions in both studies resulted in overlapping activation in frontal premotor areas, the PPC, and occipito-temporal regions. By using an ambiguous stimulus, Tipper et al. were able to control for simple visual differences that may drive differences in neural activity. They reported more clusters of activation in lateral frontal areas than we did here. Lateral frontal activity has been attributed to the need for symbol interpretation in some attention tasks (Woldorff et al.,
2004). Thus, it is possible that the recruitment of these regions was due to the symbol interpretation required to perceive their ambiguous stimulus as an eye profile. Such a cognitive requirement brings top-down processes into the task, possibly contaminating the measurement of automatic, bottom-up processes. It should be noted that their behavioral results were equivalent for both cue percepts. Yet, it is possible that the BOLD signal may have been more sensitive to additional top-down processing than reaction time performance. In our main experiment, symbol interpretation was not required, as the stimuli were automatically perceived as social or nonsocial. Thus, our nonsocial cue is purely automatic. Of course, we compared two visually different stimuli, but even so, social and nonsocial cueing demonstrated largely overlapping activity. Our results together with that of Tipper et al. suggest that social and automatic nonsocial orienting engage similar cortical mechanisms.
Overlapping activity for social and nonsocial cues further supports the idea that we automatically orient to where someone else is looking. Sudden changes in the periphery cause automatic attention shifts, allowing one to be vigilant to the environment and mobilize appropriate responses to the stimulus at hand. When another individual gazes toward a particular location, he/she may be mobilizing an appropriate response toward his/her focus of attention. It is advantageous for one to attend to the same location as others and possibly mobilize the same response. Thus, it makes sense that similar neural mechanisms were selected for responding to eye gaze cues and peripheral cues.
Specifically, overlapping activity was found in a fronto-parietal network including the FEF, SEF, and PEF. Several neuroimaging studies have found activation in this fronto-parietal network for both covert shifts in attention (without eye movement) and overt eye movements (Beauchamp, Petit, Ellmore, Ingeholm, & Haxby,
2001; Corbetta et al.,
1998; de Haan, Morgan, & Rorden,
2008; Nobre, Gitelman, Dias, & Mesulam,
2000), supporting the premotor theory of attention (Rizzolatti, Riggio, Dascola, & Umilta,
1987). This theory proposes that covert orienting and saccadic eye movements are mediated by the same neural network, and hence, covert orienting reflects planning an eye movement without proceeding with its execution. In the present study, covert orienting in response to social and nonsocial cues engaged these oculomotor regions. Thus, our results are consistent with the premotor theory of attention and suggest that it extends to social cues. As we were unable to track eye movements, it is possible that the oculomotor system was engaged due to uncontrolled eye movements. However, several studies of covert orienting have shown that participants have little trouble maintaining fixation when instructed to do so (Corbetta et al.,
1998; Corbetta, Miezin, Shulman, & Petersen,
1993). Given that such instructions were emphasized during the practice block and before each experimental run, it is unlikely that our participants moved their eyes freely.
When comparing the cue conditions, we found that the social cues yielded greater activity in extrastriate regions than the nonsocial cues. It has been proposed that these areas are specialized for processing social information and make up a network for face perception (Haxby, Hoffman, & Gobbini,
2000). Specifically, we found greater activation for the social cue condition in the fusiform gyrus, which includes the “fusiform face area” (FFA) (Kanwisher & Yovel,
2006), and in the inferior lateral occipital cortex, which has been termed the “occipital face area” (OFA) (Gauthier et al.,
2000). Both the FFA and OFA have demonstrated selectivity in responding to faces. In our social cue condition, the pupils filled in empty eyes, enhancing the stimulus’ appearance as a face. Thus, it was not surprising to find greater activity for social cueing in these regions.
As previously mentioned, it is possible that the increased extrastriate activity found for social cues was a result of the physical location of the cue (central vs. peripheral), not its social nature. In order to examine this alternative explanation, we ran another ten participants on the same task with the inclusion of central arrow cues. Again, we found greater activation in the same extrastriate regions when comparing social and peripheral cues. Examination of the BOLD signal within these regions revealed a greater increase in activity for the social cue than for the central nonsocial cue (arrow), as well as a greater increase for the central nonsocial cue than for the peripheral nonsocial cue. Thus, increased extrastriate activation was not simply driven by the central location of the cue. We used an arrow with an arrowhead on both ends in an attempt to minimize the perception of the central nonsocial cue as social. Yet, it is possible that the arrow cue, appearing in the location of a mouth, resembled a social gesture such as a smirk in one direction or the other. Thus, the cue may have been perceived as more “social” than the square cue, but less obviously “social” than the gaze cue, fitting with the intermediate engagement of extrastriate regions. Interestingly, the behavioral results of our control experiment did not differentiate between gaze and arrow cues, suggesting again that in some cases BOLD changes may have higher sensitivity than response times in differentiating mechanisms for attentional orienting.
It is worth noting that we did not find greater STS activity for the gaze cue condition compared to the nonsocial cue condition. In fact, we did not find greater STS activity for either of the cue conditions compared to rest. This may be due to the constant view of the empty face image during the resting baseline condition. It is unlikely that participants did not view the gaze cue as eyes, since very simple stimuli, like those used in our task, are easily perceived as eyes, even in nonprimate animals (for a review, see Emery,
2000). In fact, it is quite difficult
not to perceive the eyes. Thus, the continuous display of empty eyes may have subtracted out STS activity involved in gaze perception.
Since the time interval between the cue and the target (SOA) influences performance on a behavioral level, we included a short (150 ms) and a long (950 ms) SOA in order to examine the neural correlates of such behavioral differences. At the short SOA, both social and nonsocial cues elicited facilitation effects (faster reaction times to validly cued targets than invalidly cued targets). Yet, at the long SOA, behavior diverged and nonsocial cues yielded the reverse effect, IOR, while social cues showed no effect, consistent with previous findings (Friesen & Kingstone,
1998). Interestingly, our imaging data showed that brain activity diverged along with behavior. At the short SOA, nonsocial cueing did not elicit greater neural activity than social cueing. At the long SOA, however, there was increased activity in subcortical regions for the nonsocial cue. Specifically, several cerebellar regions were active, suggesting a role for the cerebellum in IOR. It is unlikely that this activation was driven by the visual differences between cue types because if that were the case, increased activity would be found regardless of SOA.
The cerebellum has a well know role in oculomotor control (Robinson & Fuchs,
2001). The posterior vermis has been called the “oculomotor vermis,” since Purkinje cell activity in this region increases during saccades, and stimulation of this region can induce saccades (Noda & Fujikado,
1987) or smooth-pursuit eye movements (Krauzlis & Miles,
1998). Evidence suggests a necessary role for the oculomotor vermis and its deep cerebellar nuclei (i.e., caudal fastigial nuclei) in both accurate saccades and smooth-pursuit eye movement (Robinson & Fuchs,
2001). A question that has arisen in the literature is the role of the cerebellum in attention. Patients with cerebellar damage have demonstrated impaired performance on neuropsychological tests of attention, and several neuroimaging studies implementing attention paradigms have found activation in the cerebellum (for a review, see Haarmeier & Thier,
2007). However, upon closer examination of the specific tests used in these studies, Haarmeier and Thier concluded that the deficits observed in patients and the activations observed with neuroimaging were a result of oculomotor demands. Thus, the cerebellar activation we observed during peripheral cueing at the long SOA is consistent with an oculomotor theory of IOR, which proposes that IOR results from oculomotor planning (Rafal, Calabresi, Brennan, & Sciolto,
1989). This inhibitory mechanism is important because of the tight link between automatic activation of motor plans and intention to act upon the world. In order to facilitate efficient and adaptive actions within the environment, it is just as necessary to inhibit oculomotor responses to repeated stimuli as it is to activate oculomotor responses to novel stimuli. Our results indicate that the cerebellum may mediate such inhibition, biasing oculomotor control away from repeated stimuli. Other types of cues, such as eye gaze, can override this mechanism on both a neural and behavioral level.
The superior colliculus (SC) is another subcortical region that has been implicated in IOR. Evidence has shown that a patient with damage to the right SC demonstrated IOR in the hemifields projecting to the left (intact) SC, but not in the hemifields projecting to the damaged SC (Sapir et al.,
1999). Single-unit recording data from SC neurons in the rhesus monkey demonstrated suppressed SC activity when a target occurred at a cued location (Dorris et al.,
2002). However, neurons were more excitable during the time window between the cue and the target. Thus, the authors suggest that higher cortical areas may be responsible for inhibiting SC activity, and this inhibition in turn leads to IOR. The PPC has been proposed as a likely candidate of this inhibition due to its role in attentional orienting and its neuroanatomical connections with the SC (Klein,
2000). Neuroimaging research has supported the involvement of the PPC and other oculomotor areas, including the SEF and FEF, in IOR (Mayer, Dorflinger, Rao, & Seidenberg,
2004; Muller & Kleinschmidt,
2007). In the present study, we did not find increased activity in the fronto-parietal network specifically for the conditions in which IOR was found (peripheral cue, long SOA). Rather, this network was engaged during facilitation and IOR, and for both social and nonsocial cueing.
Due to the evidence supporting a role for the SC in IOR, we expected to find SC activity during the conditions that yield IOR. However, activation did not survive cluster thresholding likely because of the tiny size of the SC and because our imaging protocol was not optimized to measure collicular activity. Therefore, we performed an ROI analysis of the region, which revealed a trend toward increased engagement of the SC for peripheral cue, long SOA trials. Thus, both the superior colliculus and the cerebellum demonstrated greater activity during conditions that generate IOR, suggesting that IOR indeed involves subcortical mechanisms. Therefore, it is possible that IOR is not found with social cues because social cues engage cortical structures that override the subcortical mechanisms. The activity in occipito-temporal cortical areas differentiating between social and nonsocial cues may—at least in part—reflect a modulatory role of these areas over subcortical structures.
The results of this study carry interesting evolutionary implications. The importance of eye gaze in social interactions has increased throughout evolution. We suggest that the social orienting system may have developed at a later evolutionary stage than nonsocial automatic orienting, consistent with the evolutionary shift from subcortical to cortical structures. Social orienting may have stemmed from the evolutionarily older system for automatic orienting, shifting away from subcortical mechanisms but utilizing some of the same cortical mechanisms, as indexed by overlapping cortical networks of activity. Additionally, as the relevance of social information became more prominent throughout evolution, cortical mechanisms may have evolved to override the subcortical ones that are involved in inhibitory behavior. Given the rich information conveyed by gaze direction, it may not be adaptive to inhibit an oculomotor response toward repeatedly gazed at locations. If a stimulus is worth looking at by another individual, it is important enough to override previous intentions and influence future actions.