The observer observed: Frontal EEG asymmetry and autonomic responses differentiate between another person's direct and averted gaze when the face is seen live

Abstract

Recently, we showed that another person's gaze direction influenced the perceiver's frontal EEG asymmetry and autonomic arousal in response to freely viewed real faces, but not in response to face pictures. However, the lack of a task during the viewing may have resulted in less attention allocation to face pictures vs. live faces. In the present study, the participants performed two online tasks while viewing the faces presented live through an electronic shutter and as pictures on a computer screen. The results replicated those from our previous experiment showing that direct gaze elicited greater relative left-sided frontal EEG asymmetry and autonomic arousal than averted gaze but, again, only in the live condition. However, the results also showed that two live stimulus faces (male and female) elicited differential EEG asymmetry responses in our participants (all females), and the effects of gaze direction were observed only for the (live) female faces. The results suggest that the discriminative responses to live faces vs. pictures are likely to reflect the participants' enhanced mental-state attributions and self-awareness when looking at and being looked by live faces. Thus, the motivation- and affect-related psychophysiological responses to gaze direction are most discriminative in the presence of another person, regardless of whether the face/gaze is actively monitored or not.

Introduction

The vast development of computer- and video-mediated communication devices has allowed two people to effortlessly engage in face-to-face interaction without being physically near each other. Current research also shows that impressions given through computer-mediated communication approximate those given in live interaction (Weisbuch et al., 2009). How can it be explained, then, that only a minority of significant decisions related to other people, for instance, whom one is going to hire or marry, are currently based on interaction conducted by virtual means? Apparently, something is lost in transfer when people do not share the same physical space when communicating with each other. Smooth and efficient interaction between people requires accurate perception of a host of different types of nonverbal cues. Gaze direction seems to have an important role in gathering socially relevant information from other people (George and Conty, 2008, Kleinke, 1986). Monitoring of others' gaze is needed especially in the initiation of interaction, as it aids in defining the nature of dyadic relations (Exline et al., 1965) and motor plans (Nummenmaa et al., 2009) in the first place. If the other person's gaze is turned away, it is likely to be seen as an act of aversion, whereas seeking of an eye contact is usually seen as a gesture of approach. Although these action tendencies can be communicated to some extent in virtual interaction, meeting the eyes of a person present in the same space is likely to elevate the anticipation of the other person's likely next actions and the urge to evaluate the consequences of these actions, and heighten the sense of reciprocal involvement.

In recent experiments, we have measured brain responses and autonomic responses to facial stimuli differing in preparedness for social interaction. These stimuli have been presented in two conditions, as pictures on a computer monitor and “live”, referring here to a condition in which the stimulus person has been physically present and has shown his/her face through a computer-controlled liquid crystal (LC) window (e.g., Hietanen et al., 2008). The results have shown greater visual event-related potentials (ERPs) to a human face vs. a dummy face (Pönkänen et al., 2008) and to a direct vs. an averted gaze (Pönkänen et al., 2011), but only when the faces have been presented live through the LC window. In addition to these early-stage neurocognitive responses, Hietanen et al. (2008) showed that seeing another person's direct and averted gaze activates the motivational approach–avoidance brain systems, respectively, indicated by asymmetrical alpha power distribution in the frontal electroencephalography (EEG). There is a growing body of evidence showing that approach motivation enhances relative left prefrontal cortex activity and that avoidance motivation enhances relative right prefrontal cortex activity, regardless of stimulus valence (for a review, see Harmon-Jones et al., 2010). Hietanen et al. (2008) also measured sympathetic arousal (skin conductance responses), considered to be a good index of the general energetic level (arousal) of behaviour (Andreassi, 2000), and showed greater arousal for direct gaze than averted gaze. Again, all these effects of gaze direction were observed only when the stimulus faces were presented live though the LC window. Thus, we suggested that when facing a live person, gaze direction is likely to play a greater role in influencing sensations of intimacy, experienced self-relevance, and awareness of how one is seen by another person who is physically present, as compared to seeing a picture of a face on a computer screen (Hietanen et al., 2008, Pönkänen et al., 2011).

However, there are several, different types of behavioral and neurophysiological studies showing gaze direction effects also when pictures or animations of faces are used as stimuli. Notably, in many of these studies (i.e., Conty et al., 2007, Mason et al., 2005, Sato et al., 2008, Schilbach et al., 2006) discrimination of gaze direction or some other explicit task related to face categorization was employed. In our aforementioned studies, instead, the faces were observed without a concomitant task (Hietanen et al., 2008, Pönkänen et al., 2011, Pönkänen et al., 2008). Thus, the question arises, whether the lack of a task, related to the gaze direction or not, might have contributed to the lack of gaze direction effects in the picture presentation mode in these studies.

There are several ways in which the presence of a concomitant task may contribute to the differing neurocognitive and motivational responses to gaze direction when faces are presented live vs. as pictures. With even a simple task, the participants' focus is directed according to the task requirements, whereas in free viewing of the face stimuli the participants' allocation of attention has more degrees of freedom. For example, when participants are asked to perform discrimination based on some facial attributes (e.g., expression, gender, gaze direction etc.) the participants have to pay attention to faces independent of whether they are presented live or as pictures. Instead, it is possible that participants allocate considerably less attention to faces without than with a face-related task and, moreover, that there is also a discrepancy in attention allocation between live faces and pictures of faces. A live face is potentially capable for changes (e.g., expression, gaze direction, mouth area movements) and the changes are likely to be somewhat intentional. A live face is “present”, whereas the face pictures lack the charm of immediate presence. Moreover, there may be greater spontaneous attention to the eyes presented live vs. as pictures due to social learning: when facing another person for the first time, at least in western cultures it is common to catch the eyes of the other when initiating social interaction (Argyle, 1981). Hence, as suggested also in our previous studies (Hietanen et al., 2008, Pönkänen et al., 2011), free viewing of face pictures is likely to be less motivating and less attention-demanding than free viewing of live faces. Therefore, in the present study, we continued investigating the effects of seeing another person's direct vs. averted gaze on functioning of the approach–avoidance brain systems (frontal EEG asymmetry and autonomic responses) by comparing the effects of gaze direction between live vs. picture presentation modes, but this time in a condition where the participants were performing face-related tasks during the stimulus presentation. In the present study, we used two “online” tasks: discrimination of another's gaze direction and evaluation of one's own feelings of pleasantness in response to this stimulus. By including these two tasks we aimed at explicit simulation of other- and self-related mental attributions which are suggested to interplay in social cognitive processing (Legrand and Ruby, 2009, Mitchell, 2009).

In our previous study (Hietanen et al., 2008), we suggested that the differential physiological responses between live and picture conditions could be related to differences in experienced self-awareness when facing a live person vs. a picture, i.e., differences due to the fact that, in the live condition, the perceiver knows that the other person can really see him/her, whereas in the picture condition the perceiver knows that this is not the case. In the present study, we were interested in expanding our previous results by investigating whether the gaze direction can modulate the self-awareness evaluations. In Hietanen et al. (2008) study, we compared self-awareness ratings during viewing of live vs. picture faces with a direct gaze. The results showed that subjective ratings of “public self-awareness”, related to the feelings of how one is perceived in the eyes of others (Govern and Marsch, 2001), were higher when the faces were presented live vs. as pictures. Interestingly, evaluations of the other two types of self-awareness, directed to one's inner feelings or relations to external surroundings, were not affected by the presence of a live vs. picture face. In the present study, we wanted to investigate whether public self-awareness ratings are also influenced by the gaze direction, and whether the possible influence is restricted only to live faces.

In sum, in the present study, we measured relative hemispheric asymmetry in the frontal EEG and skin conductance responses (SCRs) to another person's direct and averted gaze presented live through an LC window vs. as pictures on a computer screen. During physiological recordings, the participants were performing two tasks: gaze direction discrimination and evaluation of one's own feelings of pleasantness towards the stimulus face. We also studied whether the gaze direction has an effect on the subjective ratings of self-awareness and, especially, on self-awareness related to how one is perceived in the eyes of the other people. This was investigated by employing the Situational Self-Awareness Scale questionnaire (SSAS; Govern and Marsch, 2001). Finally, we also wanted to investigate if the sex of the stimulus face would modulate the results. In our previous study (Hietanen et al., 2008), participants were shown only female faces. However, females have been reported being more approachable than males when rating facial pictures (Campbell et al., 2010). It is thus possible that the motivation- and affect-related psychophysiological responses could be sensitive to the sex of the gazing person. In the present study, therefore, we decided to use both a female and a male model as stimuli. Because this was the first study in which we attempted to investigate the possible effects of stimulus face sex on neural approach–avoidance -related responses, we recruited female participants only. Previous research has shown that females show greater physiological responses to emotion-related facial cues than males (e.g., Anokhin and Golosheykin, 2010), and females are behaviourally more sensitive to eye gaze as compared to males (Gueguen and Jacob, 2002). Four main hypotheses were tested: i) perceiving a direct gaze would elicit relative left-sided frontal EEG asymmetry indicative of a motivational tendency to approach and averted gaze would elicit smaller relative left-sided asymmetry or even relative right-sided asymmetry indicative of avoidance, ii) the SCR would be greater for the direct vs. averted gaze, iii) public self-awareness would be heightened in response to a direct vs. averted gaze, and iv) to extend and corroborate the earlier findings (Hietanen et al., 2008), we expected to obtain all these effects in the live but not in the picture presentation mode also when we apply a concomitant behavioural task to ensure comparable attention allocation to faces in both stimulus presentation modes.