Discrimination of emotional facial expressions in a visual oddball task: an ERP study

https://doi.org/10.1016/S0301-0511(02)00005-4Get rights and content

Abstract

Several ERP studies have shown an orienting complex, the N2/P3a, associated to the detection of stimulus novelty. Its role consists in preparing the organism to process and react to biologically prepotent stimuli. Whether this N2/P3a: (1) could be obtained with complex visual stimuli, such as with emotional facial expressions; and (2) could take part in a complex discrimination process has yet to be determined. To investigate this issue, event-related potentials were recorded in response to repetitions of a particular facial expression (e.g. sadness) and in response to two different deviant (rare) stimuli, one depicting the same emotion as the frequent stimulus, while the other depicted a different facial expression (e.g. fear). As expected, deviant stimuli evoked an N2/P3a complex of larger amplitude than frequent stimuli. But more interestingly, when the deviant stimulus depicted the same emotion as the frequent stimulus the N2/P3a was delayed compared to the response elicited by the different-emotion deviant. The N2/P3a was thus implicated in the detection of physical facial changes, with a higher sensitivity to changes related to a new different emotional content, perhaps leading to faster adaptive reactions.

Introduction

Most event-related brain potential (ERP) studies have used an ‘oddball paradigm’, in which subjects have to detect, amongst a series of standard stimuli, an infrequent deviant one (Garcia-Larrea et al., 1992). The detection of stimulus change may play a role in turning attention to events of biological importance (Halgren and Marinkovic, 1995). This has been indexed by three main ERP components, in the auditory as well as in the visual modalities. First, when subjects are placed in inattentive conditions, deviant stimuli in a homogenous stimulus sequence elicit a specific negative deflection, called mismatch negativity (MMN) (Näätänen et al., 1978 for audition; Tales et al., 1999 for vision), which reflects an automatic (attention-independent) neural mechanism underlying the perception of stimulus change (Näätänen et al., 1993).1 Second, when subjects are placed in attentive conditions, deviant stimuli evoke a series of field potentials, the N2/P3a, overlapping the MMN activity described above, and called by Halgren and Marinkovic (1995) the orienting complex, because it subserves attention. Indeed, the orienting complex is defined as the mobilization of cerebral and somatic resources in order to effectively cope with a biologically important event. Third, a P3b component, recorded maximally at parietal sites and functionally related to the conscious detection of change leading subjects to respond to deviant stimuli, has also been recorded (Bentin et al., 1999, Campanella et al., 2000).

In the present study, we employed a variation of the visual oddball paradigm2 that did not manipulate attention. Our hypotheses were focused on the N2/P3a complex and the P3b component, given that the MMN is classically described in inattentive conditions. As suggested above, the N2/P3a complex could reflect the afferent (preparation-to-process) and efferent (preparation-to-respond) functions of the orienting complex (Halgren and Marinkovic, 1995), whereas the P3b component is possibly related to the conscious subjects' responses (Bentin et al., 1999). Therefore, it is plausible to think that: (1) similar N2/P3a and P3b components could be obtained in response to deviant complex visual stimuli, such as emotional facial expressions (due, for instance, to their high importance in social communication); and (2) these components could be modulated, in latency and/or in amplitude, by the categorical nature of frequent and deviant stimuli.

By using a morphing procedure, it is possible to generate continua of different morphed faces moving linearly from one facial expression (e.g. sadness) to another one (e.g. fear). Several studies have shown that two different morphed faces, perceived as sharing the same emotion (WITHIN-categorical differences), are harder to discriminate than two different morphed faces perceived as two different emotions (BETWEEN-categorical differences), even if the physical distance inside each pair is identical (Etcoff and Magee, 1992, Calder et al., 1996, Young et al., 1997).

The main purpose of the present study was to show that the visual N2/P3a complex elicited in response to deviant stimuli, would be modulated (in amplitude and/or in latency) whether or not deviant stimuli shared the same facial expression as the frequent one. We hypothesized that this modulation would be more important when the rare stimulus does not share the same informational content then the frequent stimulus. This is of the greatest relevance: (1) as expressive faces are very relevant social signals, so that we have to pay attention to them, particularly when a change in a facial emotional expression has to be detected; and (2) as it would index, in the N2/P3a, a phenomenon taking part in an active discrimination process for emotional facial expressions, in order that subjects could cope efficiently with particular (e.g. threatening) facial expressions. Indeed, in such a case, it would be necessary to analyze what can be done about this situation, in order to furnish fast and adaptive reactions (Bradley et al., 1999, Mogg et al., 2000). Moreover, as expressive morphed faces were used in the present study, we expected that the present results could lead us to learn more about the neurophysiological processes involved: (1) in facial expression analysis; and (2) in the phenomenon of categorical perception of emotion.

Section snippets

Materials and method

Two faces (A, B) with sad and fear expressions were taken from Ekman and Friesen series (1976). Two continua of faces were therefore possible (‘A sad’ to ‘A fear’ and ‘B sad’ to ‘B fear’). Four morphed images were created for each continuum. They were prepared by blending two faces in proportion 5:95 (i.e. 5% ‘A fear’ and 95% ‘A sad’), 35:65, 65:35 and 95:5. We will refer to them as 5, 35, 65 and 95% morphs along the appropriate continuum (i.e. ASF 5% refers to face A, continuum sadness to

Behavioral data

Table 2 illustrates the performance of the subjects when they had to detect deviants in the stimulus sequences. As in previous studies (Etcoff and Magee, 1992, Calder et al., 1996, Young et al., 1997), we found that subjects detected deviants more easily when they expressed a different emotion (rare BETWEEN) than when they expressed the same one (rare WITHIN). This was indexed by both greater accuracy (t(9)=4868; P=0.001) and by faster response latencies (t(9)=4223; P=0.002).

Event-related potentials

In the present

Discussion

This study was carried out in an attempt to establish whether the visual N2/P3a orienting complex evoked by deviant stimuli in a sequence of frequent ones could be modulated in amplitude and/or latency by the categorical origin of both stimuli. This goal was achieved by the use of the behavioral categorical perception effect shown on facial emotional expressions (Calder et al., 1996, Young et al., 1997). Indeed, by using a morphing technique, these authors showed that two different morphed

Conclusions

The findings of the present study leads to three main considerations. First, by using a complex visual oddball task with faces, the N2/P3a is associated with deviants and can be modulated by the categorical origin of the frequent and rare stimuli. This is of particular interest, because it suggests that the visual N2/P3a activity does not only reflect an automatic detection of the rarity of a visual event (in order to switch attention), but is also engaged in an automatic discrimination of the

Acknowledgments

This study was supported by the grant no. 95/00-189 (‘Action de Recherche Concertée’) from the Government of the French-speaking Community. The first author was supported by the Belgian Fund of the Scientific Research (FNRS).

References (33)

  • S. Bentin et al.

    Structural encoding and identification in face processing: ERP evidence for separate mechanisms

    Cognitive Neuropsychology

    (2000)
  • B.P. Bradley et al.

    Attentional bias for emotional faces in generalized anxiety disorder

    British Journal of Clinical Psychology

    (1999)
  • A.J. Calder et al.

    Categorical perception of morphed facial expressions

    Visual Cognition

    (1996)
  • S. Campanella et al.

    Right N170 modulation in a face discrimination task: an account for categorical perception of familiar faces

    Psychophysiology

    (2000)
  • D.M. Clark et al.

    A cognitive model of social phobia

  • D.M. Clark

    Anxiety disorders: why do they persist and how to treat them

    Behavior Research and Therapy

    (1999)
  • Cited by (0)

    View full text