Discrimination of emotional facial expressions in a visual oddball task: an ERP study
Introduction
Most event-related brain potential (ERP) studies have used an ‘oddball paradigm’, in which subjects have to detect, amongst a series of standard stimuli, an infrequent deviant one (Garcia-Larrea et al., 1992). The detection of stimulus change may play a role in turning attention to events of biological importance (Halgren and Marinkovic, 1995). This has been indexed by three main ERP components, in the auditory as well as in the visual modalities. First, when subjects are placed in inattentive conditions, deviant stimuli in a homogenous stimulus sequence elicit a specific negative deflection, called mismatch negativity (MMN) (Näätänen et al., 1978 for audition; Tales et al., 1999 for vision), which reflects an automatic (attention-independent) neural mechanism underlying the perception of stimulus change (Näätänen et al., 1993).1 Second, when subjects are placed in attentive conditions, deviant stimuli evoke a series of field potentials, the N2/P3a, overlapping the MMN activity described above, and called by Halgren and Marinkovic (1995) the orienting complex, because it subserves attention. Indeed, the orienting complex is defined as the mobilization of cerebral and somatic resources in order to effectively cope with a biologically important event. Third, a P3b component, recorded maximally at parietal sites and functionally related to the conscious detection of change leading subjects to respond to deviant stimuli, has also been recorded (Bentin et al., 1999, Campanella et al., 2000).
In the present study, we employed a variation of the visual oddball paradigm2 that did not manipulate attention. Our hypotheses were focused on the N2/P3a complex and the P3b component, given that the MMN is classically described in inattentive conditions. As suggested above, the N2/P3a complex could reflect the afferent (preparation-to-process) and efferent (preparation-to-respond) functions of the orienting complex (Halgren and Marinkovic, 1995), whereas the P3b component is possibly related to the conscious subjects' responses (Bentin et al., 1999). Therefore, it is plausible to think that: (1) similar N2/P3a and P3b components could be obtained in response to deviant complex visual stimuli, such as emotional facial expressions (due, for instance, to their high importance in social communication); and (2) these components could be modulated, in latency and/or in amplitude, by the categorical nature of frequent and deviant stimuli.
By using a morphing procedure, it is possible to generate continua of different morphed faces moving linearly from one facial expression (e.g. sadness) to another one (e.g. fear). Several studies have shown that two different morphed faces, perceived as sharing the same emotion (WITHIN-categorical differences), are harder to discriminate than two different morphed faces perceived as two different emotions (BETWEEN-categorical differences), even if the physical distance inside each pair is identical (Etcoff and Magee, 1992, Calder et al., 1996, Young et al., 1997).
The main purpose of the present study was to show that the visual N2/P3a complex elicited in response to deviant stimuli, would be modulated (in amplitude and/or in latency) whether or not deviant stimuli shared the same facial expression as the frequent one. We hypothesized that this modulation would be more important when the rare stimulus does not share the same informational content then the frequent stimulus. This is of the greatest relevance: (1) as expressive faces are very relevant social signals, so that we have to pay attention to them, particularly when a change in a facial emotional expression has to be detected; and (2) as it would index, in the N2/P3a, a phenomenon taking part in an active discrimination process for emotional facial expressions, in order that subjects could cope efficiently with particular (e.g. threatening) facial expressions. Indeed, in such a case, it would be necessary to analyze what can be done about this situation, in order to furnish fast and adaptive reactions (Bradley et al., 1999, Mogg et al., 2000). Moreover, as expressive morphed faces were used in the present study, we expected that the present results could lead us to learn more about the neurophysiological processes involved: (1) in facial expression analysis; and (2) in the phenomenon of categorical perception of emotion.
Section snippets
Materials and method
Two faces (A, B) with sad and fear expressions were taken from Ekman and Friesen series (1976). Two continua of faces were therefore possible (‘A sad’ to ‘A fear’ and ‘B sad’ to ‘B fear’). Four morphed images were created for each continuum. They were prepared by blending two faces in proportion 5:95 (i.e. 5% ‘A fear’ and 95% ‘A sad’), 35:65, 65:35 and 95:5. We will refer to them as 5, 35, 65 and 95% morphs along the appropriate continuum (i.e. ASF 5% refers to face A, continuum sadness to
Behavioral data
Table 2 illustrates the performance of the subjects when they had to detect deviants in the stimulus sequences. As in previous studies (Etcoff and Magee, 1992, Calder et al., 1996, Young et al., 1997), we found that subjects detected deviants more easily when they expressed a different emotion (rare BETWEEN) than when they expressed the same one (rare WITHIN). This was indexed by both greater accuracy (t(9)=4868; P=0.001) and by faster response latencies (t(9)=4223; P=0.002).
Event-related potentials
In the present
Discussion
This study was carried out in an attempt to establish whether the visual N2/P3a orienting complex evoked by deviant stimuli in a sequence of frequent ones could be modulated in amplitude and/or latency by the categorical origin of both stimuli. This goal was achieved by the use of the behavioral categorical perception effect shown on facial emotional expressions (Calder et al., 1996, Young et al., 1997). Indeed, by using a morphing technique, these authors showed that two different morphed
Conclusions
The findings of the present study leads to three main considerations. First, by using a complex visual oddball task with faces, the N2/P3a is associated with deviants and can be modulated by the categorical origin of the frequent and rare stimuli. This is of particular interest, because it suggests that the visual N2/P3a activity does not only reflect an automatic detection of the rarity of a visual event (in order to switch attention), but is also engaged in an automatic discrimination of the
Acknowledgments
This study was supported by the grant no. 95/00-189 (‘Action de Recherche Concertée’) from the Government of the French-speaking Community. The first author was supported by the Belgian Fund of the Scientific Research (FNRS).
References (33)
- et al.
A comparison of auditory and visual distraction effects: behavioural and event-related indices
Brain Research Cognitive Brain Research
(2001) - et al.
Categorical perception of facial expressions
Cognition
(1992) - et al.
Revisiting the oddball paradigm. Non-target vs. neutral stimuli and the evaluation of ERP attentional effects
Neuropsychologia
(1992) - et al.
Early selective attention effect on evoked potential reinterpreted
Acta Psychologica (Amsterdam)
(1978) - et al.
Brain activity related to normal and inverted face processing: an event-related potentials study
Biological Psychology
(1999) - et al.
Mismatch negativity to changes in a continuous tone with regularly varying frequencies
Electroencephalography and Clinical Neurophysiology
(1994) - et al.
Mapping attractor field in face space: the atypicality bias in face recognition
Cognition
(1998) - et al.
Facial expression megamix: tests of dimensional and category accounts of emotion recognition
Cognition
(1997) - et al.
Cognitive structures and anxiogenic rules
- et al.
ERP manifestations of processing printed words at different psycholinguistic levels: time course and scalp distribution
Journal of Cognitive Neuroscience
(1999)