Multilevel alterations in the processing of audio–visual emotion expressions in autism spectrum disorders
Highlights
► We assessed the recognition and integration of emotions in individuals with ASD. ► ASD discriminated emotional expressions less efficiently than controls. ► ASD needed a higher signal/noise ratio for the discrimination of emotions. ► ASD presented an alteration in the audiovisual integration of emotional information.
Introduction
The ability to recognize emotional expressions is a fundamental cognitive ability for the regulation of interpersonal interactions (Adolphs, 2002, Custrini and Feldman, 1989, Izard et al., 2001). The tone of the voice and the facial expression are two crucial cues that we constantly use to predict others' actions and to react appropriately in a social situation. An important aspect of affect perception in everyday life is that it usually involves, like speech, the activation of several sensory channels simultaneously. Therefore, the combination of information from facial expression (visual signal) and prosody (auditory signal) usually results in a unified and more optimal representation of the expressed emotion (de Gelder et al., 1999, de Gelder and Vroomen, 2000, de Gelder et al., 2005). For example, it has been shown that the multisensory integration (MSI) of these two types of information typically allows for faster and more accurate recognition of emotion expressions in human observers (Collignon et al., 2008, Collignon et al., 2010, de Gelder and Vroomen, 2000, Dolan et al., 2001, Kreifelts et al., 2007, Massaro and Egan, 1996) and in human-machine interfaces (Busso et al., 2004).
Deficits in the perception of emotion expressions have been suggested as possible causes of atypical social and communicative interactions that are a striking part of the behavioral phenotype of autistic spectrum disorders (ASD) (Bachevalier and Loveland, 2006, Monk et al., 2010, Sigman et al., 2004). However, a majority of the empirical investigations in the field have focused on the facial expression of emotions using static stimuli such as photographs (Bal et al., 2010), with only a few studies using video representing dynamic facial movements (Golan et al., 2008, Loveland et al., 2008, Loveland et al., 1997). The former static stimuli have limited ecological validity and neglect the intrinsic dynamic nature of facial expressions. Indeed, facial movements have been shown to enrich emotional expression, contributing to its identification and playing an important role in the perception of its intensity (Ambadar et al., 2005, Biele and Grabowska, 2006). Also, neuroimaging studies have shown that the brain regions involved in the processing of facial affect, such as the posterior superior temporal sulcus (pSTS), the amygdala and the insula, respond differently to dynamic than to static emotional expressions (Haxby et al., 2000, Haxby et al., 2002, LaBar et al., 2003, Miki et al., 2011). Moreover, only a few studies explored the processing of affective vocalizations in ASD (Baker et al., 2010, Hall et al., 2003, Loveland et al., 2008, Wang et al., 2007). In most cases, these studies included semantic or lexical confounds in the tasks (Lindner & Rosen, 2006) raising the possibility that the results were influenced by differences in language comprehension (Haviland et al., 1996, Paul et al., 2005). Finally, most studies investigating the recognition of emotions in autistic individuals explored a single sensory modality at a time, whereas in natural settings, emotions are expressed both facially and vocally, allowing the combination of these sources of information by human observers for optimal recognition (Collignon et al., 2008, de Gelder et al., 1999, de Gelder and Vroomen, 2000, de Gelder et al., 2005). The use of multisensory conditions to explore the recognition of emotional expressions in ASD is of particular interest since differences in multisensory processing between ASD and typically developing controls (TD) has recently been demonstrated (Collignon et al., 2012, Magnee et al., 2007; 2008; Russo et al., 2010, Russo et al., 2012).
An additional challenge associated with the processing of emotional expressions in natural settings is related to the fact that the saliency of emotional information in faces and voices is often reduced by environmental noise. In signal processing, noise can be considered unwanted data that is not being used to transmit a signal, but is simply a by-product of other activities. For example, the voice of an individual can be masked by noise from other human voices or from objects surrounding him. Similarly, a person's facial expression can be partially hidden by an object or because of the angle in which the observer is positioned. Therefore, the ability of the observer to extract efficiently emotional information from noise appears crucial for effective social interactions and therefore it is relevant to evaluate the perception of emotional expressions in noisy situations (Pelli & Farell, 1999). Some studies have suggested that ASD have a specific difficulty in perceiving speech when presented in a noisy background compared to TD (Alcantara et al., 2004, Smith and Bennetto, 2007). To our knowledge, no study has investigated the perception of visual or auditory emotional expressions in noise in ASD.
The goal of the present study was therefore to explore the perception and the integration of emotion expressions in ASD by using ecological and validated sets of dynamic visual and non-verbal vocal clips of emotional expressions (Belin et al., 2008, Simon et al., 2008). Participants were asked to categorize expressions of fear or disgust as quickly and accurately as possible when presented with auditory, visual and audio–visual stimuli. This task allowed us to compare recognition and MSI performance of emotional expressions between ASD and TD. We also compared unisensory performance of ASD and TD participants by measuring their ability to discriminate emotional expressions when presented auditorily and visually in individually adapted levels of noise. Similar paradigms have been previously successfully used to demonstrate that the perception of emotional expressions is a robust multisensory situation which follows rules that have been observed in other perceptual domains (Collignon et al., 2008) and to illustrate gender differences in the processing of emotion expressions (Collignon et al., 2010).
Section snippets
Subjects
Thirty-two autistic participants (30 males; mean age 21 years±6; range 14–32 years) and 18 TD controls (18 males; mean age 21 years±4; range 15–27 years) participated in this study. Participants were recruited from the database of the Rivière-des-Prairies Hospital's autism clinic (Montreal, Canada). ASD participants were defined using DSM-IV-TR diagnostic criteria, as operationalized by the Autism Diagnostic Interview – Revised (ADI-R) (Lord, Rutter, & Le Couteur, 1994) and the Autistic
Results
In all the analyses presented in the main manuscript, data obtained for fear and disgust stimuli are collapsed. Results (and related statistics) obtained for each emotion separately are presented in Supplementary material (SFigs. 6–8).
Discussion
Alterations in the ability to recognize emotional expressions in ASD is often suggested as a possible source for certain atypical social and communicative behaviors that characterize this population. The first aim of this study was to empirically test this hypothesis by exploring the perception of emotion in autistic individuals using ecological and validated sets of dynamic visual and non-verbal vocal clips of emotional expressions. We found a decreased performance in ASD compared to TD for
Acknowledgments
This research was supported in part by the Canada Research Chair Program (ML, FL), the Canadian Institutes of Health Research (AB, ML, FL, GC), the Natural Sciences and Engineering Research Council of Canada (ML, FL) and the research centre of the University Hospital Sainte-Justine (OC). The authors would like to thank Patricia Jelenic for her help with participant recruitment and selection.
References (136)
- et al.
The orbitofrontal-amygdala circuit and self-regulation of social–emotional behavior in autism
Neuroscience and Biobehavioral Reviews
(2006) - et al.
Multiple measures of the orienting reaction and their dissociation after amygdalectomy in monkeys
Experimental Neurology
(1968) - et al.
The neural basis of the speed-accuracy tradeoff
Trends in Neurosciences
(2010) - et al.
Enhanced pure-tone pitch discrimination among persons with autism but not Asperger syndrome
Neuropsychologia
(2010) - et al.
Audio–visual integration of emotion expression
Brain Research
(2008) - et al.
Women process multisensory emotion expressions more efficiently than men
Neuropsychologia
(2010) - et al.
A functional and structural study of emotion and face processing in children with autism
Psychiatry Research
(2009) - et al.
Predictors of amygdala activation during the processing of emotional stimuli: a meta-analysis of 385 PET and fMRI studies
Brain Research Reviews
(2008) - et al.
The combined perception of emotion from voice and face: early interaction revealed by human electric brain responses
Neuroscience Letters
(1999) - et al.
Multisensory integration of emotional faces and voices in schizophrenics
Schizophrenia Research
(2005)