Difficulties in Recognising Dynamic but not Static Emotional Body Movements in Autism Spectrum Disorder

If the difficulty in recognizing the emotional content of BM in ASD is related to the processing of BM information, it is reasonable to suppose that the recognition of static body postures will be preserved. Whereas, if the impairment encompasses also the recognizing the emotional static body postures, this will suggest a deficit related to the decoding of emotional content.

The results about the recognition of emotional BM using static stimuli are mixed. Some studies found no differences between children (Peterson et al. 2015) and adults (Libero et al. 2014) with and without ASD. In this latter study, although no differences were observed at the behavioural level, the authors found significant differences in functional connectivity between groups in motor-related areas (i.e., ventral premotor cortex, inferior frontal gyrus, and superior parietal areas) that are part of the putative mirror neuron system.

In contrast, other studies showed poorer identification of fearful body postures in participants with ASD, compared to TD controls. For instance, in adolescents with Asperger’s Syndrome (AS), Doody and Bull (2013) found poorer accuracy in verbal labelling of fearful stimuli (but no different accuracy in a nonverbal matching of the same stimuli) and longer response times for angry body posture in a nonverbal matching task, suggesting an impairment specific to threating stimuli. Consistently, Philips and collaborators (Philip et al., 2010a) compared the recognition of emotional images of faces, videos of bodies, and voices and found that participants with ASD were less accurate than TD in all the stimulus categories. Finally, Hadjikhani and collaborators (2009) found that adults with ASD were more accurate than TD in recognizing neutral body postures, but less accurate in recognizing the fearful ones. Furthermore, unlike TD participants, in participants with ASD the vision of fearful postures did not elicit response in visuo-motor areas (intra-parietal lobule and inferior frontal gyrus) and failed to elicit differential brain activity for emotional and neutral body postures.

The inconsistent results in research on static BM in ASD may be explained by some methodological differences. For instance, very small samples size (e.g. Hadjikhani et al., 2009), unbalanced numerosity between groups (Peterson, Slaughter, & Brownell, 2015), and inconsistency in matching criteria may led to failure in replicating results in heterogeneous clinical conditions, such as ASD. Specifically, in two studies the group of participants were not IQ matched (Hadjikhani et al., 2009; Philip et al., 2010b). The most of other studies used verbal (VIQ) or full-scale IQ (FSIQ) as group-matching criteria (Doody & Bull, 2013; Libero et al. 2014; Peterson et al. 2015). Critically, in FSIQ the verbal and nonverbal performance are averaged together and this may mask group-differences in cognitive profile that, in turn, can inflate the results. Moreover, deficits in facial (Bormann-Kischkel et al. 1995; Macdonald et al., 1989; Tantam et al. 1989) and bodily (Mazzoni et al. 2020) emotion recognition in ASD seems to emerge clearly when the nonverbal IQ was used as matching criteria, but to a lesser extent when verbal IQ was used (Castelli 2005; Davies et al. 1994; Loveland et al. 1997). For these reasons, in the present study, participants were matched according to nonverbal IQ and performed a nonverbal recognition task.

Inconsistency in results may also rely to the stimuli’s features, such as differences in emotional content and in lower-level visual features. Indeed, some studies presented emotional but not neutral BM (Doody & Bull, 2013; Peterson et al. 2015; Philip et al., 2010b), or neutral and negative but not positive expressions (Doody & Bull, 2013; Hadjikhani et al., 2009). Besides, portray of static emotional BM have been vary, including black and white silhouettes (Libero et al., 2014), avatars (Doody & Bull, 2013), and naturalistic images (Hadjikhani et al., 2009; Peterson, Slaughter, Brownell, et al., 2015). Possibly, the great number of details in naturalistic stimuli can have diverted attention from emotional content in ASD participants. To limit this bias, in the present study changes of not relevant features were minimized (e.g., showing actors wearing similar dresses).

Finally, inconsistency in results may be due to the type of task (e.g., verbal vs nonverbal, free vs forced choice labelling). Forced-choice paradigm (Libero et al., 2014; Peterson et al. 2015; Philip et al., 2010b) required to select a response option even though the expressed emotion is not recognized. This may artificially inflate the agreement in ASD participants and lead to different neurological activity in spite of comparable behavioural performance (Libero et al. 2014). Furthermore, in nonverbal match-to-sample task (Doody & Bull, 2013; Hadjikhani et al. 2009), participants can match the response options according to the visual features (e.g., limbs/trunk position) irrespective of emotion recognition. To prevent participants from relying on visual features, in this study we asked participants to match emotional BM with the respective facial expression.

Crucially, none of the abovementioned studies compared static and dynamic bodily expressions. To the best of our knowledge, only one study has directly compared the recognition of static and dynamic body stimuli in participants with ASD (Weisberg et al., 2014). In a fMRI task, the authors presented static images, full-colour videos, and PLDs depicting neutral human actions (social) or tools (non-social). Behavioural results revealed no group differences in distinguishing tools from human stimuli, regardless of whether the stimuli were static or dynamic. At the neural level, in TD but not in ASD group, the authors found a heightened response to social compared to non-social stimuli, specifically for dynamic ones. Notably, the human movements used in this study were emotionally neutral. To date, the direct comparison of static and dynamic bodily expressions in ASD remains unexplored.

To fill this gap, in the present study we investigated the role of motion cues (implied vs dynamic) in modelling the recognition of emotional content of BM in ASD. To this aim, we asked a group of adolescents and adults with high-functioning ASD (HFA—namely, without intellectual disability) to recognise static images and dynamic FLD and PLD videos depicting happy, fearful, and neutral body movements. Building on previous studies’ criticalities, we used a nonverbal recognition task and, consistently, we matched ASD and TD groups according to nonverbal IQ. We hypothesise that, if the difficulty in recognizing the emotional content of BM in ASD is related to the processing of motion information, we will find no group differences in recognizing static body postures, but impaired recognition of dynamic body stimuli in participants with ASD.

20 participants with HFA (age range 13–27 years, age mean = 19.85; age sd = 7.77) and 20 typically developed controls (age range 14–26 years, age mean = 20.44, age sd = 5.27) took part in this study (Table 1). This numerosity was determined taking into account the sample size of previous studies in the field (Atkinson, 2009; Annaz et al., 2012; Doody & Bull, 2013; Hubert et al., 2007; Parron et al. 2008; Philip et al. 2010). Participants with and without ASD were matched according to gender, age, and nonverbal IQ measured with the Raven’s progressive matrices (Raven, 1936) (Table 1).

All participants had normal or correct-to-normal vision. Participants with ASD were recruited at the Observation, Diagnosis, and Education Laboratory (University of Trento) and met the criteria specified in DSM-IV (American Psychiatric Association 2006) or DSM-5 (American Psychiatric Association 2013), or the Autism Diagnostic Observation Schedule (ADOS—Lord et al. 2000), or the Autism Diagnostic Interview (ADI—Lord, Rutter, & Le Couteur, 1994). Data on socioeconomic status were not recorded. Before the experiment, all participants received an exhaustive explanation of the experimental procedure and written informed consent was provided by them or their parents, according to the Declaration of Helsinki. The study was approved by the ethical committee of the University of Milano-Bicocca.

We presented neutral and emotional whole-body movements depicted as static body images (SB), and dynamic video-clips of full-light (FLDs) and point-light (PLDs) displays. The final set of stimuli included 24 stimuli (8 neutral, 8 fearful, 8 happy) for each category.

The static images were created by our group of researchers (Supplementary materials). All the actors (4 females and 4 males) wore a white T-shirt with long sleeves, blue jeans, and sneakers. All the pictures were edited with Photoshop: brightness, number of pixels and dimension were equalised among all the pictures, the background colour was turned to black, and the face was blurred (Fig. 1). The recognizability of the stimuli was assessed in a Validation study that involved 20 TD adults (16 males and 4 females, age mean(sd) = 24.65(1.66), Supplementary materials).

For the main study, we selected a total of 24 images (8 happy, 8 fearful, 8 neutral) whose accordance about the expressed emotion ranged between 80 and 100% (that is, at least 80% of participants responded that that images depicted a specific emotional expression—e.g., fear—or a specific action—e.g., push). For the emotional expressions, only the fearful and happy stimuli whose intensity was rated above 4.7 were selected; for the neutral actions, only the stimuli that were never confounded as an emotion were selected. The actors’ gender was equally represented (4 males, 4 females). The final dataset of SB images is available on request from the corresponding author.

The dynamic stimuli consisted in short video clips depicting happy, fearful, and neutral whole-BM represented as FLDs and PLDs. The stimuli were selected from a wider dataset realised by Atkinson and collaborators (Atkinson et al. 2004, 2012). The FLDs consisted of 3 s digital movies depicting a grey-scale actor moving against a black background. The actors’ face was covered. The PLDs lasted 2 s and consisted of 13 lighting dots placed over the main joints of the actor, moving against a black background, and were created by converting the FLDs stimuli to PLDs (Atkinson et al., 2012). The stimuli were created this way by Atkinson and collaborators and we maintained the original timing of presentation. The actors’ gender was equally represented (4 males, 4 females). Examples of the stimuli can be viewed in Fig. 1 and at https://​atkinsonap.​github.​io/​stimuli/​.

Participants were tested individually in a quiet room. They sat in front of a computer monitor, at a distance of 60 cm. The experimenter sat next to the participant for the entire duration of the study. Participants were presented with fearful, happy, or neutral whole-body movements depicted as SB, FLDs, and PLDs. In total, each participant saw 24 PLDs, 24 FLDs, and 24 images of whole-body expressions (8 videos or images in every emotional category). The three stimulus categories were presented in three separated blocks, each block lasted around eight minutes. Participants could take a break between the blocks, if needed. The order of blocks was counterbalanced between participants. Within each block the stimuli were presented randomly. Every block started with few practice trials. Every trial started with a one-second fixation cross, then the stimulus was presented for 2 s (PLDs) or 3 s (FLDs and static images). The question “Which emotion was expressed?” was presented on the top of the screen and lasted until participants gave the response. Sticky emoticons of happy, fearful, and neutral facial expressions were placed on the response keys. Participants were asked to categorise the observed stimulus by pressing the key with the corresponding emoticon, as accurately and fast as possible. Accuracy and response times (RT) were recorded. Responses were collected by keyboard; the order of key-emotion correspondence was randomised across participants. Stimulus presentation was controlled and behavioural responses were recorded with E-Prime 2.0 software (Psychology Software Tools, Inc). At the end of the experiment, participants were administered with the Matrix of Raven test to assess the nonverbal IQ.

All the analyses were performed using R, version 3.6.1 (R Core Team, 2019).

With regard to static body (SB), we did not find any group difference in accuracy and RTs, in line with some previous studies that have investigated the recognition of emotional static body postures in ASD (Doody & Bull, 2013; Libero et al. 2014; Peterson et al. 2015; Weisberg et al. 2014). Similarly, in dynamic stimuli we did not find between groups differences in accuracy, regardless of whether the BM were depicted as solely kinematic information (PLDs) or with visible body form (FLDs), in agreement with previous findings using PLDs in adults with ASD (Actis-Grosso et al. 2015; Murphy et al. 2009; van Boxtel, Dapretto, & Lu, 2016).

The absence of between groups differences in accuracy could be interpreted as the ability to recognise the emotional meaning of observed BM being intact in participants with HFA. Nonetheless, a number of neuroimaging studies have reported comparable behavioural performance, but different brain activation, between participants with and without ASD when presented with BM stimuli (Hubert et al. 2007; Libero et al. 2014; McKay et al. 2012). In line with that, it has been suggested that high-cognitive resources can mediate the acquisition of compensatory strategies that develops with age in individuals with HFA (McKay et al., 2012). According to these studies, an alternative explanation for our failure to find between groups differences in accuracy could be that our participants might have used different—but equally successful—strategies to recognise the emotional meaning of bodily expressions that are mediated by high-cognitive resources and have been developed at a younger age. This alternative explanation would be consistent with research showing that the accuracy in recognizing emotional BM increased with age in children with HFA (Fridenson-Hayo et al. 2016; Mazzoni et al. 2020). However, we acknowledge that in the present study we did not use longitudinal design and thus it was not possible to assess the effective acquisition of compensatory strategies during development. Therefore, this interpretation of our results remains hypothetical and would need to be further investigated. In this regards, longitudinal design would be a very interesting approach for future research as it would help to shed light on the development of emotional BM decoding in individuals with ASD.

Alternatively, another possible explanation for the lack of observed difference in accuracy between the ASD and control groups may be due to the use of the forced-choice response methodology. In TD participants, the forced-choice response has been shown to artificially inflate the rate agreement on emotions expressed by face (Frank & Stennett, 2001) and body posture (Winters, 2005). For instance, Frank and Stennett (2001) showed that, when the correct response was removed, participants tend to respond anyway and to choose an incorrect option at above chance level. The authors suggested that this problem could be resolved by including a "none of the above" label as a response option. In our study, although the correct option was always presented, we did not provide the “none of the above” option and this may have led to artificially inflated the response agreements between the ASD and TD groups. Future studies should hence consider to include the “none of the above” option to avoid this issue.

Concerning RTs, we found an impairment in HFA group specific for the dynamic body stimuli, in line with previous evidence showing that individuals with ASD at all ages struggled in processing the information conveyed by dynamic BM (Atkinson, 2009; Mazzoni et al. 2020; Nackaerts et al. 2012; Philip et al. 2010). In particular, our finding of no group differences in SB, but greater RTs in participants with HFA in recognizing PLDs and FLDs, suggested that individuals with ASD need more time to decode dynamic stimuli compared to TD. Notably, in SB, a single static frame of BM was presented for 3 s, while in dynamic stimuli a sequence of BM was presented within the stimulus duration. Therefore, according to our results, it seems that when the motion information is just implied – as it is in SB – and is presented for a relatively long duration (3 s), the individuals with HFA have enough time to decode the BM emotional meaning and could respond readily as soon as the stimulus disappeared. On the contrary, they seem to need more time to decode dynamic cues from actual motion (FLDs and PLDs videos). This result is consistent with a study on coherent-motion judgment, showing that the performance of adolescents and adults with ASD decreased by shortening the video duration (Robertson et al., 2014). Moreover, our results are in agreement with EMG findings in children with ASD that suggested an impairment in action-chaining mechanism (Cattaneo et al., 2007). This mechanism (Fogassi et al., 2005) has been hypothesised to allow the observer to infer the agent’s intention (Gallese, Fadiga, Fogassi, & Rizzolatti, 2002) and emotions (Jezzini et al., 2015). Consistently, transcranial magnetic stimulation (TMS) studies in humans have provided causal evidence for the specific involvement of parietal areas in processing emotional BM (Engelen et al. 2015, 2018; Mazzoni et al. 2017). Notably, those parietal areas are exactly the brain regions that are believed to underlie the action-chaining mechanism and are an important hub of the putative mirror neurons system (MNS). MNS’ neurons are active both during the execution and the observation of the same action (Rizzolatti et al. 2014) and allow the observer to comprehend an action through the representation of the observed movement within his/her motor system. Interestingly, a number of neuroimaging studies in ASD have showed structural, functional, and connectivity atypicalities in MNS areas during BM perception (Alaerts, Swinnen, & Wenderoth, 2017; Libero et al., 2014; McKay et al., 2012), suggesting that alteration of the MNS areas could explain the difficulty found in BM processing in ASD. In TD individuals, the internal mapping of the observed movement into the observer’s motor system occurs automatically and allows a very rapid recognition of the observed BM (Gallese et al. 2002). For instance, using MEG Meeren et al. (2016) showed a response in right posterior parietal cortex to fearful body postures as early as 80 ms after stimulus onset. Conversely, individuals with HFA may lack this internal simulation and to compensate for it they could develop alternative cognitive strategies that recruit different neural networks (Alaerts et al. 2017; McKay et al., 2012). In fact, although many individuals with HFA can achieve explicit or controlled mentalising skills, the implicit, automatic, and intuitive mechanism for emotion recognition remains impaired even in adulthood (Lai et al. 2014). Thus, although those strategies may allow participants with HFA to recognise the observed BM, they are likely not automatic and would require longer time to achieve BM recognition. In agreement with this, our results showed no group differences in accuracy, but greater RTs in participants with ASD, specifically in recognizing dynamic but not static stimuli.

On these premises, we hypothesise that—similarly to what has been described for grasping (Cattaneo et al. 2007)—our results could be explained by an impaired action-chaining mechanism that would prevent the individuals with ASD to rapidly distinguish the observed BM, e.g. jump-to-exult from jump-to-exercise, or rise the arms to self-protect from rise the arm to stretch out. Indeed, in static emotional body posture, only one single movement was displayed. As a consequence, to recognise SB images, the observers did not need to chain any movement because all the information was already available in the observed static posture. Conversely, when the BM stimulus was dynamic, in order to understand its meaning the observer needed to chain together a sequence of movements. This implies that, if the difficulty in ASD was specific to action-chaining, individuals with ASD should present difficulties in recognizing dynamic, but not static stimuli. Notably, this is exactly what we found in the present study.

However, alternative explanations should also be considered. If the use of the forced-choice paradigm did in fact inflate response agreement among the ASD participants, the slower responses in ASD group may reflect their uncertainty about what emotions the stimuli represented. Nonetheless, we found slower responses in ASD participants specifically for dynamic stimuli and this result offers a tentative support for an action-chaining mechanism deficit. Moreover, our results on emotional BM are consistent with previous evidence on static vs dynamic facial expressions. In TD population the motion information seems to facilitate the recognition of facial expressions (Tobin, Favelle, & Palermo, 2016), while previous behavioural and eye tracking studies found atypical responses to facial expressions elicited by dynamic compared to static stimuli in individuals with ASD (Tardif et al. 2007; Uono et al. 2009). Furthermore, reduced facial mimicry in high-functioning ASD was found in responses to dynamic but not to static facial expressions and this reduction was related to social dysfunction (Yoshimura et al. 2015). Finally, fMRI findings showed reduced activation and connectivity of social brain areas in response to dynamic facial expressions in in high-functioning ASD (Pelphrey et al. 2007; Sato et al. 2012). Altogether these studies seem to supports the hypothesis of impaired processing of dynamic emotional faces and are in agreement with our finding on emotional BM.

Interestingly, in SB we found a significant advantage in recognizing fearful stimuli. Accuracy was higher in fearful SB than neutral and (marginally) happy SB in both groups. The fearful-advantage has been previously described in behavioural studies on recognition of emotional BM both in individuals with TD (Atkinson et al. 2004; Bannerman et al. 2009) and ASD (Mazzoni et al., 2020; Philip et al., 2010). From an evolutionary point of view, the vision of an emotional expression triggers adaptive actions (Darwin, 1872). Frijda wrote that "Emotions exist for the sake of signalling states of the world that have to be responded to or that no longer need response and action” (Frijda, 1988, p.354). In other words, this author suggests that the emotions exist for the sake of action, for dealing with the environment, and highlighted that different emotions arise in response to different situation and prompt different reaction (Frijda, 1988; Frijda et al. 1989). Some emotions are more relevant when the observer is close to the agent (e.g., disgust or happiness) (Gelder et al. 2015), as they are aimed to trigger a behavioural response over a close source of emotion (e.g. throwing away a bad food, or getting close to something pleasant in order to enjoy it). Those emotions are more likely to be expressed and recognised better through the face, which indeed requires proximity to be perceived. Instead, some other emotions are expressed and recognised better with BM (e.g. threatening signals, such as fear and anger, e.g. Actis-Grosso et al., 2015). Those emotions communicate the presence of threats or dangers in the environment, hence their recognition is as more important for survival as they can be seen from a distance. Indeed, this enables the observer to have “enough” time for reacting promptly and adaptively (e.g. fight or flight), maximizing the chance of survival. In particular, fearful BM perception is associated with increased vigilance and attention (Bannerman et al. 2009; Kret et al. 2013; Phelps et al. 2006; Tamietto et al. 2007), improved visual processing (Borhani et al. 2015; van Heijnsbergen, Meeren, Grèzes, & de Gelder, 2007), and enhanced reactivity of the motor system (Borgomaneri, Gazzola, & Avenanti, 2012; Borgomaneri et al. 2015). Due to its critical evolutionary salience, in our opinion, it is not completely surprising that we found a fear-advantage also in individuals with high-functioning ASD.

Our results are partially in contrast with some findings that posited a deficit in decoding fearful signals in ASD, possibly related to dysfunctions in amygdala (Ashwin, Chapman, Colle, & Baron-Cohen, 2006; Hadjikhani et al., 2009; Howard et al. 2000; Schultz, 2005). However, it is important to acknowledge that our sample includes people with HFA, who might have developed compensatory mechanisms to recognise the social relevant stimuli—especially when evolutionary vital—at a TD-level.

Moreover, our stimuli depicted the emotional expressions at their peak intensity, possibly being too easily recognizable for participants with HFA. Finally, in the present study participants were asked to perform a low demanding experimental task. Indeed, it has been showed that, when the task is complex, the performance of participants with ASD might be hampered by their difficulties with attention and working memory (Barendse et al., 2013; Happé, Ronald, & Plomin, 2006). Therefore, to prevent the results from reflecting the task demand instead of the emotion recognition ability, we minimised the cognitive demands by presenting only three emotional contents. This allowed us to limit the number of response options (i.e. working memory) and the duration of the experiment (i.e. attention). Since we tested individuals with HFA, there is the possibility that we obtained a ceiling effect in the accuracy because the task was actually too simple for our participants.

To unmask differences in accuracy in recognizing the emotional expression in individuals with HFA that have been in treatment for years, future studies aimed at i) presenting the stimuli briefly, ii) using subtler expressions, and iii) using more complex tasks—e.g. with increased number of presented emotions and response options- are desirable.

As a final comment, we acknowledge the small sample size as a major limit of the study that necessarily impose caution in interpreting our results. Historically, the field of emotional BM recognition in ASD has been characterised by small-scale studies that have yielded to contradictory results. In fact, small sample size may have only partially represented the ASD population as it is characterised by a well-recognized heterogeneity in functioning and clinical profile. Although in our statistical analyses we tried to account for the individual variability, it is important to highlight that our limited sample size does not allow to draw robust conclusions. This limitation is remarkable and future research is needed to corroborate our findings. Nevertheless, the present study added some novelty to the field as it explored a new aspect of bodily emotion recognition in ASD and showed an interesting – although preliminary—differences between static and dynamic body stimuli. Our results could hence serve as an interesting starting point for future research that should necessarily involve larger sample size in order to account for heterogeneity in ASD.