No evidence for impaired multisensory integration of low-level audiovisual stimuli in adolescents and young adults with autism spectrum disorders

Highlights

• We measured multisensory processing in people with Autism Spectrum Disorders (ASD).

• ASDs were relative to controls impaired in judgments of visual temporal order.

• ASDs performed like controls in visual search and visual orienting.

• ASDs profited like controls from click sounds in all visual tasks.

• ASDs have no impairments in the integration of low-level audiovisual stimuli.

Abstract

Abrupt click sounds can improve the visual processing of flashes in several ways. Here, we examined this in high functioning adolescents with Autism Spectrum Disorders (ASD) using three tasks: (1) a task where clicks improve sensitivity for visual temporal order (temporal ventriloquism); (2) a task where a click improves visual search (pip-and-pop), and (3) a task where a click speeds up the visual orienting to a peripheral target (clock reading). Adolescents with ASD were, compared to adolescents with typical development (TD), impaired in judgments of visual temporal order, but they were unimpaired in visual search and orienting. Importantly, in all tasks visual performance of the ASD group improved by the presence of clicks by at least equal amounts as in the TD group. This suggests that adolescents and young adults with ASD show no generalized deficit in the multisensory integration of low-level audiovisual stimuli and/or the phasic alerting by abrupt sounds.

Introduction

Autism Spectrum Disorders (ASD) are characterized by deficits in social interactions, communication, and by restricted interests and/or repetitive behaviors (APA, 1994). In addition, sensory disturbances have been reported consistently in the clinical literature dating back to Kanner's original description of autism (Kanner, 1943). Indeed, several contemporary theories on ASD reflect the idea that sensory deficits are core symptoms of autism (Crane et al., 2009, Kern et al., 2007) that could have downstream effects on the development of the perceptual system (Bertone et al., 2005, Mottron and Burack, 2001). To create a unified percept of the world, the brain has to synthesize a mix of sensory information into one coherent multisensory percept. This sensory synthesis is a constantly occurring phenomenon that shapes our view of the world and it is crucial for everyday social and adaptive behavior (Wallace, 2004). Deficits in (multi)sensory processing might lead to aberrant social and adaptive behavior and interaction as known in ASD.

There is indeed evidence that people with ASD have impairments in (multi)sensory processing. One piece of evidence comes from studies on the multisensory integration (MSI) of speech and emotions as perceived from the face and the voice (Bebko et al., 2006, Charbonneau et al., 2013, de Gelder et al., 1991, Magnée et al., 2008, Megnin et al., 2012, Mongillo et al., 2008, Smith and Bennetto, 2007). These studies suggest that people with ASD have problems with audiovisual integration of social and emotional stimuli that could account for the atypical social behavior of individuals with ASD. Other studies on MSI of lower-level information like clicks and flashes, though, show opposite results (Foss-Feig et al., 2010, Grossman et al., 2009, Keane et al., 2010, Kwakye et al., 2011, Magnée et al., 2009, Mongillo et al., 2008, Van der Smagt et al., 2007). This dichotomy between the processing of complex and lower-level stimuli may be in line with the conceptualization of autism as a selective disorder of complex information processing (Minshew, Sweeney, & Luna, 2002). However, a recent study by Brandwein et al. (2012) assessed the integrity of basic audiovisual integration by recording high-density electrophysiology from high-functioning children with ASD while the children performed a simple audiovisual reaction time task. The authors found that children with ASD showed considerably less behavioral facilitation to multisensory inputs. Two other recent studies by Foss-Feig et al. (2010) and Kwakye et al. (2011) showed that, although children with ASD are well able to integrate audiovisual information, they do have an altered multisensory temporal binding window. For example, the Kwakye et al. (2011) study reported an extended window of temporal integration in children with ASD using temporal order judgment (TOJ) tasks with visual, auditory, and audiovisual stimuli. The authors reported no differences in sensitivity for visual temporal order, but thresholds were higher in ASD on the auditory TOJ task. In the multisensory TOJ task, the authors relied on the phenomenon known as temporal ventriloquism (Scheier, Nijhawan, & Shimojo, 1999), where click sounds improve sensitivity for visual temporal order if the clicks are presented within a certain time window. Children with ASD showed performance improvements over a wider range of temporal intervals than TD children, which is in line with the idea that children with ASD have a wider temporal window of multisensory integration that could serve as a possible explanation for the observed differences in MSI in people with ASD.

A recent study by de Boer-Schellekens, Eussen and Vroomen (2013) also examined whether people with ASD suffer from intersensory temporal deficits that may underlie other impairments in multisensory integration. An audiovisual TOJ task was used to study sensitivity of audiovisual temporal asynchronies in adolescents with ASD on three kinds of stimuli that are known to differ on a number of potentially relevant dimensions (a single flash/beep, the video of a handclap, or the video of a face articulating a syllable). This allowed the authors to examine whether adolescents with ASD suffered from a general or a more specific impairment in audiovisual temporal processing. The asynchrony between the audio and video was varied and participants judged whether the auditory stimulus came “early” or “late” with respect to the video. Results showed that, compared to TD controls, individuals with ASD were generally less sensitive in judgments of audiovisual temporal order, but there was no specific impairment with social stimuli (i.e., the face/speech stimulus), thus indicating that people with ASD may suffer from a more general impairment in audiovisual temporal processing.

It has also been reported that people with ASD may have problems in visual attention, especially with the (dis)engagement and orienting of attention (e.g. Courchesne et al., 1994; Harris, Courchesne, Townsend, Carper, & Lord, 1999; Landry & Bryson, 2004; Renner, Klinger, & Klinger, 2006; see Simmons et al. (2009) for a review; Van der Geest, Kemner, Camfferman, Verbaten, & van Engeland, 2001; Wainwright & Brown, 1996; Wainwright-Sharp & Bryson, 1993). These attentional deficits may also be related to poor judgments of visual temporal order, as has been suggested for dyslexia (Hari & Renvall, 2001).

In the present study we contribute to the ongoing debate on MSI and visual attention in ASD by examining the performance of adolescents and young adults with ASD on three different tasks. First, we examined MSI using a phenomenon known as ‘temporal ventriloquism’. The basic phenomenon is that an abrupt click can attract the perceived timing of a flash in time. A sound before a flash (at ~100 ms) can make the flash appear earlier, and a sound after the flash (also at ~100 ms) can make the flash appear later (Morein-Zamir et al., 2003, Scheier et al., 1999, Stekelenburg and Vroomen, 2005, Vroomen and de Gelder, 2004, Vroomen and Keetels, 2006). One way to demonstrate this is by means of a visual TOJ task where participants judge which of two flashes appeared first. When two click sounds are ‘sandwiched’, one before the first flash and one after the second flash, sensitivity to visual temporal order improves, presumably because the apparent stimulus onset asynchrony (SOA) between the two flashes is increased (see Fig. 1, panel A). The question we addressed here is whether equal amounts of temporal ventriloquism can be observed at various audiovisual SOAs in adolescents and young adults with ASD if compared to a TD control group. If people with ASD have difficulties with MSI, they should have a diminished temporal ventriloquist effect because sounds are not well-integrated with flashes. Alternatively, if they have an enlarged “window of temporal integration” (Foss-Feig et al., 2010, Kwakye et al., 2011), one may observe improvements by sounds at atypical large SOAs.

Second, we examined MSI with the ‘pip-and-pop’ task as introduced by Van der Burg, Olivers, Bronkhorst, and Theeuwes (2008). In this task, a visual target (a horizontal or vertical line) is embedded in a cluttered display of distracters (oblique lines). The targets and distracters change, on randomly determined times, color from green-to-red or red-to-green. The search time can be drastically improved if a ‘pip’-sound is synchronized with the color-change of the target: the ‘pip’ then makes the target ‘pop-out’. We used this task to examine whether the pip improves visual search time of adolescents and young adults with ASD as it does in people with TD. We hypothesized that if people with ASD have problems with the disengagement of attention from the current fixation (e.g. Landry & Bryson, 2004), one expects them in the tone-absent condition (serial search) to have longer search times than the TD group. The slope of their search-time-per-item function should then also be steeper as compared to the TD group (see also Romani et al., 2011, Sireteanu et al., 2008, Vidyasagar and Pammer, 1999). In the sound-present condition, though, the pip can make the target pop-out (parallel search), and search time may become independent of the set-size of the distracters. If MSI is unimpaired in ASD, then ultimately a single pip might compensate for their visual attention deficit. It should be noted that the pip-and-pop task uses a cluttered visual display where the target and many distracters change color at approximately the same time. Extracting audiovisual synchrony between the sound and the color change of the target seems like a key requisite that is likely challenged in this task. To the extent that the ASDs have problems extracting audiovisual synchrony, one would expect them to profit less from sound because the sound might be wrongly combined with a change of a distracter rather than the target.

The third task was motivated by the idea that poor judgments of temporal order might be a result of impairments in the shift or disengagement of visual attention (e.g. Courchesne et al., 1994, Harris et al., 1999, Landry and Bryson, 2004, Wainwright-Sharp and Bryson, 1993). We used a ‘digital clock reading’ task that allowed us to obtain insight into the orienting of attention (Keetels & Vroomen, 2011). In this task, participants viewed two streams of digits in rapid serial visual representation (RSVP), one on the left, the other on the right of fixation. After a variable time, one of the streams was cued by a placeholder turning red (an exogenous cue). Participants were asked to report the digit that they saw in the target stream at the time the placeholder turned red. The time it takes to overtly shift attention towards the stream and to read out the target digit can be estimated from the delay between the actual digit and the reported digit. This delay is referred to as the ‘visual latency’. The task contained a silent condition and two sound conditions where a short click from central location was presented either ~100 ms or ~200 ms before the cue. If people with ASD have problems with the shifting and/or disengagement of visual attention, one might expect them to have slower visual latencies than people with TD in a silent condition. Furthermore, click sounds at~200 or ~100 ms before the presentation of the cue/target have been demonstrated to improve visual latency for various reasons: A click may enhance detection of the cue, and/or it may improve the disengagement from fixation, speed-up the shift of attention, or the click may release backward masking so that the visual stimulus with which the click is synchronized becomes more visible (Chen and Spence, 2011, Vroomen and de Gelder, 2000, Vroomen and Keetels, 2009). If alerting by sound is unimpaired in people with ASD, we expected them to profit from clicks as the TD controls.