This investigation aimed to determine if children with ASD are impaired in their ability to switch attention between different tasks, and whether performance is further impaired when required to switch across two separate modalities (visual and auditory). Eighteen children with ASD (9–13 years old) were compared with 18 typically-developing children matched with the ASD group for mental age, and also with 18 subjects with learning difficulties matched with the ASD group for mental and chronological age. Individuals alternated between two different visual tasks, and between a different visual task and an auditory task. Children with ASD performed worse than both comparison groups at both switching tasks. Moreover, children with ASD had greater difficulty when different modalities were required than where only one modality was required in the switching task in comparison with participants matched in terms of mental and chronological age.