Attention during social interaction in children with autism: Comparison to specific language impairment, typical development, and links to social cognition

Highlights

• We investigated attention during real interaction in autism and in language impairment.

• We used novel eye tracking methods to measure attention during real interaction.

• We also explored links between attention and social cognitive skills.

• Gaze behaviour differentiated autism from language impairment.

• Findings highlight areas to be targeted for social skill intervention in autism.

Abstract

Eye-tracking studies have shown how people with autism spend significantly less time looking at socially relevant information on-screen compared to those developing typically. This has been suggested to impact on the development of socio-cognitive skills in autism. We present novel evidence of how attention atypicalities in children with autism extend to real-life interaction, in comparison to typically developing (TD) children and children with specific language impairment (SLI). We explored the allocation of attention during social interaction with an interlocutor, and how aspects of attention (awareness checking) related to traditional measures of social cognition (false belief attribution). We found divergent attention allocation patterns across the groups in relation to social cognition ability. Even though children with autism and SLI performed similarly on the socio-cognitive tasks, there were syndrome-specific atypicalities of their attention patterns. Children with SLI were most similar to TD children in terms of prioritising attention to socially pertinent information (eyes, face, awareness checking). Children with autism showed reduced attention to the eyes and face, and slower awareness checking. This study provides unique and timely insight into real-world social gaze (a)typicality in autism, SLI and typical development, its relationship to socio-cognitive ability, and raises important issues for intervention.

Introduction

Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterised by impairments in social and communicative functioning, as well as the presence of repetitive/stereotyped patterns of behaviour (APA, 1994). Difficulties with reciprocal social interactions are among the greatest challenges that individuals with ASD face in their everyday activities, and this type of difficulty can lead to a range of negative outcomes, such as social isolation, anxiety and depression (Bellini et al., 2007, Howlin et al., 2004). In this study we focus on the nature of gaze to socially relevant information for individuals functioning on the autism spectrum. We specifically explore the importance of the nature of experimental stimuli, the importance of comparison groups, and the relationship between gaze behaviour and other socio-cognitive capabilities.

Atypical social gaze is among the earliest clinical markers for ASD (Volkmar, Carter, Grossman, & Klin, 1997), which is important as typical social gaze is key to many early socio-developmental milestones, such as joint attention (Von Hoftsen & Gredebäack, 2009). Attention to the eyes of others is significant for social development and social adaption in early childhood, but it is also important in later life, as much of our socialisation is typically facilitated by information portrayed by the eyes (Jones et al., 2008, Klin, 2008). It has been suggested that atypical attention to social information early in development could have a cascading effect on subsequent social development, derailing typical social learning, ultimately contributing to the development of atypical social behaviours that we associate with functioning on the autism spectrum (Dawson et al., 2004, Klin and Jones, 2008, Klin, 2008, Pruett et al., 2011, Senju and Johnson, 2009). Therefore, exploration of social gaze in autism has become an important research priority, with potential theoretical and clinical implications.

Research on social gaze (looking to faces) has provided an important gateway to understand socio-cognitive difficulties in autism. For example, Dawson, Meltzoff, Osterling, Rinaldi, and Brown (1998) found that children with autism failed to orient to social stimuli (e.g. name calling), but not non-social stimuli (e.g. jack-in-the-box), highlighting the fact that social information does not get attentional priority for children with autism. Mundy, Sigman, Ungerer and Sherman (1986) showed how deficits in joint attention observed during play-based interactions best discriminated children with autism from children with developmental delay, highlighting that joint attention is a key area of difficulty for children with autism.

Research on looking to faces has allowed better definition of the social deficits in autism, and advances with eye tracking technology have been central to furthering this literature. Whereas early studies of gaze behaviour in autism relied on less precise methods, such as behavioural coding from video, eye-tracking provides an accurate method of capturing spontaneous (or task directed) attention. It gives detailed moment-to-moment recordings of where and what an individual is attending to, and thus what information they have available to them to process. By capturing spontaneous attention allocation in experiments depicting social information (faces, people, social interaction), eye tracking studies have helped to explain the everyday social interaction difficulties faced by individuals with autism in a way that was not previously possible (Boraston & Blakemore, 2007).

A range of eye tracking studies have shown how people with autism attend atypically to social stimuli (still images of faces or social scenes; clips of dynamic social interaction), mainly in terms of reduced attention to the eyes and faces of others (Corden et al., 2008, Frazier Norbury et al., 2009, Klin et al., 2002bNakano et al., 2010, Pelphrey et al., 2002, Riby and Hancock, 2009a, Riby and Hancock, 2009b). When attention is allocated to a face, it seems that it is directed to the mouth region for an atypically increased time compared to individuals who do not have autism and this may be an indicator of reliance upon speech and language to understand social interactions (Jones et al., 2008, Klin et al., 2002b). A strong pattern to have also emerged is that of atypically increased attention to bodies, objects, and background regions of stimuli (Klin et al., 2002b, Rice et al., 2012). Therefore within a stimulus image individuals who are functioning on the autism spectrum do not distribute their attention to the same information as typically developing individuals. This may be especially the case when there is competition between social and non-social information (e.g. Speer, Cook, McMahon, & Clark, 2007). Furthermore, even when overall amount of attention allocation to regions of interest is typical, other aspects of social attention are atypical in autism, such as the timing of attention to social information and following another's gaze cues in social scenes (Fletcher-Watson et al., 2009, Freeth et al., 2010). Such findings shed some light on how the social world looks to a person with autism, and can help to explain some of the social difficulties people with autism face (e.g. understanding facial expressions, inappropriate social responses, greater interest in objects rather than people).

These attention patterns highlighted in research outlined above come from studies involving samples of people with autism ranging in age from 2 years (Jones et al., 2008); to school aged children (Riby and Hancock, 2009a, Riby and Hancock, 2009b, Rice et al., 2012); to adults with autism (Hanley et al., 2012, Klin et al., 2002b); and from participants ranging in levels of cognitive functioning (Rice et al., 2012). Although very informative, important questions remain which could further the clinical utility of such research. The first is on the nature of atypicalities of social attention in autism, as although the patterns reported above are representative of the literature, there have been important inconsistencies, particularly in relation to attention the mouth in autism (Bar-Haim, Shulman, Lamy, & Reuveni, 2006, and Rutherford & Towns, 2008). Inconsistencies between reports of typical and atypical social attention are discussed in more detail in the next section. Furthermore, the way that atypical social attention in real-life contributes to the difficulties people with autism face (as opposed to attention while viewing social information on a screen) is as yet poorly understood. This is important given that the issue of ecological validity has been at the core of the debate on the nature of atypical social attention in autism.

A further key issue relates to the specificity of the atypical social attention profile associated with autism (particularly to eyes and mouth), as the majority of work in the area has involved typical comparison groups. Involving more appropriate comparison groups (who share some of the same difficulties as children with autism, e.g. communication difficulties) could help to elucidate the roots of atypical attention profiles and potential compensatory strategies. Finally, a critical next step for work in this area is to go beyond capturing atypical attention in autism, to exploring how this relates to social understanding and socio-cognitive capacities; we will return to this issue later in this introduction.

Research on social attention in autism has involved the use of a range of social stimuli. Essentially stimuli have differed in the way faces appear or are presented within them, e.g. isolated faces; faces in social scenes; moving faces in clips of social interaction. Although reduced attention to the eyes is the most commonly reported pattern associated with autism, typical amounts of time viewing of the eyes has also been found (Fletcher-Watson et al., 2009, Freeth et al., 2010, Van der Geest et al., 2002). Emerging evidence indicates that stimuli characteristics can have an influence on attention patterns. Using a range of modern colour still images, Hanley et al. (2012) have shown how attention allocation to the eyes varies for the same group of high functioning individuals with Asperger syndrome (AS) compared to typically developing (TD) participants, depending on the context within which faces are presented. Typical viewing of the eyes was reported when these participants with AS viewed faces presented in isolation (image cropped at shoulders against blank background), and reduced attention to the eyes was found when the same participants with AS viewed the same faces but in the context of a social scene (two individuals, bodies and background visible; Hanley et al., 2012). Speer et al. (2007) have also reported similar effects of stimuli characteristics on viewing patterns in autism. The evidence suggests that closer replication of realistic social information is critical in studies of social attention.

The very things that are social about social interaction have largely eluded eye tracking studies to date. Critical to social interaction is the interlocutor, engaging you in mutual gaze, taking their turn to speak to you, as well as to listen to you, all the while modulating their gaze and facial expressions accordingly (Kleinke, 1986, Riby et al., 2012a). You, as the conversational partner must take your cue to respond to a question when asked, and show interest when listening by attending to the face of the interlocutor (Doherty-Sneddon, Riby, & Whittle, 2012). Paying attention to other's faces and eyes is critical for understanding during social interaction (as opposed to being distracted by objects), and picking up on social cues at the right time is important (Doherty-Sneddon et al., 2012, Riby et al., 2012a, Riby et al., 2012b). To understand the role of social attention atypicalities in autism, and to maximise potential clinical applications, it is imperative that we study real-life scenarios. The current study aims to address some of these key issues by measuring social attention in children with autism during a real-life interaction using eye tracking techniques.

To the best of our knowledge, only one other published study has attempted this. Nadig, Lee, Singh, Bosshart and Ozonoff (2010) explored verbal exchange and eye gaze behaviour in children with autism during a conversation with an adult. They explored the quality of verbal exchange across two topics of conversation (max of 3 min for each topic), a generic topic and a topic of circumscribed interest (CI) to the children, and they measured eye gaze behaviour from a subset of their participants during the conversation. There were several aspects of verbal exchange that were atypical for the children with autism, including maintaining topic of conversation, and making a greater proportion of atypical utterances which interfered with the flow of conversation. Quality of verbal exchange was found to be less reciprocal for the children with autism during the CI topic. However, both their participants with autism and the typical comparison children increased their gaze time to the adult's face during conversation on topics of CI, and importantly, they found no group differences in the time spent attending to specified regions of interest (whole face, body, non-partner areas). This indicates that children with autism show typical social gaze during a real social scenario and seems to add to literature suggesting typical social gaze in individuals with autism even within a real-life interaction. However, the authors did not provide information on attention to face regions (eyes, nose, mouth etc.) which is very important given that we know attention to eyes and mouth has been found to be most atypical in autism (Klin et al., 2002b). Therefore, we aim to explore where children look during a real social scenario, especially in terms of face regions.

The most commonly used comparison groups in research studies on autism are groups of typically developing individuals matched for age and cognitive ability. While this method of matching can indicate the typicality of attention patterns, a pertinent question relates to the ‘uniqueness’ of atypical attention patterns in autism by comparison to other developmental conditions (Burack, Iarocci, Flangan, & Bowler, 2004). This allows us to tap into the question of syndrome-specificity. The use of appropriate cross-syndrome comparisons can help to elucidate the relationships between underlying processes (in this case social attention) and autism symptomatology (Bishop & Norbury, 2005).

A comparison that is rarely made in eye tracking research is between autism and specific language impairment (SLI). According to the diagnostic criteria outlined in DSM-IV-TR (American Psychiatric Association (APA), 2000),¹ SLI is characterised by receptive and expressive language difficulties which interfere with academic or occupational achievement or with social communication. However, the criteria for a pervasive developmental disorder must not be met (APA, 2000). Given that both groups share similar communication difficulties but differ markedly in terms of social functioning, this comparison offers an interesting way of exploring how different profiles of socio-communicative impairment impact on attention. For example, in relation to weak central coherence accounts of autism, Brock, Norbury, Einav, and Nation (2008) explored the processing of words in context by individuals with autism by comparing their attention patterns to individuals matched for language ability (including participants with language impairments). By using this comparison, it was possible to show that reduced sensitivity to sentence context was not specific to autism, but mediated by language ability more generally (Brock et al., 2008).

Thinking back to the previously mentioned atypical distribution of facial attention to the mouth by individuals with autism, a ‘mouth bias’ has been suggested to represent a strategy to compensate for social impairment because of previously reported associations with better social competence (Klin et al., 2002b). In other words, people with autism direct their attention to the mouth as a means of relying on speech and language to make sense of the social world. However, communication impairments also form part of the triad, and as such, could drive increased mouth attention. We know from the literature on typical development that visual cues from the face have an influence on speech perception. For example, focusing on lip movements can enhance the perception of speech when the quality of the speech is degraded (Varghese, Ozmeral, Best, & Shinn-Cunningham, 2012) and when talking on the telephone we need to modify our communication strategy (Doherty-Sneddon & Kent, 1996). Therefore, it may be difficulties in communication (processing of speech and language) that drive a preferential attention bias to the mouth region in autism.

Indeed, research exploring different profiles of socio-communicative impairment within ASD and social attention highlight the link between communication skills and mouth-looking. Frazier Norbury et al. (2009) compared the social attention patterns of two groups of teenagers with autism – one group with autism and language impairment (ALI), and one group with autism but no language impairment (ALN) – and TD individuals. They suggested that if a mouth bias was a compensatory strategy to accrue social information from language, it would be expected that an ALN group might exhibit increased fixation to the mouth, while an ALI group would not, as increased fixation to the mouth for these individuals would not afford them additional information. Participants in this study viewed dynamic clips of a social interaction while their eye movements were recorded, and although an attentional bias for the mouth was not found for any of the groups, more looking at the mouth was associated with better communicative competence for participants with autism. Using eye-tracking techniques and presenting different kinds of social stimuli (static emotional faces, dynamic emotional faces, dynamic clips of an actor), Falck-Ytter, Fernell, Gillberg, and von Hofsten (2010) showed how social attention patterns distinguished different subgroups of children with autism according to their profile of socio-communicative impairment. Children with autism with better non-verbal communication than social interaction skills honed in on mouths and non-verbal communicative information (from an actor's body) than on eyes. The opposite pattern was found (honing in on eyes) for children with autism who had better social interaction skills than non-verbal communication skills. Therefore, the presence of a mouth bias, its specificity to autism and its possible underlying cause remains an issue for consideration. An important way of exploring this further is to compare the attention patterns of children with autism to children with SLI using eye tracking techniques.

Hosozawa, Tanaka, Shimizu, Nakano and Kitazawa (2012) and Nakano et al. (2010) have made this comparison between autism and SLI by comparing gaze data to the same dynamic social stimuli across two studies. The first study (Nakano et al., 2010) involved children and adults with autism and typical controls, and the second (Hosozawa et al., 2012). involved children with SLI and typical controls. When they compared gaze data from both studies, they found gaze behaviour between SLI and TD children to be very similar (largely directed towards faces), contrasting with the gaze behaviour of individuals with autism (reduced attention to faces). However, although both the TD and SLI groups spent very similar percentages of time looking at faces (TD: 70%; SLI: 69%), the distribution of attention between the eyes and mouth was quite different between the groups. Whereas the TD children spent 29% attending the eye regions (for socio-emotional information) and 21% attending the mouth regions (for audio-visual information), the SLI children spent a greater percentage of time viewing the mouth regions than the eyes (27% vs. 23% respectively). The authors suggested that this might represent a compensatory strategy in SLI making up for weak audio-visual processing, which may place children with SLI at risk for social difficulties. This indicates that a mouth bias could in fact be due to communication impairments, however if that account were feasible it is surprising that Hosozawa et al. (2012) such a bias was not present for young children with autism in Hosozawa et al. (2012).

It is worth considering once more the demands of passively viewing interaction on a screen as opposed to participating in a real social interaction, which are very different. If a mouth bias exists and is related to social/communicative functioning, it may be more likely to be observed in real social interaction where it is incumbent upon the participant to process speech. Therefore, we aimed to explore gaze behaviour to an interlocutor's face (especially eyes and mouth) during a real-life social scenario. We aimed to explore this by comparing the attention patterns of children with autism to two comparison groups: a group of children with SLI matched for verbal ability; and a further group of TD children matched for non-verbal ability.

The third and final aim of this work was to explore the relationship between socially relevant gaze behaviour and socio-cognitive functioning. The assumption which drives the majority of eye tracking research is that what we look at closely corresponds to what we are thinking about (Yarbus, 1967). As well as exploring where children attend during social interaction (face vs. object; face regions) we also wanted to look at aspects of gaze behaviour that could provide insights into implicit mindreading/mentalising in real-life interactions. Eye tracking techniques are increasingly being used as a method of exploring spontaneous and implicit forms of mentalising, for example, in terms of anticipatory eye movements. Senju, Southgate, White, and Frith (2009) have shown how individuals with Asperger syndrome failed to show anticipatory gaze behaviours in line with an actor's false belief even though they passed a battery of theory of mind tests. Here, the dissociation between successful performance on verbally mediated false belief tasks (varying in complexity) and ‘unsuccessful’ performance on the video version of the false belief task provides evidence of an absence of spontaneous/implicit mindreading in autism, even when explicit mindreading is intact. Furthermore, Senju et al. (2010) showed the same absence of spontaneous false belief attribution in children with autism, through a lack of anticipatory gaze which was independent of verbal and non-verbal ability, age, and performance on standard false belief tasks.

False belief tasks, although the gold standard for measuring ToM, are very structured experimental tasks which do not represent the way we mentalise in everyday life (Klin, Jones, Schultz, Volkmar, & Cohen, 2002a). Everyday social interaction is unstructured, fluid, and spontaneous and automatic mentalising is necessary for successful social interaction (Senju et al., 2010). Understanding information from the face is a critical part of this because the face contains the information we need to attribute another's focus of attention, mood or emotion, and intention, which allows us to predict other's behaviour (Frith & Frith, 2006). The use of unexpected events has been important for studying how people use information from faces. An unexpected event creates a situation of uncertainty, and looking at another person's face allows you to check their awareness of what has happened. For example, unexpected events have been used to probe joint attention abilities in children with autism (Mundy et al., 2003). Other research has used unexpected events in visual illusions to highlight the influence of social cues from faces (Kuhn, Kourkoulou, & Leekam, 2010). Exploring the priority of attention to the face during social interaction when something unexpected happens may reveal aspects of spontaneous mentalising, specifically, the priority for monitoring another's response in terms of their facial expression.

Therefore, in addition to measuring attention allocation during social interaction, we aimed to capture the attentional response to an unexpected event. We wanted to explore how the social interaction was monitored around the time of this event, and whether the participants’ gaze behaviour might indicate ‘awareness checking’ of the experimenter. This is a novel way of capturing spontaneous real-life mentalising, circumventing the issues traditionally associated with social-cognitive tasks that load heavily on language ability, which may provide insights into importance of anticipatory gaze and the timing of gaze behaviour for understanding others.

We have summarised the literature that relates to the three key aims of this research. The first aim is to explore the allocation of attention to socially relevant information during a real life interaction. The second aim relates to the involvement of individuals with SLI as a critical comparison to those functioning on the autism spectrum. While we predicted that TD children and children with SLI would spend similar amounts of time attending the eyes, we predicted that children with autism would spend significantly less time viewing the eyes throughout the interaction. Our predictions for patterns of mouth viewing were exploratory. Increased viewing of the mouth by children with autism only would support the notion of a ‘social’ compensatory strategy. Increased viewing of the mouth by children with SLI only would support previous reports of a ‘communicative’ compensatory strategy to aid audio-visual processing (Hosozawa et al., 2012). Increased viewing of the mouth by both the autism and SLI groups would indicate that this viewing behaviour may be a consequence of developmental delay relating to social/communicative functioning.

The third aim of the work was to explore attention capture by a face vs. an object, and to use an unexpected event around the object to explore links between attention and socio-cognitive ability in understanding this event. We predicted that the children with autism would show increased attention to the object by comparison to the TD and SLI groups. The unexpected event allowed us to measure the time it took children to fixate back on the face once this had occurred, as an index of anticipatory gaze corresponding to awareness checking. We predicted that children with autism would be slower than the other two groups to make this fixation, and that this behaviour would reflect spontaneous mentalising. Therefore, we predicted that false belief attribution (measure of social-cognitive skill) would correlate with latency, but at the same time, that the latency difference between the groups would be remain even when false belief attribution was co-varied out, showing that this measure reflects a more spontaneous/automatic form of mentalising.