Elsevier

Brain and Cognition

Volume 72, Issue 2, March 2010, Pages 317-324
Brain and Cognition

Adapted to explore: Reinforcement learning in Autistic Spectrum Conditions

https://doi.org/10.1016/j.bandc.2009.10.005Get rights and content

Abstract

Recent studies have recorded a tendency of individuals with Autism Spectrum Conditions (ASC) to continually change their choices in repeated choice tasks. In the current study we examine if this finding implies that ASC individuals have a cognitive style that facilitates exploration and discovery. Six decision tasks were administered to adolescents with ASC and matched controls. Significant differences in shifting between choice options appeared in the Iowa Gambling task (Bechara, Damasio, Damasio, & Anderson, 1994). A formal cognitive modeling analysis demonstrated that for about half of the ASC participants the adaptation process did not conform to the standard reinforcement learning model. These individuals were only coarsely affected by choice-outcomes, and were more influenced by the exploratory value of choices, being attracted to previously un-explored alternatives. An examination of the five simpler decision tasks where the advantageous option was easier to determine showed no evidence of this pattern, suggesting that the shifting choice pattern is not an uncontrollable tendency independent of task outcomes. These findings suggest that ASC individuals have a unique adaptive learning style, which may be beneficial is some learning environment but maladaptive in others, particularly in social contexts.

Introduction

In 2005 Nobel laureate Vernon Smith, considered the father of experimental economics, openly discussed his diagnosed Asperger’s syndrome, and suggested that it helps improve his ability to make scientific discoveries. Asperger’s syndrome is a mild form of autism,1 which along with high functioning autism, and pervasive developmental disorder are known as Autistic Spectrum Conditions (ASC) (Baron-Cohen, 2009, Golan et al., 2007).

Researchers have suggested that many other well-known inventors and discoverers, such as Issac Newton and the mathematician Paul Erdos, had ASC (Fitzgerald, 2004). We examined whether adolescents with ASC indeed have an adaptive learning style geared towards exploration and discovery.

Recent studies revealed a unique decision making pattern in ASC, characterized by constant shifting between choice alternatives in repeated choice tasks (Johnson et al., 2006, Minassian et al., 2006). For example Johnson et al. (2006) examined the behavior of adolescents with ASC on the Iowa Gambling task (Bechara et al., 1994), a repeated choice task involving four decks of cards, in which two of the decks are advantageous and two are disadvantageous (see Section 2 and Table 1 for a complete description of the task). The individual learning curves of adolescents with ASC revealed that they tended to constantly shift from one choice alternative to the other, and this pattern did not diminish with task experience. The difference in this behavior was evaluated by calculating the average and maximal size of consecutive selections (or runs) from the same alternative. In the control group, the average run of consecutive choices was about three trials on average (implying that a person selected a deck three times in a row). In contrast, in the ASC group it was only 1.3 trials (i.e., most selections were followed by a switch). Similar differences emerged for the largest run of consecutive choices. Johnson et al. (2006) also found that within the ASC group short runs on the IGT were associated with more severe autistic syndromes on the Autism Diagnostic Interview (ADI-Revised, Lord, Rutter, & Le Couteur, 1994). Fig. 1 shows an example of this exceptional adaptive learning style in three of our study participants diagnosed with ASC (left) and three matched healthy controls (right). As one can see, the ASC participants’ adaptation pattern involves constant flipping between choice alternatives. In Johnson et al.’s (2006) study this pattern (of an average run of less than 1.5) characterized 80% of the adolescents diagnosed with ASC.

Using formal reinforcement learning models we contrasted three possible explanations of this pattern. The first explanation is that the choices of ASC individuals are simply more random, due to errors in selection. This is consistent with the idea that complex tasks that require generalization skills, such as the IGT, pose a challenge to people with ASC because of their difficulty in recognizing relationships between features of the task and forming general knowledge about categories of items and types of situations (see Klinger and Dawson, 2001, Klinger et al., 2006).

A second explanation is that the adaptation process of individuals with ASC involves more trial and error exploration. This argument is consistent with findings showing that adolescents with ASC rate high on such traits as perseverance and drive for perfection (e.g., Ashburner et al., 2009, Gillberg, 2002, Kobayashi and Murata, 1998). Under a third, related, explanation the difference in exploration capacity is qualitative rather than quantitative; that is, while individuals with ASC explore more than healthy individuals do, their exploration does not conform to the standard reinforcement learning paradigm. Rather, exploration may be the primary directive of their adaptation process. For examining this last possibility, a conventional model of reinforcement learning was contested with two novel models resting on the assumption that exploration, and not the pursuit of outcomes, is the main directive governing the learning process.

In order to examine the scope of the difference between adolescents with ASC and healthy controls, we employed a battery of decision tasks ranging in their difficulty (see Table 1). The battery included the more complex Iowa Gambling task (Bechara et al., 1994) in which the shifting choice pattern was originally found (Johnson et al., 2006). It also included five simpler variants of this decision tasks. This enabled us to assess if the tendency to continuously shift choices in ASC emerges only in relatively difficult decision tasks, or whether it is an uncontrollable tendency appearing even in simpler tasks. All tasks were then analyzed with the three classes of formal reinforcement learning model noted above. These models are explained more thoroughly in Section 3 and their mathematical details are available in the Appendix section.

Section snippets

Participants

The participants included 15 high functioning children and adolescents with formal diagnoses of ASC, who arrived to the community medical center for diagnosis, either individually or through ads placed in a patient support group center. The majority of the participants in this group (14 out of the 15) were male. The mean group age was 15.6 (SD = 2.8). Only participants diagnosed by at least one psychiatrist as specifically meeting the ICD-10 criteria for two autism spectrum disorders were

Results

Fig. 2 presents the proportion of selections from the advantageous decks of the IGT in the two groups. As can be seen, although both groups learned to make more advantageous choices with experience, learning was slower in the ASC group. To examine the statistical significance of this pattern we conducted a mixed analysis of variance (ANOVA) with group as a between-subject factor, and task block (four blocks of 25 trials) as a repeated measure. The results revealed a significant group by

General discussion

Individuals with ASC have a tendency to continuously shift between choice alternatives in complex repeated choice tasks (Johnson et al., 2006). In the current study, control participants made about 2.4 consecutive choices on each run, compared to 1.4 in the ASC sample. The ensuing pattern appears like pendulum movements from one choice alternative to the other, with choices being switched 73% of the time in the ASC group compared to 41% in the sample of healthy adolescents. An examination of

Acknowledgment

This research was supported in part by the Israel Science Foundation (Grant No. 244/06) and by the Max Wertheimer Minerva Center for Cognitive Studies.

References (58)

  • G. Baird et al.

    Screening and surveillance for autism and pervasive developmental disorders

    Archives of Disease in Childhood

    (2001)
  • S. Baron-Cohen

    Autism: The empathizing–systemizing (E–S) theory

    Annals of the New York Academy of Sciences

    (2009)
  • S. Baron-Cohen et al.

    Do children with autism use the speaker’s direction of gaze strategy to crack the code of language?

    Child Development

    (1997)
  • S. Baron-Cohen et al.

    The Autism Spectrum Quotient (AQ): Evidence from Asperger syndrome/high functioning autism, males and females, scientists and mathematicians

    Journal of Autism and Developmental Disorders

    (2001)
  • R.I. Brafman et al.

    R-MAX – A general polynomial time algorithm for near-optimal reinforcement learning

    Journal of Machine Learning Research

    (2003)
  • J.R. Busemeyer et al.

    Cognitive modeling

    (2009)
  • J.R. Busemeyer et al.

    A contribution of cognitive decision models to clinical assessment: Decomposing performance on the Bechara gambling task

    Psychological Assessment

    (2002)
  • C.R. Cloninger et al.

    The temperament and character inventory (TCI): A guide to its development and use

    (1994)
  • J. Craig et al.

    Creativity and imagination in autism and Asperger’s syndrome

    Journal of Autism and Developmental Disorders

    (1999)
  • G. Dawson et al.

    Neurocognitive function and joint attention ability in young children with autism spectrum disorder versus developmental delay

    Child Development

    (2002)
  • I. Erev et al.

    Cognitive game-theoretic analysis of attention strategies, ability and incentives

  • W.K. Estes et al.

    A theory of stimulus variability in learning

    Psychological Review

    (1953)
  • M. Fitzgerald

    Autism and creativity: Is there a link between autism in men and exceptional ability?

    (2004)
  • M. Fitzgerald et al.

    Diagnosis and differential diagnosis of Asperger syndrome

    Advances in Psychiatric Treatment

    (2001)
  • C. Gillberg

    A guide to Asperger’s syndrome

    (2002)
  • J.C. Gittins et al.

    A dynamic allocation index for the sequential design of experiments

  • M.A. Gluck et al.

    From conditioning to category learning: An adaptive network model

    Journal of Experimental Psychology: General

    (1988)
  • O. Golan et al.

    The ‘reading the mind in the voice’ test-revised: A study of complex emotion recognition in adults with and without autism spectrum conditions

    Journal of Autism and Developmental Disorders

    (2007)
  • J.P. Guilford

    The nature of human intelligence

    (1967)
  • Cited by (41)

    • Exploring maximizing, satisficing and minimizing tendency in decision-making among autistic and neurotypical individuals

      2022, Research in Autism Spectrum Disorders
      Citation Excerpt :

      As a possible explanation for why autistic individuals may also be more resolute maximizing than neurotypical individuals, there is the empirical evidence reported by Kobayashi and Murata (1998) that autistic individuals score high on drive for perfection and perseverance, two traits that can be linked to the resolute maximizing tendency in decision-making. Other studies like Yechiam et al. (2010) reported evidence of autistic individuals, displaying intensive search behaviour in complex decision tasks. This drive for perfection in combination with the tendency to engage in intensive search behaviour could explain for the high latent mean score of the group of participants with autism in resolute maximizing (next to the high score on fearful maximizing).

    • Enhanced rationality in autism spectrum disorder

      2021, Trends in Cognitive Sciences
      Citation Excerpt :

      In such tasks, several studies reported superior performance of individuals with ASD, who are less drawn to the immediate salience of rewards, and learn long-term reward contingencies more accurately from choice outcomes [21,28,29]. ( However, see [30–32] for studies that did not find this advantage.) An interesting exception to the salience of high reward is revealed in the ultimatum game, in which a person irrationally declines a reward in order to punish a partner who offers an unfair sharing of that reward [33].

    • Abnormal negative feedback processing in individuals with autistic traits in the Iowa gambling task: Evidence from behavior and event-related potentials

      2021, International Journal of Psychophysiology
      Citation Excerpt :

      It is possible that the rigid decision-making style of individuals with ASD or high autistic traits was related to their repetitive symptoms. In addition, a formal cognitive model indicates that decision making by individuals with ASD cannot be explained using standard reinforcement learning models due to deficits in terms of social interaction information processing and goal-directed learning (Johnson et al., 2006; Yechiam et al., 2010; Mussey et al., 2015; Zhang et al., 2015). Our results are consistent with those obtained in previous studies and they suggest that individuals with high autistic traits within a nonclinical population had an unusual learning style for decision making based on negative and positive feedback.

    View all citing articles on Scopus
    View full text