Adapted to explore: Reinforcement learning in Autistic Spectrum Conditions
Introduction
In 2005 Nobel laureate Vernon Smith, considered the father of experimental economics, openly discussed his diagnosed Asperger’s syndrome, and suggested that it helps improve his ability to make scientific discoveries. Asperger’s syndrome is a mild form of autism,1 which along with high functioning autism, and pervasive developmental disorder are known as Autistic Spectrum Conditions (ASC) (Baron-Cohen, 2009, Golan et al., 2007).
Researchers have suggested that many other well-known inventors and discoverers, such as Issac Newton and the mathematician Paul Erdos, had ASC (Fitzgerald, 2004). We examined whether adolescents with ASC indeed have an adaptive learning style geared towards exploration and discovery.
Recent studies revealed a unique decision making pattern in ASC, characterized by constant shifting between choice alternatives in repeated choice tasks (Johnson et al., 2006, Minassian et al., 2006). For example Johnson et al. (2006) examined the behavior of adolescents with ASC on the Iowa Gambling task (Bechara et al., 1994), a repeated choice task involving four decks of cards, in which two of the decks are advantageous and two are disadvantageous (see Section 2 and Table 1 for a complete description of the task). The individual learning curves of adolescents with ASC revealed that they tended to constantly shift from one choice alternative to the other, and this pattern did not diminish with task experience. The difference in this behavior was evaluated by calculating the average and maximal size of consecutive selections (or runs) from the same alternative. In the control group, the average run of consecutive choices was about three trials on average (implying that a person selected a deck three times in a row). In contrast, in the ASC group it was only 1.3 trials (i.e., most selections were followed by a switch). Similar differences emerged for the largest run of consecutive choices. Johnson et al. (2006) also found that within the ASC group short runs on the IGT were associated with more severe autistic syndromes on the Autism Diagnostic Interview (ADI-Revised, Lord, Rutter, & Le Couteur, 1994). Fig. 1 shows an example of this exceptional adaptive learning style in three of our study participants diagnosed with ASC (left) and three matched healthy controls (right). As one can see, the ASC participants’ adaptation pattern involves constant flipping between choice alternatives. In Johnson et al.’s (2006) study this pattern (of an average run of less than 1.5) characterized 80% of the adolescents diagnosed with ASC.
Using formal reinforcement learning models we contrasted three possible explanations of this pattern. The first explanation is that the choices of ASC individuals are simply more random, due to errors in selection. This is consistent with the idea that complex tasks that require generalization skills, such as the IGT, pose a challenge to people with ASC because of their difficulty in recognizing relationships between features of the task and forming general knowledge about categories of items and types of situations (see Klinger and Dawson, 2001, Klinger et al., 2006).
A second explanation is that the adaptation process of individuals with ASC involves more trial and error exploration. This argument is consistent with findings showing that adolescents with ASC rate high on such traits as perseverance and drive for perfection (e.g., Ashburner et al., 2009, Gillberg, 2002, Kobayashi and Murata, 1998). Under a third, related, explanation the difference in exploration capacity is qualitative rather than quantitative; that is, while individuals with ASC explore more than healthy individuals do, their exploration does not conform to the standard reinforcement learning paradigm. Rather, exploration may be the primary directive of their adaptation process. For examining this last possibility, a conventional model of reinforcement learning was contested with two novel models resting on the assumption that exploration, and not the pursuit of outcomes, is the main directive governing the learning process.
In order to examine the scope of the difference between adolescents with ASC and healthy controls, we employed a battery of decision tasks ranging in their difficulty (see Table 1). The battery included the more complex Iowa Gambling task (Bechara et al., 1994) in which the shifting choice pattern was originally found (Johnson et al., 2006). It also included five simpler variants of this decision tasks. This enabled us to assess if the tendency to continuously shift choices in ASC emerges only in relatively difficult decision tasks, or whether it is an uncontrollable tendency appearing even in simpler tasks. All tasks were then analyzed with the three classes of formal reinforcement learning model noted above. These models are explained more thoroughly in Section 3 and their mathematical details are available in the Appendix section.
Section snippets
Participants
The participants included 15 high functioning children and adolescents with formal diagnoses of ASC, who arrived to the community medical center for diagnosis, either individually or through ads placed in a patient support group center. The majority of the participants in this group (14 out of the 15) were male. The mean group age was 15.6 (SD = 2.8). Only participants diagnosed by at least one psychiatrist as specifically meeting the ICD-10 criteria for two autism spectrum disorders were
Results
Fig. 2 presents the proportion of selections from the advantageous decks of the IGT in the two groups. As can be seen, although both groups learned to make more advantageous choices with experience, learning was slower in the ASC group. To examine the statistical significance of this pattern we conducted a mixed analysis of variance (ANOVA) with group as a between-subject factor, and task block (four blocks of 25 trials) as a repeated measure. The results revealed a significant group by
General discussion
Individuals with ASC have a tendency to continuously shift between choice alternatives in complex repeated choice tasks (Johnson et al., 2006). In the current study, control participants made about 2.4 consecutive choices on each run, compared to 1.4 in the ASC sample. The ensuing pattern appears like pendulum movements from one choice alternative to the other, with choices being switched 73% of the time in the ASC group compared to 41% in the sample of healthy adolescents. An examination of
Acknowledgment
This research was supported in part by the Israel Science Foundation (Grant No. 244/06) and by the Max Wertheimer Minerva Center for Cognitive Studies.
References (58)
- et al.
Insensitivity to future consequences following damage to human prefrontal cortex
Cognition
(1994) - et al.
A model of attentional impairments in autism: First steps toward a computational theory
Cognitive Systems Research
(2005) - et al.
Resolving the paradox of the active user: Stable suboptimal performance in interactive tasks
Cognitive Science
(2004) Asperger’s and musical creativity: The case of Erik Satie
Personality and Individual Differences
(2009)- et al.
Change detection in children with autism: An auditory event-related fMRI study
Neuroimage
(2006) - et al.
Iowa Gambling Task in schizophrenia: A review and new data in patients with schizophrenia and co-occurring cannabis use disorders
Schizophrenia Research
(2007) - et al.
Striatal activity underlies novelty-based choice in humans
Neuron
(2008) - et al.
Evaluating the reliance on past choices in adaptive learning models
Journal of Mathematical Psychology
(2007) - et al.
The impact of ADHD and autism spectrum disorders on temperament, character, and personality development
American Journal of Psychiatry
(2006) - Ashburner, J., Ziviani, J., & Rodger, S. (2009). Surviving in the mainstream: Capacity of children with autism spectrum...
Screening and surveillance for autism and pervasive developmental disorders
Archives of Disease in Childhood
Autism: The empathizing–systemizing (E–S) theory
Annals of the New York Academy of Sciences
Do children with autism use the speaker’s direction of gaze strategy to crack the code of language?
Child Development
The Autism Spectrum Quotient (AQ): Evidence from Asperger syndrome/high functioning autism, males and females, scientists and mathematicians
Journal of Autism and Developmental Disorders
R-MAX – A general polynomial time algorithm for near-optimal reinforcement learning
Journal of Machine Learning Research
Cognitive modeling
A contribution of cognitive decision models to clinical assessment: Decomposing performance on the Bechara gambling task
Psychological Assessment
The temperament and character inventory (TCI): A guide to its development and use
Creativity and imagination in autism and Asperger’s syndrome
Journal of Autism and Developmental Disorders
Neurocognitive function and joint attention ability in young children with autism spectrum disorder versus developmental delay
Child Development
Cognitive game-theoretic analysis of attention strategies, ability and incentives
A theory of stimulus variability in learning
Psychological Review
Autism and creativity: Is there a link between autism in men and exceptional ability?
Diagnosis and differential diagnosis of Asperger syndrome
Advances in Psychiatric Treatment
A guide to Asperger’s syndrome
A dynamic allocation index for the sequential design of experiments
From conditioning to category learning: An adaptive network model
Journal of Experimental Psychology: General
The ‘reading the mind in the voice’ test-revised: A study of complex emotion recognition in adults with and without autism spectrum conditions
Journal of Autism and Developmental Disorders
The nature of human intelligence
Cited by (41)
To explore or exploit: Individual differences in preschool decision making
2024, Cognitive DevelopmentExploring maximizing, satisficing and minimizing tendency in decision-making among autistic and neurotypical individuals
2022, Research in Autism Spectrum DisordersCitation Excerpt :As a possible explanation for why autistic individuals may also be more resolute maximizing than neurotypical individuals, there is the empirical evidence reported by Kobayashi and Murata (1998) that autistic individuals score high on drive for perfection and perseverance, two traits that can be linked to the resolute maximizing tendency in decision-making. Other studies like Yechiam et al. (2010) reported evidence of autistic individuals, displaying intensive search behaviour in complex decision tasks. This drive for perfection in combination with the tendency to engage in intensive search behaviour could explain for the high latent mean score of the group of participants with autism in resolute maximizing (next to the high score on fearful maximizing).
Enhanced rationality in autism spectrum disorder
2021, Trends in Cognitive SciencesCitation Excerpt :In such tasks, several studies reported superior performance of individuals with ASD, who are less drawn to the immediate salience of rewards, and learn long-term reward contingencies more accurately from choice outcomes [21,28,29]. ( However, see [30–32] for studies that did not find this advantage.) An interesting exception to the salience of high reward is revealed in the ultimatum game, in which a person irrationally declines a reward in order to punish a partner who offers an unfair sharing of that reward [33].
Abnormal negative feedback processing in individuals with autistic traits in the Iowa gambling task: Evidence from behavior and event-related potentials
2021, International Journal of PsychophysiologyCitation Excerpt :It is possible that the rigid decision-making style of individuals with ASD or high autistic traits was related to their repetitive symptoms. In addition, a formal cognitive model indicates that decision making by individuals with ASD cannot be explained using standard reinforcement learning models due to deficits in terms of social interaction information processing and goal-directed learning (Johnson et al., 2006; Yechiam et al., 2010; Mussey et al., 2015; Zhang et al., 2015). Our results are consistent with those obtained in previous studies and they suggest that individuals with high autistic traits within a nonclinical population had an unusual learning style for decision making based on negative and positive feedback.
Autism is not associated with poor or enhanced performance on the Iowa Gambling Task: A Meta-Analysis
2020, Neuroscience and Biobehavioral Reviews