Determining treatment outcome in early intervention programs for autism spectrum disorders: A critical analysis of measurement issues in learning based interventions

https://doi.org/10.1016/j.ridd.2005.07.006Get rights and content

Abstract

One of the areas receiving the greatest attention from researchers studying autism spectrum disorders in recent years involves psychologically based early intervention programs. Various claims of cure, marked improvement in social and communication skills, and improved I.Q. are among the conclusions that have been drawn by various researchers. However, little has been done to analyze the dependent variables used in these studies and their impact on the conclusions reached regarding treatment effectiveness. Obviously, this set of measures is crucial since these methods define which behaviors “improved” and to what extent. The present review analyzes the current status, strengths, and weaknesses of these measurements.

Introduction

One of the most discussed issues in the child treatment literature is the success of early intervention programs with autism spectrum disorder (ASD) children (Matson & Minshawi, 2006; Moore & Goodson, 2003). These interventions, which address the range of symptoms evinced by ASD children, are behavioral/learning-based procedures. This pragmatic factor defines the parameters for the present review.

Various claims have been made about the specific effects of these programs, but a general consensus appears to be growing in the research literature. The consensus is that early intervention is valuable and, within limits, the more intense the intervention, the greater the gains despite variability in outcomes within groups of children treated (Symes, Remington, & Brown, 2006). Despite the considerable research focus on these treatment studies, however, little has been done to analyze the various dependent variables that have been used and the effect these assessment methods have on how treatment effects are perceived. When guidelines regarding assessment for children with autism spectrum disorders (ASD) are published, they tend to focus almost exclusively on differential diagnosis (AACAP, 1999).

To date, behaviorally-based interventions have been found to be by far the most effective means of intervention for children with ASD (Schreibman, 2000). Many more studies are available on behaviorally-based interventions than other topics and thus, the bulk of the review will focus on the dependent measures used in these studies. More recently pharmacological interventions have begun to appear and some guidelines for assessment have been described. Therefore, while assessment is gleaned primarily for behavioral treatments these data also have implications for pharmacology outcome studies. Other methods of intervention, some of which are quite unusual, have been proposed. Some authors have suggested that treatments have become so unusual that they refer to autism in particular as a fad magnet for off the wall interventions (Metz, Mulick, & Butter, 2005). For this review, assessment guidelines for mainstream behavioral and pharmacological treatment will be the theme for the paper.

Section snippets

Outcome measures for behavioral interventions

It would not be possible to cover the vast literature on treatment in one paper. Therefore, selective studies were reviewed to develop the general points about the type of dependent variables employed. Two methodologies are evident. The first method follows a strict behavior analytic approach. These studies employ singe case designs with one to five children being evaluated. Multiple baseline and/or reversal designs are used as the primary means of methodological control. A second, much

Behavior analysis outcome measures

Typical of these applied behavior analysis single case design studies in general are the operational definition of one to three problems/challenging behaviors such as self-injury, stereotypies, and aggression. Studies of this nature typically note in the description of participants that the children evince an ASD, typically autism. However, the purpose of the study typically, as noted, is on dealing with a few specific behaviors, usually self-injury (Mohr & Sharpley, 1985), aggression,

Program treatment outcome

As the field has advanced and matured with respect to this massive literature on testing a few specific targets, more ambitious efforts geared toward multiple behavior domains have emerged. Occurring with this shift in focus has been the use of more and more expansive dependent measures, which assess a broad range of success criteria on which individual and groups of autistic and PDD-NOS children can be measured. These two diagnostic groups among all the ASD groups have received most of the

How do researchers define effective

The assertion made here is that the outcome measures that researchers choose to include in their studies define the “universe” of potential effects. Using this definition it is clear that I.Q. and measures of language are the clear favorites to date in this regard. Interestingly, behaviorally-based programs rarely include measures of the core symptoms of autism such as the ADI-R or CARS, despite the fact that autism or PDD-NOS children, versus Rett's, Childhood Disintegrative Disorder, or

Trends

Relative to assessment of outcomes, a few observations are in order. First, group designs are becoming more common as a means of assessment versus single case research designs. Second, standardized I.Q. and adaptive measures initially introduced over 25 years ago still appear to be a primary measure of outcome (Lovaas et al., 1973). Third, operational target behaviors as outcome measures are being used less frequently. Fourth, in the place of operation target behaviors are measures of general

Concluding remarks

Despite various methodological flaws from an assessment standpoint in the existing early intervention literature with ASD, there have been a sufficient number of replications to declare the methods “promising”. This finding should be no surprise since the techniques employed are based on a massive empirical literature on applied behavior analysis (Matson & Minshawi, 2006). Despite these promising gains, one would hope and expect to see more defined and refined assessment methods as the

References (45)

  • T.M. Achenbach

    Manual for the teacher's report form and 1991 profile

    (1991)
  • T. Charman et al.

    Practitioner review: Diagnosis of autism spectrum disorder in 2- and 3-year-old children

    Journal of Child Psychology and Psychiatry

    (2002)
  • T. Charman et al.

    Outcome at 7 years of children diagnosed with autism at age 2: Predictive validity of assessments conducted at 2 and 3 years of age and pattern of symptom change over time

    Journal of Child Psychology and Psychiatry

    (2005)
  • A. Cox et al.

    Autism spectrum disorders at 20 and 42 months of age: Stability of clinical and ADI-R diagnoses

    Journal of Child Psychology and Psychiatry

    (1999)
  • S. Eikeseth et al.

    Intensive behavioral treatment at school for 4- to 7-year-old children with autism

    Behavior Modification

    (2002)
  • J. Geddes et al.

    Atypical antipsychotics in the treatment of schizophrenia: Systematic overview and meta-regression analysis

    British Medical Journal

    (2000)
  • H.M. Geurts et al.

    How specific are executive functioning deficits in attention deficit hyperactivity disorder and autism?

    Journal of Child Psychology and Psychiatry

    (2004)
  • M. Ghaziuddin et al.

    Depression in persons with autism: Implications for research and clinical care

    Journal of Autism and Developmental Disorders

    (2002)
  • P. Howlin et al.

    Adult outcome for children with autism

    Journal of Child Psychology and Psychiatry

    (2004)
  • C. Kasari

    Assessing change in early intervention programs for children with autism

    Journal of Autism and Developmental Disorders

    (2002)
  • M. Kovas

    Rating Scales to assess depression in school-aged children

    Acta Paedopsychiatrica

    (1980)
  • K. Long et al.

    Presentation, assessment and treatment of depression in a young woman with learning disability and autism

    British Journal of Learning Disabilities

    (2000)
  • Cited by (134)

    • A critical review of outcome measures used to evaluate the effectiveness of comprehensive, community based treatment for young children with ASD

      2016, Research in Autism Spectrum Disorders
      Citation Excerpt :

      Specific attention was paid to evaluating how well the tests were linked to the underlying diagnostic construct of ASD, how well the sample population was described and matched to the instruments, and if test users reported on reliability, validity, administration protocols, modifications, test bias, and minimum training requirements. As the use of different cognitive tools at pre and post test was identified as an area of concern by Matson (2007), this particular practice was also evaluated. The checklist was piloted on six studies and then revised with input from two other senior ASD researchers to ensure it was accurately capturing the intended domains of interest and that there was agreement on the correspondence between the Standards and the chosen evaluative criteria.

    • Response to Early Intensive Behavioral Intervention for autism-An umbrella approach to issues critical to treatment individualization

      2014, International Journal of Developmental Neuroscience
      Citation Excerpt :

      Language abilities of the children improve more when the children start with high adaptive behavioral skills and vice versa. Surprisingly, despite the clear call for the inclusion of measurement of ASD core symptoms and challenging behaviors, most EIBI studies failed to address these dependent variables (Matson, 2007). This is kind of disappointing, as children with severe ASD exhibit significantly more and higher levels of challenging behaviors (Matson et al., 2008) with stereotypical behaviors being the most commonly reported (Jang et al., 2011).

    View all citing articles on Scopus
    View full text