Several research-based intervention methods for children with autism spectrum disorders (ASD) have been developed and tested in controlled laboratory settings. There is increasing empirical support, for example, for child skills training and parent-mediated interventions based on principles of applied behavior analysis (see National Research Council 2001
; Rogers and Vismara 2008
) and cognitive behavioral therapy methods (e.g., Chalfant et al. 2007
; Wood et al. 2009
Despite increasing support for specific intervention models, there is limited knowledge about how these strategies may be implemented in “usual care” community-based service settings (National Advisory Mental Health Council 2001
). The limited research that has been conducted on community-based services for children with ASD has focused on early intervention/educational services and suggests that there are discrepancies between research-based practices and community-based care (e.g., McLennan et al. 2008
; Stahmer 2007
; Stahmer et al. 2005
). Further, research suggests that outcomes in community-based mental health clinics for children with other psychiatric problems are not as positive as those observed in research trials (Weisz et al. 2006
). Detailed research on usual care service context and treatment for childhood disorders is critical to bridge the gap between what is known about research-based practices and community-based practice (Bickman 2000
; Hoagwood and Kolko 2009
; McLennan et al. 2008
; Westfall et al. 2007
The limited research on community services for children with ASD suggests that services are provided in a number of different sectors including the mental retardation/developmental disability (MR/DD), special education/early intervention, medical, and mental health service systems. A child may be concurrently served in multiple sectors for different needs (Brookman-Frazee et al. 2009a
). For example, a child may receive services through the school system for social and academic issues, and the mental health system for behavioral issues. The current study focuses on services provided to children with ASD in the mental health system where co-occurring psychiatric and behavior symptoms may be targeted. Mental health services are designed to address psychiatric problems through psychosocial and pharmacological interventions. The current study focuses specifically on outpatient psychotherapy services provided to children with ASD in community-based mental health clinics.
The mental health system plays a particularly important role in caring for children with ASD for treatment of overlapping and co-morbid psychiatric problems that can occur with ASD. Although there is significant variability in the estimated prevalence rates of co-morbid psychiatric disorders among children with ASD (Lainhart 1999
), recent research suggests that approximately 70% of children with ASD meet criteria for at least one psychiatric disorder (Leyfer et al. 2006a
; Simonoff et al. 2008
). These high rates of a variety of psychiatric problems for children with ASD underscore the importance of examining mental health services.
The limited research conducted on community-based mental health services for children with ASD has examined the costs of these services (Mandell et al. 2006
), the characteristics of children receiving specific types of mental health services (e.g., Bryson et al. 2008
; Mandell 2008
; Mandell and Palmer 2005
; Mandell et al. 2005
), and general descriptions of the types of services provided (Kohler 1999
; Ruble et al. 2005
). While the data on patterns of community mental health service use are growing, there is limited information on what treatment in these settings actually entails (i.e., what types of treatment strategies are used in service sessions and what types of outcomes are obtained) and for whom these treatment strategies are being applied. The limited descriptive data on community-based practice has primarily been focused on very young children receiving early intervention services (e.g., Stahmer 2007
) and has not examined school-aged children or those served in mental health settings.
Descriptive data on the nature of mental health services and treatment outcomes is important for a number of reasons. First, while there is a growing body of research on interventions for children with ASD, including those for children with co-occurring mental health problems, it is not known how clinicians providing community care tailor mental health intervention and treatment for the unique needs and clinical presentations of children with ASD. It is likely, however, that community care may not be as effective as treatment provided in research studies since many community mental health systems were not designed to serve children with developmental disabilities, and it is likely that those providing treatment to children with ASD lack specialized training in this spectrum of disorders (Brookman-Frazee et al. 2009a
). Therapists providing outpatient psychotherapy in our community indicate that they perceive children with ASD to be particularly challenging to treat (Brookman-Frazee 2009
). Further, research on community providers’ knowledge about ASD indicates that providers (including mental health professionals) hold a number of inaccurate beliefs about ASD (Heidgerken et al. 2005
). Understanding how consistent (or inconsistent) routine community care is with interventions provided in highly structured university research settings can provide direction for efforts to improve community care (Weersing and Weisz 2002
; Weisz et al. 2004
). Further, descriptive data on usual care practice can provide baseline data for efforts to implement research-based practices in the community.
Detailed description of community services needs to include information on the context in which these services are being provided (e.g., client and provider characteristics). Understanding the characteristics of those children receiving specific services is particularly important as research suggests that there may be variability in ASD treatment response (e.g., Sherer and Schreibman 2005
). Research in the area of other childhood psychiatric disorders indicates that child sociodemographic and clinical characteristics may be associated with treatment process variables (e.g., Armbruster and Fallon 1994
; Brookman-Frazee et al. 2008
) and response to treatment variables (Beauchaine et al. 2005
; Reyno and McGrath 2006
The current study aims to address the dearth of knowledge about community-based mental health services by studying outpatient psychotherapy services for children with ASD provided in community mental health clinics. Data from this study are drawn from a large-scale longitudinal, observational study characterizing outpatient psychotherapy services for children presenting with a broad range of disruptive behavior problems, which represent the most prevalent presenting problems to outpatient mental health services (Garland et al. 2001
). The main objectives of this sub-study are to (a) describe the characteristics of children with ASD served in outpatient psychotherapy for disruptive behavior problems and (b) characterize treatment process and outcome for this group of children. In order to provide contextual information about services typically provided in this setting, a secondary aim of the study is to examine differences in sociodemographic and service entry characteristics, treatment process and outcomes for children with ASD relative to a comparison group of children without ASD treated by similar therapists.
Throughout this section, analyses of the ASD group only are presented first, followed by analyses comparing the ASD and Non-ASD groups.
Child/Family Socio-Demographic, Service Entry, and Clinical Characteristics
presents sociodemographic, service entry, and clinical characteristics for children with ASD and the non-ASD comparison group. Independent samples t
-tests and chi square analyses were used to examine differences on sociodemographic and service entry characteristics between the two groups (Differences between groups on diagnosis were not examined since they were selected based on diagnosis). There was a trend for group differences on child age. Specifically, children in the ASD group were younger than children in the non-ASD comparison group (t
= 1.95, p
= 0.06; Cohen’s d
= 0.65). There was also a trend for group differences in funding source such that children in the ASD group were more likely to receive funding through the AB2726 (educational) program, whereas children without ASD were more likely to receive funding through Medi-Cal (χ2
= 3.89, p
= .05; Phi = 0.32). There were no significant differences between the two groups on child racial/ethnic minority status, caregiver age, annual household income, referral source, and effects sizes were small (Cohen’s d
ranged from −0.25 to 0.44; Phi ranged from 0.13 to 0.23).
Children with ASD attended an average of 16.3 sessions (SD = 9.5; range = 1–29) during the 8-month study period, with 73.7% (n
= 14) still attending at 8 months. During the study period, 66.7% (n
) of the children with ASD received medication treatment for emotional/behavioral symptoms. The most common classes of drugs were stimulants (50.0%, n
= 9), antidepressants (16.7%, n
= 3), antipsychotic medications (11.1%, n
= 2), and mood stabilizers (11.1%, n
= 2). There were no significant differences between ASD and the non-ASD comparison group on number of sessions attended, the proportion who remained active in treatment at 8-month follow-up, or proportion who received medications and effect sizes were small (Cohen’s d
= 0.23; Phi ranged from 0.02 to 0.20).
Children with ASD were present in almost all of the 103 coded psychotherapy sessions (96.1%) and caregivers were present in at least part of 73.8% of all coded sessions. There were no significant differences between the ASD and Non-ASD groups in the proportions of sessions in which the child or parent were present.
Treatment Strategies Observed (Directed to Children)
shows the occurrence and average intensities (when the strategy was used) for PRAC TPOCS-S strategies directed to children for all coded tapes. For the ASD group, the rate of occurrence of each psychotherapeutic strategy directed to children varied. Strategies observed most frequently in sessions with children (>75% of sessions) included: using positive reinforcement
(87%); affect education
(80%); and psychoeducation
(78%). The strategy that was observed to occur least frequently (<25% of sessions) was assigning or reviewing homework
(18%). Average intensity ratings of the individual strategies ranged from Low to Moderate (M
= 2.6). The strategy with the highest average intensity rating was role
= 3.2; “Moderate”) and the strategy with the lowest intensity was using punishment/limit setting
= 1.9; “Low”).
Independent samples t
-tests were then conducted to examine differences in observed strategy delivery between the ASD and non-ASD comparison group. For these analyses, in order to obtain one score for each child, session data were aggregated for each child by using the maximum combined occurrence/intensity score (possible range: 0–6) across all coded sessions for each PRAC TPOCS-S strategy (not shown in Table 2
). Note: there were no significant differences in the average number of sessions coded per child between the ASD (M = 5.0) and non-ASD (M = 5.0) groups. No statistically significant differences between the groups were observed at the .05 level on any of the strategies delivered to children and effect sizes were generally small (Cohen’s d
ranged from .08 to .47).
Treatment Strategies Observed (Directed to Parents)
shows the occurrence and average intensities for strategies directed to parents. For the ASD group, the rate of occurrence of each psychotherapeutic strategy directed to parents also varied. The strategy observed most frequently in sessions with parents (>75% of sessions) was psychoeducation
(83%). Strategies that were observed to occur infrequently (<25% of sessions) included: principles of positive reinforcement
(12%) and assigning and reviewing homework
(17%). Average intensity ratings of the individual strategies ranged from Low to Moderate (M
= 2.7). The strategy with the highest average intensity rating was psychoeducation
= 3.5; “Moderate”) and the strategy with the lowest intensity was addressing parent/family issues, care
= 2.0; “Low”).
Similar to the analyses with strategies directed to children, independent samples t
-tests were conducted to examine differences in strategy delivery to parents between the ASD and non-ASD comparison group, aggregated to the child level (not shown in Table 3
). Of the eight parent strategies examined, the only significant group difference at the .05 level was on assigning and reviewing homework
such that parents of children in the ASD group (M
= 1.47; SD = 1.55) received more intensive delivery of this strategy than parents of children in the non-ASD comparison group (M
= 0.44; SD = 0.89) (t
= −2.37, p
= 0.03; Cohen’s d
= 0.93). Effect sizes for the other strategies delivered to parents were small (Cohen’s d
ranged from .03 to .16) with the exception of addressing parent issues and care
= .52) such that parents of children in the ASD group (M
= 1.35; SD = 1.73) received less intensive delivery of this strategy than parents of children in the non-ASD group (M
= 2.19; SD = 1.56).
Paired sample t
-tests were used to compare scores on the four outcome measures at the 8-month follow-up interview to baseline scores for the ASD sample. See Table 4
for means and standard deviations for each outcome measure. Results indicate significant decreases on parent reported child problem severity (ECBI) from baseline to 8-month follow-up (t
= 3.68; p
< .01; Cohen’s d
using original standard deviations = 0.65). It is important to note that 78% of the ASD group scored in the clinical range on the ECBI at the baseline interview, compared to 44% at 8-month follow-up. Parents of children with ASD also reported significantly lower caregiver strain on the CGSQ at 8-month follow-up compared to the baseline assessment (t
= 3.12; p
< .01; Cohen’s d
= 0.72). There was a trend for increased family-related empowerment on the FES at follow-up (t
= −2.08; p
= .05; Cohen’s d
= 0.60). There was no significant change on the measure of consistent discipline from baseline to follow up (t
= −0.19; p
> .05; Cohen’s d
Means and standard deviations of baseline and 8-month follow-up on the Eyberg Child Behavior Inventory, Caregiver Strain Questionnaire, Family Empowerment Scale (Family subscale), and Consistent Discipline measures
Eyberg child behavior inventory (ECBI) intensity scale
Caregiver strain questionnaire (CGSQ)
Family empowerment scale (family)a
A series of one-way between groups analyses of co-variance (ANCOVA) were conducted to examine differences in mean 8-month follow-up scores as a function of group (ASD and non-ASD comparison) while controlling for baseline score as a covariate. The assumptions of homogeneity of regression and variance were met for analyses of all outcomes measures. After controlling for baseline scores, no significant group differences were observed on the ECBI (F(1,30) = 0.03, p > .05, η2 = 0.001); the CGSQ (F(1,30) = 0.02, p > .05, η2 = 0.001); the FES-Family (F(1,28) = 0.06, p > .05, partial η2 = 0.002), or on the Consistent Discipline measure (F(1,28) = 0.36, p > .05, partial η2 = 0.013).
This study examined treatment processes and outcome for a subgroup of children with ASD and disruptive behavior problems receiving care in community mental health clinics. It represents the first research reporting detailed observational data on community mental health services for children with ASD. Further, it provides important descriptive information about the characteristics of children with ASD and rich contextual data about a matched comparison group without ASD receiving care in this setting.
The children with ASD were an average of 8 years old and primarily male, with almost all having a high functioning (i.e., Asperger’s Disorder) or non-specific (i.e., PDD-NOS) diagnosis on record (i.e., DSM-IV 299.80). This high rate of Asperger’s/PDD-NOS diagnoses may be related the eligibility requirements of publicly-funded community mental health services in our county. That is, a primary
diagnosis of autistic disorder is excluded for reimbursement in the publicly-funded mental health service system, as the MR/DD system (not the mental health system) is responsible for funding services for children with this diagnosis. Asperger’s Disorder and PDD-NOS, however, are not excluded diagnoses. Children in the ASD sample were also diagnostically complex, with over 70% labeled as having at least one co-morbid additional psychiatric diagnosis (most frequently ADHD). This rate is consistent with studies examining the prevalence of psychiatric problems in children with ASD (Leyfer et al. 2006b
; Simonoff et al. 2008
). Lastly, funding for a majority of the children with ASD was provided through the school district. This finding is consistent with national trends indicating that the education system plays a significant role in providing mental health services for children with a broad range of mental health problems (Roanes and Hoagwood 2000
The observational data on treatment strategies delivered to children with ASD indicate that therapists do frequently deliver a number of strategies conceptually consistent with research-based behavioral methods (delivering positive reinforcement and punishment/limit setting) and certain cognitive behavioral strategies (problem-solving/social skills, affect education, affect management) (e.g., Rogers and Vismara 2008
; Wood et al. 2009
). However, these strategies were delivered, on average, with low to moderate intensity (i.e. thoroughness). These findings indicate that while therapists in community mental health clinics are targeting many of the same general areas as research-based child skill-building interventions, the interventions are not being delivered in the same manner. The observed occurrence of behavioral parent training strategies (e.g., covering operant conditioning principles) and active teaching strategies typically employed in research-based parent training programs (e.g., role-play/practice, homework) was low. Likewise, when these strategies were observed to be delivered with parents, average intensity was low. While the observational measure used in this study is not a measure of fidelity to a particular treatment model, the low intensity suggests that the strategies are not being delivered as thoroughly as would likely be present in a research-based treatment model. Taken together, these findings replicate results from the larger PRAC study which indicate that certain strategies conceptually consistent with evidence-based practices for children with disruptive behavior problems are delivered with some frequency in community mental health clinics, but they are not delivered very intensively (Garland et al. in press). Further, the findings are consistent with research on community early intervention services that indicate that there are gaps between research-based practices and those that are provided in the community (Stahmer 2007
; Stahmer et al. 2005
It is important to highlight that observed treatment process was very similar for children with and without ASD who are receiving care from the same (or very similar) providers. This finding suggests that therapists may not tailor treatment to the unique characteristics of children with ASD, which may be explained by the limited specialized ASD training reported by therapists in these settings (Brookman-Frazee 2009
). It also supports the general lack of specificity of treatment strategies based on child primary diagnosis reported in analyses of the full PRAC study which indicate that variability in practice patterns is not associated with child diagnosis (Brookman-Frazee et al. 2009b
Outcomes across a number of different domains were measured. Statistically significant improvements in child behavior problems, caregiver strain, and family-related empowerment were observed at 8 month follow-up and medium effect sizes were observed. Significant improvements in parent discipline practices were not seen. These findings provide preliminary evidence of the positive impacts of community services, particularly related to child behavior problems and parent functioning, however, the effect sizes are smaller than those reported in intervention trials (e.g., Wood et al. 2009
). It is also important to interpret these findings with caution given that there was no control group. Thus, no conclusions can be drawn about the impact of treatment on these outcomes. More research is needed on the potential impact of treatment provided in this context on social and adaptive functioning. Further, examination of change in internalizing psychiatric symptoms (particularly anxiety) is warranted given that over one quarter of the ASD sample was diagnosed with an anxiety disorder.
This study adds to the literature in a number of different ways. First, it expands the literature on ASD services research. While much of the existing work in this area relies on large administrative datasets to describe the broad types of services that child with ASD receive (e.g., Mandell et al. 2005
; Mandell and Palmer 2005
; Ruble et al. 2005
) this study provides detailed observational data on the nature of outpatient treatment actually delivered in treatment sessions in community practice. These data have important implications for bridging the research-practice gap. For example, they can be used as baseline data for future studies that attempt to implement research-based ASD interventions in community settings. Further, the findings provide a “roadmap” for specific treatment strategies that should potentially be the focus of therapist training. For example, the low intensity/thoroughness of all strategies delivered suggest that increasing the intensity with which strategies are delivered may be a primary focus. Further, the low frequencies of behavioral parent training and active teaching strategies (role-play/practice and homework) need to be explicitly targeted. This is particularly important given that many of the children also had ADHD and DBD diagnoses for which there is also strong empirical support for behavioral parent training. It is important to note that certain research-based strategies were observed relatively frequently, suggesting that there is some overlap between community services and research-based care and that therapists may be open to learning more about research-based strategies. Overall, the findings highlight the heterogeneity of usual care psychotherapy. While some aspects of usual care treatment resemble research-based models, it is not entirely consistent. The findings also suggest that using treatment as usual is an important control group for future interventions studies.
This study also adds to the research on mental health services more generally. To date, there has been limited attention to ASD in mental health services research (see Brookman-Frazee et al. 2009a
). This study confirms that children with ASD are being served in community-based mental settings. That is, they represent almost 10% of children presenting with disruptive behavior problems sequentially recruited into the PRAC study. Since therapists in our community indicate that they have limited training on treatment for ASD, it suggests that efforts to improve mental health services overall should include some targeted training efforts for this population. This study also provides information about the characteristics of children with ASD served in general community mental health settings. This information is important because it facilitates our understanding of who
is receiving these services. Lastly, this study provides information on the delivery of psychotherapeutic strategies for all
children receiving care in these settings. Consistent with other analyses of the full PRAC study sample (e.g., Brookman-Frazee et al. 2009b
), the data for this subgroup indicate that overall intensity of observed strategy delivery was relatively low for strategies directed towards both children and parents. Since evidence-based practices for most childhood disorders (including ASD) typically include thoroughly pursuing specific goals (i.e., intensity), this finding suggests the need for training interventions to strengthen therapists’ use of active and directive techniques to increase the thoroughness of strategies delivered.
Some study limitations should be noted. First, the broader study from which these data were drawn was not focused exclusively on treatment for children with ASD. Thus, there are no standardized measures to validate ASD diagnoses. However, the sample does reflect children being treated in community mental health settings for ASD, as determined by the treating therapists, thus there is ecological validity. Use of these type of therapist-assigned diagnoses obtained through administrative data is a common method of identification in ASD services research (e.g., Mandell et al. 2002
; Ruble et al. 2005
). Co-occurring DSM diagnoses were also based on therapist-report and not validated by diagnostic measures. The total number of diagnoses that therapists were able to report was limited to four. Since the ASD diagnosis took up one of the possible diagnoses for the ASD group, the rates
of co-occurring diagnoses for the two groups are not directly comparable. Further, although the observational measure used in this study was developed to capture a broad range of potential strategies, it was not specifically designed to be used to characterize treatment for children with ASD.
Related, information about the therapists’ experiences related to ASD is lacking. It is not known how many children on the therapists’ caseloads have an ASD. Likewise, information for ASD training for the participating therapists is lacking. Although this information is not available for those in the current study, we have collected this information from 100 therapists practicing in 9 community-based mental health clinics in San Diego County. Results indicate that only 5% of therapists consider themselves “ASD experts” and an average of 20.7% of therapists’ current caseloads are represented by children diagnosed with an ASD or suspected of having an ASD (Brookman-Frazee 2009
). These data suggest that therapists in the current sample have limited ASD experience, but are regularly serving this population.
Another potential limitation relates to the generalizability of the therapist and child/family samples. This study was conducted in one large, geographically diverse county. While the distributions of therapist education level, gender, race/ethnicity, and trainee status is similar to other studies of community mental health providers (Glisson et al. 2008
; Hawley and Weisz 2005
), the characteristics of children with ASD in this sample may differ from children receiving community mental health services in privately funded mental health settings and/or other locations that may have different fiscal policy factors that impact service referrals for the Mental Health, Special Education, and MR/DD systems. Related, this study utilized a small sample of children ages 4–12 with ASD who were referred to treatment for disruptive behavior problems. Treatment process and outcomes for older children with ASD and those referred for other psychiatric problems (e.g., anxiety, depression) may be different.
This study is also limited by the small sample size. It may limit the power to detect differences between the ASD and comparison group. It is important to note that the lack of significant differences in these small samples is consistent with other analyses of the full PRAC sample which reveal no significant differences on treatment process based on child diagnosis (Brookman-Frazee et al. 2009b
). Further, due to the size of the ASD group, we are unable to conduct analyses which examine potential predictors of treatment delivery or outcomes. It is possible that certain therapists are more likely to deliver strategies more intensively; however, analyses of the full sample indicate that there are few therapist characteristics associated with intensity (Brookman-Frazee et al. 2009b
). It would also be interesting to determine whether certain strategies, or combinations of strategies, result in more positive changes in child or family outcomes. Based on preliminary analyses of the full PRAC sample, there are few predictors of positive outcomes.