Application of Latent Class Analysis to Identify Subgroups of Children with Autism Spectrum Disorders who Benefit from Social Skills Training

The study was based on data from an RCT on the effectiveness of group SST with and without parental and teacher involvement, with three conditions: Social Skills Training (SST; n = 47), Social Skills Training with Parent and Teacher Involvement (SST-PTI; n = 51) and CAU (n = 24) (Dekker et al. 2014, 2019). From September 2010 through September 2013, training groups had started (September and February), data collection therefore started in May 2010 (first pre test) and ended in October 2014 (last follow-up measurement).

The current study only included the children in the treatment conditions (n = 98; 47 SST and 51 SST-PTI), in order to be able to investigate their progress after SST. Three measurements were conducted: before randomization (T1), immediately after the SST (T2), and six months after the end of SST (T3). Before participation, all parents and children above 12 signed an informed consent. The study followed CONSORT guidelines for RCTs, was approved by the Institutional Review Board of the University Medical Center Groningen, and was registered in the Dutch Trial Register (NTR2405; https://​www.​trialregister.​nl). For a detailed description of study recruitment and treatment allocation, we refer to the research protocol (Dekker et al. 2014) and the report on the findings of the primary analyses (Dekker et al. 2019).

The participants of the ESTIA-study (n = 122) were recruited in four outpatient mental health care clinics in the northern part of the Netherlands. The current study only included the 98 participants who participated in the intervention (n = 24 were in the control condition). All were preadolescent high-functioning children with a clinical diagnosis of ASD (81 boys, 17 girls). As reported in Dekker et al. 2019 all participants met the following inclusion criteria, which were slightly adjusted from the original design registered in the trial register: (1) pre-existing clinician-based DSM-IV-TR ASD diagnosis (Autistic disorder, Asperger’s disorder, or Pervasive Developmental Disorder-Not Otherwise Specified [PDD-NOS]), based on developmental history, current problems, child observation (ADOS), and information from school in expert teams including at least a child psychologist and a child psychiatrist (original criterion: ASD diagnosis either supported by an Autism classification on the Autism Diagnostic Interview-Revised (ADI-R) or maximally two points below the cut off for Autism on the ADI-R but with an ASD classification on the ADOS); (2) the child’s clinician indicated SST as the first appropriate treatment due to social problems in school and other contexts; (3) parents and child were motivated for SST, as established during a meeting with the clinician, the parents, and the child; (4) preferably IQ ≥ 80 (original criterion: IQ ≥ 80); children with IQs slightly below 80 were included when therapists established they were able to follow an SST; (5) being in the last two and half years of primary education (original criterion: being in the last two years of primary education); (6) no physical condition affecting participation; and (7) the child could travel to the child mental health center for training. The original criteria were broadened to more closely approximate the regular decisions in clinical practice. As a result of the change in the first criterion, the clinical diagnoses in the sample had not been corroborated with classifications on standardized instruments. This indicates that the included sample functioned at or towards the less autistic end of the spectrum.

Of the 98 participants in the intervention, 65 had a DSM-IV-TR diagnosis of PDD-NOS (66%), 20 of Asperger’s disorder (20%), and 13 of autistic disorder (13%). ASD diagnoses did not differ between the two SST conditions (Pearson χ² 0.25; p = 0.881). One comorbid diagnosis was established in 32.7% of the children, 3.1% had two comorbid diagnoses (20 ADHD, 20.4% of all participants; 7 a tic disorder, 7.1%; 3 an Anxiety Disorder, 3.1%; 4 an Oppositional Defiant Disorder, 4.1%; 2 other, 2%). The SST conditions did not differ in comorbid secondary diagnoses (Pearson χ² 15.19; p = 0.296) or tertiary diagnoses (Pearson χ² 2.94; p = 0.402). At the start of the SST the mean age was 10.9 years (SD = 0.7; range 9.5–12.7). Most children had two Dutch parents (n = 72), all had at least one Dutch parent. The characteristics of the participants at baseline are presented in Table 1.

Children in both training conditions participated in a manualized SST, based on behavioral therapeutic principles and the social learning theory (Van Warners et al. 2010; internal publication), in groups of 4–6 children, led by two therapists. The training consisted of 18 sessions of 90 min. The first 15 were weekly sessions, followed by three booster sessions after two months, to maintain trained social skills. During the sessions, the children learned how they could perform social skills, based on behavioral therapeutic principles and the social learning theory. The therapists created a safe situation for the children so that they could experience that social interaction can be enjoyable. The sessions had a recurring structure: conversation, homework review, introducing a new topic, practice and role-play, new homework, and play-time. In the first four sessions the focus was on creating a safe environment for the children. In the other sessions specific topics were discussed, e.g. “asking something to someone”, “responding to bullying”.

In the SST with Parental and Teacher Involvement (SST-PTI; Van Warners and Vet 2010; internal publication) parents and teachers of the 51 children were additionally involved in the training, as opposed to SST only. Parents received 8 additional parent sessions, specifically related to the SST. Additionally, therapists met with teachers at the start of SST-PTI, followed by five telephone meetings during SST-PTI. The aim was to teach parents and teachers how to support children in practicing social skills in daily life. Therapists used instruction, behavioral exercises, and role-play. This involvement was added to advance the generalization of social skills to situations beyond the training. A more comprehensive description of the SST and SST-PTI can be found elsewhere (Dekker et al. 2014, 2019).

We included three parent measures as indicators of social communicative skills, pertaining to the application and the perceived difficulty of social skills by their child, at three moments: before randomization (i.e. before SST; T1), immediately after SST (T2), and six months after the end of SST (T3). The measures used in this study rate social performance rather than social competence.

We used Multilevel Latent Class Growth Analysis (MLCGA; Muthén 2004; Palardy and Vermunt 2010) to identify distinct subgroups of participants based on their level and perceived difficulty of social communicative skills as reported by parents at start of SST and their response to SST. In the MLCGA the three outcome measures (Vineland Socialization, SSRS-P, and ESTIA-TS) at the three moments (before (T1), immediately after (T2) and six months after end of SST (T3)) were jointly analyzed, using for each outcome variable a regression equation with normal error, including an intercept and two dummy variables. The latter were coded such that the intercept referred to T1, and the slopes for the dummy variables referred to the difference between T1 and T2, and to the difference between T2 and T3. MLCGA models with one to seven classes were estimated (i.e., response patterns). From these models, we selected the model with a minimal class size of 10% and the lowest value of the Bayesian Information Criterion (BIC; Lanza et al. 2007). The BIC indicates a model with an optimal balance between fit and model complexity. Of the selected model, we calculated within group effect sizes (ES) for all slopes and classes (i.e., per class change between T2-T1 and/or T3-T2, as \(d= \frac{\left({\widehat{\mu }}_{Tj}-{\widehat{\mu }}_{T(j-1)}\right)}{{\widehat{\sigma }}_{T1}}\), with j = 2,3). ES were based on the estimated model, and were interpreted as ES = 0.2 as small, ES = 0.5 medium, ES = 0.8 large and ES = 1.3 very large (Cohen 1988), and applied as indicators for the clinical relevance of the findings.

For each participant we established the posterior probability of membership of each class based on the selected model above. Each participant was assigned to the class that matched with his or her highest probability.

In a follow-up analysis, we predicted the probability of class membership with each of the selected participant and intervention characteristics via univariable regression analyses, using the so-called three-step procedure as described in Vermunt and Magidson (2013). The participant and intervention characteristics were categorized as described in Table 2. These analyses provide insight into the distribution of the characteristics in each class, indicating whether differences between classes can be detected, based on other characteristics than outcome. For the characteristics that appeared to be significant, we performed a post-hoc analysis, to examine possible differences among the different classes with paired comparisons. Additionally, the characteristics that appeared to be significantly related to class membership were included in a multivariable regression analysis, to assess their unique contribution to predict class membership. The analyses were performed with Latent Gold 5.0 (Vermunt and Magidson 2013), using all available data.

For the first measurement (T1) no missing data existed. Four participants dropped-out after the first measurement, but before the start of the actual SST, for these participants T2 and T3 data were not available. Additionally, at T2, two parents did not complete the ESTIA-TS and one did not complete the ESTIA-TS and the SSRS. At T3, for four additional participants, no data were available and two parents did not complete the ESTIA-TS and the SSRS.

A four-class model was selected and interpreted, as this model had the lowest BIC (see Table 3) and a minimal class size of 10%, with the smallest of the four classes representing 17.4% of the children.

The four classes represent four underlying subgroups of children with ASD with different levels and perceived difficulty of social skills as reported by parents at start and different patterns of response during and after SST. Table 4 and Fig. 1 present the information for interpreting and naming the four classes.

Class 2 is characterized by significantly lower scores on Vineland Socialization (age equivalent 5y2m) than the mean across all children before SST. The perceived difficulty of social skills as reported by parents was similar to the mean across all children before SST. Additionally, the children significantly improved in their social communicative skills after SST (SSRS: from percentile 2 to percentile 14 between T1 and T2, ES 1.15; and Vineland Socialization: improvement of 6–8 months between T1 and T2, to 5y8m/5y10m, ES 0.53) and their perceived difficulty as reported by parents significantly decreased after SST (ES – 0.71). The children showed no further change six months after SST. We named this class improvers (low starters, mean perceived difficulty; improvement), and it was the biggest class with 29.2% of the children.

Class 3 is characterized by significantly higher levels than the mean across all children before SST on social communicative skills (SSRS percentile 8 at T1; and Vineland Socialization age equivalent 6y6m) and significantly higher perceived difficulty of these skills, as reported by parents, than the mean across all children before SST. No significant change appeared in skills (SSRS from percentile 8 to percentile 18 between T1 and T2; Vineland Socialization: improvement of 4 months between T1 and T2, to 7y2m) or perceived difficulty as reported by parents, immediately and six months after training. We named this class stable class (high starters, perceived difficulty above the mean; no improvement), and it consisted of 24.8% of the children.

Class 4 is characterized by significantly lower levels of social communicative skills (SSRS below percentile 2 at T1; and Vineland Socialization age equivalent 4y6m) and significantly higher perceived difficulty of these skills, as reported by parents, than the mean across all children before SST. No significant change appeared in skills (SSRS: did not exceed below percentile 2 from T1 to T2; Vineland Socialization: improvement of 4 months between T1 and T2, to 4y10m) or perceived difficulty as reported by parents, immediately and six months after training. We named this class poor performers (low starters, perceived difficulty above the mean; no improvement), and it represented the smallest proportion with 17.4% of the children.

Of the four classes, the two classes with average or lower perceived difficulty of social skills as reported by parents (i.e., Classes 1 and 2) showed significant improvement of high to very high effect size during the SST. The other two classes, with higher perceived difficulty as reported by parents (i.e., Classes 3 and 4) showed no significant improvement during the SST, independent of the actual level of social communicative skills at start; yet the effect sizes of improvement during SST were small to medium, suggesting that this lack of significance may be due to a too small power. None of the four classes showed a clear change from T2-T3, as indicated by nonsignificant tests and generally very small effect sizes. See also Fig. 2.

In Table 5, we present the results of the univariable regression analyses predicting class membership based on each participant and intervention characteristic. The mean scores on these characteristics in each of the four classes is presented in supplementary Table 1. We here report the characteristics that showed a significant relationship with class membership. See also Fig. 3.

For age, ‘improvers’ were the youngest and differed significantly from ‘best performers’ and ‘poor performers’. On verbal ability, ‘best performers’ scored higher than the ‘stable class’ and ‘poor performers’. On ASD symptoms measured with the ADOS, ‘best performers’ showed the least severe symptoms and differed significantly from all other three classes. ‘Improvers’ showed the most severe symptoms, compared to all other classes. On ASD symptoms measured with the ADI-R, ‘poor performers’ showed the most severe symptoms and differed significantly from ‘best performers’ and ‘stable class’. ‘ADHD Inattention and Hyperactivity/impulsivity symptoms were most apparent among the ‘improvers’ and ‘poor performers’, compared to both other classes for Hyperactivity/impulsivity, and to best performers on Inattention. Social phobia symptoms were most present in the ‘stable class’ compared to all other classes. ‘Stable class’ also showed more total anxiety symptoms than ‘improvers’, as did ‘poor performers’ compared to ‘improvers’ and ‘best performers’. Gender was the only participant characteristic that was not significantly related to class membership.

On the intervention characteristic (parent and teacher involvement), the ‘poor performers’ participated more often in SST without parent and teacher involvement than in SST with parent and teacher involvement, compared to all other classes.

In the multivariable regression we included all participant and intervention characteristics except gender. All but ADHD inattention symptoms, appeared to be significant in the multivariable regression analysis and thus relate uniquely to class membership.

The discussion of the findings in our study indicates several strengths that contribute to the existing literature; from the manualized training with and without parent and teacher involvement, to the variety of participant characteristics that we could include in the analyses. However, the current study also had several limitations.

First, for the type of analyses, the sample size of the children included is relatively small, which limits the power to detect the role of all participant and treatment characteristics.This implies that further studies using a similar design are needed to corroborate the current findings. Second, we considered a relatively high functioning sample, both with respect to ASD symptom level and to verbal cognitive level. Although the sample represents children diagnosed with ASD (based on DSM-IV-TR criteria) who were referred for SST in clinical practice, their clinical diagnosis had not been corroborated by standardized instruments, such as the ADOS or ADI-R. Additionally, the majority of children (66%) received a clinical diagnosis of PDD-NOS. This impacts the generalizability of the findings as the results of the current study may perhaps be more applicable to children in clinical practice with autism traits and social skills impairments rather than a strict ASD diagnosis as defined in the DSM-5 criteria. Third, we could only include parent reported measures, completed by parents who were only blind to condition at pretest, which may have led to bias, since they knew in which condition their child had participated after training. We did not include objective measures, completed by independent raters. Adding blinded outcome measures by someone not involved in the training (unlike parents) is currently an important focus of research, in order to increase the highly needed insight into the objective effect of SST or other interventions for children with ASD. However, none was available at the time the current study started. For this aim we had developed a measure of naturalistic behavior in school, observed by an independent rater (Dekker et al. 2016), yet questions have risen on the feasibility during the process of data collection. Other measures are for younger children (e.g. the Brief Observation of Social Communication Change; Grzadzinski et al. 2016). The current study still is informative and adds to the meta-analysis of Gates et al. 2017, as they focused on the relation between participant characteristics and self reported improvement. Additionally, we used the ESTIA-TS to measure perceived difficulty of social performance as reported by parents, a simple, yet non-validated instrument. Fourth, we could not investigate medication as a participant characteristic, or duration and intensity as intervention characteristics. We did not have data on the exact medication at baseline of the children. Additionally, all children participated in the same 15 sessions of 90 min (22.5 h of duration), in the same weekly pattern. Our duration and the intensity belong to the less effective end according to the literature (Wolstencroft et al. 2018). Last, the concept measured as the outcome in the current study is closer to social performance than social competence. Social performance is a narrower concept, yet it is in line with the aims of the training, i.e. applying social skills in role play and real life.

The current study provides insight into differences between subgroups of children with ASD in their response to SST, even within a relatively homogeneous sample, thereby specifying the effects found on group level. By doing so the current study contributes to the desired development towards personalizing SST for children with ASD. As improvement was not related to level of social communicative skills at start of the intervention but showed to be related much more to perceived difficulty of social communicative skills as reported by parents at start, the subgroups with higher perceived difficulties and anxiety may benefit more from an SST that focuses explicitly on perceived difficulty and anxiety in interaction. Additionally, subgroups with lower verbal levels probably benefit more from a training that addresses learning of social communicative skills through other means than mainly verbally. Last, explicit parent and teacher involvement in SST only seems to enhance the outcome of training for children with low levels of social communicative skills before training. Adapting training from the perspectives of specific needs of children with ASD may add to the benefit of SST. However, the current study concerned only one training (with and without parent and teacher involvement) and a specific sample. Replication in other samples and with other training approaches is needed, refining our understanding of what works for whom in SST, in order to reach the ultimate goal of personalizing social skills treatment for children with ASD.