Assessment of anxiety in children and adolescents: A comparative study on the validity and reliability of the Spence Children's Anxiety Scale in children and adolescents with anxiety and Autism Spectrum Disorder

Highlights

• The factor structure of the SCAS-P is not identical in groups of young people with anxiety and ASD.

• The relationship between SCAS-P items and factors differs across anxious and ASD groups.

• ASD impacted on three of the six factors and eleven of the thirty-eight items of the SCAS-P.

• Results on the SCAS-P in children with ASD need to be interpreted with caution.

Abstract

Objective

The present study assessed the utility of the Spence Children's Anxiety Scale - Parent Form (SCAS-P) across parents of children with (i) anxiety and (ii) Autism Spectrum Disorder (ASD).

Method

Parents of children aged 7–18 years with anxiety or ASD completed the SCAS-P. Multiple indicator multiple cause (MIMIC) structural equation modelling was utilized to analyse the data.

Results

Analysis revealed different factor structures between the Anxious and ASD groups and evidence for measurement variance across groups in some parts of the SCAS-P.

Conclusion

Results on the SCAS-P in children with ASD need to be interpreted with caution. Some SCAS-P items cannot be interpreted in the same way in an ASD population compared to neurotypical children with anxiety.

Introduction

Young people with Autism Spectrum Disorder (ASD) commonly experience other developmental, emotional or behavioural difficulties (Roberts et al., 2016). Anxiety disorders are the most common psychiatric comorbidities seen in ASD (White et al., 2009), affecting about 40% of children and adolescents with ASD (van Steensel et al., 2011). MacNeil et al. (2009) reported that young people with ASD have higher levels of anxiety than typically developing children and comparable levels of anxiety to typically developing clinically anxious children.

The presentations of anxiety in young people with and without ASD share some common features, such as social fears that are characteristic of Social Anxiety Disorder (Settipani et al., 2012). However, other features are more unique to ASD, such as impaired social functioning and social confusion, theory of mind deficits, sensory sensitivities, and repetitive and restricted behaviours (Bellini, 2004, Bellini, 2006, Kerns et al., 2014, Ollendick and White, 2013). Individuals with ASD access, process and interpret emotions in an atypical fashion (e.g. Shalom et al., 2006, White et al., 2014), leaving them vulnerable to anxiety symptoms that result in impairment beyond that caused by the core symptoms of ASD (Ozsivadjian et al., 2012). Thus, young people with ASD may be predisposed to anxiety as a result of a range of ASD-specific factors.

The assessment of anxiety in ASD has relied mainly on measures originally developed and validated for use in typically developing populations (White et al., 2012). Current assessment tools used to assess anxiety in children and adolescents with ASD include diagnostic interviews such as the Anxiety Disorders Interview Schedule - Child and Parent Interview Schedules (ADIS-IV-C/P; Silverman and Albano, 1996) as well as parent and self-report measures such as the child- and parent- forms of the Multidimensional Anxiety Scale for Children (MASC; March et al., 1997) and the Spence Children's Anxiety Scale-Child and Parent Forms (SCAS-C/P; Spence, 1998). These measures are psychometrically sound; however, they are standardised for neurotypical children, posing a limitation to current assessment of anxiety in ASD.

The reliability and validity of anxiety measures in young people with ASD has been frequently questioned (Kaat and Lecavalier, 2015, Sterling et al., 2015; van Steensel et al., 2013). Amongst the anxiety measures assessed for suitability in the ASD population are the Multidimensional Anxiety Scale for Children (MASC; March et al., 1997), the Screen for Child Anxiety-Related Emotional Disorders (SCARED-71; Bodden et al., 2009), and the self- and parent-report Revised Child Anxiety and Depression Scale (RCADS; Chorpita et al., 2000). Findings regarding the validity and reliability of such measures in the ASD population are mixed.

As a result of challenges with self-report of anxiety in young people with ASD (Baron, 2006; Ghaziuddin et al., 2002; Green et al., 2000; Groden et al.,  2006), clinicians rely primarily on parent reports when assessing anxiety in this population. To date, little is known about the cross-diagnostic (Anxiety vs. ASD) robustness of most measures used when assessing anxiety in children and adolescents. Anxiety measures which take into consideration ASD symptomatology such as intolerance of uncertainty and sensory fears, have been developed only recently (e.g. Rodgers et al., 2016). Hence, anxiety measures originally developed for individuals without ASD, are still widely used to study anxiety in those with ASD (e.g. Clarke et al., 2016, Ooi et al., 2016).

The Spence Children's Anxiety Scale (SCAS-C; Spence, 1998) and its corresponding parent/care-giver version (SCAS-P; Spence, 1999) are amongst the measures that are routinely used in clinical and research practice as a measure of anxiety symptoms in young people with and without ASD (e.g. Chalfant et al., 2007, McConachie et al., 2014, Rodgers et al., 2012, Sung et al., 2011). The SCAS-P is a 38-item questionnaire completed by parents or care-givers to assess the severity of a range of anxiety symptoms in children ages 6–18. The SCAS-P was designed to assess children's symptoms of anxiety along the structure of the Diagnostic and Statistical Manual of Mental Disorders (4th ed., DSM-IV-TR American Psychiatric Association, 2000). Although the SCAS-P has been reported to have sound psychometric properties in typically developing children (Nauta et al., 2004), further examination of its use for both clinical and research purposes in the ASD population has been recommended (Grondhuis and Aman, 2012). Russell and Sofronoff (2004) noted significant differences in parent–child ratings on the SCAS in a clinical sample of children with Asperger Syndrome. More recently, May et al. (2015) assessed parent-child agreement using the SCAS where results indicated generally good correlations between parent and child self-reported SCAS symptoms for typically developing children but poor agreement in parent-child ASD dyads. Consistent with clinical observations, findings suggested that 8–13-year-old children with ASD may have difficulties accurately reporting their anxiety levels.

Recent studies (Jitlina et al., 2017 and Magiati et al., 2017) have shown that the original six-factor structure of the SCAS-P as suggested by several studies (e.g. Arendt et al., 2014, Ishikawa et al., 2014, Nauta et al., 2004;) was not a good fit based on confirmatory factor analysis (CFA) with large samples of young people with ASD. Suggestions for the lack of fit of the original structure of the SCAS-P in childhood ASD populations include characteristics of ASD that impact anxiety presentation and potential differences in the underlying conceptual and structural basis of the SCAS-P anxiety symptoms in ASD. To our knowledge, no replicable alternative structure for the SCAS-P has been proposed to date. It remains unclear whether the SCAS-P measures the same constructs in ASD as it does in typically developing clinically anxious children (without ASD). Moreover, the subsequent question of whether this instrument measures the construct in the same way, should also be addressed to enable valid comparisons of observed scores across groups to be made. Further investigation is required to enable confidence that the scale functions in the same way across clinical groups. In order to establish whether a given measure of a particular latent construct (such as anxiety) performs similarly across the groups, a test of measurement invariance is to be performed (Vandenberg and Lance, 2000). Only then can meaningful comparisons between groups be made as measurement invariance analysis indicates whether the instrument measures the same construct in the same way across different populations or groups (Millsap and Kwok, 2004).

To our knowledge, there has only been one very recent attempt at assessing measurement invariance using the SCAS-P to compare assessment of anxiety in separate clinical groups. Glod et al. (2017) conducted a study to determine the factor structure for the SCAS-P in a sample of young people with ASD and to compare it with the factor structure derived from a sample of clinically-anxious young people without ASD, and in the combined (anxious and ASD) sample to ensure adequate fit to consider invariance. Measurement invariance techniques were also planned to determine whether SCAS-P items function in the same way in children with ASD and anxious children without ASD, in order to establish whether cross-groups comparisons using the SCAS-P are appropriate and meaningful. Unfortunately, due to poor model fit and lack of an adequate baseline model for further between group model testing, it was not possible to perform measurement invariance analyses. Investigation of the factor structure of the SCAS-P in the anxious and ASD samples was instead pursued through exploratory factor analysis (EFA). Therefore, the present study is the first to assess the measurement invariance of the SCAS-P across a group of children with anxiety versus children with ASD.

A common limitation across studies that have assessed use of the SCAS-P in the ASD population is the identification of the specific factors or items within the SCAS-P that are performing differently across the anxious and ASD groups and where specifically these differences lie. In the present study, we aim to address this gap in the literature by assessing whether, based on parent report, items of the SCAS-P measure the same constructs (have the same factor structure) and if there is evidence of equivalent relationships to these constructs (equal factor loadings) in both subgroups. We assessed whether there are subgroup differences (Anxious versus ASD) that impede responding to the SCAS-P in comparable ways. Measurement invariance tests on the SCAS-P will shed light on the nature of group differences in child anxiety between children with and without ASD. When using the SCAS, some group variations are partially corrected by using sex- norms and cutoff scores (e.g. Albano and Krain, 2005). However, such norms and cutoff scores are not yet available for the ASD population. We believe that diagnostic group differences when measuring anxiety can be further revealed via strong invariance tests. That is, a parent of a child with ASD (with or without anxiety) will rate SCAS items, such as “My child worries that something awful will happen to someone in our family,” “My child can't seem to get bad or silly thoughts out of his / her head,” “My child worries what other people think of him/her,” at a different frequency compared to a parent of a child without ASD.

In addition to examining structure across groups, the present study tested whether the SCAS-P measures levels of anxiety similarly in children with anxiety versus children with ASD. Cross-group (Anxious, ASD) measurement invariance of anxiety symptoms based on the SCAS-P was evaluated. Invariance tests were used to compare anxiety, a latent construct that is not directly observable, across children and adolescents with anxiety and ASD based on parent report. Given our sample size, we elected to use multiple-indicators-multiple-causes (MIMIC) modelling to assess invariance across the Anxious and ASD groups when using the SCAS-P and more specifically highlight where such differences lie. MIMIC modelling has been used successfully in other studies to assess the differences in the factor structure of latent variables such as depression across two different groups of people (e.g. Skule et al., 2014). We hypothesised that anxiety presents differently across these two groups.