Anxiety and depressive disorders are highly prevalent among youths (Compas 1997; Kashani and Orvaschel 1990), afflicting 10 to 20% of children and adolescents at some point in their development (Costello and Angold 1995). Because these disorders are not only common but also associated with significant functional impairment (Bell-Dolan and Wessler 1994; Langley et al. 2004) and poor outcomes (e.g., Compton et al. 2002), their accurate diagnostic assessment is an important issue. Numerous measures have been designed for assessment of child and adolescent anxiety and depression. Among these are such instruments as the Spielberger State-Trait Anxiety Inventory for Children (Spielberger 1973), the Revised Children’s Manifest Anxiety Scale (Reynolds and Richmond 1978), the Fear Survey Schedule for Children-Revised (Ollendick 1983), the Children’s Depression Inventory (Kovacs 1981) and the Youth Self Report (Achenbach 1991). Although psychometric studies have demonstrated the utility of these instruments in multiple contexts (Ollendick 1983; Reynolds 1982; Reynolds and Paget 1983; Saylor et al. 1984), the usefulness of these instruments in a diagnostic context may be limited due to their scale development preceding the establishment of contemporary diagnostic criteria (DSM-IV-TR; APA 2000). That is, most of these instruments do not map neatly onto DSM nosology (Perrin and Last 1992), and not surprisingly, some researchers have found poor discriminant validity for several of the most widely used child self-report measures of anxiety and depression (Lonigan et al. 1994; Stark and Laurent 2001). Consequently, recent efforts have sought to supplement existing dimensional and trait measures of anxiety and depression so as to evaluate instruments that are more concordant with current DSM nosology and that have scales that correspond more closely with specific dimensions of psychiatric disorders (e.g., Birmaher et al. 1997; Muris et al. 2001). The Revised Child Anxiety and Depression Scale (RCADS), a revision of the Spence Children’s Anxiety Scale (SCAS; Spence 1997), is one such measure, providing scales that index the main features of five prominent DSM-IV anxiety disorders (Separation Anxiety Disorder, Social Phobia, Generalized Anxiety Disorder, Obsessive-Compulsive Disorder, Panic Disorder) as well as Major Depressive Disorder. Studies in both community and clinical samples have demonstrated strong psychometric properties for the RCADS as a favorable youth self-report measure of major depression and its selected DSM-IV anxiety disorders (Chorpita et al. 2005; Chorpita et al. 2000).
Although the child-reported RCADS MDD and anxiety subscales have demonstrated promising utility with respect to diagnostic assessment of internalizing mood and anxiety problems, the broader literature on psychological measurement highlights the value of additional informants (Barbosa et al. 2002)—and particularly parents—during youth clinical assessment. Although low concordance has been historically found between youth and parent reports (Achenbach et al. 1987), researchers have suggested that the disagreement between parent and child report should be viewed as different yet equally valid perceptions of youths’ problems (Kazdin et al. 1983). More recently, De Los Reyes and Kazdin (2005) commented on the importance of integrating multi-informant reports as a part of youth assessments and proposed a theoretical framework (i.e., the Attribution Bias Context Model) to help guide the integration of information from these various sources (e.g., parental reports).
As parent reports are viewed as integral to youth assessments, the parent version of the RCADS (the RCADS-P) was developed to complement the well-supported child version and to include the same MDD and five anxiety disorder subscales. Notably, a few other parent-report measures exist that yield similar DSM-oriented internalizing scales. Namely, the SCARED-parent version (SCARED-P; Birmaher et al. 1997) and the SCAS-parent version (SCAS-P; Nauta et al. 2004) yield DSM-oriented anxiety disorder subscales based on parent reports of youths anxiety symptomatology. However, a key advantage of the RCADS-P is that it also incorporates an MDD scale in addition to its five anxiety scales—whereas both the SCARED-P the SCAS-P specially target only anxiety disorders. The inclusion of both depression and anxiety scales may be particularly important and useful in clinical settings given the high comorbidity of anxiety and depression in youth (Brady and Kendall 1992). The CBCL is another parent-report measure that recently added DSM-Oriented scales, including both a DSM-Oriented Anxiety Problems Scale and a DSM-Oriented Affective Problems Scale. Although the CBCL yields multiple DSM-Oriented scales that target not only anxiety and depression, but also externalizing disorders (i.e., oppositional and conduct problems), the CBCL DSM-Oriented Scales were not designed to map to specific DSM anxiety and affective disorders. That is, the DSM-Oriented Anxiety Problems Scale was designed to target the anxiety cluster of GAD, SAD and Specific Phobia, and the DSM-Oriented Affective Problems Scale was designed to target the affective cluster of MDD and Dysthymic Disorder (Achenbach et al. 2001).
The RCADS-P thus has the unique advantage of measuring dimensions corresponding to specific anxiety disorders (GAD, SAD, OCD, social phobia and panic disorder), as well as major depressive disorder. Although completely supplanting diagnostic interviews with self-report measures alone to determine diagnostic status has generally not been recommended (e.g., Hodges 1993; Sitarenios and Kovacs 1999), measures such as the RCADS-P offer the ability to provide suitable screens for additional assessment of specific youth internalizing problems as well as the potential to enhance time efficiency when used in conjunction with subsequently administered diagnostic interviews (e.g., Chorpita and Nakamura 2008). Questionnaires have become widely used in child psychiatry research (Brandenburg et al. 1990), and are seen as valuable in both research and clinical contexts due to their relatively quick and inexpensive way of collecting data in a standardized manner. As the psychometric properties of the RCADS-P have never been evaluated, however, the present study sought to examine the reliability and convergent, divergent, and discriminant validity of this instrument in a large, clinic-referred sample of children and adolescents.
Method
Participants
Youth in the present sample were 490 of 557 consecutively referred children and adolescents to two mental health clinics (the Center for Cognitive Behavioral Therapy, in Honolulu, Hawaii; and Judge Baker’s Children Hospital, in Boston, Massachusetts) to receive mental health diagnoses. Eligibility for youths in the present study included being 6–18 years old and having an available RCADS-P form filled out by their caregiver. Of the 557 consecutively referred youths, three children (0.5%) were younger than 6 years old, and 64 youths (11.5%) did not have an available RCADS-P (due to their caretakers choosing to not fill out the assessment measures). These 67 youths were thus not included in the study, leaving 490 eligible youths whose caregivers filled out the RCADS-P. To help ensure that all RCADS-P forms comprised valid responses, inclusion into the study also required RCADS-P forms having 90% or more completed data (i.e., fewer than 6 RCADS-P items missing). All available RCADS-P forms met this criterion (i.e., 83.5% of the 490 available RCADS-P forms had no missing data, 10.4% had only 1 missing item, and the remaining 6.1% had 2–5 missing items). All other forms in the study (i.e, RCADS, CBCL) also had to have 90% or more completed data to be included in analyses. Twenty-nine RCADS (5.9%) and twelve CBCL forms (2.4%) had more than 90% missing data and thus were excluded from analyses involving these measures. Information about the total number of diagnoses present in the sample (primary and anywhere in the diagnostic profile) appears in Table 1. Youth ages ranged from 6.55 to 18.97 years (M = 11.5, SD = 2.5). Additional youth and primary caregiver demographic information appears in Table 2. All children and parents were fluent in English.
Table 1
Number of Principal and Anywhere Diagnoses Among Study Participants (N = 490)
Diagnoses | Anywhere | Principal |
|---|---|---|
Major depressive disorder | 69 | 27 |
Dysthymic disorder | 33 | 11 |
Depressive Disorder NOS | 4 | 4 |
Panic disorder with agoraphobia | 1 | 0 |
Specific Phobia | 75 | 8 |
Generalized anxiety disorder | 75 | 24 |
Separation anxiety disorder | 74 | 26 |
Social phobia | 41 | 10 |
Obsessive-compulsive disorder | 18 | 7 |
PTSD/ASD | 22 | 8 |
Anxiety NOS | 1 | 1 |
ADHD-Combined Type | 112 | 33 |
ADHD- Predominantly Inattentive Type | 86 | 40 |
ADHD-Predominantly Hyperactive-Impulsive Type | 6 | 4 |
ADHD-NOS | 50 | 23 |
Oppositional defiant disorder | 176 | 118 |
Conduct disorder | 99 | 59 |
Disruptive behavior disorder NOS | 7 | 6 |
Bipolar | 4 | 1 |
Schizophrenia | 12 | 6 |
PDD | 0 | 0 |
Other | 41 | 21 |
No Diagnosis | 68 | 68 |
Table 2
Youth and Caregiver Demographic Information
n | Percentage | |
|---|---|---|
Youth Gender | ||
Male | 332 | 67.8 |
Female | 158 | 32.2 |
Youth Ethnicity | ||
Multiethnic | 191 | 39.0 |
White | 170 | 34.7 |
African American | 36 | 7.3 |
Asian American | 28 | 5.7 |
Latino/Hispanic | 25 | 5.1 |
Other | 27 | 5.5 |
Missing | 13 | 2.7 |
Caregiver type | ||
Parents | 414 | 84.5 |
Grandparents | 33 | 6.7 |
Other | 15 | 3.1 |
Missing | 28 | 5.7 |
Caregiver marital status | ||
Married | 202 | 41.2 |
Divorced, separated, single, or widowed | 252 | 51.5 |
Missing | 36 | 7.3 |
Caregiver highest level of education | ||
No high school | 41 | 8.4 |
High school | 165 | 33.7 |
1 to 2 Year College | 131 | 26.7 |
4 Year College | 67 | 13.7 |
Graduate School | 37 | 7.5 |
Missing | 49 | 10.0 |
Family income | ||
$0–$19,000 | 106 | 21.6 |
$20,000–$39,000 | 142 | 29.0 |
$40,000–$59,000 | 65 | 13.3 |
$60,000–$79,000 | 53 | 10.8 |
$80,000–$99,000 | 29 | 5.9 |
$100,000 or more | 51 | 10.4 |
Missing | 44 | 9.0 |
Measures
Child Behavior Checklist for Ages 6–18 (CBCL/6-18; Achenbach and Rescorla 2001)
The 120 items on the CBCL are rated as Not True (0), Somewhat or Sometimes True (1), or Very True or Often True (2). Items are summed to yield (a) Competence and Adaptive Scale scores, (b) eight Syndrome Scale scores, (c) six DSM-Oriented Scale scores, and (d) Total Problems Scale scores (including Internalizing and Externalizing Total Scores). Validity and reliability of the Syndrome and DSM-Oriented Scales have been documented (Achenbach and Rescorla 2001; Achenbach et al. 2003), and extensive normative data are available for children ranging from 6 to 18. All analyses in the present study were conducted using raw CBCL scale scores, as Achenbach and Rescorla’s ASEBA (2001) manual recommends using raw scores in order to account for the full range of variation.
The CBCL DSM-Oriented Scales (i.e., Affective Problems, Anxiety Problems, Attention/Deficit/Hyperactivity Problems, Conduct Problems, Oppositional Defiant Problems, and Somatic Problems) were recently constructed through clinician agreement among 22 highly experienced child psychiatrists and psychologists on the extent to which the CBCL/6-18 items represented each of these DSM-oriented problem areas in accordance with current DSM nosology (Achenbach et al. 2001). In addition to the Syndrome Scales, the DSM-Oriented Scales were also included as criterion measures of their targeted constructs in the present study.
The P-ChIPS is a semi-structured interview designed to be administered to the parents of youths (aged 6–18 years) by trained interviewers. The interviews screen for 20 different mood, anxiety, behavioral, substance, eating, and elimination Axis I disorders as well as psychosocial stressors, and are based on the DSM-IV classification criteria. Content and concurrent validity, as well as interrater reliability of the P-ChIPS have been demonstrated in previous studies. Fristad et al. (1998) showed that the P-ChIPS demonstrated moderate levels of agreement with clinician diagnoses (mean kappa = 0.49). Across diagnostic categories, sensitivity averaged 87% and specificity averaged 76%, suggesting that the P-ChIPS is similar to other structured interviews administered to parents regarding the accuracy of both positive and negative findings (Fristad et al. 1998).
Revised Child Anxiety and Depression Scales, child and parent versions (RCADS/ RCADS-P; Chorpita et al. 2000)
The RCADS and RCADS-P are each 47-item questionnaires designed to assess for the same DSM-IV depression and anxiety disorders in children and adolescents. The RCADS and RCADS-P are composed of six subscales: Separation Anxiety Disorder, Social Phobia, Generalized Anxiety Disorder, Obsessive-Compulsive Disorder, Panic Disorder, and Major Depressive Disorder, and also yields an Anxiety Total Score (sum of all five anxiety scales) and Total Score (sum of all six subscales). The RCADS and RCADS-P items were adapted from previous measures as well as evaluation of DSM-IV diagnostic criteria. A majority of items come from the SCAS (Spence 1997), which itself generated items from a combination of literature review, existing measures, structured interviews, and DSM diagnostic criteria. Additional items on the RCADS and RCADS-P were adapted from questionnaires related to GAD criteria, pathological worry, and depression (Chorpita et al. 1997; Reynolds and Richmond 1978; Kovacs 1981). The RCADS and RCADS-P measures ask youths and their parents to rate items according to how often each applies to the child. Responses range from 0–3, corresponding to “never”, “sometimes”, “often”, and “always”. The RCADS has been shown to have good internal consistency, high convergent and discriminant validity, and an adequate factor structure in both community and clinical samples of children and adolescents aged seven to seventeen (Chorpita et al. 2000; Chorpita et al. 2005). The RCADS-P instructions and items are the same as those of the RCADS, with the wording of RCADS-P item stems modified to suit parent informants (e.g., “I worry about things” modified to “My child worries about things”). The RCADS-P psychometric properties are still under empirical investigation.1
Procedure
At the start of the initial assessment visit, all participants and their legal guardians underwent standardized Institutional Review Board-approved notice of privacy and consent procedures prior to any data collection. Both child and primary caregivers completed questionnaires (including the RCADS, RCADS-P and CBCL) in English and parents participated in separate structured diagnostic interviews (i.e., the P-ChIPS). Assessors consisted of Ph.D. level clinical child psychologists and senior doctoral students in clinical psychology. Although inter-rater reliability data of these structured interviews were not gathered, assessors in the present study were trained to reliability using the P-ChIPS. Becoming trained to reliability involved (a) observation of three P-ChIPS interviews conducted by trained assessors, (b) conducting a series of five P-ChIPS interviews while being observed by a criterion-trained assessor, and (c) matching the experienced assessor on all clinical diagnoses in three of the five interviews. Expert diagnosticians also provided supervision and feedback to assessors when making diagnoses. A small portion of diagnoses were for problem areas not assessed by the P-ChIPS diagnostic interviews (e.g., trichotillomania, body dysmorphic disorder). In order to make such diagnostic determinations, assessors acquired information from the parents regarding these problems areas according to DSM diagnostic criteria (APA 2000) and also discussed diagnostic formulation with available supervisors and expert consultants.
Data preparation
Although missing data levels were low in the present sample, missing data on all scales were handled using the Missing Value Analysis (MVA) module of SPSS 15.0 (SPSS 2006). The SPSS MVA module examines missing data patterns and imputes missing values for continuous (missing at random) variables through a maximum likelihood method based on expectation-maximization algorithms (Little and Rubin 1987). Each scale used in the present study was calculated only if it had fewer than 20% of its scale items missing. Twenty percent instead of 10% was used as the cut-off for inclusion to allow scales with low item counts (e.g., CBCL DSM-Oriented Anxiety Problems Scale, DSM-Oriented Oppositional Problems Scale) to have one item missing and still be calculated. Few scales scores were excluded due to missing data. The number of scales excluded from analyses due to having more than 20% missing items ranged from 0 to 9 (0.0–1.8%) for the RCADS-P Scales; from 3 to 6 (0.6–1.3%) for the RCADS Scales; and from 0 to 2 (0.0–0.4%) for the CBCL Scales.
Data Analytic Approach
A confirmatory factor analysis (CFA), using LISREL 8.8 (Jöreskog and Sörbom 2006), was used to assess how well the 47 items comprising the RCADS-P fit the hypothesized six-factor structure. A confirmatory approach was used because previous research found support for the hypothesized structure with the RCADS (Chorpita et al. 2005). Alternative factor structures to the hypothesized six-factor structure were also examined. Notably, a five-factor structure was examined, combining GAD and MDD into a single factor, given recent research suggesting that GAD and MDD together constitute a single “distress” factor (e.g., Lahey et al. 2008; Watson 2005) as opposed to two separate constructs.
The fit of these factor structures was evaluated via various fit indices as well as through the interpretation of modification indices. Fit indices included the Comparative Fit Index (CFI; Bentler 1990), Akaike Information Criterion (AIC; Akaike 1987), and the Root Mean Square Error of Approximation (RMSEA; Steiger 1990). For the RMSEA index, values of 0.08 or lower indicate adequate fit, and RMSEA values of 0.05 or lower indicate excellent fit (Browne and Cudeck 1993). For the CFI, scores of 0.90 and above conventionally represent good model fit (e.g., Jöreskog et al. 2000). Modification indices indicate the presence of model parameters that would improve the model’s fit if freed. Since modification indices can be viewed as χ2 statistics with 1 df, an identified modification index of 10.83 or greater (indicative of a significant χ2 statistic with 1 df, p < 0.001) suggests that the overall fit of the model could be significantly improved if the fixed or constrained parameter was freely estimated. However, before a suggested path (from an identified item to a suggested factor) was allowed to be added to the original six-factor model to improve model fit, the suggested path also had to be associated with a factor loading that was equal to or greater than the factor loading of the original path specified, indicating a suggested path that is superior to the path specified by the original model.
The reliability of the RCADS-P was next evaluated through scale consistency for the RCADS-P Total Score as well as each of its subscales. To assess the validity of the RCADS-P, four main questions were asked with increasing levels of specificity related to the targeted constructs of the RCADS-P. In order from least to most specific, these four questions were: (a) Does the RCADS-P Total Score provide an index of general internalizing problems (e.g., negative affectivity)? (b) Does the RCADS-P Total Score provide a measure of anxiety and depression specifically, apart from other internalizing problems? (c) Do the RCADS-P Total Anxiety Scale and the RCADS-P MDD Scale provide measures specifically of anxiety and depression, respectively, and (d) Do the RCADS-P subscales (i.e., GAD, SAD, OCD, SOC, PD, and MDD) provide measures of the specifically targeted anxiety disorders as well as of Major Depressive Disorder? If so, the RCADS-P may prove to not only serve as a screen for general negative affectivity and internalizing problems, but may also help inform clinical diagnoses of these specific disorders.
For tests evaluating divergent validity, it is notable that the divergent validity criteria (e.g., CBCL externalizing scales) were often not orthogonal to the constructs targeted by the RCADS-P depression and anxiety scales. Internalizing and externalizing scales have also been found to be moderately correlated in previous studies (e.g., Achenbach et al. 2001; Goodman and Scott 1999). As a result, we did not expect divergent validity coefficients (i.e., correlations of RCADS-P internalizing scales with externalizing criterion scales) to be zero. Instead, divergence was evaluated in two specific ways. First, when significant and positive correlations were expected to emerge between an RCADS-P scale and a divergent criterion measure, Fisher’s z-tests (Meng et al. 1992) were employed to determine whether that correlation was significantly smaller than the correlation between the RCADS-P scale and its convergent criterion measure. Second, zero-order calculations of divergent validity correlations were supplemented with separate tests controlling for non-target variance in the divergent criteria. For example, before subjecting the CBCL Externalizing Scale to divergent validity tests, the divergent validity criteria were regressed on the CBCL Internalizing Total Score. The residuals of this regression represent “internalizing-free” criterion scores. These residuals were then used as “adjusted” divergent validity criteria. It was predicted that correlations of the RCADS-P Scales with these “adjusted” divergent validity criteria would not be positive and significant.
To assess the discriminative and classification properties of the RCADS-P subscales, between group analyses (i.e., one-way ANOVAs) as well as receiver operating characteristic (ROC) analyses were performed using parent-based diagnoses derived from the P-ChIPS as the criterion for diagnostic status. These diagnoses were determined independently of the RCADS-P information and other assessment measures (i.e., RCADS, CBCL). These analyses examined the utility of each RCADS-P subscale to differentiate (a) youths receiving the targeted diagnosis of the scale from youths not receiving that diagnosis (e.g., differentiate youths receiving a diagnosis of GAD from youths not receiving a diagnosis of GAD, when evaluating the GAD Scale); (b) youths receiving the targeted diagnosis of each of the anxiety scales from youths with any mood diagnosis (i.e., MDD, dysthymic disorder, depressive disorder not otherwise specified) but without that targeted anxiety diagnosis (e.g., differentiating youths receiving a diagnosis of GAD from youths receiving any mood diagnosis [but no GAD], when evaluating the GAD Scale); (c) youths receiving diagnoses of MDD from youths receiving any anxiety diagnosis (with no MDD); (d) youths receiving diagnoses of MDD from youths receiving any non-MDD mood diagnoses; (e) youths receiving any mood diagnosis from youths receiving any anxiety diagnosis (with no mood diagnosis); and (f) youths receiving the targeted anxiety diagnoses from youths receiving any other anxiety diagnosis (e.g., differentiating youths receiving a diagnosis of GAD from youths receiving a diagnosis of SAD, SOC, OCD, PD, PTSD and/or Specific Phobia [but no GAD], when evaluating the GAD Scale). The RCADS-P PD Scale could not be included in this set of discriminative analyses due to a lack of youths in the present sample with this diagnosis (n = 1). Due to the number of analyses conducted (i.e., 17 ANOVAs), the significance level was set at p < 0.001 to reduce type-1 error rates.
Results
Factorial Validity
Fit statistics from the CFA conducted on the full sample2 appear in Table 3 and represent adequate model fit for the six-factor model. All factor loadings were statistically significant (p < 0.05). Loadings ranged from 0.49 to 0.73 (Separation Anxiety factor), 0.54 to 0.80 (Social Anxiety factor), 0.59 to 0.82 (Obsessive–compulsive factor), 0.41 to 0.73 (Panic factor), 0.61 to 0.84 (Generalized Anxiety factor), and 0.38 to 0.74 (Depression factor). The maximum modification index identified in the Lambda-X (factor loading) matrix for the six-factor solution was 61.90, suggesting that a path could be added from the item “I worry about things” to the Social Anxiety latent factor, to significantly improve model fit. However, the completely standardized expected change associated with this fixed path indicated that its estimation would result in a factor loading of 0.43 on the Social Anxiety factor, which is considerably less than the item’s original completely standardized loading on the GAD factor (0.70). As a result, this suggested path was not added. Other modification indices greater than 10.83 were identified; however, as with the case above, each suggested path resulted in completely standardized estimated factor loading on the suggested factor that was less than item’s completely standardized loading on the originally specified factor. As a result, no additional paths were added to the original six-factor solution.
Table 3
Fit Statistics for the Confirmatory Factor Analytic Models
Difference from 6 Factor | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
Model | χ2
| df | p | GFI | RMSEA | SRMR | CFI | AIC | χ2
| df |
6 Factor | ||||||||||
(MDD, GAD, SOC, SAD, OCD, PD) | 3768.14 | 1019 | <0.001 | 0.74 | 0.079 | 0.071 | 0.94 | 4349.85 | ||
5 Factor | ||||||||||
(MDD/GAD, SOC, SAD,OCD, PD) | 4375.92 | 1024 | <0.001 | 0.68 | 0.093 | 0.075 | 0.93 | 5548.02 | 607.78* | 5 |
2 Factor | ||||||||||
(Anxiety/Depression) | 5391.70 | 1033 | <0.001 | 0.62 | 0.110 | 0.083 | 0.91 | 7202.20 | 1623.56 * | 14 |
1 Factor | ||||||||||
(General Negative Affectivity) | 5935.89 | 1034 | <0.001 | 0.59 | 0.120 | 0.086 | 0.90 | 8207.66 | 2167.75* | 15 |
The six-factor solution was tested against an alternative single factor model (a general negative affectivity factor), and a two-factor model (separate anxiety and depression factors), collapsing the five anxiety scales into a single anxiety factor. Also, given that researchers have recently suggested that MDD and GAD may in fact not fall into distinct constructs (Krueger 1999; Lahey et al. 2008; Watson 2005), the original six-factor model was tested against a five–factor model, collapsing MDD and GAD into a single MDD/GAD “distress” factor. The fit statistics for all three competing models appear in Table 3 and represent significantly degraded model fit compared to the original six-factor model, lending support to the hypothesized six-factor structure of the RCADS-P, as well as the distinction made between MDD and GAD. The correlations between factors were also calculated to examine the degree of covariation between GAD and MDD. The Phi (latent) and observed correlations between GAD and MDD were 0.57 and 0.56, respectively. The Phi and observed correlations between the other anxiety disorders (i.e., SAD, SOC, OCD, PD) and MDD ranged from 0.42 to 0.62 and from 0.39 to 0.56, respectively. These correlations among factors indicate that the GAD scale is not more closely related to the MDD scale than are the other anxiety scales. Consistent with the CFA findings above, these results also suggest that it is not warranted to collapse the MDD and GAD scales into a single MDD/GAD “distress” factor.
Reliability and Validity
Internal Consistency
All RCADS-P scales were found to have good internal consistency (α MDD = 0.83; α SAD = 0.83; α SOC = 0.88; α GAD = 0.88 α PD = 0.81; α OCD = 0.84; α Anxiety Total = 0.94; α Total Score = 0.95).
Internalizing Problems
Although the present CFA did not evidence strong factorial support for the one-factor model (i.e., the Total Problems Scale) of the RCADS-P, further tests were conducted to determine if the RCADS-P Total Score may be used and interpreted as a general screening for internalizing problems—a potentially useful feature for clinical settings. Zero-order bivariate correlations of the RCADS-P Total Score with convergent and divergent validity criteria were thus examined. We hypothesized that the RCADS-P Total Score would converge (i.e., correlate significantly and positively) with the CBCL Internalizing Scale. In addition, we hypothesized that the RCADS-P Total Score would evidence divergence with criterion measures of externalizing problems (i.e., the CBCL Externalizing Scale, Rule-Breaking Behavior Syndrome Scale, Aggressive Behavior Syndrome Scale, DSM-Oriented Oppositional Defiant Problems Scale, and the DSM-Oriented Conduct Problems Scale). Regarding convergence, as predicted, we found that the RCADS-P Total Score correlated significantly and positively with the CBCL Internalizing Scale (r = 0.78, p < 0.01).
Divergence with the various divergent validity criteria were evidenced in the following two ways. First, the correlation of the RCADS-P Total Score with the CBCL Aggressive Behavior Scale (r = 0.30; the largest correlation of the RCADS-P Total Score with any of the externalizing divergent validity criterion measures) was significantly smaller than the correlation between the RCADS-P Total Score and the CBCL Internalizing Scale (i.e., r = 0.78; the convergent validity criterion). The difference between these correlations was significant (i.e., for the difference between rRCADS-P Total – CBCL Int and rRCADS-P Total –CBCL A/B, z (477) = 13.33, p < 0.001, two-tailed). Second, none of the correlations of the RCADS-P Total Score with any of the “adjusted” divergent validity criteria (correlations ranging from −0.05 to −0.12) were significant and positive. These results indicate that the RCADS-P can serve as a screen for general internalizing (anxiety and depressive) problems.
Anxiety and Depression
To evaluate whether the RCADS-P Total Score provides a measure of anxiety and depression, specifically, apart from other types of internalizing problems—and thus a screen for anxiety and depressive problems—we employed Fisher’s z-tests to compare various zero-order bivariate correlations. Specifically, given that the RCADS-P was designed to measure anxiety and depression, we predicted that the RCADS-P Total Score would be significantly more correlated with the CBCL Anxious/Depressed Syndrome Scale than with any other CBCL Syndrome Scales related to internalizing problems (i.e., CBCL Withdrawn/Depressed, Social Problems and Somatic Complaints Scales). As predicted, we found that the RCADS-P Total Score was significantly more correlated with the CBCL Anxious/Depressed Syndrome Scale than with the CBCL Withdrawn/Depressed, Social Problems and Somatic Complaints Syndrome Scale (i.e., for the difference between rRCADS-P Total –CBCL A/D and rRCADS-P Total –CBCL W/D, z (474) = 8.53, p < 0.001, two-tailed; for the difference between rRCADS-P Total –CBCL A/D and rRCADS-P Total –CBCL S/P, z (474) = 9.42, p < 0.001, two-tailed; for the difference between rRCADS-P Total –CBCL A/D and rRCADS-P Total –CBCL S/C, z (474) = 7.99, p < 0.001, two-tailed). These correlations appear in Table 4, top row, and suggest that the RCADS-P Total Score does measure anxiety and depression specifically, as opposed to other constructs within the broader internalizing dimension.
Table 4
Correlations of RCADS-P with Convergent and Divergent Validity Criteria (N = 474)
CBCL | ||||||
|---|---|---|---|---|---|---|
RCADS-P | A/D | W/D | S/P | S/C | Anxiety problems | Affective problems |
Total Score | 0.78 | 0.54 | 0.53 | 0.54 | 0.77 | 0.66 |
Total Anxiety | 0.75 | 0.46 | 0.46 | 0.48 | 0.76 | 0.56 |
MDD | 0.68 | 0.68 | 0.59 | 0.57 | 0.58 | 0.83 |
OCD | 0.40 | 0.25 | 0.26 | 0.27 | 0.40 | 0.32 |
GAD | 0.71 | 0.35 | 0.42 | 0.45 | 0.73 | 0.53 |
SAD | 0.59 | 0.28 | 0.39 | 0.40 | 0.69 | 0.42 |
SOC | 0.65 | 0.50 | 0.40 | 0.34 | 0.60 | 0.46 |
PD | 0.57 | 0.37 | 0.35 | 0.48 | 0.60 | 0.47 |
Total Anxiety and Depression
To evaluate whether the RCADS-P Total Anxiety Scale and MDD Scale represent measures specific to anxiety and depression, we evaluated zero-order bivariate correlations of the RCADS-P Total Anxiety Scale and RCADS-P MDD Scale with convergent and divergent validity criteria (i.e., the CBCL DSM-Oriented Anxiety Problems and Affective Problems Scale). However, given that these convergent and divergent validity criteria were not orthogonal (i.e., anxiety and depression are believed to correlate in nature; Brady and Kendall 1992) and the CBCL DSM-Oriented Anxiety Problems and Affective Problems Scale correlated at r = 0.61 in the present sample, we did not expect these divergent validity coefficients to be zero. Rather, we predicted that the RCADS-P Total Anxiety Problems Scale would correlate significantly and positively with the CBCL DSM-Oriented Affective Problems Scale, though significantly more with the CBCL DSM-Oriented Anxiety Problems Scale, as evidenced by a significant Fisher’s z-test. This prediction was confirmed [z (474) = 7.25, p < 0.001]. Similarly, with respect to the RCADS-P MDD Scale, we predicted that the RCADS-P MDD Scale would correlate significantly and positively with the CBCL DSM-Oriented Anxiety Problems Scale, though significantly more with the CBCL DSM-Oriented Affective Problems Scale, as evidenced by a significant Fisher’s z-test. This prediction was also confirmed [z (247) = 10.12, p < 0.001]. These results appear in Table 4, second and third rows, and suggest that the RCADS-P Total Anxiety Scale and MDD Scale do specifically target anxiety and depression, respectively.
We also conducted parallel analyses on the individual RCADS-P anxiety subscales. These correlations of the RCADS-P anxiety subscales with both the CBCL DSM-Oriented Anxiety Problems and Affective Problems Scale appear in Table 4. As seen with the RCADS-P Total Anxiety Scale, each of the RCADS-P anxiety subscales was significantly more correlated with the CBCL DSM-Oriented Anxiety Problems Scale than with the CBCL DSM-Oriented Affective Problems Scale (GAD: z (474) = 6.89, p < 0.001; SAD: z (474) = 8.49, p < 0.001; SOC: z (471) = 4.22, p < 0.001; OCD: z (472) = 2.14, p < 0.05; PD: z (465) = 3.87, p < 0.001). In summary, the results shown in Table 4 illustrate that the RCADS-P Anxiety Total Score and anxiety subscales more closely measure anxiety (as opposed to depression) and that the RCADS-P MDD subscale more closely measures depression (as opposed to anxiety).
Specific Anxiety Disorders and MDD
To evaluate whether the RCADS-P subscales provide measures of their specifically targeted anxiety disorders and Major Depressive Disorder, zero-order bivariate correlations were examined between the RCADS-P subscales and corresponding subscales of an established measure (i.e., the RCADS-child version) containing the same DSM-Oriented MDD and anxiety subscales. Although it was predicted that correlations between all corresponding subscales (e.g., the RCADS-P GAD and the RCADS GAD Scale) would be positive and significant, it is notable that these correlations are cross-informant (i.e., parent vs. child report). Given that children and parents typically show low agreement in the literature (e.g., r = 0.29; Meyer et al. 2001), significant, though relatively low correlations, were expected. As predicted, all correlations between corresponding RCADS-P and RCADS scales were relatively low, yet significant using a 99% confidence interval (i.e., rSAD = 0.40 , rSOC = 0.25, rMDD = 0.22, rGAD = 0.26 , rOCD = 0.23 , rPD = 0.16 , rANXIETY TOTAL = 0.27 , rTOTAL SCORE = 0.25 ).
Discriminant Validity
As shown in Table 5, all RCADS-P scales successfully discriminated youths with and without disorders corresponding to the RCADS-P subscales. Further, all examined RCADS-P anxiety subscales successfully discriminated youths with pertinent anxiety diagnoses from youths with any mood diagnoses, as well as from youths with other anxiety diagnoses. Similarly, the RCADS-P MDD Scale successfully discriminated youths with MDD diagnoses from all youths with any anxiety diagnosis (without a diagnosis of MDD), as well as youths with any mood diagnosis from youths with any anxiety diagnosis (without a mood diagnosis), as evidenced by significant ANOVAs. The RCADS-P MDD Scale, however, did not perform as strongly with respect to discriminating youths with MDD from youths with non-MDD mood disorders (i.e., dysthymic disorder, mood disorder not otherwise specified).
Table 5
RCADS-P Scale Means, Standard Deviations, and Area Under the Curve (AUC) Values for Children with and without Selected Diagnoses Anywhere in their Diagnostic Profile
RCADS-P scale | Target group/non-target group | M | SD | n | F | p | d |
|---|---|---|---|---|---|---|---|
SAD | SAD | 8.6 | 3.8 | 74 | |||
No SAD | 2.6 | 2.9 | 414 | 235.20 | <0.001 | 1.78 | |
Any Mood Disorder (no SAD) | 3.3 | 3.1 | 73 | 84.95 | <0.001 | 1.53 | |
All Other (non-SAD) Anxiety Disorders | 4.6 | 3.4 | 109 | 55.52 | <0.001 | 1.11 | |
GAD | GAD | 10.0 | 4.5 | 75 | |||
No GAD | 4.6 | 4.1 | 413 | 109.85 | <0.001 | 1.25 | |
Any Mood Disorder (no GAD) | 6.5 | 4.7 | 73 | 21.71 | <0.001 | 0.76 | |
All Other (non-GAD) Anxiety Disorders | 7.7 | 4.6 | 108 | 11.81 | <0.001 | 0.51 | |
OCD | OCD | 9.9 | 5.9 | 17 | |||
No OCD | 1.5 | 2.3 | 469 | 186.63 | <0.001 | 1.88 | |
Any Mood Disorder (no OCD) | 1.9 | 2.5 | 98 | 89.79 | <0.001 | 1.77 | |
All Other (non-OCD) Anxiety Disorders | 2.2 | 2.8 | 165 | 89.74 | <0.001 | 1.67 | |
SOC | SOC | 15.4 | 5.4 | 44 | |||
No SOC | 8.5 | 5.3 | 441 | 62.10 | <0.001 | 1.29 | |
Any Mood Disorder (no SOC) | 10.1 | 4.9 | 81 | 30.15 | <0.001 | 1.03 | |
All Other (non-SOC) Anxiety Disorders | 11.3 | 5.5 | 140 | 18.01 | <0.001 | 0.75 | |
MDD | MDD | 11.0 | 5.6 | 69 | |||
No MDD | 5.8 | 4.3 | 417 | 79.01 | <0.001 | 1.04 | |
Any Anxiety Disorders (no MDD) | 7.3 | 4.9 | 133 | 23.57 | <0.001 | 0.70 | |
All Other (non-MDD) Mood Disorders | 8.0 | 4.3 | 36 | 7.79 | 0.006 | 0.60 | |
MDD | Any Mood Disorders | 10.0 | 5.3 | 105 | |||
No Mood Disorders | 5.5 | 4.3 | 383 | 79.13 | <0.001 | 0.93 | |
Any Anxiety Disorders (no Mood) | 6.9 | 4.9 | 115 | 19.73 | <0.001 | 0.61 |
Receiver Operating Characteristic (ROC) Analyses
As diagnostic measures often aim to optimally balance both sensitivity and specificity, we identified “optimal” cut-offs for each scale that maximized sensitivity and specificity. Since there was only one youth with a diagnosis of panic disorder in the present sample, we did not conduct ROC analyses on the RCADS-P PD Scale. For the MDD Scale, a score of 7 appeared to optimize sensitivity and specificity, yielding a sensitivity of 0.67 and a specificity of 0.72, and an accurate classification percentage of 71.3% for the prediction of major depressive disorder. For the GAD Scale, a score of 6 appeared to optimize sensitivity and specificity, yielding a sensitivity of 0.83 and a specificity of 0.73, and an accurate classification percentage of 74.9% for the prediction of generalized anxiety disorder. For the SAD Scale, a score of 4 appeared to optimize sensitivity and specificity, yielding a sensitivity of 0.92 and a specificity of 0.73, and an accurate classification percentage of 75.6% for the prediction of separation anxiety disorder. For the OCD Scale, a score of 4 appeared to optimize sensitivity and specificity, yielding a sensitivity of 0.88 and a specificity of 0.86, and an accurate classification percentage of 85.4% for the prediction of obsessive-compulsive disorder. For the SOC Scale, a score of 12 appeared to optimize sensitivity and specificity, yielding a sensitivity of 0.71 and a specificity of 0.76, and an accurate classification percentage of 75.1% for the prediction of social phobia.
Discussion
The results of the current study demonstrated that the RCADS-P possesses favorable psychometric properties in a sample of clinic-referred children and adolescents. High internal consistency for all subscales, as well as high convergent, divergent, and discriminant validity with interview, youth-self-report, and parent-self-report measures of related constructs were evidenced, supporting the validity of the RCADS-P MDD and anxiety scales. Interestingly, CFA analyses and examination of factor correlations did not support collapsing MDD and GAD into a single MDD/GAD “distress” factor. Given recent findings that MDD and GAD appear to tap the same construct (e.g., see Watson 2005), more research is needed to clarify the relationship between these disorders. Based on the present findings, however, the RCADS-P MDD and GAD subscales appear to measure two separate—albeit related—constructs.
A notable feature of the RCADS-P psychometric properties evidenced in the present study is that all examined RCADS-P anxiety subscales not only demonstrated the ability to discriminate anxiety from non-anxiety disorders (e.g., anxiety from depression), but also the ability to discriminate within anxiety disorders (i.e., youths with the targeted anxiety diagnoses from youths with any other anxiety diagnoses). These discriminant validity results appear at least as strong, and in some cases stronger, than results from other parent-report measures that also contain DSM anxiety subscales (e.g., SCARED-R parent version; Muris et al. 2004). The RCADS-P MDD Scale, however, did not perform as well as the RCADS-P anxiety subscales in these contexts. Although the RCADS-P MDD Scale was predictive of MDD in the full sample, it did not perform as strongly with respect to discriminating MDD from other mood disorders. This may not be surprising, however, given that dysthymic disorder (the most represented non-MDD mood disorder in the present sample) shares several DSM diagnostic criteria with MDD (e.g., depressed mood, changes in appetite, sleep problems; APA 2000). Although the RCADS-P MDD Scale was less able to discriminate MDD from other mood disorders, this scale did evidence the ability to discriminate youths with MDD from youths with anxiety.
The RCADS-P thus appears to be a valuable measure for identifying youths with depression and specific anxiety disorders, and may be used as a screen to inform further assessment of these problem areas. The RCADS-P also has other clinically useful features worth noting. Not only is the RCADS-P relatively quick to administer (i.e., contains only 47 items), it is also available for free. Finding ways to assist in formulating reliable diagnoses without time-intensive (and thus expensive) structured interviews is needed given recent pressures from managed care to increase time-efficiency and cost effectiveness of services provided (e.g., Piotrowski 1999). Additional work, however, could make this measure even more clinically informative. For instance, the collection of normative data could enhance clinicians’ ability to make diagnostic inferences from RCADS-P scores.
Although the results of the present study are promising in their support of the six anxiety and depression scales of the RCADS-P, a few caveats are in order. First, the lack of sufficient number of youths with a diagnosis of panic disorder prevented the testing of the RCADS-P Panic Scale via discriminative ANOVA and ROC analyses. Future studies that have adequate sample size of youths with panic disorder should examine this scale. Future studies could investigate other clinically relevant questions. For example, given the need for more studies to derive empirically- and theoretically-based methods for incorporating child and parent reports (De Los Reyes and Kazdin 2005), future studies could examine whether the incremental validity of child and parent self-reports (e.g., RCADS and RCADS-P subscales) vary as a function of anxiety disorder type. For instance, the child-reported RCADS SAD Scale may provide little incremental predictive power of SAD diagnostic status beyond that of the parent-reported RCADS-P SAD Scale given the observable nature of SAD. Oppositely, the child-reported RCADS GAD Scale may provide significant incremental predictive power for GAD diagnostic status beyond that of the RCADS-P SAD Scale given the more internal and less observable nature of GAD. In a separate area, the utility of the RCADS-P may further be strengthened through the collection of data from children and adolescents in non-referred and community settings. Examination of the factor structure, internal consistency, test-retest reliability, and validity of the RCADS-P from parents of non-referred youths may inform the potential use of the RCADS-P as a screening measure in school settings. In summary, the performance of the RCADS-P in this study indicates that its use with clinical youth populations is warranted. Parents’ reports on this instrument gave significantly accurate and reliable accounts of their children’s internalizing states and problems. Whether it is used as a screening measure or as a diagnostic aid of depression and specific anxiety disorders in clinical settings, the RCADS-P may be used to provide valuable supplementary information on youth anxiety and depression to be considered alongside youths’ self-reports of these constructs.
Open Access
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Open AccessThis is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.