Introduction
Young children (0–6 years old) are at particularly high risk of exposure to potentially traumatic events (PTEs) such as abuse, neglect, violence, and accidents, with many young children experiencing more than one PTE (Lieberman, Van Horn, & Harris, 2011). Studies demonstrate high levels of internalizing and externalizing behaviors as well as posttraumatic stress symptoms (PTSS) of re-experiencing, avoidance, and hyperarousal in young children following trauma exposure (De Young & Landolt, 2018). Between 11.9% and 30.5% of trauma-exposed young children develop posttraumatic stress disorder (PTSD) (Woolgar et al., 2021). PTSD can have a debilitating impact on a young child’s functioning in different domains of daily life, including the relationship to their parents, siblings, and peers (American Psychiatry Association (APA), 2013). For instance, ongoing temper tantrums may disrupt play activities with other children in daycare and thus may hinder the development of first peer relationships. Further, previous work on PTEs in early childhood documented increasing odds for poor physical health (Flaherty et al., 2006; Kerker et al., 2015), future academic difficulties (Jimenez et al., 2016; McKelvey et al., 2018), and attachment insecurities (van Duin et al., 2018).
To prevent potential long-lasting adverse effects of exposure to PTEs into later childhood, adolescence and adulthood, early recognition of the sequelae of PTEs in young children is critical for timely referral for further assessment and trauma-focused interventions (Cohen & Scheeringa, 2009; Conradi et al., 2011). However, few empirically validated tools exist that are designed explicitly to screen for PTSD in young children. In search for a screening instrument for PTSS and PTSD in children, previous studies explored the Child Behavior Checklist (CBCL; Achenbach & Rescorla, 2000) as a useful measure. While some research efforts investigated whether established CBCL syndrome scales can successfully distinguish between children with and without PTSD (e.g., Saigh et al., 2002), others proposed specific sets of symptoms as CBCL-PTSD subscales (e.g., Ruggiero & McLeer 2000; Wolfe et al., 1989). Utilizing the CBCL or creating a CBCL-PTSD subscale can be seen as an efficient strategy for screening children at risk for PTSD. The CBCL is widely used as part of a standard assessment battery across various medical/clinical settings and child welfare or health services worldwide, who act as “gatekeepers” between families and mental health care specialists (Costello et al., 1988; Gudiño et al., 2012) imposed with the task to make aimed referrals to further assessment and specialized treatment. In these settings, professionals interact with children exposed to PTEs on an everyday basis.
Dehon & Scheeringa (2006) proposed a subset of 15 CBCL-PTSD items (referred to as CBCL-PTSD) embedded within the full measure (CBCL 1.5–5; Achenbach & Rescorla, 2000) as a developmentally appropriate PTSD screener for young children. Past results concerning the validity of the CBCL-PTSD by two studies (Dehon & Scheeringa, 2006; Loeb et al., 2011) were promising, though yielded inconsistent results with regard to sensitivity. Aiming to identify the best cutoff score, Dehon & Scheeringa (2006) considered a balance between sensitivity and specificity, as well as positive predictive and negative predictive values (C. Dehon, personal communication, September 15, 2021). Results revealed that the most appropriate cutoff score was 9. This cutoff score accurately classified 75% of young children who met the alternative algorithm diagnostic PTSD criteria (PTSD-AA; Scheeringa et al., 2003). However, the maximum sensitivity found by Loeb et al. (2011) was only 67% for the PTSD criteria of the Diagnostic Classification of Mental Health and Developmental Disorders of Infancy and Early Childhood (DC:0-3R; Zero to Three 2005). A retrospective study by Fraser et al. (2019) found a high detection rate for young children identified as needing further in-depth assessment or trauma-focused intervention at intake into mental health services using the Young Child PTSD Screen (YCPS; Scheeringa 2019) but not for the CBCL-PTSD, suggesting that the items of the CBCL-PTSD do not include the most salient trauma symptoms in trauma-exposed young children. To date, no efforts however have been undertaken to develop and evaluate an alternative CBCL-PTSD subscale in light of the past findings by Dehon & Scheeringa (2006), Loeb et al. (2011), and Fraser et al. (2019). While only a minority of young children exposed to trauma develop full DSM-5 PTSD, a clinically significant proportion suffer from subthreshold PTSD symptomatology followed by impairment in every-day functioning. A study by Scheeringa et al. (2005) indicates that two to three times as many trauma-exposed young children demonstrate functional impairment from PTSS as are meeting diagnostic criteria. Their study also showed that functional impairment associated with PTSS may not remit spontaneously, even when young children no longer meet the diagnostic criteria for PTSD. This underscores not only the need for early recognition of young children at risk for PTSD, but also the need to identify trauma-exposed young children at risk for functional impairment. Therefore, being able to use a CBCL-PTSD subscale also as a screener for PTSS-related functional impairment would be twice as valuable. However, to the best of our knowledge, no studies have yet investigated the utility of a CBCL-PTSD subscale as screening tool for functional impairment.
To address these gaps and extend upon the previous validation studies, the aim of the present study was threefold: (1) to examine the criterion validity of the CBCL-PTSD, (2) to develop alternative CBCL-PTSD subscales and examine their criterion validity, and (3) to examine the utility of all CBCL-PTSD subscales, i.e., the existing and alternative CBCL-PTSD subscales, for screening for functional impairment.
Method
Participants and procedure
Data on trauma-exposed young children in the present study were drawn and merged from two completed studies (De Young et al., 2016; Salloum et al., 2022). Inclusion criteria of the present study were that (a) young children must have met the age range for the CBCL 1.5–5; (b) caregivers must have been administered a clinical interview for PTSD based on DSM-5 diagnostic criteria for children six years and younger (PTSD ≤ 6), and (c) caregivers must have completed the CBCL 1.5–5. The first part of the present study’s sample consisted of 60 trauma-exposed young children recruited from community mental health nonprofit agencies, located in the southeast of the United States. Participating families were recruited for a randomized clinical trial examining the effectiveness of stepped care Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) for children ages 4 to 12 (Salloum et al., 2022). Data included in the present study were assessed at pre-treatment. Out of 60 young children ages 4 to 6, 20 (33.3%) participants were excluded due to being 6 years old. The final US sample comprised 40 trauma-exposed young children (Mage = 4.55 years, SDage = 0.50 years, rangeage = 4–5 years). A total of 20 (50.0%) young children experienced more than one PTE, and 31 (77.5%) young children experienced some type of interpersonal trauma. For information on the lead parent who took part in the assessments and intervention, see supplementary material (SM) 1. The study was approved by the Institutional Review Board of the University of South Florida. All assessment measures used in this study were administered before randomization to stepped care TF-CBT and standard TF-CBT. The clinical trial was registered at ClincialTrials.gov (NCT02537678). Written informed consent was obtained from all participating parents/guardians.
The second part of the presents study’s full sample comprised 133 young children exposed to unintentional injuries from the Coping with Accident Reaction (CARE) intervention randomized controlled trial (RCT) conducted in Switzerland and Australia (see Haag et al. (2020) for details on sample, procedures, and results). More specifically, participants screened as high-risk for developing PTSD at baseline two weeks after the accident, i.e., before randomization into control and intervention group, were included in the present study along with their data from the 3-months follow-up assessment. Young children screened as low-risk for developing PTSD were not included in the present study because no 3-months follow-up assessment was conducted within this group.
Out of 133 young children screened as high-risk, 76 young children (Mage = 2.84 years, SDage = 1.05 years, rangeage = 1.50–5.54 years) met the inclusion criteria for the present study. Out of the 76 young children, 40 (52.6%) young children were in the control group and 36 (47.4%) young children were in the intervention group (randomized). For information on the caregivers see SM2.
The multi-site CARE trial was registered with the Australian New Zealand Clinical Trials Registry (ACTRN12614000325606) and ClinicalTrials.gov (NCT02088814). Ethics approval was obtained at both study sites. Written informed consent was obtained from all participating parents. The final overall sample of the present study comprised 116 trauma-exposed young children with an average age of 3.42 years old (SD = 1.21). Information on gender, PTEs, and race/ethnicity are presented in Table 1.
Table 1
Descriptives on gender and overview of index PTEs and race/ ethnicity for the CARE, US, and overall sample
CARE sample (n = 76) | US sample (n = 40) | Overall sample (n = 116) | |
|---|---|---|---|
Gender (n(%)) | |||
Female | 34(44.7) | 23(57.5) | 57(49.1) |
Male | 42(55.3) | 17(42.5) | 59(50.9) |
Potentially traumatic events (n(%)) | |||
Accident | 2(5.0) | 2(1.7) | |
Animal attack | 1(2.5) | 1(0.9) | |
Man-made disasters | |||
Natural disasters | |||
Witness of serious injury/deatha | |||
Domestic violence/ witness of abuseb | 10(25) | 10(8.6) | |
Physical abuse | 4(10) | 4(3.5) | |
Sexual abuse | 16(40) | 16(13.8) | |
Accidental burning | 60(78.9) | 60(51.7) | |
Near drowning | |||
Hospitalization, invasive medical procedure | 16(21.05) | 1(2.5) | 17(14.7) |
Fracture | 12(15.8) | ||
Laceration | 2(2.6) | ||
Head injury | 2(2.6) | ||
Kidnappeda | |||
Learned of traumatic eventb | |||
Death of parent/relative | 4 (10) | 4(3.5) | |
Other | 2(5.0) | 2(1.7) | |
Witness of crime | 1(2.5) | ||
Verbal abuse | 1(2.5) | ||
Race/Ethnicity (n(%)) | |||
Anglo/Europeana | 70(92.1) | 70(60.3) | |
Whiteb | 20(50) | 20(17.2) | |
Black African Americanb | 12(30) | 12(10.3) | |
Asian | 2(2.6) | 2(1.7) | |
Mixed raceb | 7(17.5) | 7(6.0) | |
Hispanic or Latinob | 9(22.5) | 9(7.8) | |
Othera | 3(4.0) | 3(2.6) | |
Missing information | 1(1.3) | 1(2.5) | 2(1.7) |
Measures
Diagnostic infant and Preschool Assessment
PTEs, PTSD symptoms, and PTSD diagnosis were assessed using the PTSD module from the Diagnostic Infant and Preschool Assessment-Likert version (DIPA-L; Haag et al., 2014; Scheeringa, 2020). This semi-structured diagnostic interview is conducted with the primary caregiver of children 1–6 years old. The PTSD module of the DIPA strictly rests on the PTSD ≤ 6 diagnostic criteria of the DSM, 5th edition (APA, 2013), and all trauma events assessed fall under the A criterion of the PTSD criteria. Reliability and validity of this measure have been confirmed (Løkkegaard et al., 2019; Scheeringa, 2020). To obtain symptom severity ratings, the frequency and intensity of the 16 PTSD ≤ 6 symptoms are rated on a 5-point Likert scale. The total symptom severity score ranges from 0 to 128. A PTSD ≤ 6 symptom was endorsed in the CARE study if its frequency was rated ≥ 1 and its intensity ≥ 2. In the US study, the previous (i.e., dichotomous) DIPA version was used (Scheeringa, 2004). This version of the DIPA also followed the DSM-5 PTSD ≤ 6 conceptualization and has demonstrated acceptable test-retest reliability (Scheeringa & Haslett, 2010). Functional impairment was considered to be present, when young children showed impairment in at least one of five domains of daily functioning: with parents, with siblings, at daycare/school, with peers, and in public. Internal consistency of the DIPA PTSD ≤ 6 PTSD symptom scale (dichotomous items) was 0.88 in the present study.
CBCL-PTSD subscale by Dehon & Scheeringa (2006)
The CBCL-PTSD (Dehon & Scheeringa, 2006) comprises 15 items from the CBCL 1.5–5 version (Achenbach & Rescorla, 2000). The CBCL-PTSD items are presented in Table 2. The CBCL 1.5–5 is a standardized 100-item caregiver-report checklist to assess behavioral problems in children, demonstrating good psychometric properties (Achenbach & Rescorla, 2000). The items are rated on a 3-point Likert scale (0 = “not true” to 2 = “very true or often true”). Previous reports found adequate internal consistencies for the CBCL-PTSD, ranging from 0.79 to 0.83 (Dehon & Scheeringa, 2006; Loeb et al., 2011). Cronbach’s α of the CBCL-PTSD in the presents study’s sample was 0.87.
Table 2
Items of the CBCL-PTSD, the CBCL-PTSD-17, and the CBCL-PTSD-6
CBCL-PTSD (Dehon & Scheeringa, 2006) | CBCL-PTSD-17 (+ corresponding DSM-5 PTSD ≤ 6 symptoms) | CBCL-PTSD-6 | ||
|---|---|---|---|---|
5. Cannot concentrate or cannot pay attention for long | 5. Cannot concentrate or cannot pay attention for long* (D3: concentration problems) | 5. Cannot concentrate or cannot pay attention for long* | ||
10. Clings to adult or too dependent | 29. Easily frustrated (D2: irritability, angry outbursts, temper tantrums) | 10. Clings to adult or too dependent* | ||
15. Defiant | 32. Fears of certain animals, situations, or places* (C1: avoidance) | 48. Nightmares* | ||
32. Fears certain animals, situations, or places | 38. Has trouble getting to sleep (D1: Sleep problems) | 71. Loss of interest | ||
45. Nausea and feels sick | 43. Looks unhappy without good reason (C3: persistent negative emotional state & C6: reduction in expression of positive feelings) | 78. Stomachaches and cramps (without medical cause)* | ||
47. Nervous, high-strung, or tense | 44. Angry moods (D2: irritability, angry outbursts, temper tantrums) | 87. Too fearful or anxious* | ||
48. Nightmares | 47. Nervous, high-strung, or tense* (D-cluster related symptom of arousal and reactivity) | |||
78. Stomachaches and cramps (without medical cause) | 48. Nightmares* (B2: distressing dreams) | |||
81. Stubborn, sullen, or irritable | 51. Shows panic for no good reason (B4: psychological distress at reminders) | |||
82. Sudden changes in mood or feelings | 71. Loss of interest (C4: diminished interest in activities) | |||
87. Too fearful or anxious | 81. Stubborn, sullen, or irritable* (D2: irritability, angry outbursts, temper tantrums) | |||
90. Unhappy, sad, or depressed | 84. Talks or cries out in sleep (B2: distressing dreams) | |||
93. Vomiting and throwing up (without medical cause) | 85. Temper tantrums or hot temper (D2: irritability, angry outbursts, temper tantrums) | |||
94. Wakes up often at night | 87. Too fearful or anxious* (C3: increase in negative emotional states) | |||
98. Withdrawn or does not get involved with others | 90. Unhappy, sad, or depressed* (C3: increase in negative emotional states) | |||
94. Wakes up often at night* (D1: Sleep problems) | ||||
98. Withdrawn or does not get involved with others* (C5: social withdrawal) |
Development of alternative CBCL-PTSD subscales
CBCL-PTSD subscale based on expert rating. For the present study, an alternative CBCL-PTSD subscale (17 items), oriented on the DSM-5 PTSD ≤ 6 criteria, was developed by an expert panel comprised of nine experts in the field of trauma in young children. First, the experts were asked to rate whether each of the 100 CBCL items captured one of the 16 DSM-5 PTSD ≤ 6 symptoms or not. Answers were coded as “consistent with DSM-5 PTSD ≤ 6 criteria, “not consistent with DSM-5 PTSD ≤ 6 criteria”, and “not sure if consistent with DSM-5 PTSD ≤ 6 criteria”. After the first round, CBCL items were included in the alternative CBCL-PTSD subscale when seven out of nine experts stated that a given item captured a DSM-5 PTSD ≤ 6 symptom. In cases where one of the experts stated that a given CBCL item did not capture a PTSD symptom, experts who had answered with “not sure” where given the possibility to revise their decision. Consequently, items which received seven positive ratings and only one negative rating after the second step were also included in the alternative CBCL-PTSD subscale.
CBCL-PTSD subscale based on variable importance. Because in many cases not all predictors are equally important for prediction of a given outcome (Hastie et al., 2009), a second alternative CBCL-PTSD subscale (six items) was developed and validated. This subscale was based on variable importance, i.e., this subscale comprised a subset of the most important CBCL items for the prediction of PTSD included in the CBCL-PTSD and the first alternative CBCL-PTSD subscale. We used the online application SSVSforPsych (available at: https://ssvsforpsych.shinyapps.io/ssvsforpsych/) for stochastic search variable selection (SSVS), a prominent Bayesian variable-selection method that enables one to identify important predictors for further consideration. The online application was developed by Bainter et al. (2020). By answering the question if a given variable is a constant predictor of the outcome variable, accounting for uncertainty in the other predictors included in the model, SSVS provides information about the relative importance of predictors. In order to account for uncertainty in predictor sets and regression parameters, SSVS samples thousands of high probability Bayesian regression models made up of varying subsets of the predictors. Returned are the proportion of models that include each predictor. These proportion of how often each predictor was included in the sampled models are called Marginal Inclusion Probabilities (MIPs). The higher the MIP, the more often included was a predictor in the sampled regression models. In other words, the reliability that a given variable X is a predictor of the criterion Y is higher for variables with high MIPs. Markov chain Monte Carlo (MCMC) estimation is used to estimate the parameters of the regression models, which draws samples from the posterior distribution. In addition, a prior distribution for each regression parameter is specified (Bainter et al., 2020). A prior inclusion probability of 0.5 was recommended in case of the present study by one of the developers of the online-tool, considering the smaller sample size (S. Bainter, personal communication, August 13, 2021). This prior indicates the belief that each of the 23 CBCL items had a 50/50 probability of being included in the model. This value also reflects the prior belief that the model should include 0.5*23 (11.5) CBCL items. Further, 20,000 MCMC samples were generated, including 10,000 burn iterations to ensure convergence and stable results. For a more detailed description of SSVS, see Bainter et al. (2020).
Statistical analysis for validation of the CBCL-PTSD subscales
Following previous studies (Dehon & Scheeringa, 2006; Loeb et al., 2011), correlations between number of PTSD ≤ 6 PTSS and CBCL-PTSD subscale scores were calculated and t-tests and χ2 analysis were performed to examine whether children with and without PTSD ≤ 6 differed in age, gender, and CBCL internalizing scale scores, externalizing scale scores, and total scores, as well as CBCL-PTSD subscale scores. The same analyses were performed for functional impairment as the outcome variable. To assess the detection accuracy of the CBCL-PTSD subscales, receiver operating characteristic (ROC) analyses were performed against PTSD ≤ 6 diagnosis and functional impairment, using the pROC R-package (Robin et al., 2011). Sensitivity (Se) and specificity (Sp) combined for all possible cutoff scores are indicated by one ROC curve each. The area under the curve (AUC) was assessed for all subscales to examine diagnostic accuracy. The interpretation of the AUC values were as follows: ≥0.9 = “excellent”, ≥ 0.8 = “good”, ≥ 0.7 = “fair”, ≥ 0.6 = “poor”, and 0.5 = “recognition by chance” (Rice & Harris, 2005). The AUCs were then compared using a bootstrapping approach (10,000 bootstraps). Further, positive predictive value (PPV; the probability that young children with a positive screen result indeed have PTSD or show functional impairment) and negative predictive values (NPV; the probability that young children with a negative screen result indeed do not have PTSD or do not show functional impairment) were calculated for all possible cutoff points on the subscales. In a 2-stage screening approach for mental disorders, a screening tool should demonstrate higher sensitivity than specificity because over-identifying young children has no severe or harmful consequences than under-detecting children in need of help. It was therefore decided a priori to prioritize sensitivity over specificity. However, given the paucity of resources in mental health care services (McBain et al., 2019; O’Brien et al., 2016), we also decided a priori that the most appropriate cutoff score had to demonstrate a minimum specificity of ≥ 0.60. Further, it was decided a priori that a cutoff score of at least 3 must be scored, indicating the presence of at least two symptoms. A cutoff score of 1 or 2 could be achieved by parents mistakenly marking one symptom as endorsed due to misunderstanding or misperception of the item, which could result in a false positive.
Contrary to Dehon & Scheeringa (2006) and Loeb et al. (2011) we did not dichotomize the scoring of the CBCL items (i.e., 0 = “absence of symptom” and 1/2 = “presence of symptoms”), but used the original continuous subscale scores to have subscales which can be used by clinicians in the easiest and most practicable fashion. Data were analyzed using R (R CoreTeam, 2014).
The CBCL items in the US sample contained no missing data. Overall, 0.72% of data were missing on CBCL item-level in the CARE sample. According to Little’s MCAR test (χ2 = 3289.650, df = 5582, p = 1.000), the data of the CARE sample were assumed to be missing completely at random and missing values for each subscale were imputed using Expectation-Maximization in SPSS (IBM Corp., 2013). Imputed item values < 0 or > 2 were set to exactly 0 or exactly 2, respectively.
Results
On average, young children exhibited 4.43 PTSD ≤ 6 symptoms (SD = 4.16). Out of 116 young children, 41 (35.3%) met the PTSD ≤ 6 criteria and 52 (44.8%) met the functional impairment criteria. Results indicate that PTSD and the presence of functional impairment did not significantly differ with regards to gender (PTSD: χ2 (1, N = 116) = 0.28, p = .60; functional impairment: χ2 (1, N = 116) = 1.21, p = .27), but was different across age and all CBCL (sub-) scales investigated (see Table 3).
Table 3
Descriptives of age and various CBCL (sub-)scales and comparison across young children with and without PTSD and functional impairment
PTSD | Functional Impairment | ||||||||
|---|---|---|---|---|---|---|---|---|---|
No PTSD (n = 75) M (SD) | PTSD (n = 41) M (SD) |
t
| Cohen‘s d
[95%CI]
| No functional Impairment (n = 64) M (SD) | Functional Impairment (n = 52) M (SD) |
t
| Cohen‘s d
[95%CI]
| ||
Age | 2.88 (1.03) | 4.43 (0.84) | -8.78*** | -1.65 [-2.09; -1.21] | 2.91 (1.01) | 4.06 (1.15) | -5.73*** | -1.07 [-1.46; -0.68] | |
Internalizing CBCL subscale score | 5.58 (5.75) | 21.79 (11.55) | -8.44*** | 1.78 [-2.31; -1.23] | 4.54 (4.69) | 19.64 (11.56) | -8.85*** | -1.71 [-2.19; -1.23] | |
Externalizing CBCL subscale score | 9.28 (9.57) | 21.44 (10.02) | -6.44*** | -1.25 [-1.66; -0.83] | 7.62 (7.59) | 20.90 (10.86) | -7.46*** | -1.42 [-1.84; -0.99] | |
Total CBCL score | 23.56 (21.62) | 66.49 (26.70) | -9.40*** | -1.83 [-2.27; -1.37] | 19.39 (16.68) | 62.54 (28.36) | -9.69*** | -1.85 [-2.32; -1.38] | |
CBCL-PTSD score | 3.40 (3.29) | 12.00 (4.63) | -11.62*** | -2.26 [-2.73; -1.77] | 2.92 (2.85) | 10.77 (5.12) | -9.88*** | -1.89 [-2.37; -1.41] | |
CBCL-PTSD-17 score | 4.34 (4.33) | 14.50 (5.38) | -11.58*** | -2.25 [-2.73; -1.77] | 3.66 (3.58) | 13.56 (6.16) | -10.27*** | -1.97 [-2.45; -1.48] | |
CBCL-PTSD-6 score | 1.18 (1.33) | 5.32 (1.95) | -12.10*** | -2.47 [-3.06, -1.87] | 1.03 (1.26) | 4.63 (2.28) | -10.24*** | -1.96 [-2.45; -1.47] | |
CBCL-PTSD subscale by Dehon & Scheeringa (2006)
Significant positive associations were found between the number of PTSD ≤ 6 symptoms exhibited and the CBCL-PTSD scores (r = .82, p < .001). The CBCL-PTSD demonstrated excellent discriminatory accuracy for PTSD with an AUC = 0.94 (10,000 bootstrapped 95%CI [0.890; 0.980]). A cutoff score of ≥ 5 with a sensitivity of 98% (Sp = 71%, PPV = 65%, NPV = 98%) appeared to be the most appropriate.
Moreover, significant positive associations were found between the presence of functional impairment and the CBCL-PTSD (r = .70, p < .001). The CBCL-PTSD demonstrated excellent discriminatory accuracy for functional impairment with AUC = 0.91 (10,000 bootstrapped 95%CI [0.845; 0.957]). A cutoff score of ≥ 5 with a sensitivity of 90% (Sp = 77%, PPV = 76%, NPV = 91%) appeared to be the most appropriate.
CBCL-PTSD subscale based on expert rating
The CBCL-PTSD subscale based on expert ratings, named CBCL-PTSD-17, consisted of 17 items (see Table 2). Compared to the CBCL-PTSD, the CBCL items related to somatic complaints nausea and feels sick (CBCL45), stomachaches and cramps (without medical cause) (CBCL78), and vomiting and throwing up (without medical cause) (CBCL93); and the CBCL items clings to adult or too dependent (CBCL10), defiant (CBCL15), and sudden changes in mood or feelings (CBCL82) were not included in the new CBCL-PTSD-17. The other nine items were identical. New to the CBCL-PTSD-17 were the CBCL items easily frustrated (CBCL29), has trouble getting to sleep (CBCL38), looks unhappy without good reason (CBCL43), angry moods (CBCL44), shows panic for no good reason (CBCL51), loss of interest (CBCL71), talks or cries out in sleep (CBCL84), and temper tantrums or hot temper (CBCL85). Cronbach’s α of the CBCL-PTSD-17 in the present study was 0.91.
Significant positive associations were found between the number of PTSD ≤ 6 symptoms exhibited and the CBCL-PTSD-17 scores (r = .80, p < .001). The CBCL-PTSD-17 demonstrated excellent discriminatory accuracy for PTSD with an AUC = 0.94 (10,000 bootstrapped 95%CI [0.890; 0.973]). Further, the CBCL-PTSD-17 did not differ significantly in AUC for PTSD from the CBCL-PTSD (bootstrap = 10,000, D = 0.21, p = .83). A cutoff score of ≥ 7 with a sensitivity of 98% (Sp = 73%, PPV = 67%, NPV = 98%) appeared to be the most appropriate.
Significant positive associations were found between the presence of functional impairment and the CBCL-PTSD-17 scores (r = .71, p < .001). Moreover, the CBCL-PTSD-17 demonstrated excellent discriminatory accuracy for functional impairment with an AUC = 0.91 (10,000 bootstrapped 95%CI [0.849; 0.962]). Further, the CBCL-PTSD-17 did not differ significantly in AUC for functional impairment from the CBCL-PTSD (bootstrap = 10,000, D = − 0.31, p = .76). A cutoff score of ≥ 5 with a sensitivity of 92% (Sp = 66%, PPV = 69%, NPV = 91%) appeared to be the most appropriate.
CBCL-PTSD subscale based on variable importance
The results of the SSVS analysis indicated a cutoff between the 6th (MIP = 0.33) and 7th (MIP = 0.19) highest MIP. Therefore, it was decided to determine the most important predictors for PTSD diagnosis as those with an MIP ≥ 0.3, which means that these variables were included in 30% or more of the sampled regression models. The six CBCL items falling into this category were (ranked from the highest to the lowest MIP): too fearful or anxious (CBCL87; MIP = 0.82), nightmares (CBCL48, MIP = 0.79), loss of interest (CBCL71, MIP = 0.41), can’t concentrate (CBCL5, MIP = 0.38), clings to adults or too dependent (CBCL10, MIP = 0.34), and stomach aches or cramps (CBCL78, MIP = 0.33). These six items made up the second alternative CBCL-PTSD subscale, named CBCL-PTSD-6 (Table 2). See SM3 for an overview of MIPs for each of the 23 CBCL items. Cronbach’s α of the CBCL-PTSD-6 in the present study was 0.75. All items but loss of interest were included in the CBLC-PTSD.
The CBCL-PTSD-6 scores were significantly positively associated with the number of PTSD ≤ 6 symptoms exhibited (r = .86, p < .001). The CBCL-PTSD-6 demonstrated excellent discriminatory accuracy for PTSD with AUC = 0.96 (10,000 bootstrapped 95%CI [0.913; 0.986]). The CBCL-PTSD-6 did not differ significantly in AUC for PTSD from the CBCL-PTSD (bootstrap = 10,000, D = -1.16, p = .25) and CBCL-PTSD-17 (bootstrap = 10,000, D = -1.11, p = .27). A cutoff score of ≥ 3 with a sensitivity of 95% (Sp = 84%, PPV = 77%, NPV = 97%) appeared to be the most appropriate.
Significant positive associations were found between the presence of functional impairment and the CBCL-PTSD-6 scores (r = .71, p < .001). Further, the CBCL-PTSD-6 demonstrated excellent discriminatory accuracy for functional impairment with an AUC = 0.91 (10,000 bootstrapped 95%CI [0.849; 0.959]) for functional impairment. Moreover, the CBCL-PTSD-6 did not differ significantly in AUC for functional impairment from the CBCL-PTSD (bootstrap = 10,000, D = 0.21, p = .84) and CBCL-PTSD-17 (bootstrap = 10,000, D = − 0.09, p = .93). A cutoff score of ≥ 3 with a sensitivity of 85% (Sp = 89%, PPV = 86%, NPV = 88%) appeared to be the most appropriate. Means and standard deviations of age and the internalizing CBCL subscale, externalizing CBCL subscale, total CBCL scale, CBCL-PTSD, CBCL-PTSD-17, and CBCL-PTSD-6 in young children with PTSD and without PTSD and young children with and without functional impairment are displayed in Table 3. An overview of psychometric properties for various cutoff scores for the three CBCL-PTSD subscales with regard to PTSD and functional impairment is presented in SM4a/b/c.
Discussion
The present study extended the previous evidence base on the screening utility of the CBCL-PTSD subscale by Dehon & Scheeringa (2006). Further, this study evaluated two newly developed alternative CBCL-PTSD subscales: (1) the CBCL-PTSD-17, a subscale oriented on the DSM-5 PTSD ≤ 6 criteria based on expert rating; and (2) the CBCL-PTSD-6, a subscale based on empirically derived variable importance based on the items of the CBCL-PTSD and the CBCL-PTSD-17. The findings demonstrated good to excellent discriminatory accuracy and high levels of sensitivity for all three CBCL-PTSD subscales investigated. Moreover, the present study is the first to demonstrate that a CBCL subscale for PTSD could be useful in screening for PTSS-related functional impairment.
In the present study, sensitivity of the CBCL-PTSD for PTSD was 98% (specificity of 71%) whereas the results of Dehon & Scheeringa (2006) showed a sensitivity of 75% (specificity of 84.4%). However, the comparison of these different results might be limited because contrary to the present study, Dehon & Scheeringa (2006) chose a cutoff score based on a balance between sensitivity and specificity, as well as PPV and NPV (C. Dehon, personal communication, September 15, 2021). Therefore, their sensitivity results might have been comparable, i.e., higher, if they had also prioritized sensitivity as we have done in the present study.
Loeb et al. (2011) reported a maximum sensitivity of 67% (specificity of 63%) for the CBCL-PTSD for the DC:0–3 PTSD diagnosis, an algorithm very similar to the PTSD-AA and to the DSM-5 PTSD ≤ 6 criteria. Differences in sensitivity levels could be explained by differences in the patient spectrum. The sample in the current study included referred young children due to a potentially traumatic experience, whereas young children in Loeb et al. (2011) where referred for various developmental delays and/or behavioral and emotional difficulties, which might overlap with symptoms of PTSD in young children. When underlying conditions look alike, the target condition can also be recognized as being a comorbidity, resulting in a lower sensitivity (Leeflang et al., 2009).
The performance of the CBCL-PTSD-17 for PTSD was very similar to the CBCL-PTSD, i.e., strong correlation between subscale scores and number of DSM-5 PTSD ≤ 6 symptoms exhibited and higher subscale scores in young children who met the clinician-based diagnosis. Further, the CBCL-PTSD-17 achieved an equally high level of sensitivity, indicating that this alternative subscale oriented on the PTSD ≤ 6 criteria based on expert ratings might be equally useful in screening for PTSD ≤ 6. The comparable psychometric properties could be explained by the large number of overlapping CBCL items, i.e., nine, which can all be considered strong indicators of potential PTSD. However, neither the CBCL-PTSD nor the CBCL-PTSD-17 can be seen as a short or rapid screen for PTSD when compared to existing checklists that cover all 16 DSM-5 PTSD ≤ 6 symptoms (e.g., Child and Adolescent Trauma Screen version 3–6; Sachser et al., 2017). Instead, the CBCL-PTSD-6, which was derived from the exploratory SSVS analysis based on the items from both the CBCL-PTSD and CBCL-PTSD-17, comprised only six CBCL items and distinguished just as well between young children with or without PTSD, with an equally high sensitivity and higher specificity for PTSD compared to the CBCL-PTSD and CBCL-PTSD-17. Interestingly, the composition of the six items indicates that both symptoms included in the DSM-5 PTSD ≤ 6 diagnostic criteria (i.e., nightmares, concentration problems, loss of interest) and symptoms not included in the diagnostic criteria but possessing face validity as posttraumatic symptoms (i.e., clinginess, stomachache or cramps, fear and anxiousness) seem to be relevant in identifying children at risk for PTSD (Dehon & Scheeringa, 2006). For comparison, the only other screen as short as the CBCL-PTSD-6 is the YCPS (Scheeringa, 2019). The CBCL-PTSD-6 overlaps with the YCPS in only one item, i.e., nightmares. The otherwise different composition is mainly because the CBCL does not include specific trauma-related symptoms such as intrusive thoughts or memories or exaggerated startle response. However, the YCPS and CBCL-PTSD-6 screening performances for PTSD appear to be equivalent (YCPS: Se = 100%, Sp = 42%, PPV = 71%, NPV = 100%; Scheeringa 2019).
NPVs of all three CBCL-PTSD subscales for PTSD were ≥ 97%, which provides confidence that a negative screening result is true. Moreover, PPVs for all three subscales ranged from 65 to 77%, indicating that false positives occurred. However, in the case of screening for PTSD, this should be of relatively low concern as pointed out by Scheeringa (2019). The consequence of a young child falsely classified as at risk for PTSD would only initiate a referral for more in-depth assessment (e.g., usage of a clinical interview by a mental health care specialist) but would mean less potential for harm (Scheeringa, 2019). Further, it is possible that a seemingly unnecessary in-depth assessment for a false-positive result for PTSD could lead to the detection of another mental disorder that was not screened for and can therefore help to focus intervention efforts. It is worth noting that the downside of false positives might involve additional challenges on the already strained child mental health service system. The present study tried to consider the impact of false positives on the health care system by not completely disregarding specificity.
Aside from PTSD, the present study investigated the usefulness of identifying trauma-exposed young children at risk for functional impairment using the three CBCL-PTSD subscales. Findings of the present study indicated that scores on the CBCL-PTSD, the CBCL-PTSD-17, and the CBCL-PTSD-6 correlated highly with the presence of functional impairment. Further, the results of high sensitivity indicate that all three investigated subscales detected a large number of functionally impaired participants. Therefore, each of these subscales demonstrate usefulness in screening for functional impairment as they succeed to adequately distinguish between trauma-exposed young children functionally impaired and young children not functionally impaired. For all three subscales, false positives occurred. Like for PTSD, false positives in this case would be of relatively low concern. Early recognition of functional impairment related to PTSS however, may help to prevent disturbances in development such as insecure attachments to caregivers and impaired peer relationships, which are considered fundamental milestones of development in early childhood.
In light of the present study’s findings for both PTSD and functional impairment, we consider the CBCL-PTSD-6 subscale the most promising and clinically useful screener among the three investigated CBCL-PTSD subscales. With regard to PTSD, it could be argued that the CBCL-PTSD-17 has the advantage of providing more clinically relevant information than the CBCL-PTSD-6, considering its items represent a closer reflection of the PTSD ≤ 6 PTSD criteria. However, the usage of the CBCL-PTSD-17 would not increase the ability to identify young children at risk for PTSD compared to the CBCL-PTSD-6, given that the PTSD ≤ 6 criteria would be assessed by a mental health specialist in a further in-depth assessment in any case. The usage of the CBCL-PTSD-6 however would allow employees across medical settings and child health services and agencies to screen for PTSD and PTSS-related functional impairment in an efficient manner in a population, where PTEs occur frequently, and consequences of such events can potentially put a child on a very negative developmental trajectory. An advantage in comparison to the YCPS would be that the CBCL-PTSD-6 is already embedded in the CBCL 1.5–5 and thus no further instrument needs to be used to identify young children at risk for PTSD. The recommended cutoff for PTSD and functional impairment would be a continuous score of 3 or higher. It should be noted that if the CBCL-PTSD by Dehon & Scheeringa (2006) is to be used for PTSD and functional impairment, we recommend using a cutoff score of 5 or higher (continuous subscale score) based on the present findings.
Overall, the present study has significant clinical implications, contributes to the research evidence base and has several strengths. The findings are based on a large international mixed trauma sample from a population which is difficult to recruit. Moreover, differential, and advanced methodological approaches were adopted to develop and investigate alternative CBCL-PTSD subscales. Further, results of the present study expand the utility of the CBCL 1.5–5 in screening for mental disorders in young children. The results of this study should be interpreted in light of several limitations. The merging of the data represents a strength of the study in that sample size and variability in trauma exposure was increased, which in turn strengthens the generalizability of the findings. However, it is unclear whether differences between the two subsamples due to the nature and aims of the studies in which data were collected might have had an unknown impact on the results. Therefore, future research endeavors must replicate the utility of the CBCL-PTSD-6 across different samples and various types of trauma.
Due to the fact that comorbidity was not assessed in the two subsamples, the analyses focused solely on criterion validity (i.e., PTSD). Therefore, it might be possible that the CBCL-PTSD-6 predicts other disorders due to their overlapping internalizing and externalizing symptomatology with PTSD. However, from a clinical point of view this seems of less concern. A positive screening result for PTSD with a following in-depth assessment leading to the detection of other mental disorders, that were not screened for, is still helpful for identification of young children in need and can therefore help to focus intervention efforts. Further, a large proportion of young children were White or Anglo/European and diagnosed children were likely to be older. Moreover, a large proportion of caregivers were well-educated (see SM1 and SM2). Hence, generalizability of the findings may be reduced with regard to very young children, ethnic minorities, and families with lower socioeconomic status. It will be important for future studies to investigate the utility of the CBCL-PTSD-6 in populations of ethnic/racial minorities since they are at an increased risk for trauma exposure (e.g., Sedlak et al., 2010).
To evaluate the validity of the screener further, the CBCL-PTSD-6 subscale should be investigated for its test-retest-reliability to ensure that the psychometric properties can be maintained. It is also important for future studies to investigate discriminant validity by comparing the utility of the CBCL-PTSD-6 between groups of young children with and without trauma histories. Because most of the functionally impaired young children in the present study also received a PTSD diagnosis, future studies need to investigate whether the CBCL-PTSD-6 can successfully distinguish between functionally impaired young children due to PTSS, but not fulfilling the PTSD diagnostic criteria and young children not impaired in functioning and not meeting the PTSD criteria. Future research endeavors should also explore the importance of symptoms for PTSD in children via different statistical approaches such as item-response theory to examine whether different methods highlight the same symptoms.
The results of the present study support the utility of a new, shorter CBCL-PTSD subscale – the CBCL-PTSD-6 – as a screening tool for DSM-5 PTSD ≤ 6 and for PTSS-related functional impairment in trauma-exposed young children. While the CBCL-PTSD by Dehon & Scheeringa (2006) and the CBCL-PTSD-17, as an alternative developed based on expert-ratings performed comparably well, the major strength of the CBCL-PTSD-6 is characterized by its quick and practicable application for health professionals across various medical settings, health services and agencies, as well as clinicians. Given that the CBCL 1.5–5 is probably the most used screener worldwide to assess young children, the newly developed CBCL-PTSD-6 increases the usefulness of the overall instrument by including a subscale that provides valid information about a child’s likelihood of meeting the full PTSD ≤ 6 criteria and exhibiting PTSS-related functional impairment.
Supplementary Information
The online version contains supplementary material available at https://doi.org/10.1007/s10862-022-09985-5.
Conflict of interest
Dr. Alison Salloum receives grant support from Patient-Centered Outcomes Research Institute, The Norwegian Centre for Violence and Traumatic Stress Studies, Department of Education, Office of Special Education and Rehabilitative Services and previously from NIMH. Dr. Salloum receives royalties from Taylor and Francis. Dr. Salloum is a co-author of Stepping Together, a treatment manual used in stepped care TF-CBT and holds a licensing agreement with Guilford Press for adaptation of some of the handouts and worksheets in the manual. Dr. Salloum is also a national trainer for TF-CBT.
Consent to participate
Written informed consent was obtained from all participating parents/caregivers/guardians.
Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.