Abstract
Increasingly, adversity-focused assessment tools are being introduced into preventive mental health screening protocols. However, few studies have explicitly examined whether use of these instruments serves as equitable, clinically useful measures of mental health risk in adolescents. In response, the present study examined whether an adverse childhood experiences (ACEs) measure was accurate and fair as an index of environmental risk for adolescent mental health diagnoses. Secondary data analyses were conducted on the National Comorbidity Survey-Adolescent Supplement. Adolescents (N = 10,148; AgeMean = 15.20; 51.3% male; 65.6% White, 15.1% Black, and 14.4% Hispanic) answered ten questions concerning childhood adversities and completed diagnostic interviews for PTSD, depression, and externalizing disorders. In the overall sample, ACEs showed some clinical utility (e.g., area under the curve (AUCs) ≥ 0.64), diagnostic likelihood ratios (DLRs) > 4.0) and acceptable calibration (i.e., expected/observed indices’ confidence intervals included 1) across diagnoses. Within subpopulations, however, predictive validity varied. The AUCs were lower for multiple diagnoses in Black male and Hispanic female adolescents and DLRs suggested greater clinical utility for indexing mental health in White, female adolescents. Finally, models were not well-calibrated between adolescent subpopulations, suggesting recommended ACEs screening can potentially produce biased results when used to inform mental health policy and prevention. Reasons for why results from ACEs screening may vary across adolescent subpopulations and the importance of testing statistical fairness for preventive mental health screening are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
A total of 8 groups were compared for these analyses. Adolescents who did not identify as White, Black, or Hispanic were grouped into an “other” category. These four racial/ethnic groups were then grouped as male or female. Gender was coded as a binary category in the NCS-A study.
Of note, the E/O index for threshold scores did not include 1 (see Table 3). This suggests we were not able to replicate the rate at which White female adolescents report depression within the context of above threshold ACEs score. Similarly, the E/O index for threshold scores did not include 1 for depression in Hispanic males.
References
Alisic, E., Zalta, A., Van Wesel, F., Larsen, S., Hafstad, G., Hassanpour, K., & Smid, G. (2014). Rates of post-traumatic stress disorder in trauma-exposed children and adolescents: Meta-analysis. British Journal of Psychiatry, 204, 335–340. https://doi.org/10.1192/bjp.bp.113.131227
Anda, R. F., Porter, L. E., & Brown, D. W. (2020). Inside the adverse childhood experience score: Strengths, limitations, and misapplications. American Journal of Preventive Medicine, 59, 293–295. https://doi.org/10.1016/j.amepre.2020.01.009
Baldwin, J. R., Caspi, A., Meehan, A. J., Ambler, A., Arseneault, L., Fisher, H. L., & Danese, A. (2021). Population vs individual prediction of poor health from results of adverse childhood experiences screening. JAMA Pediatrics, 175, 385–393.
Balistreri, K. S., & Alvira-Hammond, M. (2016). Adverse childhood experiences, family functioning and adolescent health and emotional well-being. Public Health, 132, 72–78. https://doi.org/10.1016/j.puhe.2015.10.034
Berk, R., Heidari, H., Jabbari, S., Kearns, M., & Roth, A. (2018). Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research. https://doi.org/10.1177/0049124118782533
Bernard, D. L., Calhoun, C. D., Banks, D. E., Halliday, C. A., Hughes-Halbert, C., Danielson, C. K. (2020). Making the “c-ACE” for a culturally-informed adverse childhood experiences framework to understand the pervasive mental health impact of racism on Black youth. Journal of Child & Adolescent Trauma, 1–15. https://doi.org/10.1007/s40653-020-00319-9
Bernard, D. L., Calhoun, C. D., Banks, D. E., Halliday, C. A., Hughes-Halbert, C., & Danielson, C. K. (2021). Making the “C-ACE” for a culturally-informed adverse childhood experiences framework to understand the pervasive mental health impact of racism on Black youth. Journal of Child & Adolescent Trauma, 14(2), 233–247.
Bomysoad, R. N., & Francis, L. A. (2020). Adverse childhood experiences and mental health conditions among adolescents. Journal of Adolescent Health, 67, 868–870. https://doi.org/10.1016/j.jadohealth.2020.04.013
Brockie, T. N., Dana-Sacco, G., Wallen, G. R., Wilcox, H. C., & Campbell, J. C. (2015). The relationship of adverse childhood experiences to PTSD, depression, poly-drug use and suicide attempt in reservation-based Native American adolescents and young adults. American Journal of Community Psychology, 55, 411–421. https://doi.org/10.1007/s10464-015-9721-3
Chouldechova, A. (2017). Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5, 153–163. https://doi.org/10.1089/big.2016.0047
Cohen, J. R., So, F. K., Young, J. F., Hankin, B. L., & Lee, B. A. (2019a). Youth depression screening with parent and self-reports: Assessing current and prospective depression risk. Child Psychiatry & Human Development, 50, 647–660. https://doi.org/10.1007/s10578-019-00869-6
Cohen, J. R., Thakur, H., Burkhouse, K. L., & Gibb, B. E. (2019b). A multimethod screening approach for pediatric depression onset: An incremental validity study. Journal of Consulting and Clinical Psychology, 87, 184–197. https://doi.org/10.1037/ccp0000364
Cohen, J. R., Thakur, H., Young, J. F., & Hankin, B. L. (2020). The development and validation of an algorithm to predict future depression onset in unselected youth. Psychological Medicine, 50, 2548–2556. https://doi.org/10.1017/S0033291719002691
Crenshaw, K. (1989). Demarginalizing the intersection of race and sex: A black feminist critique of antidiscrimination doctrine, feminist theory and antiracist politics. University of Chicago Legal Forum, 139.
Dube, S. R. (2018). Continuing conversations about adverse childhood experiences (ACEs) screening: A public health perspective. Child Abuse & Neglect, 85, 180–184. https://doi.org/10.1016/j.chiabu.2018.03.007
Elkins, J., Briggs, H. E., Miller, K. M., Kim, I., Orellana, R., & Mowbray, O. (2019). Racial/ethnic differences in the impact of adverse childhood experiences on posttraumatic stress disorder in a nationally representative sample of adolescents. Child and Adolescent Social Work Journal, 36, 449–457. https://doi.org/10.1007/s10560-018-0585-x
Felitti, V. J. (1991). Long-term medical consequences of incest, rape, and molestation. Southern Medical Journal, 84, 328–331. https://doi.org/10.1097/00007611-199103000-00008
Felitti, V. J., Anda, R. F., Nordenberg, D., & Williamson, D. F. (1998a). Adverse childhood experiences and health outcomes in adults: The Ace study. Journal of Family and Consumer Sciences, 90(3), 31.
Felitti, V. J., Anda, R. F., Nordenberg, D., Williamson, D. F., Spitz, A. M., Edwards, V., & Marks, J. S. (1998b). Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults: The Adverse Childhood Experiences (ACE) Study. American Journal of Preventive Medicine, 14, 245–258. https://doi.org/10.1016/S0749-3797(98)00017-8
Finkelhor, D. (2018). Screening for adverse childhood experiences (ACEs): Cautions and suggestions. Child Abuse & Neglect, 85, 174–179. https://doi.org/10.1016/j.chiabu.2017.07.016
Fripp, J. A., & Carlson, R. G. (2017). Exploring the influence of attitude and stigma on participation of African American and Latino populations in mental health services. Journal of Multicultural Counseling and Development, 45, 80–94. https://doi.org/10.1002/jmcd.1206
Green, J. G., Gruber, M. J., Kessler, R. C., Lin, J. Y., Mclaughlin, K. A., Sampson, N. A., Zaslavsky, A. M., & Alegria, M. (2012). Diagnostic validity across racial and ethnic groups in the assessment of adolescent DSM-IV disorders. International Journal of Methods in Psychiatric Research, 21, 311–320. https://doi.org/10.1002/mpr.1371
Green, J. G., McLaughlin, K. A., Berglund, P. A., Gruber, M. J., Sampson, N. A., Zaslavsky, A. M., & Kessler, R. C. (2010). Childhood adversities and adult psychiatric disorders in the national comorbidity survey replication I: Associations with first onset of DSM-IV disorders. Archives of General Psychiatry, 67, 113–123. https://doi.org/10.1001/archgenpsychiatry.2009.186
Hanson, R. K. (2017). Assessing the calibration of actuarial risk scales: A primer on the E/O index. Criminal Justice and Behavior, 44(1), 26–39.
Harrell, F. E. (2001). Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis (Vol. 608). New York: springer.
Hudziak, J. J., Copeland, W., Stanger, C., & Wadsworth, M. (2004). Screening for DSM-IV externalizing disorders with the Child Behavior Checklist: A receiver operating characteristic analysis. Journal of Child Psychology and Psychiatry, 45, 1299–1307. https://doi.org/10.1111/j.1469-7610.2004.00314.x
Hughes, K., Bellis, M. A., Hardcastle, K. A., Sethi, D., Butchart, A., Mikton, C., ... & Dunne, M. P. (2017). The effect of multiple adverse childhood experiences on health: a systematic review and meta-analysis. The Lancet Public Health, 2(8), e356–e366.
Humphreys, K. L., LeMoult, J., Wear, J. G., Piersiak, H. A., Lee, A., & Gotlib, I. H. (2020). Child maltreatment and depression: A meta-analysis of studies using the Childhood Trauma Questionnaire. Child Abuse & Neglect, 102, 104361. https://doi.org/10.1016/j.chiabu.2020.104361
Infurna, M. R., Reichl, C., Parzer, P., Schimmenti, A., Bifulco, A., & Kaess, M. (2016). Associations between depression and specific childhood experiences of abuse and neglect: A meta-analysis. Journal of Affective Disorders, 190, 47–55. https://doi.org/10.1016/j.jad.2015.09.006
Kaiser Health. (2020). Medicaid Enrollment by Race/Ethnicity. https://www.kff.org/medicaid/state-indicator/medicaid-enrollment-by-raceethnicity/. Accessed 15 Aug 2020.
Kessler, R. C., & Üstün, T. B. (2004). The world mental health (WMH) survey initiative version of the world health organization (WHO) composite international diagnostic interview (CIDI). International Journal of Methods in Psychiatric Research, 13, 93–121. https://doi.org/10.1002/mpr.168
Kessler, R. C., Avenevoli, S., Costello, E. J., Green, J. G., .Zaslavsky, A. M. (2009). National comorbidity survey replication adolescent supplement (NCS-A): II. Overview and design. Journal of the American Academy of Child & Adolescent Psychiatry, 48(4), 380–385. https://doi.org/10.1097/CHI.0b013e3181999705
Leffler, J. M., Riebel, J., & Hughes, H. M. (2015). A review of child and adolescent diagnostic interviews for clinical practitioners. Assessment, 22, 690–703. https://doi.org/10.1177/1073191114561253
Lindhiem, O., Petersen, I. T., Mentch, L. K., & Youngstrom, E. A. (2020). The importance of calibration in clinical psychology. Assessment, 27, 840–854. https://doi.org/10.1177/1073191117752055
López, C. M., Andrews, A. R., III., Chisolm, A. M., De Arellano, M. A., Saunders, B., & Kilpatrick, D. (2017). Racial/ethnic differences in trauma exposure and mental health disorders in adolescents. Cultural Diversity and Ethnic Minority Psychology, 23, 382. https://doi.org/10.1037/cdp0000126
Marie-Mitchell, A., Studer, K. R., & O’Connor, T. G. (2016). How knowledge of adverse childhood experiences can help pediatricians prevent mental health problems. Families, Systems, & Health, 34, 128–135. https://doi.org/10.1037/fsh0000179
Matchinger, E. L., Lieberman, A., & Lightfoot, M. (2021). Research, practice, and policy implications of Adverse Childhood Events. JAMA Pediatrics. https://doi.org/10.1001/jamapediatrics.2021.0810
McFall, R. M., & Treat, T. A. (1999). Quantifying the information value of clinical assessments with signal detection theory. Annual Review of Psychology, 50, 215–241. https://doi.org/10.1146/annurev.psych.50.1.215
McLaughlin, K. A., Green, J. G., Gruber, M. J., Sampson, N. A., Zaslavsky, A. M., & Kessler, R. C. (2012). Childhood adversities and first onset of psychiatric disorders in a national sample of US adolescents. Archives of General Psychiatry, 69(11), 1151–1160. https://doi.org/10.1001/archgenpsychiatry.2011.2277
McLennan, J. D., MacMillan, H. L., & Afifi, T. O. (2020). Questioning the use of adverse childhood experiences (ACEs) questionnaires. Child Abuse & Neglect, 101, 104331. https://doi.org/10.1016/j.chiabu.2019.104331
Meehan, A. J., Baldwin, J. R., Lewis, S. J., MacLeod, J. G., & Danese, A. (2022). Poor individual risk classification from adverse childhood experiences screening. American Journal of Preventive Medicine, 62, 427–432. https://doi.org/10.1016/j.amepre.2021.08.008
Merikangas, K. R., Avenevoli, S., Costello, E. J., Koretz, D., & Kessler, R. C. (2009). National comorbidity survey replication adolescent supplement (NCS-A): I. Background and measures. Journal of the American Academy of Child & Adolescent Psychiatry, 48(4), 367–379. https://doi.org/10.1097/CHI.0b013e31819996f1
Merikangas, K. R., He, J. P., Burstein, M., Swendsen, J., Avenevoli, S., Case, B., Georgiades, K., Heaton, L., Swanson, S., & Olfson, M. (2011). Service utilization for lifetime mental disorders in US adolescents: Results of the National Comorbidity Survey-Adolescent Supplement (NCS-A). Journal of the American Academy of Child & Adolescent Psychiatry, 50, 32–45. https://doi.org/10.1016/j.jaac.2010.10.006
Nanni, V., Uher, R., & Danese, A. (2012). Childhood maltreatment predicts unfavorable course of illness and treatment outcome in depression: A meta-analysis. American Journal of Psychiatry, 169, 141–151. https://doi.org/10.1176/appi.ajp.2011.11020335
Negriff, S. (2020). ACEs are not equal: Examining the relative impact of household dysfunction versus childhood maltreatment on mental health in adolescence. Social Science & Medicine, 245, 112696. https://doi.org/10.1016/j.socscimed.2019.112696
Pepe, M. S., Janes, H., Longton, G., Leisenring, W., & Newcomb, P. (2004). Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. American Journal of Epidemiology, 159, 882–890. https://doi.org/10.1093/aje/kwh101
Petruccelli, K., Davis, J., & Berman, T. (2019). Adverse childhood experiences and associated health outcomes: A systematic review and meta-analysis. Child Abuse & Neglect, 97, 104127. https://doi.org/10.1016/j.chiabu.2019.104127
Racine, N., Killam, T., & Madigan, S. (2020). Trauma-informed care as a universal precaution: Beyond the adverse childhood experiences questionnaire. JAMA Pediatrics, 174, 5–6. https://doi.org/10.1001/jamapediatrics.2019.3866
Salcedo, S., Chen, Y. L., Youngstrom, E. A., Fristad, M. A., Gadow, K. D., Horwitz, S. M., & Findling, R. L. (2018). Diagnostic efficiency of the Child and Adolescent Symptom Inventory (CASI-4R) depression subscale for identifying youth mood disorders. Journal of Clinical Child & Adolescent Psychology, 47(5), 832–846. https://doi.org/10.1080/15374416.2017.12808
Straus, S., Glasziou, P., Richardson, W., & Haynes, R. (2019). Evidence-based medicine : How to practice and teach EBM (5th edition.). Elsevier.
Turner, H. A., Finkelhor, D., Mitchell, K. J., Jones, L. M., & Henly, M. (2020). Strengthening the predictive power of screening for adverse childhood experiences (ACEs) in younger and older children. Child Abuse & Neglect, 107, 104522. https://doi.org/10.1016/j.chiabu.2020.104522
Turner, H. A., Finkelhor, D., & Ormrod, R. (2010). Poly-victimization in a national sample of children and youth. American Journal of Preventive Medicine, 38(3), 323–330.
United States Preventive Services Task Force (USPSTF). (2017). Factors to consider when recommending starting and stopping times for screening (Procedure Manual Appendix IX). Retrieved May 4, 2022 from https://uspreventiveservicestaskforce.org/uspstf/about-uspstf/methods-and-processes/procedure-manual/procedure-manual-appendix-ix-factors-consider-when-recommending-starting-and-stopping-times-screening. Accessed 4 May 2022.
Underwood, E. (2020). Screen for childhood trauma triggers debate. Science (New York, N.Y.), 367(6477), 498. https://doi.org/10.1126/science.367.6477.498
Wade, R., Shea, J. A., Rubin, D., & Wood, J. (2014). Adverse childhood experiences of low-income urban youth. Pediatrics, 134, e13–e20. https://doi.org/10.1542/peds.2013-2475
Widom, C. S., Czaja, S., Wilson, H. W., Allwood, M., & Chauhan, P. (2013). Do the long-term consequences of neglect differ for children of different races and ethnic backgrounds? Child Maltreatment, 18, 42–55. https://doi.org/10.1177/1077559512460728
You, D. S., Youngstrom, E. A., Feeny, N. C., Youngstrom, J. K., & Findling, R. L. (2017). Comparing the diagnostic accuracy of five instruments for detecting posttraumatic stress disorder in youth. Journal of Clinical Child & Adolescent Psychology, 46, 511–522. https://doi.org/10.1080/15374416.2015.1030754
Youngstrom, E. A. (2014). A primer on receiver operating characteristic analysis and diagnostic efficiency statistics for pediatric psychology: We are ready to ROC. Journal of Pediatric Psychology, 39, 204–221. https://doi.org/10.1093/jpepsy/jst062
Zink, D., Lee, B., & Allen, D. (2015). Structured and semistructured clinical interviews available for use among African American clients: Cultural considerations in the diagnostic interview process. In Guide to psychological assessment with African Americans (pp. 19–42). Springer, New York, NY.
Funding
The National Comorbidity Survey-Adolescent Supplement was supported by the National Institute of Mental Health (NIMH; U01-MH60220) with supplemental support from NIDA, SAMHSA, the Robert Wood Johnson Foundation, and the John W. Alden Trust. JRC’s time on this manuscript was funded by the National Institute of Justice (2018-R2-CX-0022).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethical Approval
The recruitment and consent procedures were approved by the human subjects committees of both Harvard Medical School and the University of Michigan. The secondary data analysis on this study was approved by the first author’s Institutional Review Board.
Informed Consent
Written informed parental consent and written informed adolescent assent was obtained from all participants prior to participation in the study.
Conflict of Interest
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Cohen, J.R., Choi, J.W. Is ACEs Screening for Adolescent Mental Health Accurate and Fair?. Prev Sci 23, 1216–1229 (2022). https://doi.org/10.1007/s11121-022-01391-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11121-022-01391-3