Skip to main content

Advertisement

SpringerLink
Log in

Diagnostic efficiency of the CBCL thought problems and DSM-oriented psychotic symptoms scales for pediatric psychotic symptoms

  • Original Contribution
  • Published:
European Child & Adolescent Psychiatry Aims and scope Submit manuscript

Abstract

We compared the diagnostic efficiency of the Child Behavior Checklist (CBCL) Thought Problems subscale and the rationally derived DSM-oriented psychotic symptoms scale (DOPSS) to identify clinically concerning psychosis in a multi-site sample of youths seeking outpatient mental health services (N = 694). We operationally defined clinically concerning psychosis as the presence of clinically significant hallucinations or delusions, assessed by the Schedule for Affective Disorders and Schizophrenia psychosis items. Both the Thought Problems and DOPSS scores showed significant areas under the curve (AUCs = 0.65 and 0.70, respectively), but the briefer DOPSS showed statistically significantly better diagnostic efficiency for any clinically concerning psychosis, but the difference was small enough that it would not be clinically meaningful. The optimal psychosis screening cut-off score (maximizing sensitivity and specificity) was 68.5+ [corresponding diagnostic likelihood ratio (DiLR) = 1.59] for the Thought Problems subscale and 1.67+ (DiLR = 1.97) for the DOPSS. Both the CBCL Thought Problems and DOPSS are clinically useful for identifying psychotic symptoms in children, and although the DOPSS showed statistically better discriminating power, the difference was small so we would not necessarily recommend the DOPSS over standard scoring.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Harrison G, Hopper K, Craig T, Laska E, Siegel C, Wanderling J, Dube KC, Ganev K, Giel R, Der Heiden WA, Holmberg SK, Janca A, Lee PW, Leon CA, Malhotra S, Marsella AJ, Nakane Y, Sartorius N, Shen Y, Skoda C, Thara R, Tsirkin SJ, Varma VK, Walsh D, Wiersma D (2001) Recovery from psychotic illness: a 15- and 25-year international follow-up study. Br J Psychiatry 178:506–517

    Article  CAS  Google Scholar 

  2. Wu EQ, Birnbaum HG, Shi L, Ball DE, Kessler RC, Moulis M, Aggarwal J (2005) The economic burden of schizophrenia in the United States in 2002. J Clin Psychiatry 66(9):1122–1129

    Article  Google Scholar 

  3. Stafford MR, Jackson H, Mayo-Wilson E, Morrison AP, Kendall T (2013) Early interventions to prevent psychosis: systematic review and meta-analysis. BMJ 346:f185. https://doi.org/10.1136/bmj.f185

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Kelleher I, Connor D, Clarke MC, Devlin N, Harley M, Cannon M (2012) Prevalence of psychotic symptoms in childhood and adolescence: a systematic review and meta-analysis of population-based studies. Psychol Med 42(9):1857–1863. https://doi.org/10.1017/s0033291711002960

    Article  CAS  PubMed  Google Scholar 

  5. Kelleher I, Harley M, Murtagh A, Cannon M (2011) Are screening instruments valid for psychotic-like experiences? A validation study of screening questions for psychotic-like experiences using in-depth clinical interview. Schizophr Bull 37(2):362–369. https://doi.org/10.1093/schbul/sbp057

    Article  PubMed  Google Scholar 

  6. Kline E, Schiffman J (2014) Psychosis risk screening: a systematic review. Schizophr Res 158(1–3):11–18. https://doi.org/10.1016/j.schres.2014.06.036

    Article  PubMed  Google Scholar 

  7. Thompson E, Kline E, Reeves G, Pitts SC, Bussell K, Schiffman J (2014) Using parent and youth reports from the Behavior Assessment System for Children, second edition to identify individuals at clinical high-risk for psychosis. Schizophr Res 154(1–3):107–112. https://doi.org/10.1016/j.schres.2014.02.009

    Article  PubMed  Google Scholar 

  8. Thompson E, Kline E, Reeves G, Pitts SC, Schiffman J (2013) Identifying youth at risk for psychosis using the Behavior Assessment System for Children. Second edition. Schizophr Res 151(1–3):238–244. https://doi.org/10.1016/j.schres.2013.09.022

    Article  PubMed  Google Scholar 

  9. Youngstrom EA (2013) A primer on receiver operating characteristic analysis and diagnostic efficiency statistics for pediatric psychology: we are ready to ROC. J Pediatr Psychol. https://doi.org/10.1093/jpepsy/jst062

    Article  PubMed  PubMed Central  Google Scholar 

  10. Achenbach TM, Rescorla LA (2001) Manual for the ASEBA school-age forms and profiles. University of Vermont, Burlington. ISBN 0-938565-73-7

    Google Scholar 

  11. Lengua LJ, Sadowski CA, Friedrich WN, Fisher J (2001) Rationally and empirically derived dimensions of children’s symptomatology: expert ratings and confirmatory factor analyses of the CBCL. J Consult Clin Psychol 69(4):683–698

    Article  CAS  Google Scholar 

  12. Macmann GM, Barnett DW, Lopez EJ (1993) The Child Behavior Checklist/4–28 and related materials: reliability and validity of syndromal assessment. School Psych Rev 22(2):322–333

    Google Scholar 

  13. Jensen PS, Watanabe HK, Richters JE, Roper M, Hibbs ED, Salzberg AD, Liu S (1996) Scales, diagnoses, and child psychopathology: II. Comparing the CBCL and the DISC against external validators. J Abnorm Child Psychol 24(2):151–168. https://doi.org/10.1007/BF01441482

    Article  CAS  PubMed  Google Scholar 

  14. Gadow KD, Sprafkin J (1994) Child symptom inventories manual. Checkmate Plus, Stony Brook

    Google Scholar 

  15. Rizvi SH, Salcedo S, Youngstrom EA, Freeman LK, Gadow KD, Fristad MA, Birmaher B, Kowatch RA, Horwitz SM, Frazier TW, Arnold LE, Taylor HG, Findling RL (2018) Diagnostic accuracy of the CASI-4R psychosis subscale in pediatric outpatient mental health clinics. J Clin Child Adolesc Psychol. https://doi.org/10.1080/15374416.2017.1410824

    Article  PubMed  Google Scholar 

  16. Youngstrom EA, Meyers OI, Demeter C, Kogos Youngstrom J, Morello L, Piiparinen R, Feeny NC, Findling RL, Calabrese JR (2005) Comparing diagnostic checklists for pediatric bipolar disorder in academic and community mental health settings. Bipolar Disord 7:507–517 (special issue: pediatric bipolar disorder)

    Article  Google Scholar 

  17. Kaufman J, Birmaher B, Brent D, Rao U, Flynn C, Moreci P, Williamson D, Ryan N (1997) Schedule for affective disorders and schizophrenia for school-age children-present and lifetime version (K-SADS-PL): initial reliability and validity data. J Am Acad Child Adolesc Psychiatry 36(7):980–988. https://doi.org/10.1097/00004583-199707000-00021

    Article  CAS  Google Scholar 

  18. Youngstrom EA, Meyers O, Demeter C, Youngstrom JK, Morello L, Piiparinen R, Feeny N, Calabrese JR, Findling RL (2005) Comparing diagnostic checklists for pediatric bipolar disorder in academic and community mental health settings. Bipolar Disord 7(6):507–517. https://doi.org/10.1111/j.1399-5618.2005.00269.x

    Article  PubMed  Google Scholar 

  19. Ulloa RE, Birmaher B, Axelson D, Williamson DE, Brent DA, Ryan ND, Bridge J, Baugher M (2000) Psychosis in a pediatric mood and anxiety disorders clinic: phenomenology and correlates. J Am Acad Child Adolesc Psychiatry 39(3):337–345

    Article  CAS  Google Scholar 

  20. Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez J-C, Muller M (2011) pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinform 12(1):77

    Article  Google Scholar 

  21. Rice ME, Harris GT (2005) Comparing effect sizes in follow-up studies: ROC Area, Cohen’s d, and r. Law Hum Behav 29(5):615–620. https://doi.org/10.1007/s10979-005-6832-7

    Article  PubMed  Google Scholar 

  22. Venkatraman ES (2000) A permutation test to compare receiver operating characteristic curves. Biometrics 56(4):1134–1138

    Article  CAS  Google Scholar 

  23. Hanley JA, McNeil BJ (1983) A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 148:839–843

    Article  CAS  Google Scholar 

  24. Findling RL, Youngstrom EA, Fristad MA, Birmaher B, Kowatch RA, Arnold LE, Frazier TW, Axelson D, Ryan N, Demeter CA, Gill MK, Fields B, Depew J, Kennedy SM, Marsh L, Rowles BM, Horwitz SM (2010) Characteristics of children with elevated symptoms of mania: the Longitudinal Assessment of Manic Symptoms (LAMS) study. J Clin Psychiatry 71(12):1664–1672. https://doi.org/10.4088/JCP.09m05859yel

    Article  PubMed  PubMed Central  Google Scholar 

  25. Straus SE, Glasziou P, Richardson WS, Haynes RB (2011) Evidence-based medicine: How to practice and teach it, vol, 4th edn. Churchill Livingstone Elsevier, New York

    Google Scholar 

  26. Straus SE, Glasziou P, Richardson WS, Haynes RB (2011) Evidence-based medicine: How to practice and teach EBM, vol, 4th edn. Churchill Livingstone, New York

    Google Scholar 

  27. Zhou X, Obuchowski N, McClish D (2002) The sensitivity and specificity of clustered binary data. Statistical methods in diagnostic medicine. Wiley, New York, pp 104–106

    Chapter  Google Scholar 

  28. Youngstrom EA, Egerton GA, Rizvi SH, Freeman LK, Genzlinger JL, Van Meter A (2018) Improving the global identification of bipolar spectrum disorders: meta-analysis of the diagnostic accuracy of checklists. Psychol Bull. https://doi.org/10.1037/bul0000137

    Article  PubMed  Google Scholar 

  29. Moreira ALR, Van Meter A, Genzlinger J, Youngstrom EA (2017) Review and meta-analysis of epidemiologic studies of adult bipolar disorder. J Clin Psychiatry 78(9):e1259–e1269. https://doi.org/10.4088/JCP.16r11165

    Article  PubMed  Google Scholar 

  30. Birmaher B, Axelson D, Monk K, Kalas C, Goldstein B, Hickey MB, Obreja M, Ehmann M, Iyengar S, Shamseddeen W, Kupfer D, Brent D (2009) Lifetime psychiatric disorders in school-aged offspring of parents with bipolar disorder: the Pittsburgh Bipolar Offspring study. Arch Gen Psychiatry 66(3):287–296. https://doi.org/10.1001/archgenpsychiatry.2008.546

    Article  PubMed  PubMed Central  Google Scholar 

  31. Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Lawrence Earlbaum Associates, Hilsdale

    Google Scholar 

Download references

Acknowledgements

The main outcome study was funded by NIH R01MH066647 from the National Institute of Mental Health (PI: Eric A. Youngstrom). Dr. Eric Youngstrom served as a statistical expert for the study.

Author information

Authors and Affiliations

  1. Department of Psychology and Neuroscience, University of North Carolina at Chapel Hill, CB #3270, Davie Hall, Chapel Hill, NC, 27599-3270, USA

    Stephanie Salcedo, Sabeen H. Rizvi, Jennifer K. Youngstrom & Eric A. Youngstrom

  2. Department of Psychology, Gargi College, Delhi University, New Delhi, India

    Sabeen H. Rizvi

  3. Department of Psychological Sciences, University of Missouri, Columbia, MO, USA

    Lindsey K. Freeman

  4. Department of Psychiatry, Johns Hopkins University, Maryland, USA

    Robert L. Findling

Authors
  1. Stephanie Salcedo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sabeen H. Rizvi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lindsey K. Freeman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jennifer K. Youngstrom
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Robert L. Findling
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Eric A. Youngstrom
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eric A. Youngstrom.

Ethics declarations

Conflict of interest

Dr. Findling receives or has received research support, acted as a consultant and/or served on a speaker’s bureau for Actavis, Akili, Alcobra, American Academy of Child & Adolescent Psychiatry, American Psychiatric Press, Bracket, CogCubed, Cognition Group, Coronado Biosciences, Elsevier, Epharma Solutions, Forest, Genentech, GlaxoSmithKline, Guilford Press, Ironshore, Johns Hopkins University Press, KemPharm, Lundbeck, Medgenics, Merck, NIH, Neurim, Novartis, Otsuka, PCORI, Pfizer, Physicians Postgraduate Press, Purdue, Rhodes Pharmaceuticals, Roche, Sage, Shire, Sunovion, Supernus Pharmaceuticals, Syneurx, Takeda, Teva, Tris, Validus, and WebMD. Dr. E. Youngstrom has consulted with Pearson, Lundbeck, Janssen, Western Psychological Services, and Joe Startup Technologies about psychological assessment. Ms. Salcedo, Dr. Rizvi, Ms. Freeman, and Dr. J. Youngstrom have no conflicts to disclose.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 7948 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Salcedo, S., Rizvi, S.H., Freeman, L.K. et al. Diagnostic efficiency of the CBCL thought problems and DSM-oriented psychotic symptoms scales for pediatric psychotic symptoms. Eur Child Adolesc Psychiatry 27, 1491–1498 (2018). https://doi.org/10.1007/s00787-018-1140-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00787-018-1140-1

Keywords

Search

Navigation