Skip to main content

Advertisement

SpringerLink
Log in

Psychometric Evaluation of the Depressive Severity Index (DSI) Among Children and Youth Using the interRAI Child and Youth Mental Health (ChYMH) Assessment Tool

  • Original Paper
  • Published:
Child & Youth Care Forum Aims and scope Submit manuscript

Abstract

Background

While empirical testing of many of the interRAI Child and Youth Mental Health (ChYMH) assessment scales have been found to be reliable and valid across both child and adult samples, other scales have yet to be psychometrically evaluated within child populations.

Objective

The current study evaluates the psychometric properties of the Depressive Severity Index (DSI) embedded in the ChYMH instrument in a child sample.

Method

ChYMH data from 6093 children (aged 4–18) across mental health agencies in Ontario, Canada, were obtained.

Results

The scale showed good inter-item reliability with a Cronbach’s alpha of 0.86 and high inter-item correlations. The scale also demonstrated good convergent validity when tested against DSM-IV mood disorders diagnosis, Child Behaviour Checklist Internalizing scores, and Behaviour Assessment System for Children, Third Edition-Depression scores. Furthermore, children’s DSI scores significantly predicted their self-injury attempts and suicide intent using receiver operating characteristic (ROC) analysis, showing good criterion validity of the scale.

Conclusions

Overall, the results were in support of the reliability, convergent validity, and criterion validity of the DSI in detecting depression symptoms in the children and youth population.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. The original scoring contains both 3 (“Exhibited daily in last 3 days, 1–2 episodes”) and 4 (“Exhibited daily in last 3 days, 3 or more episodes or continuously”). These two responses were grouped into 3 (“Exhibited daily in last 3 days”) to be consistent with the response keys used in Perlman et al. (26).

References

  • Achenbach, T. M., & Edelbrock, C. (1991). Child behaviour checklist. In Burlington VT.

  • Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101(2), 213–232. https://doi.org/10.1037/0033-2909.101.2.213.

    Article  PubMed  Google Scholar 

  • Achenbach, T. M., & Rescorla, L. (2001). Manual for the ASEBA school-age forms & profiles: An integrated system of multi-informant assessment. Burlington, VT: Aseba.

    Google Scholar 

  • Achenbach, T. M., & Ruffle, T. M. (2000). The child behavior checklist and related forms for assessing behavioral/emotional problems and competencies. Pediatrics in Review, 21(8), 265–271. https://doi.org/10.1542/pir.21-8-265.

    Article  PubMed  Google Scholar 

  • Akobeng, A. K. (2007). Understanding diagnostic tests 3: Receiver operating characteristic curves. Acta Paediatrica, 96(5), 644–647. https://doi.org/10.1111/j.1651-2227.2006.00178.x.

    Article  PubMed  Google Scholar 

  • Altman, D. G., & Bland, J. M. (1994). Diagnostic tests. 1: Sensitivity and specificity. BMJ, 308(6943), 1552. https://doi.org/10.1136/bmj.308.6943.1552.

    Article  PubMed  PubMed Central  Google Scholar 

  • American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders: DSM-IV-TR. Philadelphia: American Psychiatric Association. https://doi.org/10.1176/appi.books.9780890425596.744053.

    Book  Google Scholar 

  • Baiden, P., Stewart, S. L., & Fallon, B. (2017). The mediating effect of depressive symptoms on the relationship between bullying victimization and non-suicidal self-injury among adolescents: Findings from community and inpatient mental health settings in Ontario, Canada. Psychiatry Research, 255, 238–247. https://doi.org/10.1016/j.psychres.2017.05.018.

    Article  PubMed  Google Scholar 

  • Breslau, J., Miller, E., Breslau, N., Bohnert, K., Lucia, V., & Schweitzer, J. (2009). The impact of early behavior disturbances on academic achievement in high school. Pediatrics, 123(6), 1472–1476. https://doi.org/10.1542/peds.2008-1406.

    Article  PubMed  Google Scholar 

  • Burrows, A., Morris, J. N., Simons, S. E., Hirdes, J. P., & Phillips, C. (2000). Development of a minimum data set-based depression rating scale for use in nursing homes. Age and Ageing, 29(2), 165–172. https://doi.org/10.1093/ageing/29.2.165.

    Article  PubMed  Google Scholar 

  • Butcher, J. N., Williams, C. L., Graham, J. R., Archer, R. P., Tellegen, A., Ben-Porath, Y., et al. (1992). Minnesota Multiphasic Personlity Inventory-Adolescent: Manual for administration, scoring, and interpretation. Minneapolis: University of Minnesota Press.

    Google Scholar 

  • Chronis-Tuscano, A., Molina, B. S. G., Pelham, W. E., Applegate, B., Dahlke, A., Overmyer, M., & Lahey, B. B. (2010). Very early predictors of adolescent depression and suicide attempts in children with attention-deficit/hyperactivity disorder. Archives of General Psychiatry, 67(10), 1044. https://doi.org/10.1001/archgenpsychiatry.2010.127.

    Article  PubMed  PubMed Central  Google Scholar 

  • Cunningham, C. E., Boyle, M. H., Hong, S., Pettingill, P., & Bohaychuk, D. (2009). The Brief Child and Family Phone Interview (BCFPI): 1. Rationale, development, and description of a computerized children’s mental health intake and outcome assessment tool. Journal of Child Psychology and Psychiatry, 50(4), 416–423. https://doi.org/10.1111/j.1469-7610.2008.01970.x.

    Article  PubMed  Google Scholar 

  • Doyle, A., Ostrander, R., Skare, S., Crosby, R. D., & August, G. J. (1997). Convergent and criterion-related validity of the behavior assessment system for children-parent rating scale. Journal of Clinical Child Psychology, 26(3), 276–284. https://doi.org/10.1207/s15374424jccp2603_6.

    Article  PubMed  Google Scholar 

  • Fisher, K. A., Seow, H., Brazil, K., Smith, T. F., & Guthrie, D. M. (2015). Dimensionality, reliability and validity of the InterRAI depression rating scale in a Canadian palliative care population. Social Indicators Research, 122(1), 297–314. https://doi.org/10.1007/s11205-014-0685-8.

    Article  Google Scholar 

  • Gadermann, A. M., Guhn, M., & Zumbo, B. D. (2012). Estimating ordinal reliability for likert-type and ordinal item response data: A conceptual, empirical, and practical guide. Practical Assessment, Research & Evaluation, 17(3), 1–13.

    Google Scholar 

  • Gray, L. C., Berg, K., Fries, B. E., Henrard, J.-C., Hirdes, J. P., Steel, K., & Morris, J. N. (2009). Sharing clinical information across care settings: The birth of an integrated assessment system. BMC Health Services Research, 9(1), 71. https://doi.org/10.1186/1472-6963-9-71.

    Article  PubMed  PubMed Central  Google Scholar 

  • Greshsm, F., & Elliott, S. N. (2008). Social skills improvement system (SSIS) rating scales. Minneapolis: Pearson Assessments.

    Google Scholar 

  • Gross, D., Fogg, L., Young, M., Ridge, A., Cowell, J. M., Richardson, R., & Sivan, A. (2006). The equivalence of the Child Behavior Checklist/1 1/2-5 across parent race/ethnicity, income level, and language. Psychological Assessment, 18(3), 313–323. https://doi.org/10.1037/1040-3590.18.3.313.

    Article  PubMed  Google Scholar 

  • Gruber, J., & Weinstock, L. M. (2018). Interrater reliability in bipolar disorder research: Current practices and suggestions for enhancing best practices. International Journal of Bipolar Disorders, 6(1), 1. https://doi.org/10.1186/s40345-017-0111-7.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hamza, C. A., Stewart, S. L., & Willoughby, T. (2012). Examining the link between nonsuicidal self-injury and suicidal behavior: A review of the literature and an integrated model. Clinical Psychology Review, 32(6), 482–495. https://doi.org/10.1016/j.cpr.2012.05.003.

    Article  PubMed  Google Scholar 

  • Hawton, K., Saunders, K. E., & O’Connor, R. C. (2012). Self-harm and suicide in adolescents. Lancet, 379(9834), 2373–2382. https://doi.org/10.1016/S0140-6736(12)60322-5.

    Article  PubMed  Google Scholar 

  • Hemphälä, M., & Hodgins, S. (2014). Do psychopathic traits assessed in mid-adolescence predict mental health, psychosocial, and antisocial, including criminal outcomes, over the subsequent 5 years? Canadian Journal of Psychiatry, 59(1), 40–49. https://doi.org/10.1177/070674371405900108.

    Article  PubMed  PubMed Central  Google Scholar 

  • Henry, K. L., Knight, K. E., & Thornberry, T. P. (2012). School disengagement as a predictor of dropout, delinquency, and problem substance use during adolescence and early adulthood. Journal of Youth and Adolescence, 41(2), 156–166. https://doi.org/10.1007/s10964-011-9665-3.

    Article  PubMed  Google Scholar 

  • Hilden, J. (1991). The area under the ROC curve and Its competitors. Medical Decision Making, 11(2), 95–101. https://doi.org/10.1177/0272989X9101100204.

    Article  PubMed  Google Scholar 

  • Hirdes, J. P., Ljunggren, G., Morris, J. N., Frijters, D. H., Finne Soveri, H., Gray, L., et al. (2008). Reliability of the interRAI suite of assessment instruments: A 12-country study of an integrated health information system. BMC Health Services Research, 8(1), 277. https://doi.org/10.1186/1472-6963-8-277.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hirdes, J. P., van Everdingen, C., Ferris, J., Franco-Martin, M., Fries, B. E., Heikkilä, J., et al. (2020). The interRAI suite of mental health assessment instruments: An integrated system for the continuum of care. Frontiers in Psychiatry, 10, 926. https://doi.org/10.3389/fpsyt.2019.00926.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hodges, K. (2004). The Child and Adolescent Functional Assessment Scale (CAFAS). In M. E. Maruish (Ed.), The use of psychological testing for treatment planning and outcomes assessment: Instruments for children and adolescents, Vol. 2 (3rd ed., pp. 405–441). Abingdon-on-Thames: Routledge.

    Google Scholar 

  • Kessler, R., Berglund, P., Demler, O., Jin, R., Merikangas, K. R., & Walters, E. E. (2005). Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in National Comorbidity Survey Replication. Archives of Geneneral Psychiatry, 62(6), 593–602. https://doi.org/10.1001/archpsyc.62.6.593.

    Article  Google Scholar 

  • Kirby, M., & Keon, W. J. (2004). Mental health, mental illness and addiction: Overview of policies and programs in Canada. Senate of Canada, Standing Committee on Social Affairs, Science and Technology. www.parl.gc.ca.

  • Klassen, J. A., Hamza, C. A., & Stewart, S. L. (2017). An Examination of correlates for adolescent engagement in nonsuicidal self-injury. Suicidal Self-Injury, and Substance Use. https://doi.org/10.1111/jora.12333.

    Article  Google Scholar 

  • Lau, C., Stewart, S. L., Saklofske, D. H., & Hirdes, J. (2020). Development and psychometric validation of the interRAI ChYMH externalizing subscale. Clinical Child Psychology and Psychiatry. https://doi.org/10.1177/1359104520963143.

    Article  PubMed  Google Scholar 

  • Lau, C., Stewart, S. L., Saklofske, D. H., Tremblay, P. F., & Hirdes, J. (2018). Psychometric evaluation of the interRAI Child and Youth Mental Health Disruptive/Aggression Behaviour Scale (DABS) and Hyperactive/Distraction Scale (HDS). Child Psychiatry & Human Development, 49(2), 279–289. https://doi.org/10.1007/s10578-017-0751-y.

    Article  Google Scholar 

  • Lorenzo-Seva, U., & Ferrando, P. J. (2006). FACTOR: A computer program to fit the exploratory factor analysis model. Behavior Research Methods, 38(1), 88–91. https://doi.org/10.3758/BF03192753.

    Article  PubMed  Google Scholar 

  • Martin, L., Hirdes, J. P., Fries, B. E., & Smith, T. F. (2007). Development and psychometric properties of an assessment for persons with intellectual disability—The interRAI ID. Journal of Policy and Practice in Intellectual Disabilities, 4(1), 23–29. https://doi.org/10.1111/j.1741-1130.2006.00094.x.

    Article  Google Scholar 

  • Mulsant, B. H., Kastango, K. B., Rosen, J., Stone, R. A., Mazumdar, S., & Pollock, B. G. (2002). Interrater reliability in clinical trials of depressive disorders. American Journal of Psychiatry, 159(9), 1598–1600. https://doi.org/10.1176/appi.ajp.159.9.1598.

    Article  Google Scholar 

  • Nock, M. K. (2010). Self-injury. Annual Review of Clinical Psychology, 6(1), 339–363. https://doi.org/10.1146/annurev.clinpsy.121208.131258.

    Article  PubMed  Google Scholar 

  • Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.

    Google Scholar 

  • Perlman, C. M., Hirdes, J. P., Barbaree, H., Fries, B. E., McKillop, I., Morris, J. N., & Rabinowitz, T. (2013). Development of mental health quality indicators (MHQIs) for inpatient psychiatry based on the interRAI mental health assessment. BMC Health Services Research, 13(1), 15. https://doi.org/10.1186/1472-6963-13-15.

    Article  PubMed  PubMed Central  Google Scholar 

  • Peterson, R. A. (1994). A meta-analysis of Cronbach’s coefficient alpha. Journal of Consumer Research, 21(2), 381. https://doi.org/10.1086/209405.

    Article  Google Scholar 

  • Phillips, C. D., Patnaik, A., Dyer, J. A., Naiser, E., Hawes, C., Fournier, C. J., & Elliott, T. R. (2011). Reliability and the measurement of activity limitations (ADLs) for children with special health care needs (CSHCN) living in the community. Disability and Rehabilitation, 33(21–22), 2013–2022. https://doi.org/10.3109/09638288.2011.555596.

    Article  PubMed  Google Scholar 

  • Phillips, C. D., Patnaik, A., Moudouni, D. K., Naiser, E., Dyer, J. A., Hawes, C., et al. (2012). Summarizing activity limitations in children with chronic illnesses living in the community: A measurement study of scales using supplemented interRAI items. BMC Health Services Research, 12(1), 19. https://doi.org/10.1186/1472-6963-12-19.

    Article  PubMed  PubMed Central  Google Scholar 

  • Porche, M. V., Fortuna, L. R., Lin, J., & Alegria, M. (2011). Childhood trauma and psychiatric disorders as correlates of school dropout in a national sample of young adults. Child Development, 82(3), 982–998. https://doi.org/10.1111/j.1467-8624.2010.01534.x.

    Article  PubMed  PubMed Central  Google Scholar 

  • Redquest, B. K., Stewart, S. L., Bryden, P. J., & Fletcher, P. C. (2020). Assessing the wellbeing of individuals with autism using the interRAI Child and Youth Mental Health (ChYMH) and the interRAI Child and Youth Mental Health—Developmental disabilities (ChYMH-DD) tools. Pediatric Nursing, 46(2), 83–91.

    Google Scholar 

  • Reynolds, C. R., & Kamphaus, R. W. (1998). Behavior Assessment system for children: Manual. Circle Pines: American Guidance Service Publishing.

    Google Scholar 

  • Reynolds, C. R., & Kamphaus, R. W. (2004). Behavior assessment for children, (BASC-2). Circle Pines: American Guidance Service Publishing.

    Google Scholar 

  • Reynolds, C. R., Kamphaus, R. W., & Vannest, K. J. (2015). BASC3: Behavior assessment system for children. PsychCorp.

  • Stanger, C., & Lewis, M. (1993). Agreement among parents, teachers, and children on internalizing and externalizing behavior problems. Journal of Clinical Child Psychology, 22(1), 107–116. https://doi.org/10.1207/s15374424jccp2201_11.

    Article  Google Scholar 

  • Stewart, S. L., & Babcock, S. E. (2020). InterRAI Child and Youth Mental Health-Screener (ChYMH-S): A psychometric evaluation and validation study. Child Psychiatry and Human Development. https://doi.org/10.1007/s10578-020-01003-7.

    Article  PubMed  PubMed Central  Google Scholar 

  • Stewart, S. L., Babcock, S. E., Li, Y., & Dave, H. P. (2020). A psychometric evaluation of the interRAI Child and Youth Mental Health instruments (ChYMH) anxiety scale in children with and without developmental disabilities. BMC Psychiatry, 20(1), 390. https://doi.org/10.1186/s12888-020-02785-9.

    Article  PubMed  PubMed Central  Google Scholar 

  • Stewart, S. L., Baiden, P., & Theall-Honey, L. (2014). Examining non-suicidal self-injury among adolescents with mental health needs, in Ontario, Canada. Archives of Suicide Research, 18(4), 392–409. https://doi.org/10.1080/13811118.2013.824838.

    Article  PubMed  Google Scholar 

  • Stewart, S. L., Celebre, A., Hirdes, J. P., & Poss, J. W. (2020). Risk of suicide and self-harm in kids: The development of an algorithm to identify high-risk individuals within the Children’s Mental Health System. Child Psychiatry and Human Development. https://doi.org/10.1007/s10578-020-00968-9.

    Article  PubMed  PubMed Central  Google Scholar 

  • Stewart, S. L., Currie, M., Arbeau, K., Leschied, A., & Kerry, A. (2015a). Assessment and planning for community and custodial services: The application of interRAI assessment in the youth justice system. In Serious and violent young offenders and youth criminal justice: A Canadian perspective (pp. 355–397).

  • Stewart, S. L., & Hamza, C. A. (2017). The Child and Youth Mental Health Assessment (ChYMH): An examination of the psychometric properties of an integrated assessment developed for clinically referred children and youth. BMC Health Services Research, 17(1), 82. https://doi.org/10.1186/s12913-016-1970-9.

    Article  PubMed  PubMed Central  Google Scholar 

  • Stewart, S. L., & Hirdes, J. P. (2015). Identifying mental health symptoms in children and youth in residential and in-patient care settings. Healthcare Management Forum/Canadian College of Health Service Executives = Forum Gestion Des Soins De Santé/Collège Canadien Des Directeurs De Services De Santé, 28(4), 150–156. https://doi.org/10.1177/0840470415581240.

    Article  Google Scholar 

  • Stewart, S. L., Hirdes, J. P., Curtin-Telegdi, N., Perlman, C. M., McKnight, M., MacLeod, K., et al. (2015b). InterRAI Child and Youth Mental Health (ChYMH) assessment form and user’s manual: For use with in-patient and community-based assessment. interRAI.

  • Stewart, S. L., Hirdes, J. P., Curtin-Telegdi, N., Perlman, C. M., McKnight, M., & MacLeod, K. (2018a). Santé mentale des enfants et des adolescents (SMEA) interRAI manuel de l’utilisateur et formulaire d’évaluation: Pour utilisation avec les évaluations en milieu hospitalier et en milieu communau-taire. Version 9.3, Édition canadienne-française. interRAI.

  • Stewart, S. L., Hirdes, J. P., & McKnight, M. (2015c). InterRAI Child and Youth Mental Health-Screener (ChYMH-Screener) assessment form and user’s manual. Version 1.1. InterRAI.

  • Stewart, S. L., Hirdes, J. P., McKnight, M., Curtin-Telegdi, N., Perlman, C. M., MacLeod, K., et al. (2017). interRAI Child and Youth Mental Health Screener (ChYMH-S) assessment form and user’s manual. Version 9.3. interRAI.

  • Stewart, S. L., Morris, J. N., Asare-Bediako, Y. A., & Toohey, A. (2019). Examining the structure of a new pediatric measure of functional independence using the interRAI Child and Youth Mental Health assessment system. Developmental Neurorehabilitation. https://doi.org/10.1080/17518423.2019.1698070.

    Article  PubMed  Google Scholar 

  • Stewart, S. L., Theall, L. A., Morris, J. N., Berg, K., Björkgren, M., & Declercg, A. (2018b). Guides d’analyse par domaine d’intervention (GAD) pour la santé mentale des enfants et des adolescents interRAI: Pour utilisation avec l’instrument d’évaluation de la santé mentale des enfants et des adolescents. Version 9.3, Édition canadienne-française. interRAI.

  • Veldman, K., Reijneveld, S. A., Ortiz, J. A., Verhulst, F. C., & Bültmann, U. (2015). Mental health trajectories from childhood to young adulthood affect the educational and employment status of young adults: Results from the TRAILS study. Journal of Epidemiology and Community Health, 69(6), 588–593. https://doi.org/10.1136/jech-2014-204421.

    Article  PubMed  Google Scholar 

  • Waddell, C., McEwan, K., Shepherd, C. A., Offord, D. R., & Hua, J. M. (2005). A public health strategy to improve the mental health of Canadian children. The Canadian Journal of Psychiatry, 50(4), 226–233. https://doi.org/10.1177/070674370505000406.

    Article  PubMed  Google Scholar 

  • Weis, R., & Smenner, L. (2007). Construct validity of the behavior assessment system for children (BASC) self-report of personality. Journal of Psychoeducational Assessment, 25(2), 111–126. https://doi.org/10.1177/0734282906293762.

    Article  Google Scholar 

  • Zweig, M. H., & Campbell, G. (1993). Receiver-operating characteristic (ROC) plots: A fundamental evaluation tool in clinical medicine. Clinical Chemistry, 39(4), 561–577.

    Article  Google Scholar 

Download references

Acknowledgements

This project was funded by the London Community Foundation. The funders did not participate in the study design, data analysis, or preparation of the manuscript.

Author information

Authors and Affiliations

  1. Department of Psychology, Faculty of Social Sciences, Western University, 1151 Richmond St., London, ON, N6A 3K7, Canada

    Yixian Li & Sarah E. Babcock

  2. Faculty of Education, Western University, London, ON, Canada

    Shannon L. Stewart & Vicki L. Schwean

  3. School of Public Health and Health Systems, University of Waterloo, 200 University Ave W, Waterloo, ON, N2L 3G1, Canada

    John P. Hirdes

Authors
  1. Yixian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sarah E. Babcock
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shannon L. Stewart
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. John P. Hirdes
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Vicki L. Schwean
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yixian Li.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethics Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from the primary caregivers of all the children and youth included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Babcock, S.E., Stewart, S.L. et al. Psychometric Evaluation of the Depressive Severity Index (DSI) Among Children and Youth Using the interRAI Child and Youth Mental Health (ChYMH) Assessment Tool. Child Youth Care Forum 50, 611–630 (2021). https://doi.org/10.1007/s10566-020-09592-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10566-020-09592-z

Keywords

Search

Navigation