Skip to main content
Top
Gepubliceerd in:

01-02-2023

Multi-informant Assessment of Internalizing Concerns: Rater Concordance and Implications for Decision-Making

Auteurs: Nathaniel von der Embse, Eunsook Kim, Dorie Ross, Stephen Kilgus, Thomas Koza

Gepubliceerd in: Journal of Psychopathology and Behavioral Assessment | Uitgave 1/2023

Log in om toegang te krijgen
share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail

Abstract

Rising rates of mental health challenges among youths have become a significant concern following the COVID 19 pandemic. Although strong evidence supports the implementation of universal screening as a preventative approach to address unmet mental health concerns, the research is less clear surrounding the use of such data in decision-making processes when significant discrepancies between informants (e.g., students and teachers) exist. The purpose of the study was twofold. First, the study aimed to determine the degree of rater concordance between teachers and students on students’ internalizing concerns. The second objective was to determine whether concordance on internalizing behaviors differs across ages/grades and if this differentially impacts distal (i.e., academic) outcomes. Results indicated that teachers and students demonstrated limited agreement on ratings of internalizing behaviors. However, when students and teachers agreed, higher and more positive emotional behaviors were linked to higher reading/math performance. Furthermore, patterns of informant dis/agreement and relationships between internalizing concerns and academic outcomes were similar across grade levels. Implications and areas for future research are discussed.
Literatuur
go back to reference Anderson, J. K., Ford, T., Soneson, E., Coon, J. T., Humphrey, A., Rogers, M., Moore, D., Jones, P. B., Clarke, E., & Howarth, E. (2019). A systematic review of effectiveness and cost-effectiveness of school-based identification of children and young people at risk of, or currently experiencing mental health difficulties. Psychological Medicine, 49(1), 9–19. https://doi.org/10.1017/S0033291718002490.CrossRefPubMed Anderson, J. K., Ford, T., Soneson, E., Coon, J. T., Humphrey, A., Rogers, M., Moore, D., Jones, P. B., Clarke, E., & Howarth, E. (2019). A systematic review of effectiveness and cost-effectiveness of school-based identification of children and young people at risk of, or currently experiencing mental health difficulties. Psychological Medicine, 49(1), 9–19. https://​doi.​org/​10.​1017/​S003329171800249​0.CrossRefPubMed
go back to reference Baumgartner, N., Häberling, I., Emery, S., Strumberger, M., Nalani, K., Erb, S., Bachmann, S., Wöckel, L., Müller-Knapp, U., Rhiner, B., Contin-Waldvogel, B., Schmeck, K., Walitza, S., & Berger, G. (2020). When parents and children disagree: informant discrepancies in reports of depressive symptoms in clinical interviews. Journal of Affective Disorders, 272, 223–230. https://doi.org/10.1016/j.jad.2020.04.008.CrossRefPubMed Baumgartner, N., Häberling, I., Emery, S., Strumberger, M., Nalani, K., Erb, S., Bachmann, S., Wöckel, L., Müller-Knapp, U., Rhiner, B., Contin-Waldvogel, B., Schmeck, K., Walitza, S., & Berger, G. (2020). When parents and children disagree: informant discrepancies in reports of depressive symptoms in clinical interviews. Journal of Affective Disorders, 272, 223–230. https://​doi.​org/​10.​1016/​j.​jad.​2020.​04.​008.CrossRefPubMed
go back to reference Christ, T. J., Aranas, Y., Johnson, L., Kember, J., Kilgus, S., Kiss, A., McCarthy Trentman, A., Monaghen, B., Nelson, G., Nelson, P., Newell, K., Van Norman, E., White, M., & Windram, H. (2014). Formative Assessment System for Teachers Technical Manual. Minneapolis, MN: FastBridge Learning. Christ, T. J., Aranas, Y., Johnson, L., Kember, J., Kilgus, S., Kiss, A., McCarthy Trentman, A., Monaghen, B., Nelson, G., Nelson, P., Newell, K., Van Norman, E., White, M., & Windram, H. (2014). Formative Assessment System for Teachers Technical Manual. Minneapolis, MN: FastBridge Learning.
go back to reference Collins, T. A., Dart, E. H., & Arora, P. G. (2019). Addressing the internalizing behavior of students in schools: Applications of the MTSS model. School Mental Health, 11(2), 191–193.CrossRef Collins, T. A., Dart, E. H., & Arora, P. G. (2019). Addressing the internalizing behavior of students in schools: Applications of the MTSS model. School Mental Health, 11(2), 191–193.CrossRef
go back to reference De Los Reyes, A., Augenstein, T. M., Wang, M., Thomas, S. A., Drabick, D. A. G., Burgers, D. E., & Rabinowitz, J. (2015). The validity of the multi-informant approach to assessing child and adolescent mental health. Psychological Bulletin, 141(4), 858–900.CrossRefPubMedCentral De Los Reyes, A., Augenstein, T. M., Wang, M., Thomas, S. A., Drabick, D. A. G., Burgers, D. E., & Rabinowitz, J. (2015). The validity of the multi-informant approach to assessing child and adolescent mental health. Psychological Bulletin, 141(4), 858–900.CrossRefPubMedCentral
go back to reference Evans, S. W., Simonsen, B., & Dolan, G. (2013). School-level practices. In L. Eber, S. Barrett, & M. Weist (Eds.), Advancing education effectiveness: interconnecting school mental health and school-wide positive behavioral interventions and supports (pp. 86–107). Eugene, OR: University of Oregon, College of Education. Evans, S. W., Simonsen, B., & Dolan, G. (2013). School-level practices. In L. Eber, S. Barrett, & M. Weist (Eds.), Advancing education effectiveness: interconnecting school mental health and school-wide positive behavioral interventions and supports (pp. 86–107). Eugene, OR: University of Oregon, College of Education.
go back to reference Kilgus, S. P., Bowman, N. A., Christ, T. J., & Taylor, C. N. (2017). Predicting academics via behavior within an elementary sample: an evaluation of the Social, Academic, and emotional behavior risk screener (SAEBRS). Psychology in the Schools, 54(3), 246–260. https://doi.org/10.1002/pits.21995.CrossRef Kilgus, S. P., Bowman, N. A., Christ, T. J., & Taylor, C. N. (2017). Predicting academics via behavior within an elementary sample: an evaluation of the Social, Academic, and emotional behavior risk screener (SAEBRS). Psychology in the Schools, 54(3), 246–260. https://​doi.​org/​10.​1002/​pits.​21995.CrossRef
go back to reference Kilgus, S. P., Eklund, K., von der Embse, N. P., Taylor, C. N., & Sims, W. A. (2016). Psychometric defensibility of the Social, Academic, and emotional behavior risk screener (SAEBRS) teacher rating scale and multiple gating procedure within elementary and middle school samples. Journal of School Psychology, 58, 21–39. https://doi.org/10.1016/j.jsp.2016.07.001.CrossRefPubMed Kilgus, S. P., Eklund, K., von der Embse, N. P., Taylor, C. N., & Sims, W. A. (2016). Psychometric defensibility of the Social, Academic, and emotional behavior risk screener (SAEBRS) teacher rating scale and multiple gating procedure within elementary and middle school samples. Journal of School Psychology, 58, 21–39. https://​doi.​org/​10.​1016/​j.​jsp.​2016.​07.​001.CrossRefPubMed
go back to reference Kilgus, S. P., & von der Embse, N. P. (2014). Social, Academic, and Emotional Behavior Risk Screener (SAEBRS). Unpublished technical manual. Kilgus, S. P., & von der Embse, N. P. (2014). Social, Academic, and Emotional Behavior Risk Screener (SAEBRS). Unpublished technical manual.
go back to reference Makol, B. A., Youngstrom, E. A., Racz, S. J., Qasmieh, N., Glenn, L. E., & De Los Reyes, A. (2020). Integrating multiple Informants’ reports: how conceptual and measurement models may address long-standing problems in clinical decision-making. Clinical Psychological Science, 8(6), 953–970. https://doi.org/10.1177/2167702620924439.CrossRef Makol, B. A., Youngstrom, E. A., Racz, S. J., Qasmieh, N., Glenn, L. E., & De Los Reyes, A. (2020). Integrating multiple Informants’ reports: how conceptual and measurement models may address long-standing problems in clinical decision-making. Clinical Psychological Science, 8(6), 953–970. https://​doi.​org/​10.​1177/​2167702620924439​.CrossRef
go back to reference McDonald, R. P. (1999). Test theory: a unified treatment. Mahwah, NJ: Erlbaum. McDonald, R. P. (1999). Test theory: a unified treatment. Mahwah, NJ: Erlbaum.
go back to reference Merikangas, K. R., He, J., Burstein, M., Swanson, S. A., Avenevoli, S., Cui, L., Benjet, C., Georgiades, K., & Swendsen, J. (2010). Lifetime prevalence of Mental Disorders in US adolescents: results from the National Comorbidity Survey replication–adolescent supplement (NCS-A). Journal of the American Academy of Child & Adolescent Psychiatry, 49(10), 980–989. https://doi.org/10.1016/j.jaac.2010.05.017.CrossRef Merikangas, K. R., He, J., Burstein, M., Swanson, S. A., Avenevoli, S., Cui, L., Benjet, C., Georgiades, K., & Swendsen, J. (2010). Lifetime prevalence of Mental Disorders in US adolescents: results from the National Comorbidity Survey replication–adolescent supplement (NCS-A). Journal of the American Academy of Child & Adolescent Psychiatry, 49(10), 980–989. https://​doi.​org/​10.​1016/​j.​jaac.​2010.​05.​017.CrossRef
go back to reference Perfect, M. M., Turley, M. R., Carlson, J. S., Yohanna, J., & Saint Gilles, M. P. (2016). School-Related outcomes of traumatic event exposure and traumatic stress symptoms in students: a systematic review of Research from 1990 to 2015. School Mental Health, 8(1), 7–43. https://doi.org/10.1007/s12310-016-9175-2.CrossRef Perfect, M. M., Turley, M. R., Carlson, J. S., Yohanna, J., & Saint Gilles, M. P. (2016). School-Related outcomes of traumatic event exposure and traumatic stress symptoms in students: a systematic review of Research from 1990 to 2015. School Mental Health, 8(1), 7–43. https://​doi.​org/​10.​1007/​s12310-016-9175-2.CrossRef
go back to reference Radhakrishnan, L., Leeb, R. T., Bitsko, R. H., Carey, K., Gates, A., Holland, K. M., Hartnett, K. P., Kite-Powell, A., DeVies, J., Smith, A. R., van Santen, K. L., Crossen, S., Sheppard, M., Wotiz, S., Lane, R. I., Njai, R., Johnson, A. G., Winn, A., & Kirking,. Anderson, K. N. (2022). Pediatric Emergency Department Visits Associated with Mental Health Conditions Before and During the COVID-19 Pandemic — United States, January 2019–January 2022. MMWR Morb Mortal Wkly Rep 2022, 71(8), 319–324. https://doi.org/10.15585/mmwr.mm7108e2 Radhakrishnan, L., Leeb, R. T., Bitsko, R. H., Carey, K., Gates, A., Holland, K. M., Hartnett, K. P., Kite-Powell, A., DeVies, J., Smith, A. R., van Santen, K. L., Crossen, S., Sheppard, M., Wotiz, S., Lane, R. I., Njai, R., Johnson, A. G., Winn, A., & Kirking,. Anderson, K. N. (2022). Pediatric Emergency Department Visits Associated with Mental Health Conditions Before and During the COVID-19 Pandemic — United States, January 2019–January 2022. MMWR Morb Mortal Wkly Rep 2022, 71(8), 319–324. https://​doi.​org/​10.​15585/​mmwr.​mm7108e2
go back to reference von der Romer, N., Eklund, K., Kilgus, S., Perales, K., Splett, J. W., Sudlo, S., & Wheeler, D. (2020). Best Practices in Social, Emotional, and Behavioral Screening: An Implementation Guide. Version 2.0. Retrieved from smhcollaborative.org/universalscreening von der Romer, N., Eklund, K., Kilgus, S., Perales, K., Splett, J. W., Sudlo, S., & Wheeler, D. (2020). Best Practices in Social, Emotional, and Behavioral Screening: An Implementation Guide. Version 2.0. Retrieved from smhcollaborative.org/universalscreening
go back to reference Twenge, J. M., Cooper, A. B., Joiner, T. E., Duffy, M. E., & Binau, S. G. (2019). Age, period, and cohort trends in mood disorder indicators and suicide-related outcomes in a nationally representative dataset, 2005–2017. Journal of Abnormal Psychology, 128(3), 185–199. https://doi.org/10.1037/abn0000410.CrossRefPubMed Twenge, J. M., Cooper, A. B., Joiner, T. E., Duffy, M. E., & Binau, S. G. (2019). Age, period, and cohort trends in mood disorder indicators and suicide-related outcomes in a nationally representative dataset, 2005–2017. Journal of Abnormal Psychology, 128(3), 185–199. https://​doi.​org/​10.​1037/​abn0000410.CrossRefPubMed
go back to reference von der Embse, N.P., Kilgus, S.P., Iaccarino, S., & Levi-Nielsen, S. (2017). Screening for student mental health risk: Diagnostic accuracy, measurement, invariance, and predictive validity of the Social, Academic, and Emotional Behavior Risk Screener-Student Rating Scale (SAEBRS-SRS). School Mental Health, 9,273–283. https://doi.org/10.1007/s12310-017-9214-7. von der Embse, N.P., Kilgus, S.P., Iaccarino, S., & Levi-Nielsen, S. (2017). Screening for student mental health risk: Diagnostic accuracy, measurement, invariance, and predictive validity of the Social, Academic, and Emotional Behavior Risk Screener-Student Rating Scale (SAEBRS-SRS). School Mental Health, 9,273–283. https://​doi.​org/​10.​1007/​s12310-017-9214-7.
go back to reference von der Embse, N. P., Pendergast, L. L., Kilgus, S. P., & Eklund, K. R. (2016). Evaluating the applied use of a mental health screener: structural validity of the Social, Academic, and emotional behavior risk screener. Psychological Assessment, 28(10), 1265–1275. https://doi.org/10.1037/pas0000253.CrossRefPubMed von der Embse, N. P., Pendergast, L. L., Kilgus, S. P., & Eklund, K. R. (2016). Evaluating the applied use of a mental health screener: structural validity of the Social, Academic, and emotional behavior risk screener. Psychological Assessment, 28(10), 1265–1275. https://​doi.​org/​10.​1037/​pas0000253.CrossRefPubMed
Metagegevens
Titel
Multi-informant Assessment of Internalizing Concerns: Rater Concordance and Implications for Decision-Making
Auteurs
Nathaniel von der Embse
Eunsook Kim
Dorie Ross
Stephen Kilgus
Thomas Koza
Publicatiedatum
01-02-2023
Uitgeverij
Springer US
Gepubliceerd in
Journal of Psychopathology and Behavioral Assessment / Uitgave 1/2023
Print ISSN: 0882-2689
Elektronisch ISSN: 1573-3505
DOI
https://doi.org/10.1007/s10862-023-10026-y

Andere artikelen Uitgave 1/2023

Journal of Psychopathology and Behavioral Assessment 1/2023 Naar de uitgave