Skip to main content
main-content
Top

Tip

Swipe om te navigeren naar een ander artikel

Gepubliceerd in: Journal of Psychopathology and Behavioral Assessment 3/2021

04-02-2021

Profiles of Rater Dis/Agreement within Universal Screening in Predicting Distal Outcomes

Auteurs: Nathaniel von der Embse, Eunsook Kim, Andrew Jenkins, Alexis Sanchez, Stephen P. Kilgus, Katie Eklund

Gepubliceerd in: Journal of Psychopathology and Behavioral Assessment | Uitgave 3/2021

Log in om toegang te krijgen
share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail

Abstract

The success of a comprehensive and integrated school mental health system is dependent on data to inform decision-making. Universal screening is one such method utilized by schools to inform treatment across universal, targeted, and individual levels of service. However, the use of multiple informants (teacher and student) remains elusive due to a lack of practical and research based-aggregation guidance. The purpose of the current study is to identify the optimal numbers of rater agreement profiles, determine demographic predictors of profiles membership, and if profiles differentially predict distal academic outcomes. Nearly 24,000 teacher-student dyads were included in this study. Results suggested four profiles emerged with significant differences in academic outcomes. Considerations for research and practice are presented.

Met onderstaand(e) abonnement(en) heeft u direct toegang:

BSL Psychologie Totaal

Met BSL Psychologie Totaal blijf je als professional steeds op de hoogte van de nieuwste ontwikkelingen binnen jouw vak. Met het online abonnement heb je toegang tot een groot aantal boeken, protocollen, vaktijdschriften en e-learnings op het gebied van psychologie en psychiatrie. Zo kun je op je gemak en wanneer het jou het beste uitkomt verdiepen in jouw vakgebied.

Voetnoten
1
We also conducted Little’s MCAR tests: χ2(317) = 746.88, p < .001 for student and teacher SAEBRS scores; χ2(317) = 700.51, p < .001 with gender and race covariates additionally; and χ2(507) = 3645.74, p < .001 with reading and math outcomes additionally. However, given large sample size in this study, we majorly interpreted the relationship between missingness and other variables with correlation coefficients for the missing mechanism.
 
2
As reported earlier, the missingness of gender and race were not associated with any variables in the model and thus possibly happened completely at random. When we compared the results of a model with special education status and grade which were complete (N = 24,094) to those of a model with all four predictors (N = 16,710), general findings in terms of special education status and grade remained consistent although their coefficients changed by controlling for the other predictors. Hence, we concluded that listwise deletion less likely made any notable differences in the findings. In addition, we did not consider multiple imputation which generally assumes multivariate normality that is not relevant for binary variables, gender and race.
 
Literatuur
go back to reference Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioural and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101, 213–232. Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioural and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101, 213–232.
go back to reference Borsboom, D. (2005). Measuring the mind: Conceptual issues in contemporary psychometrics . Cambridge University Press. Borsboom, D. (2005). Measuring the mind: Conceptual issues in contemporary psychometrics . Cambridge University Press.
go back to reference Christ, T. J., Aranas, Y., Johnson, L., Kember, J., Kilgus, S., Kiss, A., et al. (2014). Formative Assessment System for Teachers Technical Manual. Minneapolis, MN: FastBridge Learning. Christ, T. J., Aranas, Y., Johnson, L., Kember, J., Kilgus, S., Kiss, A., et al. (2014). Formative Assessment System for Teachers Technical Manual. Minneapolis, MN: FastBridge Learning.
go back to reference Collins, L. M., & Lanza, S. T. (2010). Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences . John Wiley & Sons, Inc. Collins, L. M., & Lanza, S. T. (2010). Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences . John Wiley & Sons, Inc.
go back to reference De Los Reyes, A. (2013). Strategic objectives for improving understanding of informant discrepancies in developmental psychopathology research. Development and Psychopathology, 25(3), 669–682. CrossRef De Los Reyes, A. (2013). Strategic objectives for improving understanding of informant discrepancies in developmental psychopathology research. Development and Psychopathology, 25(3), 669–682. CrossRef
go back to reference DiStefano, C. (2012). Cluster analysis and latent class clustering techniques. In B. Laursen, T. D. Little, & N. A. Card (Eds.), Handbook of developmental research methods (pp. 645–666). Guilford Publications: New York. DiStefano, C. (2012). Cluster analysis and latent class clustering techniques. In B. Laursen, T. D. Little, & N. A. Card (Eds.), Handbook of developmental research methods (pp. 645–666). Guilford Publications: New York.
go back to reference Dowdy, E., & Kim, R. (2012). Choosing informants when conducting a universal screening for behavioral and emotional risk. School Psychology Forum, 6(4), 1–10. Dowdy, E., & Kim, R. (2012). Choosing informants when conducting a universal screening for behavioral and emotional risk. School Psychology Forum, 6(4), 1–10.
go back to reference Duckworth, A. L., & Yeager, D. S. (2015). Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44, 237–251. CrossRef Duckworth, A. L., & Yeager, D. S. (2015). Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44, 237–251. CrossRef
go back to reference Elliott, S. N., & Gresham, F. (2008). Social Skills Improvement System (SSiS)—Performance Screening Guide. San Antonio, TX: PsychCorp Pearson Education. Elliott, S. N., & Gresham, F. (2008). Social Skills Improvement System (SSiS)—Performance Screening Guide. San Antonio, TX: PsychCorp Pearson Education.
go back to reference Goolsby, C., Jacobs, L., Hunt, R. C., Goralnick, E., Singletary, E. M., Levy, M. J., Goodloe, J. M., Epstein, J. L., Strauss-Riggs, K., Seitz, S. R., Krohmer, J. R., Nemeth, I., Rowe, D. W., Bradley, R. N., Gestring, M. L., & Kirsch, T. D. (2018). Stop the bleed education consortium. Journal of Trauma and Acute Care Surgery, 84(1), 205–210. https://​doi.​org/​10.​1097/​TA.​0000000000001732​. CrossRef Goolsby, C., Jacobs, L., Hunt, R. C., Goralnick, E., Singletary, E. M., Levy, M. J., Goodloe, J. M., Epstein, J. L., Strauss-Riggs, K., Seitz, S. R., Krohmer, J. R., Nemeth, I., Rowe, D. W., Bradley, R. N., Gestring, M. L., & Kirsch, T. D. (2018). Stop the bleed education consortium. Journal of Trauma and Acute Care Surgery, 84(1), 205–210. https://​doi.​org/​10.​1097/​TA.​0000000000001732​. CrossRef
go back to reference Kamphaus, R. W. (2012). Screening for behavioral and emotional risk: Constructs and practicalities. School Psychology Forum, 6(4), 89–97. Kamphaus, R. W. (2012). Screening for behavioral and emotional risk: Constructs and practicalities. School Psychology Forum, 6(4), 89–97.
go back to reference Kilgus, S. P., Bowman, N. A., Christ, T. J., & Taylor, C. N. (2017). Predicting academics via behavior within an elementary sample: An evaluation of the social, academic, and emotional behavior risk screener (SAEBRS). Psychology in the Schools, 54, 246–260. CrossRef Kilgus, S. P., Bowman, N. A., Christ, T. J., & Taylor, C. N. (2017). Predicting academics via behavior within an elementary sample: An evaluation of the social, academic, and emotional behavior risk screener (SAEBRS). Psychology in the Schools, 54, 246–260. CrossRef
go back to reference Kilgus, S. P., Chafouleas, S. M., Riley-Tillman, T. C., & von der Embse, N. P. (2014). Social, academic, and emotional behavior risk screener (SAEBRS). Minneapolis: Theodore J. Christ & Colleagues. Kilgus, S. P., Chafouleas, S. M., Riley-Tillman, T. C., & von der Embse, N. P. (2014). Social, academic, and emotional behavior risk screener (SAEBRS). Minneapolis: Theodore J. Christ & Colleagues.
go back to reference Kilgus, S. P., & von der Embse, N. P. (2014). Unpublished technical manual of the social, academic, and emotional behavior risk screener . Kilgus, S. P., & von der Embse, N. P. (2014). Unpublished technical manual of the social, academic, and emotional behavior risk screener .
go back to reference Laursen, B., & Hoff, E. (2006). Person-centered and variable-centered approaches to longitudinal data. Merrill-Palmer Quarterly, 52(3), 377–389. CrossRef Laursen, B., & Hoff, E. (2006). Person-centered and variable-centered approaches to longitudinal data. Merrill-Palmer Quarterly, 52(3), 377–389. CrossRef
go back to reference Makol, B. A., Youngstrom, E. A., Racz, S. J., Qasmieh, N., Glenn, L. E., & De Los Reyes, A. (2020). Integrating multiple informants’ reports: How conceptual and measurement models may address long-standing problems in clinical decision-making. Clinical Psychological Science. Makol, B. A., Youngstrom, E. A., Racz, S. J., Qasmieh, N., Glenn, L. E., & De Los Reyes, A. (2020). Integrating multiple informants’ reports: How conceptual and measurement models may address long-standing problems in clinical decision-making. Clinical Psychological Science.
go back to reference Masyn, K. E. (2013). 25 latent class analysis and finite mixture modeling. The Oxford handbook of quantitative methods, 551 Masyn, K. E. (2013). 25 latent class analysis and finite mixture modeling. The Oxford handbook of quantitative methods, 551
go back to reference Marsh, H. W., Lüdtke, O., Trautwein, U., & Morin, A. J. S. (2009). Classical latent profile analysis of academic self-concept dimensions: Synergy of person- and variable-centered approaches to theoretical models of self-concept. Structural Equation Modeling, 16, 191–225. CrossRef Marsh, H. W., Lüdtke, O., Trautwein, U., & Morin, A. J. S. (2009). Classical latent profile analysis of academic self-concept dimensions: Synergy of person- and variable-centered approaches to theoretical models of self-concept. Structural Equation Modeling, 16, 191–225. CrossRef
go back to reference Pastor, D. A., Barron, K. E., Miller, B. J., & Davis, S. L. (2007). A latent profile analysis of college students’ achievement goal orientation. Contemporary Educational Psychology, 32, 8–47. CrossRef Pastor, D. A., Barron, K. E., Miller, B. J., & Davis, S. L. (2007). A latent profile analysis of college students’ achievement goal orientation. Contemporary Educational Psychology, 32, 8–47. CrossRef
go back to reference Pendergast, L., von der Embse, N. P., Kilgus, S. P., & Eklund, K. (2017). Measurement equivalence in school psychology research: A primer and illustrated example of multi-group confirmatory factor analysis for non-statisticians. Journal of School Psychology, 60, 65–82. CrossRef Pendergast, L., von der Embse, N. P., Kilgus, S. P., & Eklund, K. (2017). Measurement equivalence in school psychology research: A primer and illustrated example of multi-group confirmatory factor analysis for non-statisticians. Journal of School Psychology, 60, 65–82. CrossRef
go back to reference Perou, R., Bitsko, R. H., Blumberg, S. J., Pastor, P., Ghandour, R. M., Gfroerer, J. C., Hedden, S. L., Crosby, A. E., Visser, S. N., Schieve, L. A., Parks, S. E., Hall, J. E., Brody, D., Simile, C. M., Thompson, W. W., Baio, J., Avenevoli, S., Kogan, M. D., & Huang, L. N. (2013). Mental health surveillance among children--United States, 2005-2011. Centers for Disease Control and Prevention Morbidity and Mortality Weekly Report, 62(2), 1–35. Perou, R., Bitsko, R. H., Blumberg, S. J., Pastor, P., Ghandour, R. M., Gfroerer, J. C., Hedden, S. L., Crosby, A. E., Visser, S. N., Schieve, L. A., Parks, S. E., Hall, J. E., Brody, D., Simile, C. M., Thompson, W. W., Baio, J., Avenevoli, S., Kogan, M. D., & Huang, L. N. (2013). Mental health surveillance among children--United States, 2005-2011. Centers for Disease Control and Prevention Morbidity and Mortality Weekly Report, 62(2), 1–35.
go back to reference Romer, N., von der Embse, N., Eklund, K., Kilgus, S., Perales, K., Splett, J. W., Sudlo, S., Wheeler, D. (2020). Best Practices in Social, Emotional, and Behavioral Screening: An Implementation Guide. Version 2.0. From www.smhcollaborative.org/universalscreening Romer, N., von der Embse, N., Eklund, K., Kilgus, S., Perales, K., Splett, J. W., Sudlo, S., Wheeler, D. (2020). Best Practices in Social, Emotional, and Behavioral Screening: An Implementation Guide. Version 2.0. From www.smhcollaborative.org/universalscreening
go back to reference Shelton, A. J., Wang, C. D., & Zhu, W. (2017). Perceived social support and mental health: Cultural orientations as moderators. Journal of College Counseling, 20(3), 194–207. CrossRef Shelton, A. J., Wang, C. D., & Zhu, W. (2017). Perceived social support and mental health: Cultural orientations as moderators. Journal of College Counseling, 20(3), 194–207. CrossRef
go back to reference Skiba, R. J., Michael, R. S., Nardo, A. C., & Peterson, R. L. (2002). The color of discipline: Sources of racial and gender disproportionality in school punishment. The Urban Review, 34(4), 317–342. CrossRef Skiba, R. J., Michael, R. S., Nardo, A. C., & Peterson, R. L. (2002). The color of discipline: Sources of racial and gender disproportionality in school punishment. The Urban Review, 34(4), 317–342. CrossRef
go back to reference Sklad, M., Diekstra, R., Ritter, M., Ben, J., & Gravesteijn, C. (2012). Effectiveness of school-based universal social, emotional, behavioral programs: Do they enhance students’ development in the area of skill, behavior, and adjustment? Psychology in the Schools, 49(9), 892–909. https://​doi.​org/​10.​1002/​pits.​21641. CrossRef Sklad, M., Diekstra, R., Ritter, M., Ben, J., & Gravesteijn, C. (2012). Effectiveness of school-based universal social, emotional, behavioral programs: Do they enhance students’ development in the area of skill, behavior, and adjustment? Psychology in the Schools, 49(9), 892–909. https://​doi.​org/​10.​1002/​pits.​21641. CrossRef
go back to reference von der Embse, N. P., Kilgus, S. P., Iaccarino, S., & Levi-Nielsen, S. (2017b). Screening for student mental health risk: Diagnostic accuracy, measurement invariance, and predictive validity of the social, academic, and emotional behavior risk screener-student rating scale (SAEBRS-SRS). School Mental Health, 9(3), 273–283. https://​doi.​org/​10.​1007/​s12310-017-9214-7. CrossRef von der Embse, N. P., Kilgus, S. P., Iaccarino, S., & Levi-Nielsen, S. (2017b). Screening for student mental health risk: Diagnostic accuracy, measurement invariance, and predictive validity of the social, academic, and emotional behavior risk screener-student rating scale (SAEBRS-SRS). School Mental Health, 9(3), 273–283. https://​doi.​org/​10.​1007/​s12310-017-9214-7. CrossRef
Metagegevens
Titel
Profiles of Rater Dis/Agreement within Universal Screening in Predicting Distal Outcomes
Auteurs
Nathaniel von der Embse
Eunsook Kim
Andrew Jenkins
Alexis Sanchez
Stephen P. Kilgus
Katie Eklund
Publicatiedatum
04-02-2021
Uitgeverij
Springer US
Gepubliceerd in
Journal of Psychopathology and Behavioral Assessment / Uitgave 3/2021
Print ISSN: 0882-2689
Elektronisch ISSN: 1573-3505
DOI
https://doi.org/10.1007/s10862-021-09869-0