Skip to main content

Advertisement

SpringerLink
Log in

Knowing Every Child: Validation of the Holistic Student Assessment (HSA) as a Measure of Social-Emotional Development

  • Published:
Prevention Science Aims and scope Submit manuscript

A Correction to this article was published on 25 January 2018

This article has been updated

Abstract

Knowing every child’s social-emotional development is important as it can support prevention and intervention approaches to meet the developmental needs and strengths of children. Here, we discuss the role of social-emotional assessment tools in planning, implementing, and evaluating preventative strategies to promote mental health in all children and adolescents. We, first, selectively review existing tools and identify current gaps in the measurement literature. Next, we introduce the Holistic Student Assessment (HSA), a tool that is based in our social-emotional developmental theory, The Clover Model, and designed to measure social-emotional development in children and adolescents. Using a sample of 5946 students (51% boys, M age = 13.16 years), we provide evidence for the psychometric validity of the self-report version of the HSA. First, we document the theoretically expected 7-dimension factor structure in a calibration sub-sample (n = 984) and cross-validate its structure in a validation sub-sample (n = 4962). Next, we show measurement invariance across development, i.e., late childhood (9- to 11-year-olds), early adolescence (12- to 14-year-olds), and middle adolescence (15- to 18-year-olds), and evidence for the HSA’s construct validity in each age group. The findings support the robustness of the factor structure and confirm its developmental sensitivity. Structural equation modeling validity analysis in a multiple-group framework indicates that the HSA is associated with mental health in expected directions across ages. Overall, these findings show the psychometric properties of the tool, and we discuss how social-emotional tools such as the HSA can guide future research and inform large-scale dissemination of preventive strategies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Change history

  • 25 January 2018

    The Holistic Student Report was reported online as open source. It is not. Any use in part or in whole in any form or version has to be approved in writing.

Notes

  1. Items for this subscale were reversely coded (see Table 1).

  2. Compared with classical alpha reliability estimates, ω has the advantage of taking into account both the strength of relation between items and constructs (λs) and measurement error, while relaxing the assumption that the items are tau-equivalent.

  3. With ordered-categorical data, the threshold parameter represents the expected value on the underlying latent variable, which indicates the shift from one response category (e.g., 0 = not at all) to another one (e.g., 1 = sometimes or higher; see Muthén and Muthén 2012).

  4. Given that χ2 values are not exact using WLSMV as method of parameter estimation, χ2 and resulting CFI values can be non-monotonic with model complexity (Muthén and Muthén 2012).

References

  • Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA school-age forms and profiles. Burlington: University of Vermont, Research Center for Children, Youth, and Families.

    Google Scholar 

  • Angold, A. E. J., Erkanli, A., Copeland, W., Goodman, R., Fisher, P. W., & Costello, E. J. (2012). Psychiatric diagnostic interviews for children and adolescents: A comparative study. Journal of the American Academy of Child & Adolescent Psychiatry, 51, 506–517. doi:10.1016/j.jaac.2012.02.020.

  • Beauducel, A., & Herzberg, P. Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling, 13, 186–203. doi:10.1207/s15328007sem1302_2.

  • Caci, H., Morin, A. J. S., & Tran, A. (2015). Investigation of a bifactor model of the strengths and Difficulties questionnaire. European Child & Adolescent Psychiatry, 24, 1291–1301. doi:10.1007/s00787-015-0679-3.

  • Casey, B. J., Jones, R. M., & Hare, T. A. (2008). The adolescent brain. Annals of the New York Academy of Sciences, 1124, 111–126. doi:10.1196/annals.1440.010.

  • Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14, 464–504. doi:10.1080/10705510701301834.

  • Cohn, B. (2011). Innovations in strength-based social-emotional assessment: Factor analysis, psychometric analysis, and cross-informant comparisons with the SEARS-T (Unpublished doctoral dissertation). University of Oregon, Eugene.

  • Cohn, B., Merrell, K. W., Felver-Grant, J., Tom, K., & Endrulat, N. (2009, February). Strength-based assessment of social and emotional functioning: SEARS-C and SEARS-A. Boston: Presented at the Meeting of the National Association of School Psychologists.

    Google Scholar 

  • Costello, J. (2016). Early detection and prevention of mental health problems: Developmental epidemiology and systems of support. Journal of Clinical Child & Adolescent Psychology, 45, 710–717. doi:10.1080/15374416.2016.1236728.

  • Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405–432. doi:10.1111/j.1467-8624.2010.01564.x.

  • Eisenberg, N. (2000). Emotion, regulation, and moral development. Annual Review of Psychology, 51, 665–697. doi:10.1146/annurev.psych.51.1.665.

  • Eisenberg, N., Fabes, R. A., Murphy, B., Karbon, M., Smith, M., & Maszk, P. (1996). The relations of children's dispositional empathy-related responding to their emotionality, regulation, and social functioning. Developmental Psychology, 32, 195–209.

    Article  Google Scholar 

  • Eisenberg, N., Spinrad, T. L., & Eggum, N. D. (2010). Emotion-related self-regulation and its relation to children’s maladjustment. Annual Review of Clinical Psychology, 6, 495–525. doi:10.1146/annurev.clinpsy.121208.131208.

  • Eisenberg, N., Spinrad, T. L., & Knafo-Noam, A. (2015). Prosocial development. In M. E. Lamb & R. M. Lerner (Eds.), Handbook of hild psychology and developmental science, Vol. 3: Social, emotional and personality development (7th ed., pp. 610–656). New York: Wiley.

    Google Scholar 

  • Eisner, M. P., & Malti, T. (2015). Aggressive and violent behavior. In M. E. Lamb & R. M. Lerner (Eds.), Handbook of child psychology and developmental science, Vol. 3: Social, emotional and personality development (7th ed., pp. 795–884). New York: Wiley.

    Google Scholar 

  • Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4, 272–299. doi:10.1037/1082-989X.4.3.272.

  • Flook, L., Goldberg, S. B., Pinger, L., & Davidson, R. J. (2015). Promoting prosocial behavior and self-regulatory skills in preschool children through a mindfulness-based kindness curriculum. Developmental Psychology, 51, 44–51. doi:10.1037/a0038256.

  • Goodman, R. (1997). The strengths and Difficulties questionnaire: A research note. Journal of Child Psychology and Psychiatry, 38, 581–586. doi:10.1111/j.1469-7610.1997.tb01545.x.

  • Greenberg, M. T., Weissberg, R. P., O’Brien, M. U., Zins, J. E., Fredericks, L., Resnik, H., & Elias, M. J. (2003). Enhancing school-based prevention and youth development through coordinated social, emotional, and academic learning. American Psychologist, 58, 466–474. doi:10.1037/0003-066X.58.6-7.466.

  • Guhn, M., & Goelman, H. (2011). Bioecological theory, early child development and the validation of the population-level early development instrument. Social Indicators Research, 103, 193–217. doi:10.1007/s11205-011-9842-5.

  • Killen, M., & Malti, T. (2015). Moral judgments and emotions in contexts of peer exclusion and victimization. Advances in Child Development and Behavior, 48, 249–276. doi:10.1016/bs.acdb.2014.11.007.

  • LeBuffe, P. A., Shapiro, V. B., & Naglieri, J. A. (2009). The Devereux student strengths assessment (DESSA). Lewisville: Kaplan Press.

    Google Scholar 

  • Malti, T., & Noam, G. G. (2008). Where youth development meets mental health and education: The RALLY approach. New Directions for Youth Development, No. 120.

  • Malti, T., & Noam, G. G. (2009). A developmental approach to the prevention of adolescents’ aggressive behavior and the promotion of resilience. European Journal of Developmental Science, 3, 235–246. doi:10.3233/DEV-2009-3303.

  • Malti, T., & Noam, G. G. (2016). Social-emotional development: From theory to practice. European Journal of Developmental Psychology, 13, 652–665. doi:10.1080/17405629.2016.1196178.

    Article  Google Scholar 

  • Malti, T., Chaparro, M. P., Zuffianò, A., & Colasante, T. (2016). School-based interventions to promote empathy-related responding in children and adolescents: A developmental analysis. Journal of Clinical Child and Adolescent Psychology, 1–14. doi:10.1080/15374416.2015.1121822.

  • McDonald, R. P. (1970). The theoretical foundations of principal factor analysis, canonical factor analysis, and alpha factor analysis. British Journal of Mathematical and Statistical Psychology, 23, 1–21. doi:10.1111/j.2044-8317.1970.tb00432.x.

  • Meredith, W. (1993). Measurement invariance, factor analysis, and factorial invariance. Psychometrika, 58, 525–543. doi:10.1007/BF02294825.

  • Millsap, R. E. (2011). Statistical approaches to measurement invariance. New York: Routledge.

    Google Scholar 

  • Millsap, R. E., & Yun-Tein, J. (2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39, 479–515. doi:10.1207/S15327906MBR3903_4.

  • Muthén, L. K., & Muthén, B. O. (2012). Mplus User's Guide (Seventh ed.). Los Angeles: Muthén & Muthén.

    Google Scholar 

  • Nickerson, A. B., & Fishman, C. (2009). Convergent, divergent, and predictive validity of the Devereux elementary student strengths assessment. School Psychology Quarterly, 24, 48–59. doi:10.1037/a0015147.

  • Noam, G. G., & Goldstein, L. S. (1998). The resilience inventory. Unpublished research report, Harvard Medical School and McLean Hospital.

  • Noam, G. G., Malti, T., & Guhn, M. (2012). From clinical-developmental theory to assessment: The holistic student assessment tool. International Journal of Conflict and Violence, 6, 201–213.

    Google Scholar 

  • Noam, G. G., Malti, T., & Karcher, M. J. (2013). Mentoring relationships in developmental perspective. In D. DuBois & M. J. Karcher (Eds.), Handbook of youth mentoring (pp. 99–116). Thousand Oaks: Sage.

    Google Scholar 

  • Payton, J. W., Weissberg, R. P., Durlak, J. A., Dymnicki, A. B., Taylor, R. D., Schellinger, K. B., & Pachan, M. (2008). Positive impact of social and emotional learning for kindergarten to eighth-grade students: Findings from three scientific reviews (report). Chicago: Collaborative for Academic, Social, and Emotional Learning.

    Google Scholar 

  • Ruini, C., Ottolini, F., Tomba, E., Belaise, C., Albieri, E., Visani, D., Offidani, E., Caffo, E., & Fava, G. A. (2009). School intervention for promoting psychological well-being in adolescence. Journal of Behavior Therapy and Experimental Psychiatry, 40, 522–532.

    Article  PubMed  Google Scholar 

  • Schaefer, L. M., Burke, N. L., Thompson, J. K., Dedrick, R. F., Heinberg, L. J., Calogero, R. M.,. .. Svami, V. (2015). Development and validation of the Sociocultural Attitudes Towards Appearance Questionnaire-4 (SATAQ-4). Psychological Assessment, 27(1), 54–67. doi:10.1037/a0037917.

  • Song, M. (2003). Two studies on the Resilience Inventory (RI): Toward the goal of creating a culturally sensitive measure of adolescence resilience. Unpublished doctoral dissertation, Harvard University, Cambridge.

  • Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Boston: Pearson.

    Google Scholar 

  • Vandenberg, R., & Lance, C. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–70. doi:10.1177/109442810031002.

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, University of Toronto, Deerfield Hall, 3359 Mississauga Road North, Mississauga, ON, L5L 1C6, Canada

    Tina Malti

  2. Department of Psychology, Liverpool Hope University, Liverpool, UK

    Antonio Zuffianò

  3. Harvard Medical School, McLean Hospital, Belmont, MA, USA

    Gil G. Noam

Authors
  1. Tina Malti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonio Zuffianò
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gil G. Noam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tina Malti.

Ethics declarations

Funding

The first author was funded, in part, by a New Investigator Award from the Canadian Institutes of Health Research. In addition, the authors wish to thank the Leon Lowenstein Foundation and the Noyce Foundation for their support.

Conflict of Interest

The authors declare that they do not have a conflict of interest.

Ethical Approval

All procedures were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendment.

Informed Consent

Informed consent of participants was obtained.

Additional information

A correction to this article is available online at https://doi.org/10.1007/s11121-018-0873-x.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Malti, T., Zuffianò, A. & Noam, G.G. Knowing Every Child: Validation of the Holistic Student Assessment (HSA) as a Measure of Social-Emotional Development. Prev Sci 19, 306–317 (2018). https://doi.org/10.1007/s11121-017-0794-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11121-017-0794-0

Keywords

Search

Navigation