Skip to main content
Top
Gepubliceerd in: Journal of Child and Family Studies 1/2016

06-05-2015 | Original Paper

Convergent, Criterion and Social Validity of the Emotional and Behavioral Screener

Auteurs: Corey D. Pierce, Matthew Lambert, Hussain Alamer

Gepubliceerd in: Journal of Child and Family Studies | Uitgave 1/2016

Log in om toegang te krijgen
share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail

Abstract

Recent reports suggest that up to 20 % of all school-aged students demonstrate characteristics related to emotional disturbance. However, <1 % of these students actually receive special education services for ED, a percentage that has remained unchanged for nearly 30 years. Screening procedures have been developed to identify the behavioral needs of students within school-wide systems such as Multitiered System of Support, Positive Behavior Support, and Response to Intervention. While several screening tools are available at this time, many require significant time, training, and/or effort, factors which limit their practical use in schools. A new efficient assessment tool Emotional and Behavioral Screener (EBS) has recently been developed as a universal screening instrument. The purpose of this study was to examine the validity of the EBS scores, with a sample of over 600 elementary students, in comparison to scores from the Social Skills Improvement System assessment tool. Results indicated high levels of convergent validity between the scores (social skills; ρ = −0.70; and problem behavior; ρ = 0.79); strong criterion validity as assessed by differences in EBS scores between individuals with normal versus problematic scores (based on established cut-off scores) on the SSIS (social skills; d = 2.23; and problem behavior; d = 3.58) and assessed by binary classification between cut-off scores of the EBS and SSIS (AUROC range .86–.88); and adequate social validity. Implications and limitations of these results are discussed.
Literatuur
go back to reference American Educational Research Association, American Psychological Association, & National Council of Measurement in Education. (1999). The standards for educational and psychological testing. Washington, DC: American Educational Research Association. American Educational Research Association, American Psychological Association, & National Council of Measurement in Education. (1999). The standards for educational and psychological testing. Washington, DC: American Educational Research Association.
go back to reference Bradshaw, C. P., Buckley, J., & Lalongo, N. (2008). School-based service utilization among urban children with early-onset educational and mental health problems: The squeaky wheel phenomenon. School Psychology Quarterly, 23, 169–186.CrossRef Bradshaw, C. P., Buckley, J., & Lalongo, N. (2008). School-based service utilization among urban children with early-onset educational and mental health problems: The squeaky wheel phenomenon. School Psychology Quarterly, 23, 169–186.CrossRef
go back to reference Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York City: Guilford Press. Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York City: Guilford Press.
go back to reference Cohen, J. (1988). Statistical power analysis for the behavioral sciences. New Jersey: Lawrence Erlbaum Associates. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. New Jersey: Lawrence Erlbaum Associates.
go back to reference Compton, D. L., Fuchs, D., Fuchs, L. S., & Bryant, J. D. (2006). Selecting at-risk readers in first grade for early intervention: A two-year longitudinal study of decision rules and procedures. Journal of Educational Psychology, 98, 394–409.CrossRef Compton, D. L., Fuchs, D., Fuchs, L. S., & Bryant, J. D. (2006). Selecting at-risk readers in first grade for early intervention: A two-year longitudinal study of decision rules and procedures. Journal of Educational Psychology, 98, 394–409.CrossRef
go back to reference Conroy, M. A., Hendrickson, J. M., & Hester, P. P. (2004). Early identification and prevention of emotional and behavioral disorders. In R. B. Rutherford Jr, M. M. Quinn, & S. Mather (Eds.), Handbook of research in behavior disorders (pp. 199–215). New York, NY: Guilford Press. Conroy, M. A., Hendrickson, J. M., & Hester, P. P. (2004). Early identification and prevention of emotional and behavioral disorders. In R. B. Rutherford Jr, M. M. Quinn, & S. Mather (Eds.), Handbook of research in behavior disorders (pp. 199–215). New York, NY: Guilford Press.
go back to reference Costello, E. J., Mustillo, S., Erkanli, A., Keeler, G., & Angold, A. (2003). Prevalence and development of psychiatric disorders in childhood and adolescence. Archives of General Psychiatry, 60, 837–844.CrossRefPubMed Costello, E. J., Mustillo, S., Erkanli, A., Keeler, G., & Angold, A. (2003). Prevalence and development of psychiatric disorders in childhood and adolescence. Archives of General Psychiatry, 60, 837–844.CrossRefPubMed
go back to reference Crick, N. R., Grotpeter, J. K., & Bigbee, M. A. (2002). Relationally and physically aggressive children’s intent attributions and feelings for relational and instrumental peer provocations. Child Development, 73, 1134–1142.CrossRefPubMed Crick, N. R., Grotpeter, J. K., & Bigbee, M. A. (2002). Relationally and physically aggressive children’s intent attributions and feelings for relational and instrumental peer provocations. Child Development, 73, 1134–1142.CrossRefPubMed
go back to reference Cullinan, D., & Epstein, M. H. (2013a). Development, reliability, and construct validity of the emotional and behavioral screener. Preventing School Failure, 57, 223–230.CrossRef Cullinan, D., & Epstein, M. H. (2013a). Development, reliability, and construct validity of the emotional and behavioral screener. Preventing School Failure, 57, 223–230.CrossRef
go back to reference Cullinan, D., & Epstein, M. H. (2013b). Emotional and behavioral screener (EBS). Austin, TX: PRO-ED. Cullinan, D., & Epstein, M. H. (2013b). Emotional and behavioral screener (EBS). Austin, TX: PRO-ED.
go back to reference Eisenberg, N., Valiente, C., Spinrad, T. L., Cumbers-land, A., Liew, J., Reiser, M., et al. (2009). Longitudinal relations of children’s effortful control, impulsivity, and negative emotionality to their externalizing, internalizing and co- occurring behavior problems. Developmental Psychology, 45, 988–1008.CrossRefPubMedPubMedCentral Eisenberg, N., Valiente, C., Spinrad, T. L., Cumbers-land, A., Liew, J., Reiser, M., et al. (2009). Longitudinal relations of children’s effortful control, impulsivity, and negative emotionality to their externalizing, internalizing and co- occurring behavior problems. Developmental Psychology, 45, 988–1008.CrossRefPubMedPubMedCentral
go back to reference Epstein, M. H., & Cullinan, D. (2010). Scales for assessing emotional disturbance (2nd ed.). Austin, TX: PRO-ED. Epstein, M. H., & Cullinan, D. (2010). Scales for assessing emotional disturbance (2nd ed.). Austin, TX: PRO-ED.
go back to reference Essex, M., Kraemer, H. C., Slattery, M. J., Burk, L. R., Boyce, W. T., Woodward, H. R., & Kupfer, D. J. (2009). Screening for childhood mental health problems: Outcomes and early identification. Journal of Child Psychology and Psychiatry, 50, 562–570.CrossRefPubMedPubMedCentral Essex, M., Kraemer, H. C., Slattery, M. J., Burk, L. R., Boyce, W. T., Woodward, H. R., & Kupfer, D. J. (2009). Screening for childhood mental health problems: Outcomes and early identification. Journal of Child Psychology and Psychiatry, 50, 562–570.CrossRefPubMedPubMedCentral
go back to reference Federal Register. (2006). Assistance to states for the education of children with disabilities and preschool grants for children with disabilities; Final Rule 34 (CFR Parts 300 and 301, 71 Fed. Reg. 46540). Washington, DC: U.S. Department of Education. Federal Register. (2006). Assistance to states for the education of children with disabilities and preschool grants for children with disabilities; Final Rule 34 (CFR Parts 300 and 301, 71 Fed. Reg. 46540). Washington, DC: U.S. Department of Education.
go back to reference Feldt, L. S. (1969). A test of the hypothesis that Cronbach’s alpha or Kuder-Richardson coefficient twenty is the same for two tests. Psychometrika, 34, 363–373.CrossRef Feldt, L. S. (1969). A test of the hypothesis that Cronbach’s alpha or Kuder-Richardson coefficient twenty is the same for two tests. Psychometrika, 34, 363–373.CrossRef
go back to reference Green, S. B., & Hershberger, S. L. (2000). Correlated errors in true score models and their effect on coefficient alpha. Structural Equation Modeling, 7, 251–270.CrossRef Green, S. B., & Hershberger, S. L. (2000). Correlated errors in true score models and their effect on coefficient alpha. Structural Equation Modeling, 7, 251–270.CrossRef
go back to reference Gresham, F. M., & Elliott, S. N. (2008). Social skills rating system—rating scales. Minneapolis: NCS Pearson Inc. Gresham, F. M., & Elliott, S. N. (2008). Social skills rating system—rating scales. Minneapolis: NCS Pearson Inc.
go back to reference Gresham, F. M., Elliott, S. N., & Kettler, R. J. (2010). Base rates of social skills acquisition/performance deficit, strengths, and problem behaviors: An analysis of the social skills improvement system—rating scales. Psychological Assessment, 22, 809–815.CrossRefPubMed Gresham, F. M., Elliott, S. N., & Kettler, R. J. (2010). Base rates of social skills acquisition/performance deficit, strengths, and problem behaviors: An analysis of the social skills improvement system—rating scales. Psychological Assessment, 22, 809–815.CrossRefPubMed
go back to reference Gresham, F. M., Elliott, S. N., Vance, M. J., & Cook, C. R. (2011). Comparability of the social skills rating system to the social skills improvement system: Content and psychometric comparisons across elementary and secondary age levels. School Psychology Quarterly, 26, 27–44. doi:10.1037/a0022662.CrossRef Gresham, F. M., Elliott, S. N., Vance, M. J., & Cook, C. R. (2011). Comparability of the social skills rating system to the social skills improvement system: Content and psychometric comparisons across elementary and secondary age levels. School Psychology Quarterly, 26, 27–44. doi:10.​1037/​a0022662.CrossRef
go back to reference Hammill, D. D., Brown, L., Bryant, B. R., & Cullinan, D. (1989). A consumer’s guide to tests in print (2nd ed.). Austin, TX: PRO-ED. Hammill, D. D., Brown, L., Bryant, B. R., & Cullinan, D. (1989). A consumer’s guide to tests in print (2nd ed.). Austin, TX: PRO-ED.
go back to reference Hawken, L. S., Vincent, C. G., & Schumann, J. (2008). Response to intervention for social behavior: Challenges and opportunities. Journal of Emotional and Behavioral Disorders, 16, 213–225.CrossRef Hawken, L. S., Vincent, C. G., & Schumann, J. (2008). Response to intervention for social behavior: Challenges and opportunities. Journal of Emotional and Behavioral Disorders, 16, 213–225.CrossRef
go back to reference Horner, R. H., Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A. W., et al. (2009). A randomized, wait-list controlled effectiveness trial assessing schoolwide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133–144.CrossRef Horner, R. H., Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A. W., et al. (2009). A randomized, wait-list controlled effectiveness trial assessing schoolwide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133–144.CrossRef
go back to reference Johnson, E. S., Jenkins, J. R., Petscher, Y., & Catts, H. W. (2009). How can we improve the accuracy of screening instruments? Learning Disabilities Research & Practice, 24, 174–185.CrossRef Johnson, E. S., Jenkins, J. R., Petscher, Y., & Catts, H. W. (2009). How can we improve the accuracy of screening instruments? Learning Disabilities Research & Practice, 24, 174–185.CrossRef
go back to reference Joint Committee on Standards for Educational and Psychological Testing. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Joint Committee on Standards for Educational and Psychological Testing. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
go back to reference Kamphaus, R. W., & Reynolds, C. R. (2007). BASC-2: Behavioral and emotional screening system. Minneapolis, MN: Pearson. Kamphaus, R. W., & Reynolds, C. R. (2007). BASC-2: Behavioral and emotional screening system. Minneapolis, MN: Pearson.
go back to reference Kamphaus, R. W., Thorpe, J. S., Winsor, A. P., Kroncke, A. P., Dowdy, E. T., & VanDeventer, M. (2007). Development and predictive validity of a teacher screener for child behavioral and emotional problems at school. Educational and Psychological Measurement, 6, 1–15. Kamphaus, R. W., Thorpe, J. S., Winsor, A. P., Kroncke, A. P., Dowdy, E. T., & VanDeventer, M. (2007). Development and predictive validity of a teacher screener for child behavioral and emotional problems at school. Educational and Psychological Measurement, 6, 1–15.
go back to reference Kauffman, J. M., & Landrum, T. J. (2009). Characteristics of emotional and behavioral disorders of children and youth (9th ed.). Upper Saddle River, NJ: Pearson Education Inc. Kauffman, J. M., & Landrum, T. J. (2009). Characteristics of emotional and behavioral disorders of children and youth (9th ed.). Upper Saddle River, NJ: Pearson Education Inc.
go back to reference Lambert, M. C., Epstein, M., Ingram, S., Simpson, A., & Bernstein, S. (2014b). Psychometrics and measurement invariance of the emotional and behavioral screener. Behavior Disorders, 39, 89–101. Lambert, M. C., Epstein, M., Ingram, S., Simpson, A., & Bernstein, S. (2014b). Psychometrics and measurement invariance of the emotional and behavioral screener. Behavior Disorders, 39, 89–101.
go back to reference Lane, K. L., Menzies, H. M., Oakes, W. P., Lambert, W., Cox, M., & Hankins, K. (2012). A validation of the student risk screening scale for internalizing and externalizing behaviors: Patterns in rural and urban elementary schools. Behavioral Disorders, 37(4), 244–270. Lane, K. L., Menzies, H. M., Oakes, W. P., Lambert, W., Cox, M., & Hankins, K. (2012). A validation of the student risk screening scale for internalizing and externalizing behaviors: Patterns in rural and urban elementary schools. Behavioral Disorders, 37(4), 244–270.
go back to reference Lane, K. L., Oakes, W., & Menzies, H. (2010). Systematic screenings to prevent the development of learning and behavioral problems: Considerations for practitioners, researchers, and policy makers. Journal of Disability Policy Studies, 21, 160–172.CrossRef Lane, K. L., Oakes, W., & Menzies, H. (2010). Systematic screenings to prevent the development of learning and behavioral problems: Considerations for practitioners, researchers, and policy makers. Journal of Disability Policy Studies, 21, 160–172.CrossRef
go back to reference McDonald, R. P. (1978). Generalizability in factorable domains: Domain validity and generalizability. Educational and Psychological Measurement, 38, 75–79.CrossRef McDonald, R. P. (1978). Generalizability in factorable domains: Domain validity and generalizability. Educational and Psychological Measurement, 38, 75–79.CrossRef
go back to reference Morris, R. J., Shah, K., & Morris, Y. P. (2002). Internalizing behavior disorders. In K. Lane, F. Gresham, & T. O’Shaughnessy (Eds.), Interventions for children with or at risk for emotional and behavioral disorders (pp. 223–241). Boston: Allyn & Bacon. Morris, R. J., Shah, K., & Morris, Y. P. (2002). Internalizing behavior disorders. In K. Lane, F. Gresham, & T. O’Shaughnessy (Eds.), Interventions for children with or at risk for emotional and behavioral disorders (pp. 223–241). Boston: Allyn & Bacon.
go back to reference Mrazek, D., & Mrazek, P. J. (2005). Prevention of psychiatric disorders in children and adolescents. In B. J. Sadock & V. A. Sadock (Eds.), Kaplan & Sadock’s comprehensive textbook of psychiatry (Vol. II, pp. 3513–3518). Philadelphia, PA: Lippincott Williams & Wilkins. Mrazek, D., & Mrazek, P. J. (2005). Prevention of psychiatric disorders in children and adolescents. In B. J. Sadock & V. A. Sadock (Eds.), Kaplan & Sadock’s comprehensive textbook of psychiatry (Vol. II, pp. 3513–3518). Philadelphia, PA: Lippincott Williams & Wilkins.
go back to reference National Research Council and Institute of Medicine. (2009). Preventing mental, emotional, and behavioraldisorders among young people: Progress and possibilities. Committee on the Prevention of Mental Disorders and Substance Abuse Among Children, Youth, and Young Adults: Research Advances and Promising Interventions. In M. E. O’Connell, T. Boat, & K. E. Warner (Eds.). Board on Children, Youth, and Families, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academies Press. National Research Council and Institute of Medicine. (2009). Preventing mental, emotional, and behavioraldisorders among young people: Progress and possibilities. Committee on the Prevention of Mental Disorders and Substance Abuse Among Children, Youth, and Young Adults: Research Advances and Promising Interventions. In M. E. O’Connell, T. Boat, & K. E. Warner (Eds.). Board on Children, Youth, and Families, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academies Press.
go back to reference Nordness, P. D., Epstein, M. H., Cullinan, D., & Pierce, C. D. (2014). Emotional and behavioral screener: Test-retest reliability, inter-rater reliability, and convergent validity. Remedial and Special Education., 35, 211–217.CrossRef Nordness, P. D., Epstein, M. H., Cullinan, D., & Pierce, C. D. (2014). Emotional and behavioral screener: Test-retest reliability, inter-rater reliability, and convergent validity. Remedial and Special Education., 35, 211–217.CrossRef
go back to reference Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.
go back to reference Pastor, P. N., Reuben, C. A., & Duran, C. R. (2012). Identifying emotional and behavioral problems in children aged 4–17 years: United States, 2001–2007. National Health Statistics Reports. Number 48. National Center for Health Statistics. Pastor, P. N., Reuben, C. A., & Duran, C. R. (2012). Identifying emotional and behavioral problems in children aged 4–17 years: United States, 2001–2007. National Health Statistics Reports. Number 48. National Center for Health Statistics.
go back to reference Salvia, J., Yesseldyke, J., & Bolt, S. (2013). Assessment in special and inclusive education (12th ed.). Belmont, CA: Wadsworth. Salvia, J., Yesseldyke, J., & Bolt, S. (2013). Assessment in special and inclusive education (12th ed.). Belmont, CA: Wadsworth.
go back to reference Severson, H. H., Walker, H. M., Hope-Doolittle, J., Kratochwill, T. R., & Gresham, F. M. (2007). Proactive, early screening to detect behaviorally at-risk students: Issues, approaches, emerging innovations, and professional practices. Journal of School Psychology, 45, 193–223.CrossRef Severson, H. H., Walker, H. M., Hope-Doolittle, J., Kratochwill, T. R., & Gresham, F. M. (2007). Proactive, early screening to detect behaviorally at-risk students: Issues, approaches, emerging innovations, and professional practices. Journal of School Psychology, 45, 193–223.CrossRef
go back to reference Spearman, C. C. (1910). Correlation calculated from faulty data. British Journal of Psychology, 3, 271–295. Spearman, C. C. (1910). Correlation calculated from faulty data. British Journal of Psychology, 3, 271–295.
go back to reference Steiger, J. H. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87, 245–251.CrossRef Steiger, J. H. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87, 245–251.CrossRef
go back to reference Sugai, G., & Horner, R. (2006). A promising approach for expanding and sustaining the implementation of school-wide positive behavior support. School Psychology Review, 35, 245–259. Sugai, G., & Horner, R. (2006). A promising approach for expanding and sustaining the implementation of school-wide positive behavior support. School Psychology Review, 35, 245–259.
go back to reference Sugai, G., & Horner, R. H. (2009). Defining and describing schoolwide positive behavior support. In W. Sailor, G. Dunlap, G. Sugai, & R. H. Horner (Eds.), Handbook of positive behavior support (pp. 307–326). New York, NY: Springer. doi:10.1007/978-0-387-09632-2_13.CrossRef Sugai, G., & Horner, R. H. (2009). Defining and describing schoolwide positive behavior support. In W. Sailor, G. Dunlap, G. Sugai, & R. H. Horner (Eds.), Handbook of positive behavior support (pp. 307–326). New York, NY: Springer. doi:10.​1007/​978-0-387-09632-2_​13.CrossRef
go back to reference Walker, H. M., & Severson, H. (1992). Systematic screening for behavior disorders: User’s guide and technical manual. Longmont, CO: Sopris West. Walker, H. M., & Severson, H. (1992). Systematic screening for behavior disorders: User’s guide and technical manual. Longmont, CO: Sopris West.
go back to reference Wood, F., Flowers, L., Meyer, M., & Hill, D. (2002). How to evaluate and compare screening tests: Principles of science and good sense. Paper presented at the meeting of the International Dyslexia Association, Atlanta, GA. Wood, F., Flowers, L., Meyer, M., & Hill, D. (2002). How to evaluate and compare screening tests: Principles of science and good sense. Paper presented at the meeting of the International Dyslexia Association, Atlanta, GA.
Metagegevens
Titel
Convergent, Criterion and Social Validity of the Emotional and Behavioral Screener
Auteurs
Corey D. Pierce
Matthew Lambert
Hussain Alamer
Publicatiedatum
06-05-2015
Uitgeverij
Springer US
Gepubliceerd in
Journal of Child and Family Studies / Uitgave 1/2016
Print ISSN: 1062-1024
Elektronisch ISSN: 1573-2843
DOI
https://doi.org/10.1007/s10826-015-0206-7

Andere artikelen Uitgave 1/2016

Journal of Child and Family Studies 1/2016 Naar de uitgave