Skip to main content

Advertisement

SpringerLink
Log in

Parent and Teacher Identification of Children at Risk of Developing Internalizing or Externalizing Mental Health Problems: A Comparison of Screening Methods

  • Original Article
  • Published:
Prevention Science Aims and scope Submit manuscript

The accuracy of screening methods for predicting children's internalizing, externalizing, or total behavior problems by 1-year follow-up were compared. Parents and teachers completed two screening measures each: a measure of exposure to multiple family risk factors (the Family Risk Factor Checklist: FRFC) and a simple nomination question of child risk status, which involved informants answering one question concerning whether they believed a child was at higher risk than average of developing a mental health problem in the future. Both parents and teachers were more accurate at identifying children at risk of externalizing than internalizing disorders. For parents, the FRFC showed similar sensitivity to simple nomination as a screen for predicting later total behavior problems, lower sensitivity for internalizing problems, and there was a nonsignificant trend for the FRFC to have higher sensitivity for externalizing problems than simple nomination. The specificity of simple nomination was somewhat higher than that of the FRFC for all three mental health outcomes. For teachers, simple nomination showed higher sensitivity than the FRFC for predicting internalizing, externalizing, and total behavior problems, but the difference was nonsignificant for internalizing problems. The specificity for both teacher screens was similar across mental health outcomes. For a population base rate of externalizing problems of 13%, teacher nominations for children at risk of externalizing problems had a sensitivity of 69%, specificity of 78%, positive predictive value (PPV) of 32%, and negative predictive value (NPV) of 94%. Teacher nominations of at-risk children had greater predictive accuracy than parent nominations. The simple nomination method shows promise as a cost-efficient screen. When screening children who did not yet have behavioral symptoms, both parent and teacher measures resulted in substantial misclassification errors. However, the use of these screening measures at a population level could benefit large numbers of children who subsequently receive selective preventive interventions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. The SES of a school was defined by its ‘Index of Relative Socioeconomic Disadvantage’ (IRSED). The quartile of schools with the highest SES (IRSED≥214) were excluded from the sample. IRSEDs were based on 1991 Australian Census data and are derived from community attributes such as average income, educational attainment, unemployment rate, and occupational prestige. IRSEDs range from 40 to 240, and roughly 80% of schools have IRSEDs between 190 and 210. Schools with IRSEDs of 190 or lower are generally considered to be of low SES, while schools with IRSEDs of 210 or higher are considered to be of high SES (personal communication, Performance Measurement Office, Education Queensland, 1998).

  2. Predictive accuracy was determined from the numbers of true positives (TP: those identified as high risk who were subsequently in the clinical range), false positives (FP: those identified as high risk who were subsequently in the nonclinical range), true negatives (TN: those identified as low risk who were subsequently in the nonclinical range) and false negatives (FN: those identified as low risk who were subsequently in the clinical range), as follows: Sensitivity = TP/(TP + FN); and Specificity = TN/(TN + FP) (Gordis, 1996).

REFERENCES

  • Achenbach, T. M. (1991a). Manual for the Child Behavior Checklist/4–18 and 1991 profile. Burlington, VT: University of Vermont, Department of Psychiatry.

    Google Scholar 

  • Achenbach, T. M. (1991b). Manual for the Teacher's Report Form and 1991 Profile. Burlington, VT: University of Vermont, Department of Psychiatry.

    Google Scholar 

  • Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioural and emotional problems: Implication of cross-informant correlations for situational specificity. Psychological Bulletin, 101, 213–232.

    Article  CAS  PubMed  Google Scholar 

  • Angold, A., Weissman, M. M., John, K., Merikangas, K. R., Prusoff, B. A., Wickramaratne, P., Gammon, G. D., & Warner, V. (1987). Parent and child reports of depressive symptoms in children at low and high risk for depression. Journal of Child Psychology and Psychiatry, 28(6), 901–915.

    CAS  PubMed  Google Scholar 

  • Auger, R. W. (2004). The accuracy of teacher reports in the identification of middle school students with depressive symptomatology. Psychology in the Schools, 41(3), 379–389.

    Article  Google Scholar 

  • Australian Bureau of Statistics, & McLennan, W. (1997). ASCO: Australian Standard Classification of Occupations (2nd ed.). Canberra: Australian Government Publishing Service.

    Google Scholar 

  • Bennett, K. J., Lipman, E. L., Brown, S., Racine, Y., Boyle, M. H., & Offord, D. R. (1999). Predicting conduct problems: Can high-risk children be identified in kindergarten and grade 1? Journal of Consulting and Clinical Psychology, 67(4), 470–480.

    Article  CAS  PubMed  Google Scholar 

  • Bennett, K. J., Lipman, E. L., Racine, Y., & Offord, D. R. (1998). Annotation: Do measures of externalizing behavior in normal populations predict later outcome?: Implications for targeted interventions to prevent conduct disorder. Journal of Child Psychology and Psychiatry, 39(8), 1059–1070.

    Article  CAS  PubMed  Google Scholar 

  • Bennett, K. J., & Offord, D. R. (2001). Screening for conduct problems: Does the predictive accuracy of conduct disorder symptoms improve with age? Journal of the American Academy of Child and Adolescent Psychiatry, 40(12), 1418–1425.

    Article  CAS  PubMed  Google Scholar 

  • Bickman, L. (1996). A continuum of care: More is not always better. American Psychologist, 51(7), 689–701.

    Article  CAS  PubMed  Google Scholar 

  • Coie, J. D., Watt, N. F., West, S. G., Hawkins, J. D., Asarnow, J. R., Markman, H. J., Ramey, S. L., Shure, M. B., & Long, B. (1993). The science of prevention: A conceptual framework and some directions for a national research program. American Psychologist, 48(10), 1013–1022.

    Article  CAS  PubMed  Google Scholar 

  • Dadds, M. R., Spence, S. H., Holland, D. E., Barrett, P. M., & Laurens, K. R. (1997). Prevention and early intervention for anxiety disorders: A controlled trial. Journal of Consulting and Clinical Psychology, 65(4), 627–635.

    Article  CAS  PubMed  Google Scholar 

  • Dishion, T. J., & Andrews, D. W. (1995). Preventing escalation in problem behaviors with high-risk young adolescents: Immediate and 1-year outcomes. Journal of Consulting and Clinical Psychology, 63, 538–548.

    Article  CAS  PubMed  Google Scholar 

  • Dwyer, S. (2002). Identifying children at risk of developing mental health problems: Screening for family risk factors in the school setting. Unpublished PhD thesis, Queensland University of Technology, Brisbane, Australia.

  • Dwyer, S. B., Nicholson, J. M., & Battistutta, D. (2003). Population level assessment of the family risk factors related to the onset or persistence of children's mental health problems. Journal of Child Psychology and Psychiatry, 44(5), 699–711.

    Article  PubMed  Google Scholar 

  • Dwyer, S. B., Nicholson, J. M., Battistutta, D., & Oldenburg, B. F. (2005). Teachers’ knowledge of children's exposure to family risk factors: Accuracy and usefulness. Journal of School Psychology, 43, 23–38.

    Article  Google Scholar 

  • Fischer, M., Rolf, J. E., Hasazi, J. E., & Cummings, L. (1984). Follow-up of a preschool epidemiological sample: Cross-age continuities and predictions of later adjustment with internalizing and externalizing dimensions of behavior. Child Development, 55(1), 137–150.

    Article  CAS  PubMed  Google Scholar 

  • Gordis, L. (1996). Epidemiology. Philadelphia: W.B. Saunders Company.

    Google Scholar 

  • Gordon, R. (1987). An operational classification of disease prevention. In J. A. Steinberg & M. M. Silverman (Eds.), Preventing mental disorders (pp. 20–26). Rockville, MD: Department of Health and Human Services.

    Google Scholar 

  • Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143, 29–36.

    CAS  PubMed  Google Scholar 

  • Herjanic, B., & Reich, W. (1997). Development of a structured psychiatric interview for children: Agreement between child and parent on individual symptoms. Journal of Abnormal Child Psychology, 25, 21–31.

    Article  CAS  PubMed  Google Scholar 

  • Hill, L. G., Lochman, J. E., Coie, J. D., & Greenberg, M. T. (2004). Effectiveness of early screening for externalizing problems: Issues of screening accuracy and utility. Journal of Consulting and Clinical Psychology, 72(5), 809–820.

    Article  PubMed  Google Scholar 

  • Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19.

    Article  CAS  PubMed  Google Scholar 

  • Jensen, P. S., & Watanabe, H. (1999). Sherlock Holmes and child psychopathology assessment approaches: The case of the false-positive. Journal of the American Academy of Child and Adolescent Psychiatry, 38(2), 138–146.

    Article  CAS  PubMed  Google Scholar 

  • Jensen, A. L., & Weisz, J. R. (2002). Assessing match and mismatch between practitioner-generated and standardized interview-generated diagnoses for clinic-referred children and adolescents. Journal of Consulting and Clinical Psychology, 70(1), 158–168.

    Article  PubMed  Google Scholar 

  • Jesness, C. F. (1987). Early identification of delinquent-prone children: An overview. In J. D. Burchard & S. N. Burchard (Eds.), Prevention of delinquent behavior (pp. 140–158). Newbury Park: Sage.

    Google Scholar 

  • Jones, D., Dodge, K. A., Foster, E. M., Nix, R., & the Conduct Problems Prevention Research Group. (2002). Early identification of children at risk for costly mental health service use. Prevention Science, 3(4), 247–256.

    Article  PubMed  Google Scholar 

  • Keenan, K., Shaw, D., Delliquadri, E., Giovannelli, J., & Walsh, B. (1998). Evidence for the continuity of early problem behaviors: Application of a developmental model. Journal of Abnormal Child Psychology, 26(6), 441–454.

    Article  CAS  PubMed  Google Scholar 

  • Kolko, D. J., & Kazdin, A. E. (1993). Emotional/behavioral problems in clinic and nonclinic children: Correspondence among child, parent and teacher reports. Journal of Child Psychology and Psychiatry, 34, 991–1006.

    CAS  PubMed  Google Scholar 

  • Lane, K. L. (2003). Identifying young students at risk for antisocial behavior: The utility of “teachers as tests.” Behavioral Disorders, 28(4), 360–369.

    Google Scholar 

  • Lavigne, J. V., Arend, R., Rosenbaum, D., Binns, H. J., Kaufer-Cristoffel, K., & Gibbons, R. D. (1998). Psychiatric disorders with onset in the preschool years: I. Stability of diagnoses. Journal of the American Academy of Child and Adolescent Psychiatry, 37(12), 1246–1254.

    Article  CAS  PubMed  Google Scholar 

  • Lewinsohn, P. M., Hops, H., Roberts, R. E., Seeley, J. R., & Andrews, J. A. (1993). Adolescent psychopathology: I. Prevalence and incidence of depression and other DSM-III-R disorders in high school students. Journal of Abnormal Psychology, 102, 133–144.

    Article  CAS  PubMed  Google Scholar 

  • Lochman, J. E., & The Conduct Problems Prevention Research Group. (1995). Screening of child behavior problems for prevention programs at school entry. Journal of Consulting and Clinical Psychology, 63(4), 549–559.

    Article  CAS  PubMed  Google Scholar 

  • Mesman, J., & Koot, H. M. (2000). Child-reported depression and anxiety in preadolescence: I. Associations with parent- and teacher-reported problems. Journal of the American Academy of Child and Adolescent Psychiatry, 39(11), 1371–1378.

    Article  CAS  PubMed  Google Scholar 

  • Mesman, J., & Koot, H. M. (2001). Early preschool predictors of preadolescent internalizing and externalizing DSM-IV diagnoses. Journal of the American Academy of Child and Adolescent Psychiatry, 40(9), 1029–1036.

    Article  CAS  PubMed  Google Scholar 

  • Mitchell, S., & Shepherd, M. A. (1966). A comparative study of children's behavior at home and at school. British Journal of Educational Psychology, 36, 248–254.

    CAS  PubMed  Google Scholar 

  • Mrazek, P. J., & Haggerty, R. J. (Eds.). (1994). Reducing risks for mental disorders: Frontiers for preventive intervention research. Washington, DC: National Academy Press.

    Google Scholar 

  • Nicholson, J. M., McFarland, M. L., & Oldenburg, B. (1999). Detection of child mental health problems in the school setting. The Australian Educational and Developmental Psychologist, 16(1), 66–77.

    Google Scholar 

  • Nicholson, J. M., Oldenburg, B., Dwyer, S. B., & Battistutta, D. (2005). The Promoting Adjustment in Schools Project (PROMAS): Study design, methods and baseline findings. Manuscript submitted for publication.

  • Offord, D. R., Boyle, M. H., Racine, Y., Szatmari, P., Fleming, J. E., Sanford, M., & Lipman, E. (1996). Integrating assessment data from multiple informants. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 1078–1085.

    Article  CAS  PubMed  Google Scholar 

  • Ollendick, T. H., Greene, R. W., Weist, M. D., & Oswald, D. P. (1990). The predictive validity of teacher nominations: A five-year followup of at-risk youth. Journal of Abnormal Child Psychology, 18, 699–713.

    Article  CAS  PubMed  Google Scholar 

  • Pedro-Carroll, J. L., & Cowen, E. L. (1985). The Children of Divorce Intervention Program: An investigation of the efficacy of a school-based prevention program. Journal of Consulting and Clinical Psychology, 53, 603–611.

    Article  CAS  PubMed  Google Scholar 

  • Pillow, D. R., Sandler, I. N., Braver, S. L., Wolchik, S. A., & Gersten, J. C. (1991). Theory-based screening for prevention: Focusing on mediating processes in children of divorce. American Journal of Community Psychology, 19(6), 809–836.

    Article  CAS  PubMed  Google Scholar 

  • Prior, M., Smart, D., Sanson, A., & Oberklaid, F. (1993). Sex differences in psychological adjustment from infancy to 8 years. Journal of the American Academy of Child and Adolescent Psychiatry, 32(2), 291–304.

    Article  CAS  PubMed  Google Scholar 

  • Reynolds, W. M. (1990). Introduction to the nature and study of internalizing disorders in children and adolescents. School Psychology Review, 19(2), 137–141.

    Google Scholar 

  • Richman, N., Stevenson, J., & Graham, P. J. (1982). Preschool to school: A behavioral study. San Diego, CA: Academic Press.

    Google Scholar 

  • Ritholz, S. (1959). Children's behavior. New York: Bookman Associates.

    Google Scholar 

  • Ritter, D. R. (1989). Teachers' perceptions of problem behavior in general and special education. Exceptional Children, 55(6), 559–564.

    CAS  PubMed  Google Scholar 

  • Rutter, M. (1978). Family, area and school influences in the genesis of conduct disorders. In L. Hersov, M. Berger, & D. Shaffer (Eds.), Aggression and antisocial behaviour in childhood and adolescence (pp. 95–113). Oxford: Pergamon.

    Google Scholar 

  • Rutter, M. (1979). Protective factors in children's responses to stress and disadvantage. In M. Whalen & J. E. Rolf (Eds.), Primary prevention of psychopathology: Vol. 3. Social competence in children (pp. 49–74). Hanover, NH: University Press of New England.

    Google Scholar 

  • Sanders, M. R., Markie-Dadds, C., Tully, L., & Bor, B. (2000). The Triple P—Positive Parenting Program: A comparison of enhanced, standard and self-directed behavioural family intervention for parents of children with early onset conduct problems. Journal of Consulting and Clinical Psychology, 68, 624–640.

    Article  CAS  PubMed  Google Scholar 

  • Sawyer, M. G., Arney, F. M., Baghurst, P. A., Clark, J. J., Graetz, B. W., Kosky, R. J., Nurcombe, B., Patton, G. C., Prior, M. R., Raphael, B., Rey, J., Whaites, L. C., & Zubrick, S. R. (2000). Child and adolescent component of the National Survey of Mental Health and Well-being: Mental health of young people in Australia. Canberra: Mental Health and Special Programs Branch, Commonwealth Department of Health and Aged Care.

    Google Scholar 

  • Weissman, M. M., Gammon, G. D., John, K., Merikangas, K. R., Warner, V., Prusoff, B. A., & Sholomskas, D. (1987). Children of depressed parents. Increased psychopathology and early onset of major depression. Archives of General Psychiatry, 44(10), 847–853.

    CAS  PubMed  Google Scholar 

  • Weist, M. D. (1997). Expanded school mental health services: A national movement in progress. In T. H. Ollendick & R. J. Prinz (Eds.), Advances in clinical child psychology (Vol. 19, pp. 319–352). New York: Plenum Press.

  • White, J. L., Moffitt, T., Earls, F., Robins, L., & Silva, P. A. (1990). How early can we tell? Predictors of childhood conduct disorder and adolescent delinquency. Criminology, 28, 507–533.

    Article  Google Scholar 

  • Wickman, E. K. (1928). Children's behavior and teachers' attitudes. New York: The Commonwealth Fund.

    Google Scholar 

Download references

ACKNOWLEDGMENTS

This study was supported by a National Health and Medical Research Council (NH&MRC) Project Grant (990241), an NH&MRC Research Scholarship (987409), and an Australian Research Council (ARC) Small Grant. Special thanks to Michelle Gill, Bonnie MacFarlane, Jyai Allen, and Chris Sibthorpe for assistance with data collection. We are grateful to all the parents and teachers who participated in the PROMAS Project. Ethical approval was obtained for this research from the QUT Human Research Ethics Committee (Ref. No. 1353/2 H), and all participants provided informed consent.

Author information

Authors and Affiliations

  1. Centre for Public Health Research, School of Public Health, Queensland University of Technology, Brisbane, Queensland, Australia

    Sarah B. Dwyer, Jan M. Nicholson & Diana Battistutta

  2. Roy McKenzie Centre for the Study of Families, PO Box 600, Victoria University of Wellington, Tawa Wellington, New Zealand

    Sarah B. Dwyer

Authors
  1. Sarah B. Dwyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan M. Nicholson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Diana Battistutta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sarah B. Dwyer.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dwyer, S.B., Nicholson, J.M. & Battistutta, D. Parent and Teacher Identification of Children at Risk of Developing Internalizing or Externalizing Mental Health Problems: A Comparison of Screening Methods. Prev Sci 7, 343–357 (2006). https://doi.org/10.1007/s11121-006-0026-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11121-006-0026-5

KEY WORDS:

Search

Navigation