Skip to main content
SpringerLink
Log in

A Comparison of Seminar and Computer Based Training on the Accuracy and Reliability of Raters Using the Children’s Global Assessment Scale (CGAS)

  • Original Paper
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

The aim of this study was to compare two methods to conduct CGAS rater training. A total of 648 raters were randomized to training (CD or seminar), and rated five cases before and 12 months after training. The ICC at baseline/end of study was 0.71/0.78 (seminar), 0.76/0.78 (CD), and 0.67/0.79 (comparison). There were no differences in training effect in terms of agreement with expert ratings, which speaks in favor of using the less resource-demanding CD. However, the effect was modest in both groups, and untrained comparison group improved of the same order of magnitude, which proposes more extensive training.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Bates, L. W., Lyons, J. A., & Shaw, J. B. (2002). Effects of brief training on application of the Global Assessment of Functioning Scale. Psychological Reports, 91(3 Pt 1), 999–1006.

    PubMed  Google Scholar 

  • Bero, L. A., Grilli, R., Grimshaw, J. M., Harvey, E., Oxman, A. D., & Thomson, M. A. (1998). Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. The Cochrane Effective Practice and Organization of Care Review Group. British Medical Journal, 317(7156), 465–468.

    Article  PubMed  CAS  Google Scholar 

  • Bird, H. R., Canino, G., Rubiostipec, M., & Ribera, J. C. (1987). Further measures of the psychometric properties of The Childrens Global Assessment Scale. Archives of General Psychiatry, 44(9), 821–824.

    Article  PubMed  CAS  Google Scholar 

  • Bland, J. M., & Altman, D. G. (1996). Measurement error and correlation coefficients. British Medical Journal, 313(7048), 41–42.

    Article  PubMed  CAS  Google Scholar 

  • Davis, D. A., Thomson, M. A., Oxman, A. D., & Haynes, R. B. (1995). Changing physician performance. A systematic review of the effect of continuing medical education strategies. Journal of the American Medical Association, 274(9), 700–705.

    Article  PubMed  CAS  Google Scholar 

  • DeMaso, D. R., Twente, A. W., Spratt, E. G., & O’Brien, P. (1995). Impact of psychologic functioning, medical severity, and family functioning in pediatric heart transplantation. The Journal of Heart and Lung Transplantation, 14(6 Pt 1), 1102–1108.

    PubMed  CAS  Google Scholar 

  • Dyrborg, J., Larsen, F. W., Nielsen, S., Byman, J., Nielsen, B. B., & Gautre-Delay, F. (2000). The Children’s Global Assessment Scale (CGAS) and Global Assessment of Psychosocial Disability (GAPD) in clinical practice—substance and reliability as judged by intraclass correlations. European Child and Adolescent Psychiatry, 9(3), 195–201.

    Article  PubMed  CAS  Google Scholar 

  • Fernando, T., Mellsop, G., Nelson, K., Peace, K., & Wilson, J. (1986). The reliability of axis V of DSM-III. American Journal of Psychiatry, 143(6), 752–755.

    PubMed  CAS  Google Scholar 

  • Galanter, C. A., & Patel, V. L. (2005). Medical decision making: a selective review for child psychiatrists and psychologists. Journal of Child Psychology and Psychiatry and Allied Disciplines, 46(7), 675–689.

    Article  Google Scholar 

  • Gilbody, S. M., House, A. O., & Sheldon, T. A. (2002). Outcomes research in mental health. Systematic review. British Journal of Psychiatry, 181, 8–16.

    Article  PubMed  Google Scholar 

  • Gold, J., Buonopane, R. J., Caggiano, R. A., Picciotto, M., Vogeli, C., Kanner, N. T., et al. (2009). Assessing outcomes in child psychiatry. American Journal of Managed Care, 15(4), 210–216.

    PubMed  Google Scholar 

  • Goldman, H. H., Skodol, A. E., & Lave, T. R. (1992). Revising axis V for DSM-IV: A review of measures of social functioning. American Journal of Psychiatry, 149(9), 1148–1156.

    PubMed  CAS  Google Scholar 

  • Green, B., Shirk, S., Hanze, D., & Wanstrath, J. (1994). The Children’s Global Assessment Scale in clinical practice: An empirical evaluation. Journal of the American Academy of Child and Adolescent Psychiatry, 33(8), 1158–1164.

    Article  PubMed  CAS  Google Scholar 

  • Hanssen-Bauer, K., Aalen, O. O., Ruud, T., & Heyerdahl, S. (2007). Inter-rater reliability of clinician-rated outcome measures in child and adolescent mental health services. Administration and Policy In Mental Health, 34(6), 504–512.

    Article  PubMed  Google Scholar 

  • Hilsenroth, M. J., Ackerman, S. J., Blagys, M. D., Baumann, B. D., Baity, M. R., Smith, S. R., et al. (2000). Reliability and validity of DSM-IV axis V. American Journal of Psychiatry, 157(11), 1858–1863.

    Article  PubMed  CAS  Google Scholar 

  • Hoagwood, K., Hibbs, E., Brent, D., & Jensen, P. (1995). Introduction to the special section: efficacy and effectiveness in studies of child and adolescent psychotherapy. Journal of Consulting and Clinical Psychology, 63(5), 683–687.

    Article  PubMed  CAS  Google Scholar 

  • Hoagwood, K., Jensen, P. S., Petti, T., & Burns, B. J. (1996). Outcomes of mental health care for children and adolescents: I. A comprehensive conceptual model. Journal of the American Academy of Child and Adolescent Psychiatry, 35(8), 1055–1063.

    Article  PubMed  CAS  Google Scholar 

  • Hodges, K. (1993). Structured interviews for assessing children. Journal of Child Psychology and Psychiatry and Allied Disciplines, 34(1), 49–68.

    Article  CAS  Google Scholar 

  • Kirkpatrick, D. L., & Kirkpatrick, J. D. (2005). Evaluating training programs : The four levels (3rd ed.). San Francisco: Berrett-Koehler.

    Google Scholar 

  • Kobak, K. A., Engelhardt, N., & Lipsitz, J. D. (2006). Enriched rater training using Internet based technologies: A comparison to traditional rater training in a multi-site depression trial. Journal of Psychiatric Research, 40(3), 192–199.

    Article  PubMed  Google Scholar 

  • Kobak, K. A., Lipsitz, J. D., & Feiger, A. (2003). Development of a standardized training program for the Hamilton Depression Scale using internet-based technologies: Results from a pilot study. Journal of Psychiatric Research, 37(6), 509–515.

    Article  PubMed  Google Scholar 

  • Kobak, K. A., Opler, M. G. A., & Engelhardt, N. (2007). PANSS rater training using Internet and videoconference: Results from a pilot study. Schizophrenia Research, 92(1–3), 63–67.

    Article  PubMed  Google Scholar 

  • Lundh, A., Kowalski, J., Sundberg, C. J., Gumpert, C., & Landén, M. (2010). Children’s Global Assessment Scale (CGAS) in a naturalistic clinical setting: Inter-rater reliability and comparison with expert ratings. Psychiatry Research, 177(1–2), 206–210.

    Article  PubMed  Google Scholar 

  • Mazade, N. A., & Glover, R. W. (2007). State mental health policy: Critical priorities confronting state mental health agencies. Psychiatric Services, 58(9), 1148–1150.

    Article  PubMed  Google Scholar 

  • Müller, M. J., & Szegedi, A. (2002). Effects of interrater reliability of psychopathologic assessment on power and sample size calculations in clinical trials. Journal of Clinical Psychopharmacology, 22(3), 318–325.

    Article  PubMed  Google Scholar 

  • Muratori, F., Picchi, L., Bruni, G., Patarnello, M., & Romagnoli, G. (2003). A two-year follow-up of psychodynamic psychotherapy for internalizing disorders in children. Journal of the American Academy of Child and Adolescent Psychiatry, 42(3), 331–339.

    Article  PubMed  Google Scholar 

  • Myers, K., & Winters, N. C. (2002). Ten-year review of rating scales. I: Overview of scale functioning, psychometric properties, and selection. Journal of the American Academy of Child and Adolescent Psychiatry, 41(2), 114–122.

    Article  PubMed  Google Scholar 

  • Peabody, J. W., Luck, J., Glassman, P., Dresselhaus, T. R., & Lee, M. (2000). Comparison of vignettes, standardized patients, and chart abstraction: A prospective validation study of 3 methods for measuring quality. Journal of the American Medical Association, 283(13), 1715–1722.

    Article  PubMed  CAS  Google Scholar 

  • Rey, J. M., Starling, J., Wever, C., Dossetor, D. R., & Plapp, J. M. (1995). Inter-rater reliability of global assessment of functioning in a clinical setting. Journal of Child Psychology and Psychiatry and Allied Disciplines, 36(5), 787–792.

    Article  CAS  Google Scholar 

  • Rosen, J., Mulsant, B. H., Marino, P., Groening, C., Young, R. C., & Fox, D. (2008). Web-based training and interrater reliability testing for scoring the Hamilton Depression Rating Scale. Psychiatry Research, 161(1), 126–130.

    Article  PubMed  Google Scholar 

  • Schorre, B. E., & Vandvik, I. H. (2004). Global assessment of psychosocial functioning in child and adolescent psychiatry. A review of three unidimensional scales (CGAS, GAF, GAPD). European Child and Adolescent Psychiatry, 13(5), 273–286.

    Article  PubMed  Google Scholar 

  • Shaffer, D., Gould, M. S., Brasic, J., Ambrosini, P., Fisher, P., Bird, H., et al. (1983). A children’s global assessment scale (CGAS). Archives of General Psychiatry, 40(11), 1228–1231.

    Article  PubMed  CAS  Google Scholar 

  • Shaffer, D., Lucas, C. P., & Richters, J. E. (1999). Diagnostic assessment in child and adolescent psychopathology. New York: Guilford Press.

    Google Scholar 

  • Shrout, P. E. (1998). Measurement reliability and agreement in psychiatry. Statistical Methods in Medical Research, 7(3), 301–317.

    Article  PubMed  CAS  Google Scholar 

  • Weissman, M. M., Warner, V., & Fendrich, M. (1990). Applying impairment criteria to children’s psychiatric diagnosis. Journal of the American Academy of Child and Adolescent Psychiatry, 29(5), 789–795.

    Article  PubMed  CAS  Google Scholar 

  • Winters, N. C., Collett, B. R., & Myers, K. M. (2005). Ten-year review of rating scales, VII: Scales assessing functional impairment. Journal of the American Academy of Child and Adolescent Psychiatry, 44(4), 309–342.

    Article  PubMed  Google Scholar 

  • Wolpert, M., Cooper, L., Tingay, K., Young, K., & Svanberg, E. (2007). CAMHS Outcomes Research Consortium (CORC) Handbook Available from http://www.corc.uk.net/media/File/CORC%20Resources/Handbook/HANDBOOK%202007.pdf.

  • Yavorsky, C., Opler, M., Ivanova, E., Gordon, J., Jovic, S., & Yang, L. (2010). Quantifying rater drift in an International sample of investigators participating in standardized rater training events: Is PANSS reliability maintained over time? Paper presented at the ISCTM 6th Annual Scientific Meeting. http://isctm.org/public_access/2010_22Feb_Posters_Abstracts.pdf.

  • Zanarini, M. C., Skodol, A. E., Bender, D., Dolan, R., Sanislow, C., Schaefer, E., et al. (2000). The collaborative longitudinal personality disorders study: Reliability of axis I and II diagnoses. Journal of Personality Disorders, 14(4), 291–299.

    Article  PubMed  CAS  Google Scholar 

Download references

Acknowledgments

This study was made possible thanks to support from Stockholm County Council’s Child and Adolescent Mental Health Services and the Department of Clinical Neuroscience, Karolinska Institutet, Sweden. Mikael Landén is funded through the regional agreement on medical training and clinical research between Stockholm County Council and the Karolinska Institutet (ALF 20100305); and through grants from the Swedish Medical Research Council (K2008-62x-14647-06-3, K2010-61X-21569-01-1, and K2010-61P-21568-01-4). We wish to thank Anders Bolin, D.D.S. Ph.D., who did the programming in PowerPoint. We are also grateful for generous and helpful suggestions on the project from Prudence W. Fisher, Ph.D., Yanling Huo, M.S., David Shaffer, F.R.C.P. (Lond), F.R.C. (Lond), and J. Blake Turner, Ph.D., at the Division of Child and Adolescent Psychiatry, New York State Psychiatric Institute/Columbia University.

Author information

Authors and Affiliations

  1. Section of Psychiatry, Department of Clinical Neuroscience, Karolinska Institutet, BUP-divisionen, PO Box 17564, SE-11891, Stockholm, Sweden

    Anna Lundh

  2. JK Biostatistics, Stockholm, Sweden

    Jan Kowalski

  3. Department of Physiology and Pharmacology, Karolinska Institutet, Stockholm, Sweden

    Carl Johan Sundberg

  4. Unit for Bioentrepreneurship, Department of Learning, Informatics, Management, and Ethics (LIME), Karolinska Institutet, Stockholm, Sweden

    Carl Johan Sundberg

  5. Institute of Neuroscience and Physiology, The Sahlgrenska Academy at Gothenburg University, Gothenburg, Sweden

    Mikael Landén

  6. Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden

    Mikael Landén

Authors
  1. Anna Lundh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Kowalski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Carl Johan Sundberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mikael Landén
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anna Lundh.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lundh, A., Kowalski, J., Sundberg, C.J. et al. A Comparison of Seminar and Computer Based Training on the Accuracy and Reliability of Raters Using the Children’s Global Assessment Scale (CGAS). Adm Policy Ment Health 39, 458–465 (2012). https://doi.org/10.1007/s10488-011-0369-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-011-0369-5

Keywords

Search

Navigation