Skip to main content

Advertisement

SpringerLink
Log in

The Incremental Value of Self-Reported Mental Health Measures in Predicting Functional Outcomes of Veterans

  • Published:
The Journal of Behavioral Health Services & Research Aims and scope Submit manuscript

Abstract

Research on patient-centered care supports use of patient/consumer self-report measures in monitoring health outcomes. This study examined the incremental value of self-report mental health measures relative to a clinician-rated measure in predicting functional outcomes among mental health service recipients. Participants (n = 446) completed the Behavior and Symptom Identification Scale, the Brief Symptom Inventory, and the Veterans/Rand Short Form-36 at enrollment in the study (T1) and 3 months later (T2). Global Assessment of Functioning (GAF) ratings, mental health service utilization, and psychiatric diagnoses were obtained from administrative data files. Controlling for demographic and clinical variables, results indicated that improvement based on the self-report measures significantly predicted one or more functional outcomes (i.e., decreased likelihood of post-enrollment psychiatric hospitalization and increased likelihood of paid employment), above and beyond the predictive value of the GAF. Inclusion of self-report measures may be a useful addition to performance measurement efforts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Donabedian A: Explorations in Quality Assessment and Monitoring, Vol. I: The Definition of Quality and Approaches to its Assessment, Ann Arbor, MI, Health Administration Press, 1980.

  2. Eisen SV, Grob MC, Dill DL. Outcome Measurement: Tapping the Patient’s Perspective. In Psychiatric Treatment, Advances in Outcome Research, Mirin SM, Gossett J, Grob MC (eds.) Washington, DC: American Psychiatric Press, 213-235, 1991

    Google Scholar 

  3. Young A, Grusky O, Jordan D, et al. Routine Outcome Monitoring in a Public Mental Health System: The Impact of Patients Who Leave Care. Psychiatric Services 2000; 51: 85-91.

    CAS  Google Scholar 

  4. Deegan PE, Drake RE. Shared Decision Making and Medication Management in the Recovery Process. Psychiatric Services 2006; 57:1636-1639.

    Article  PubMed  Google Scholar 

  5. Gertais M, Edgman-Levitan S, Daley J, et al. (Eds.) Through the Patient’s Eyes. Understanding and Promoting Patient-Centered Care. San Francisco: Jossey-Bass, 1993.

  6. Schauer C, Everett A, del Vecchio P, et al. Promoting the value and practice of shared decision-making in mental health care. Psychiatric Rehabilitation Journal 2007; 31: 54-61.

    Article  PubMed  Google Scholar 

  7. Sechrest L, McKnight P, McKnight K. Calibration of measures for psychotherapy outcome research. American Psychologist 1996; 51: 1065-1071.

    Article  PubMed  CAS  Google Scholar 

  8. Campbell JF. The Consumer Movement and implications for vocational rehabilitation services. Journal of Vocational Rehabilitation 1:67-75, 1991.

    Google Scholar 

  9. Ellison ML, Danley KS, Crean T, et al. Involvement of people with psychiatric disabilities in state agencies of vocational rehabilitation: state agency survey. Journal of Rehabilitation Administration 20:319-333, 1996.

    Google Scholar 

  10. Chinman M, Allende M, Weingarten R, et al. On the road to collaborative treatment planning: consumer and provider perspective. Journal of Behavioral Health Services & Research 1999; 26: 211-218.

    Article  CAS  Google Scholar 

  11. Chinman M, Young AS, Hassell J, et al. Toward the implementation of mental health consumer provider services. Journal of Behavioral Health Services & Research 2006; 33:176-195.

    Article  Google Scholar 

  12. Hansen T, Hatling T, Lidal E, et al. The user perspective: respected or reject in mental health care? Journal of Psychiatric and Mental Health Nursing 2004; 11: 292-297.

    Article  PubMed  CAS  Google Scholar 

  13. Guthrie D, McIntosh M, Callaly T, et al. Consumer attitudes towards the use of routine outcome measures in a public mental health service: a consumer-driven study. International Journal of Mental Health Nursing 2008; 17: 92-97.

    Article  PubMed  Google Scholar 

  14. Garland AF, Kruse M, & Aarons GA. Clinicians and outcome measurement: what’s the use? Journal of Behavioral Health Services & Research 2003; 30: 393-405.

    Article  Google Scholar 

  15. Gilbody SM, House AO, & Sheldon, TA. Outcomes research in mental health. British Journal of Psychiatry 2002; 181: 8-16.

    Article  PubMed  Google Scholar 

  16. Gilbody SM, House AO, & Sheldon, TA. Routine administration of Health Related Quality of Life (HRQoL) and needs assessment instruments to improve psychological outcome—a systematic review. Psychological Medicine 2002; 32: 1345-1356.

    Article  PubMed  CAS  Google Scholar 

  17. Slade M. What outcomes to measure in routine mental health services, and how to assess them: a systematic review. Australian and New Zealand Journal of Psychiatry 2002; 36: 743-753.

    Article  PubMed  Google Scholar 

  18. Slade M. Routine outcome assessment in mental health services. Psychological Medicine, 2002.

  19. Garb HN. Incremental validity and the assessment of psychopathology in adults. Psychological Assessment 2003; 15: 508-520.

    Article  PubMed  Google Scholar 

  20. Haynes SN & Lench HC. Incremental validity of new clinical assessment measures. Psychological Assessment 2003; 15: 456-466.

    Article  PubMed  Google Scholar 

  21. Hunsley J. Introduction to the special section on incremental validity and utility in clinical assessment. Psychological Assessment 2003; 15: 443-445.

    Article  PubMed  Google Scholar 

  22. Overall JE, Gorham DR. The Brief Psychiatric Rating Scale. Psychological Reports 1962; 10: 799-812.

    Article  Google Scholar 

  23. Spitzer RL, Gibbon M, Williams JB, et al. Global Assessment of Functioning (GAF) scale. In: Sederer LI, Dickey B (eds). Outcomes Assessment in Clinical Practice. Baltimore, Williams & Wilkins, 1996.

    Google Scholar 

  24. American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders (Text Revision): DSM-IV-TR (ed 4). Washington, DC: American Psychiatric Association, 2000.

    Book  Google Scholar 

  25. Greenberg GA, Rosenheck, RA. Using the GAF as a national mental health outcome measure in the Department of Veterans Affairs. Psychiatric Services 2005; 56: 420-426.

    Article  PubMed  Google Scholar 

  26. Greenberg GA, Rosenheck RA. Continuity of care and clinical outcomes in a national health system. Psychiatric Services 2005; 56: 427-433.

    Article  PubMed  Google Scholar 

  27. Jones SH, Thornicroft G, Coffey M et al. A Brief Mental Health Outcome Scale: reliability and validity. British Journal of Psychology 1995; 166: 654-659.

    Article  CAS  Google Scholar 

  28. Jovanovi AA, Gasi MJ, Ivkovi M, et al. Reliability and validity of DSM-IV Axis V scales in a clinical sample of veterans with posttraumatic stress disorder. Psychiatria Danubina 2008; 20: 286-300.

    Google Scholar 

  29. Schwartz, RC. Concurrent validity of the Global Assessment of Functioning Scale for clients with schizophrenia. Psychological Reports 2007; 100: 571-574.

    Article  PubMed  Google Scholar 

  30. Sernyak MJ, Leslie D, Rosenheck R. Use of system-wide outcomes monitoring data to compare the effectiveness of atypical neuroleptic medications. American Journal of Psychiatry 2003; 160: 310-315.

    Article  PubMed  Google Scholar 

  31. Startup M, Jackson MC, Bendix S. The concurrent validity of the Global Assessment of Functioning. British Journal of Clinical Psychology 2002; 41: 417-422.

    Article  PubMed  Google Scholar 

  32. Bates LW, Lyons JA, Shaw JB. Effects of Brief Training on Application of the Global Assessment of Functioning Scale. Psychological Reports 2002; 91: 999-1006.

    Article  PubMed  Google Scholar 

  33. Moos RH, McCoy L, Moos BS. Global Assessment of Functioning (GAF) ratings: determinants and role as predictors of one-year treatment outcomes. Journal of Clinical Psychology 2000; 56: 449-461.

    Article  PubMed  CAS  Google Scholar 

  34. Moos RH, Nichol AC, Moos BS. Global Assessment of Functioning ratings and the allocation and outcomes of mental health services. Psychiatric Services 2002; 53: 730-737.

    Article  PubMed  Google Scholar 

  35. Roy-Byrne P, Dagadakis C, Unutzer J, et al. Evidence for limited validity of the Revised Global Assessment of Functioning Scale. Psychiatric Services 1996; 47: 864-866.

    PubMed  CAS  Google Scholar 

  36. Söderberg P, Tungström S, Armelius BA. Reliability of Global Assessment of Functioning ratings made by clinical psychiatric staff. Psychiatric Services 2005; 56: 434-438.

    Article  PubMed  Google Scholar 

  37. Zastrow A, Faude V, Seyboth F, et al. Risk factors of symptom underestimation by physicians. Journal of Psychosomatic Research 2008; 64: 543-551.

    Article  PubMed  Google Scholar 

  38. Ellwood P, Lundberg G. Managed care: a work in progress. Journal of the American Medical Association 1996; 276: 1083-1086.

    Article  PubMed  Google Scholar 

  39. Hughner RS & Kleine SS. Variations in Lay Health Theories: Implications for Consumer Health Care Decision Making. Qualitative Health Research 2008; 18: 1687-1703.

    Article  Google Scholar 

  40. Atkinson M, Zibin S, Chuang H. Characterizing quality of life among patients with chronic mental illness: a critical examination of the self-report methodology. American Journal of Psychiatry 1997; 154: 99-105.

    PubMed  CAS  Google Scholar 

  41. Dickey B, Wagenaar H. Evaluating mental health care reform: including the clinician, client and family perspective. Journal of Mental Health Administration 1994; 21: 313-319.

    Article  PubMed  CAS  Google Scholar 

  42. Hamera EK, Schneider JK, Potocky M, et al. Validity of self-administered symptom scales in clients with schizophrenia and schizoaffective disorders. Schizophrenia Research 1996; 19: 213-219.

    Article  PubMed  CAS  Google Scholar 

  43. Niv N, Cohen AN, Sullivan G, et al. The MIRECC version of the Global Assessment of Functioning scale: reliability and validity. Psychiatric Services 2007; 58: 529-535.

    Article  PubMed  Google Scholar 

  44. Hunter R, Cameron R, Norrie J. Using patient-reported outcomes in schizophrenia: the Scottish Schizophrenia Outcomes Study. Psychiatric Services 2009; 60: 240-245.

    Article  PubMed  Google Scholar 

  45. Zuvekas SH, Fleishman JA. Self-rated mental health and racial/ethnic disparities in mental health service use. Medical Care 2008; 46: 915-923.

    Article  PubMed  Google Scholar 

  46. Statement of The Honorable Robert H. Roswell, MD, Under Secretary for Health, Dept. of Veterans Affairs, March 2, 2004

  47. VA Mental Health Consumer Council. VHA mental health strategic plan. Consumer Council Newsletter 2004; 7: 1-2.

    Google Scholar 

  48. The President’s New Freedom Commission on Mental Health Report. www.mentalhealthcommission.gov/

  49. Zimmerman M, McGlinchey JB, Posternak MA, et al. Remission in depressed outpatients: more than just symptom resolution?. Journal of Psychiatric Research 2008; 42: 797-801.

    Article  PubMed  Google Scholar 

  50. Zimmerman M, Ruggero CJ, Chelminski, I, et al. Developing brief scales for use in clinical practice: the reliability and validity of single-item self-report measures of depressive symptom severity, psychosocial impairment due to depression, and quality of life. Journal of Clinical Psychiatry 2006; 67: 1536-1541.

    Article  PubMed  Google Scholar 

  51. Burns RB, Moskowitz MA, Ash A, et al. Self-report versus medical record functional status. Medical Care 1992; 30: MS85-MS95.

    Article  PubMed  CAS  Google Scholar 

  52. Distefano MK, Pryer MW, Garrison JL. Validity of psychiatric patient’s self-reports of rehospitalization. Hospital & Community Psychiatry 1991; 12: 849-850.

    Google Scholar 

  53. Russo J, Roy-Byrne P, Jaffe C, et al. The relationship of patient-administered outcome assessments to quality of life and physician ratings: validity of the BASIS-32. Journal of Mental Health Administration 1997; 22:200-214.

    Article  Google Scholar 

  54. Mausbach BT, Harvey PD, Goldman SR, et al. Development of a brief scale of everyday functioning in persons with serious mental illness. Schizophrenia Bulletin 2007; 33:1364-1372.

    Article  PubMed  Google Scholar 

  55. Mundt JC, Marks IM, Shear MK, et al. The Work and Social Adjustment Scale: a simple measure of impairment in functioning. British Journal of Psychiatry 2002; 180:461-464.

    Article  PubMed  Google Scholar 

  56. Henry AD, & Lucca AM. Facilitators and barriers to employment: the perspectives of people with psychiatric disabilities and employment service providers. Work 2004; 22:169-182.

    PubMed  Google Scholar 

  57. Hendryx MS, Russo JE, Stegner B, et al. Predicting rehospitalization and outpatient services from administration and clinical databases. The Journal of Behavioral Health Services & Research 2003; 30: 342-351.

    Article  Google Scholar 

  58. Stergiopoulos V, Dewa CS, Rouleau K, et al. Collaborative mental health care for the homeless: the role of psychiatry in positive housing and mental health outcomes. Canadian Journal of Psychiatry - Revue Canadienne de Psychiatrie 2008; 53: 61-67.

    PubMed  Google Scholar 

  59. Rosenheck R, Lam JA. Homeless mentally ill clients’ and providers’ perceptions of service needs and clients’ use of services. Psychiatric Services 1997; 48: 381-386.

    PubMed  CAS  Google Scholar 

  60. Clements KM, Murphy JM, Eisen SV, et al. Comparison of self-report and clinician-rated measures of psychiatric symptoms and functioning in predicting 1-year hospital readmission. Administration and Policy in Mental Health & Mental Health Services Research 2006; 33: 568-577.

    Article  Google Scholar 

  61. Salyers MP, Godfrey JL, Mueser KT, et al. Measuring illness management outcomes: a psychometric study of clinician and consumer rating scales for illness self-management and recovery. Community Mental Health Journal 2007; 43:459-480.

    Article  PubMed  Google Scholar 

  62. Eisen SV, Normand SLT, Belanger A. et al. The revised Behavior and Symptom Identification Scale (BASIS-24): reliability and validity. Medical Care 2004; 42: 1230-1241.

    Article  PubMed  Google Scholar 

  63. Derogatis LR. Brief Symptom Inventory (BSI)—Administration, Scoring and Procedures Manual (3 rd edition). Minneapolis: National Computer systems, 1993.

    Google Scholar 

  64. Derogatis LR, Savitz KL. The SCL-90-R, Brief Symptom Inventory, and Matching Clinical Rating Scales. In: Maruish M. (ed). The Use of Psychological Testing for Treatment Planning and Outcome Assessment. New Jersey: Lawrence Erlbaum Associates, 1999, 679-724.

    Google Scholar 

  65. Ware JE, Sherbourne CD. The MOS 36-item Short-Form Health Survey (SF-36): I conceptual framework and items selection. Medical Care, 1992; 30: 473-483.

    Article  PubMed  Google Scholar 

  66. Kazis LE, Miller DR, Clark JA et al. Improving the response choices on the veterans SF-36 health survey role functioning scales. Results from the Veterans Health Study. Journal of Ambulatory Care Management, 2004; 27: 263-280.

    PubMed  Google Scholar 

  67. Spitzer RL Gibbon M, Williams JB, et al. Global Assessment of Functioning (GAF) scale in Outcomes Assessment in Clinical Practice. In: Sederer LI, Dickey B (eds) Baltimore, Williams & Wilkins, 1996.

    Google Scholar 

  68. Tabachnick BG, Fidell LS. Using Multivariate Statistics (5th Edition). Boston: Allyn & Bacon, 2006.

    Google Scholar 

  69. Barry MM, Zissi A. Quality of life as an outcome measure in evaluating mental health services: a review of the empirical evidence. Social Psychiatry & Psychiatric Epidemiology 1997; 32: 38-47.

    Article  CAS  Google Scholar 

  70. Ramirez A, Ekeselius L, Ramklint M. Axis V–Global Assessment of Functioning scale (GAF): further evaluation of the self-report version. European Psychiatry 2008; 23: 575-579.

    Article  PubMed  CAS  Google Scholar 

  71. Slade M, McCrone P, Kuipers E, et al. Use of standardised outcome measures in adult mental health services: a randomized controlled trial. British Journal of Psychiatry 2006; 18: 330-336.

    Article  Google Scholar 

  72. Rosen AK, Chatterjee S, Glickman ME, Spiro AS, Seal P, Eisen SV. Improving Risk Adjustment of Self-Reported Mental Health Outcomes. Journal of Behavioral Health Services & Research 2010; 37: 291-306.

    Google Scholar 

  73. Adler DA, Bungay KM, Cynn DJ, et al. Patient-based health status assessments in an outpatient psychiatry setting. Psychiatric Services 2000;51:341-348.

    Article  PubMed  CAS  Google Scholar 

  74. Clardy JA, Booth BM, Smith LG, et al. Implementing a statewide outcomes management system for consumers of public mental health services. Psychiatric Services 1998;49:191-195.

    PubMed  CAS  Google Scholar 

Download references

Acknowledgements

This research was supported by VA grant IIR 02-290 from the Veterans Administration Health Services Research & Development (HSR&D) Service. The work was conducted at the Center for Health Quality, Outcomes and Economic Research, a Health Services Research and Development Center of Excellence of the Veterans Administration located at the Edith Nourse Rogers Memorial Veterans Hospital, Bedford, MA. The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs. We thank Nicole DelVecchio for her assistance with manuscript preparation.

RH: Incremental Value of Self-Reported Mental Health Measures

Author information

Authors and Affiliations

  1. Center for Health Quality, Outcomes and Economic Research (CHQOER), Edith Nourse Rogers Memorial Veterans Hospital, 200 Springs Road (152), Bedford, MA, 01730, USA

    Susan V. Eisen PhD, Mark E. Glickman PhD & Mark R. Schultz PhD

  2. Charlie Norwood VAMC, 1 Freedom Way (261), Augusta, GA, 30904, USA

    Kathryn A. Bottonari PhD

  3. Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), Boston University School of Public Health, VA Boston Healthcare System, 150 South Huntington Avenue, Boston, MA, 02130, USA

    Avron Spiro III PhD

  4. NEPEC, VA Connecticut Healthcare System, 950 Campbell Ave, West Haven, CT, USA

    Lawrence Herz MD & Robert Rosenheck MD

  5. VISN 1, Edith Nourse Rogers Memorial Veterans Hospital, 200 Springs Road (116A), Bedford, MA, 01730, USA

    Ethan S. Rofman MD

Authors
  1. Susan V. Eisen PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kathryn A. Bottonari PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mark E. Glickman PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Avron Spiro III PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mark R. Schultz PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lawrence Herz MD
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Robert Rosenheck MD
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Ethan S. Rofman MD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Susan V. Eisen PhD.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Eisen, S.V., Bottonari, K.A., Glickman, M.E. et al. The Incremental Value of Self-Reported Mental Health Measures in Predicting Functional Outcomes of Veterans. J Behav Health Serv Res 38, 170–190 (2011). https://doi.org/10.1007/s11414-010-9216-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11414-010-9216-9

Keywords

Search

Navigation