Abstract
This article discusses outcome evaluation systems for mental health programs. It reviews and critically evaluates design and analysis methods for strengthening the validity of such uncontrolled comparisons. The article examines methods for statistically adjusting preexisting groups, now referred to as risk adjustment or case-mix adjustment, and offers guidelines for determining when this procedure is appropriate. Then, analyses on two dependent variables—a global rating of functioning and a consumer satisfaction measure—available from an outcomes evaluation system currently underway in Florida are used to demonstrate the proposed method of risk adjustment. Results for 24 providers of mental health services showed that while risk adjustment only made a small difference in the overall provider rankings, the ranking of some specific providers changed considerably. The article concludes with a discussion of the implications of this research.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Lyons JS, Howard KI, O'Mahoney MT, et al.The Measurement and Management of Clinical Outcomes in Mental Health. New York: Wiley; 1997.
Relman AS. Assessment and accountability: the third revolution in medical care.New England Journal of Medicine. 1988;319:1197–1202.
Hendryx MS. Testing competing risk-adjustment models. Presented at the 6th Annual Florida Conference on Behavioral Healthcare Evaluation; December 9, 1999; Orlando, FL.
Ettner SL, Frank RG.Risk Adjustment for Mental Health and Substance Abuse. Rockville, MD: Survey and Analysis Branch, Division of State and Community Systems Development, Center for Mental Health Services; 1999.
Iezzoni L.Risk Adjustment for Measuring Health Care Outcomes. Ann Arbor, MI: Health Administration Press; 1994.
Weisberg HI. Statistical adjustments and uncontrolled studies.Psychological Bulletin. 1979;86:1149–1164.
Shadish WS, Ragsdale K. Random vs. nonrandom assignment in controlled experiments: do you get the same answer?Journal of Consulting and Clinical Psychology. 1996;64:1290–1305.
Lord FM. A paradox in the interpretation of group comparisons.Psychological Bulletin. 1967;68:304–305.
Lord FM. Statistical adjustments when comparing preexisting groups.Psychological Bulletin. 1969;72:336–337.
Cronbach LJ, Furby L. How we should measure “change”—or should we?Psychological Bulletin. 1970;74:68–80.
Hohmann A. A contextual model for clinical mental health effectiveness research.Mental Health Services Research. 1999;1:83–91.
Hornbrook MC, Goodman MJ. Chronic disease, functional health status, and demographics: a multidimensional approach to risk adjustment.Health Services Research. 1996;31:283–307.
Fowles JB, Weiner JP, Knutson D, et al. Taking health status into account when setting capitation rates.Journal of the American Medical Association. 1996;276:1316–1321.
Salem-Schatz S, Moore G, Rucker M, et al. The case for case-mix adjustment in practice profiling: when good apples look bad.Journal of the American Medical Association. 1994;272:871–874.
Cunningham WE, Tisnado DM, Lui HH, et al. The effect of hospital experience on mortality among patients hospitalized with acquired immunodeficiency syndrome in California.Journal of the American Medical Association. 1999;107:137–143.
DesHarnais SI, McMahon LF, Wrobleweski RT, et al. Measuring hospital performance.Medical Care. 1990;28:1127–1141.
Ettner SL, Frank RG, McGuire TG, et al. Risk adjustment of mental health and substance abuse payments.Inquiry. 1998;35:223–239.
Banks SM, Pandiani JA, Schacht L, et al. A risk-adjusted measure of hospitalization rates for evaluating community mental health program performance.Administration and Policy in Mental Health. 1999;26:269–279.
Pandiani JA, Banks SM, Schacht LM. Using incarceration rates to measure mental health program performance.The Journal of Behavioral Health Services & Research. 1998;25:300–311.
Hendryx MS, Dyck DG, Srebnik D. Risk-adjusted outcome models for public mental health outpatient programs.Health Services Research. 1999;34:171–195.
Dow MG, Boaz TL. Alternative procedures for risk adjusting the Florida outcomes: methods of analysis. Presented at the national conference on Mental Health Outcomes Risk Adjustment; August 17, 1999; Spokane, WA.
Dow MG, Ward JC.Behavioral Healthcare Rating of Satisfaction (BHRS). Odessa, FL: Psychological Assessment Resources; 1996.
Ward JC, Dow MG.Functional Assessment Rating Scale (FARS). Odessa, FL: Psychological Assessment Resources; 1996.
Dow MG, Ward JC, Saunders TL, et al.Program Evaluation and Outcome Assessment Project, HRS District 7, Phase Three Summary. Tampa, FL: Department of Community Mental Health, Florida Mental Health Institute, University of South Florida; 1996.
Ellis R, Wackwitz J, Foster M.Colorado Client Assessment Record (CCAR). Denver, CO: Office of Mental Health Services; 1991.
Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition. Washington, DC: American Psychiatric Association; 1994.
Dow MG, Boaz TL. Assisting clients of community mental health centers to secure SSI benefits: a controlled evaluation.Community Mental Health Journal. 1994;30:429–440.
DuBois PH.Multivariate Correlational Analysis. New York: Harper; 1957.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Dow, M.G., Boaz, T.L. & Thornton, D. Risk adjustment of Florida mental health outcomes data: Concepts, methods, and results. The Journal of Behavioral Health Services & Research 28, 258–272 (2001). https://doi.org/10.1007/BF02287243
Issue Date:
DOI: https://doi.org/10.1007/BF02287243