Abstract
For over two decades, disease management (DM) has been touted as an intervention capable of producing large scale cost savings for health care purchasers. However, the preponderance of scientific evidence suggests that these programs do not save money. This finding is not surprising given that the theorized causal mechanism by which the intervention supposedly influences the outcome has not been systematically assessed. Mediation analysis is a statistical approach to identifying causal pathways by testing the relationships between the treatment, the outcome, and an intermediate variable that is posited to mediate the relationship between the treatment and outcome. This analysis can therefore help identify how to make DM interventions effective by determining the causal mechanisms between intervention components and the desired outcome. DM interventions can then be optimized by eliminating those activities that are ineffective or even counter-productive. In this article we seek to promote the application of mediation analysis to DM program evaluation by describing the two principal frameworks generally followed in causal mediation analysis; structural equation modeling and potential outcomes. After comparing several approaches within these frameworks using real and simulated data, we find that some methods perform better than others under the conditions imposed upon the models. We conclude that mediation analysis can assist DM programs in developing and testing the causal pathways that enable interventions to be effective in achieving desired outcomes.
Similar content being viewed by others
Notes
Linden and Adler-Milstein (2008) highlight many additional factors that explain why the standard DM approach is not effective, based on the best available evidence from the literature.
In observational studies the assumption of no residual confounding or no selection bias cannot be tested, and so causal effects are only identified under this assumption.
“Similar” refers to comparability on both observed and unobserved characteristics, because we assume no residual confounding (ignorability) once we have conditioned on pre-treatment covariates and treatment.
Mediators or outcomes may also be ordered, such as rating of perceived health status or satisfaction on a Likert-type scale (e.g., 1 through 5), which can be modeled using ordered logit or probit models (which are natural extensions of the logit or probit models).
Breen et al. (Forthcoming) develop the method further and suggest some further identities we refer to here.
An additional assumption required here for causal inference is that each individual’s potential outcomes are unrelated to the treatment status of any other individual under study (Rubin 1978; Manski, Forthcoming). In an evaluation of a DM intervention, this assumption could be violated if members of the same household were enrolled in the intervention, possibly influencing the outcomes of one another.
This holds under the no interaction assumption in which direct and indirect effects are assumed to be identical between treatment and control groups (see Imai et al. 2010, p. 312).
See Morgan and Todd (2008) for a discussion on assessing the effects of the ATT versus ATC estimators.
The method is implemented in both the R Language (Imai et al. 2010) and in Stata (Hicks and Tingley 2011).
However, in the software (Hicks and Tingley 2011), the reported “percent mediated” is the median of a simulated distribution of “percent mediated,” and thus may not provide the same result as that derived by dividing Eq. 10 by Eq. 11. We return to this point in our Monte Carlo study.
MacKinnon et al. (1995) demonstrate that mediation percentages are unstable for smaller sample sizes. We nevertheless choose to report these percentages here, because they are widely used in applied research and because they provide a sensible metric for comparing results in nonlinear probability models in which point estimates of total, direct, and effects are identified up to an arbitrary scale.
We dichotomize these variables strictly to illustrate the modeling approach. In practice, converting continuous variables to dichotomous or categorical variables should be avoided, as it leads to a loss of information and reduces power (Royston et al. 2006).
We first estimated an interaction model which produced a non-significant interaction effect (p = 0.230, CI: −0.044, 0.18), followed by a review of the contrasts between groups at each level of the mediator which supported the no-interaction effect assumption.
Because the true mediation effect cannot be analytically derived in this setup, we obtain the true percent mediated using a Monte Carlo study with 100 replications and 1,000,000 observations per draw, which essentially provides us with a population estimate.
We report both mean and median given the skewed distribution of the mediation percentages across the 72 scenarios. Although the level of bias differs between the two central tendency measures, the overall pattern of results is very similar whether one uses the mean or the median as the basis of evaluation. We nevertheless report both central tendencies in order for the reader to properly assess the results.
Comparing the method by Imai et al. (2010a, b, c) using a logit link for the outcome and a linear model for the mediator and the method by Karlson et al. (2012) in a Monte Carlo study, Breen et al. (Forthcoming) found the methods to yield highly similar results; corroborating the results we report here.
References
Alwin, D.F., Hauser, R.M.: The decomposition of effects in path analysis. Am. Sociol. Rev. 40, 37–47 (1975)
Antonakis, J., Bendahan, S., Jacquart, P., Lalive, R.: On making causal claims: a review and recommendations. Leadersh. Q. 21, 1086–1120 (2010)
Baron, R.M., Kenny, D.A.: The moderator–mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. J. Pers. Soc. Psychol. 51, 118–1173 (1986)
Bauer, D.J., Preacher, K.J., Gil, K.M.: Conceptualizing and testing random indirect effects and moderated mediation in multilevel models: new procedures and recommendations. Psychol. Methods 11, 142–163 (2006)
Bodenheimer, T., Lorig, K., Holman, H., Grumbach, K.: Patient self-management of chronic disease in primary care. J. Am. Med. Assoc. 288, 2469–2475 (2002)
Bollen, K.A.: Structural equations with latent variables. Wiley, New York (1989)
Breen, R.B., Karlson, K.B., Holm, A.: Total, direct, and indirect in logit and probit models. Sociol. Methods Res. (Forthcoming)
Butterworth, S.W., Andersen, B.T.: Health Coaching Performance Assessment™ (HCPA): a new tool for benchmarking and improving effectiveness. HealthSciences Institute. http://healthsciences.org/health-coaching-performance-assessment-hcpa-white-paper(2011). Accessed 13 Feb 2012
Butterworth, S., Linden, A., McClay, W.: Health coaching as an intervention in health management programs. Dis. Manag. Health Outcomes 15, 299–307 (2007)
Cheong, J., MacKinnon, D.P., Khoo, S.T.: Investigation of meditational process using parallel process latent growth curve modeling. Struct. Equ. Model. 10, 238–262 (2003)
Cole, D.A., Maxwell, S.E.: Testing meditational models with longitudinal data: questions and tips in the use of structural equation modeling. J. Abnorm. Psychol. 112, 558–577 (2003)
Congressional Budget Office: an analysis of the literature on disease management programs. Washington DC: Congressional Budget Office. http://www.cbo.gov/sites/default/files/cbofiles/ftpdocs/59xx/doc5909/10-13-diseasemngmnt.pdf(2004). Accessed 19 Oct 2012
Cramer, J.S.: Logit models. From economics and other fields. Cambridge University Press, Cambridge (2003)
Duncan, O.D.: Path analysis: sociological examples. Am. J. Sociol. 72, 1–16 (1966)
Efron, B., Tibshirani, R.: An introduction to the bootstrap. Chapman and Hall, New York (1993)
Frangakis, C.E., Rubin, D.B.: Principal stratification in causal inference. Biometrics 58, 21–29 (2002)
Freedman, L.S., Schatzkin, A.: Sample size for studying intermediate endpoints within intervention trials of observational studies. Am. J. Epidemiol. 136, 1148–1159 (1992)
Gennetian, L.A., Magnuson, K., Morris, P.A.: From statistical associations to causation: what developmentalists can learn from instrumental variables techniques coupled with experimental data. Dev. Psychol. 44, 381–394 (2008)
Glynn, A.N.: The product and difference fallacies for indirect effects. Am. J. Political Sci. 56, 257–269 (2012)
Goetzel, R.Z., Ozminkowski, R.J., Villagra, V.G., Duffy, J.: Return on investment on disease management: a review. Health Care Financ. Rev. 26, 1–19 (2005)
Hafeman, D.M.: Confounding of indirect effects: a sensitivity analysis exploring the range of bias due to a cause common to both the mediator and the outcome. Am. J. Epidemiol. 174, 710–717 (2011)
Hafeman, D.M., Schwartz, S.: Opening the black box: a motivation for the assessment of mediation. Int. J. Epidemiol. 38, 838–845 (2009)
Hibbard, J.H., Stockard, J., Mahoney, E.R., Tusler, M.: Development of the patient activation measure (PAM): conceptualizing and measuring activation in patients and consumers. Health Serv. Res. 39, 1026–1105 (2004)
Hicks, R., Tingley, D.: Casual mediation analysis. Stata J. 11, 605–619 (2011)
Hill, J., Waldfogel, J., Brooks-Gunn, J.: Sustained effects of high participation in an early intervention for low-birth-weight premature infants. Dev. Psychol. 39, 730–744 (2003)
Hirano, K., Imbens, G.W.: The propensity score with continuous treatments. In: Gelman, A., Meng, X.-L. (eds.) Applied Bayesian Modeling and Causal Inference from Incomplete-Data Perspectives, pp. 73–84. Wiley InterScience, West Sussex (2004)
Holland, P.W.: Statistics and causal inference. J. Am. Stat. Assoc. 81, 945–960 (1986)
Holland, P.W.: Causal inference, path analysis, and recursive structural equation models. In: Clogg, C.C. (ed.) Sociological Methodology, pp. 449–484. American Sociological Association, Washington, DC (1988)
Hong, G.: Ratio of mediator probability weighting for estimating natural direct and indirect effects. In: 2010 Proceedings of the American Statistical Association, Biometrics Section, pp. 2401–2415. American Statistical Association, Alexandria (2010)
Imai, K., Keele, L., Tingley, D.: A general approach to causal mediation analysis. Psychol. Methods 15, 309–334 (2010a)
Imai, K., Keele, L., Yamamoto, T.: Identification, inference, and sensitivity analysis for causal mediation effects. Stat. Sci. 25, 51–71 (2010b)
Imai, K., Keele, L., Tingley, D., Yamamoto, T.: Advances in social science research using R. In: Vinod, H.D. (ed.) Causal Mediation Analysis Using R, pp. 129–154. Springer, New York (2010c)
Imai, K., Tingley, D., Yamamoto, T.: Experimental designs for identifying causal mechanisms. J. R. Stat. Soc. A 176(1), 5–51 (2013)
Imai, K., van Dyke, D.A.: Causal inference with general treatment regimes: generalizing the propensity score. J. Am. Stat. Assoc. 99, 854–866 (2004)
Jo, B.: Causal inference in randomized experiments with mediational processes. Psychol. Methods 13, 314–336 (2008)
Jo, B., Stuart, E.A.: Comments: causal interpretations of mediation effects. J. Res. Educ. Eff. 5, 250–253 (2012)
Jo, B., Stuart, E.A., MacKinnon, D.P., Vinokur, A.D.: The use of propensity scores in mediation analysis. Multivar. Behav. Res. 46, 425–452 (2011)
Jo, B., Vinokur, A.D.: Sensitivity analysis and bounding of causal effects with alternative identifying assumptions. J. Educ. Behav. Stat. 36, 415–440 (2011)
Joffe, M.M., Rosenbaum, P.R.: Invited commentary: propensity scores. Am. J. Epidemiol. 150, 327–333 (1999)
Judd, C.M., Kenny, D.A.: Process analysis: estimating mediation in treatment evaluations. Eval. Rev. 5, 602–619 (1981)
Karlson, K.B., Holm, A.: Decomposing primary and secondary effects: a new decomposition method. Res. Stratif. Soc. Mobil. 29, 221–237 (2011)
Karlson, K.B., Holm, A., Breen, R.: Comparing regression coefficients between models using logit and probit: a new method. Sociol. Methodol. 42, 274–301 (2012)
Kohler, U., Karlson, K.B., Holm, A.: Comparing coefficients of nested nonlinear probability models. Stata J. 11, 420–438 (2011)
Kraemer, H.C., Kiernan, M., Essex, M.J., Kupfer, D.J.: How and why criteria defining moderators and mediators differ between the Baron and Kenny and MacArthur approaches. Health Psychol. 27, 101–108 (2008)
Krull, J.L., MacKinnon, D.P.: Multilevel modeling of individual and group level mediated effects. Multivar. Behav. Res. 36, 249–277 (2001)
Linden, A., Adler-Milstein, J.: Medicare disease management in a policy context. Health Care Financ. Rev. 29, 1–11 (2008)
Linden, A., Adams, J.L.: Using propensity score-based weighting in the evaluation of health management programme effectiveness. J. Eval. Clin. Pract. 16, 175–179 (2010a)
Linden, A., Adams, J.L.: Evaluating health management programmes over time: application of propensity score-based weighting to longitudinal data. J. Eval. Clin. Pract. 16, 180–185 (2010b)
Linden, A., Roberts, N.: Disease management interventions: what’s in the black box? Dis. Manag. 7, 275–291 (2004)
Linden, A., Butterworth, S., Roberts, N.: Disease management interventions II: what else is in the black box? Dis. Manag. 9, 73–85 (2006)
Long, J.S.: Regression models for categorical and limited dependent variables. Sage, Thousand Oaks (1997)
Lorig, K.R., Holman, H.: Self-management education: history, definition, outcomes, and mechanisms. Ann. Behav. Med. 26, 1–7 (2003)
Lu, B., Zanutto, E., Hornik, R., Rosenbaum, P.R.: Matching with doses in an observational study of a media campaign against drug abuse. J. Am. Stat. Assoc. 96, 1245–1253 (2001)
MacKinnon, D.P.: Introduction to Statistical Mediation Analysis. Erlbaum, Mahwah, NJ (2008)
MacKinnon, D.P., Dwyer, J.H.: Estimation of mediated effects in prevention studies. Eval. Rev. 17, 144–158 (1993)
MacKinnon, D.P., Warsi, G., Dwyer, J.H.: A simulation study of mediated effect measures. Multivar. Behav. Res. 30, 41–62 (1995)
MacKinnon, D.P., Lockwood, C.M., Brown, C.H., Wang, W., Hoffman, J.M.: The intermediate endpoint effect in logistic and probit regression. Clin. Trials 4, 499–513 (2007)
MacKinnon, D.P., Lockwood, C.M., Hoffman, J.M., West, S.G., Sheets, V.: A comparison of methods to test mediation and other intervening variable effects. Psychol. Methods 7, 83–104 (2002)
Manski, C.F.: Identification of treatment response with social interactions. Econ. J. (Forthcoming)
Marks, R., Allegrante, J.P., Lorig, K.L.: A review and synthesis of research evidence for self-efficacy-enhancing interventions for reducing chronic disability: implications for health education practice (Part I). Health Promot. Pract. 6, 37–43 (2005)
Matheson, D., Wilkins, A., Psacharopoulos, D.: Realizing the promise of disease management: payer trends and opportunities in the United States. Boston Consulting Group, Boston (2006)
Mathieu, J.E., Taylor, S.R.: A framework for testing meso-mediational relationships in organizational behavior. J. Organ. Behav. 28, 141–172 (2007)
Mattke, S., Seid, M., Ma, S.: Evidence for the effect of disease management: is $1 billion a year a good investment? Am. J. Manag. Care 13, 670–676 (2007)
Maxwell, S.E., Cole, D.A.: Bias in cross-sectional analyses of longitudinal mediation. Psychol. Methods 12, 23–44 (2007)
Maxwell, S.E., Cole, D.A., Mitchell, M.A.: Bias in cross-sectional analyses of longitudinal mediation: partial and complete mediation under an autoregressive model. Multivar. Behav. Res. 46, 816–841 (2011)
Mays, G.P., Au, M., Claxton, G.: Convergence and dissonance: evolution in private-sector approaches to disease management and care coordination. Health Aff. 26, 1683–1691 (2007)
McKelvey, R.D., Zavoina, W.: A statistical model for the analysis of ordinal level dependent variables. J. Math. Sociol. 4, 103–120 (1975)
Miller, W.R., Rose, G.S.: Toward a theory of motivational interviewing. Am. Psychol. 64, 527–537 (2009)
Mirowsky, J., Ross, C.E.: Eliminating defense and agreement bias from measures of the sense of control: a 2 × 2 index. Soc. Psychol. Q. 54, 127–145 (1991)
Morgan, S.L., Todd, J.J.: A diagnostic routine for the detection of consequential heterogeneity of causal effects. Sociol. Methodol. 38, 231–281 (2008)
Nelson, L.: Lessons from medicare’s demonstration projects on disease management and care coordination. Congressional Budget Office Working Paper 2012-01. http://www.cbo.gov/ftpdocs/126xx/doc12664/WP2012-01_Nelson_Medicare_DMCC_Demonstrations.pdf.(2012). Accessed 11 Feb 2012
Ofman, J.J., Badamgarav, E., Henning, J.M., Knight, K., Gano Jr, A.D., Levan, R.K., Gur-Arie, S., Richards, M.S., Hasselblad, V., Weingarten, S.R.: Does disease management improve clinical and economic outcomes in patients with chronic diseases? A systematic review. Am. J. Med. 117, 182–192 (2004)
Pearl, J.: Direct and indirect effects. In: Proceedings of the Seventeenth Conference on Uncertainty and Artificial Intelligence. pp. 411–420. Morgan Kaufmann, San Francisco (2001)
Pearl, J.: The mediation formula: a guide to the assessment of causal pathways in non-linear models. Technical report R-363, University of California, Los Angeles (2011)
Pearl, J.: The causal foundations of structural equation modeling. In: Hoyle, R.H. (ed.) Handbook of Structural Equation Modeling, pp. 68–91. Guilford Press, New York (2012)
Peterson, M.L., Sinisi, S.E., van der Laan, M.J.: Estimation of direct causal effects. Epidemiology 17, 276–284 (2006)
Robins, J.M.: Marginal structural models. In: 1997 Proceedings of the Section on Bayesian Statistical Science, pp. 1–10. American Statistical Association, Alexandria (1998)
Robins, J.M., Greenland, S.: Identifiability and exchangeability for direct and indirect effects. Epidemiology 3, 143–155 (1992)
Robins, J.M., Hernan, M.A., Brumback, B.: Marginal structural models and causal inference in epidemiology. Epidemiology 11, 550–560 (2000)
Rosenbaum, P.R., Rubin, D.B.: The central role of the propensity score in observational studies for causal effects. Biometrika 70, 41–55 (1983)
Royston, P., Altman, D.G., Sauerbrei, W.: Dichotomizing continuous predictors in multiple regression: a bad idea. Stat. Med. 25, 127–141 (2006)
Rubin, D.B.: Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66, 688–701 (1974)
Rubin, D.B.: Bayesian inference for causal effects: the role of randomization. Ann Stat 6, 34–58 (1978)
Shrout, P., Bolger, N.: Mediation in experimental and nonexperimental studies: new procedures and recommendations. Psychol. Methods 7, 422–445 (2002)
Selig, J.P., Preacher, K.J.: Mediation models for longitudinal data in developmental research. Res. Hum. Dev. 6, 144–164 (2009)
Sobel, M.E.: Asymptotic confidence intervals for indirect effects in structural equation models. In: Leinhardt, S. (ed.) Sociological Methodology, pp. 290–312. American Sociological Association, Washington, DC (1982)
Sobel, M.E.: Identification of causal parameters in randomized studies with mediating variables. J. Educ. Behav. Stat. 33, 230–251 (2008)
Sousa, V.D., Zauszniewski, J.A., Bergquist-Beringer, S., Musil, C.M., Neese, J.B., Jaber, A.F.: Reliability, validity and factor structure of the Appraisal of Self-Care Agency Scale—Revised (ASAS-R). J. Eval. Clin. Pract. 16, 1031–1040 (2010)
Stolzenberg, R.M.: The measurement and decomposition of causal effects in nonlinear and nonadditive models. Sociol. Methodol. 11, 459–488 (1980)
VanderWeele, T.J.: Marginal structural models for the estimation of direct and indirect effects. Epidemiology 20, 18–26 (2009)
VanderWeele, T.J.: Bias formulas for sensitivity analysis for direct and indirect effects. Epidemiology 21, 540–551 (2010)
Winship, C., Mare, R.D.: Structural equations and path analysis for discrete data. Am. J. Sociol. 89, 54–110 (1983)
Winship, C., Mare, R.D.: Regression models with ordinal variables. Am. Sociol. Rev. 49, 512–525 (1984)
Wooldridge, J.M.: Econometric analysis of cross section and panel data. MIT Press, Cambridge (2002)
Zanutto, E., Lu, B., Hornik, R.: Using propensity score subclassification for multiple treatment doses to evaluate a national antidrug media campaign. J. Educ. Behav. Stat. 30, 59–73 (2005)
Zhang, Z., Zyphur, M.J., Preacher, K.J.: Testing multilevel mediation using hierarchical linear models: problems and solutions. Organ. Res. Methods 12, 695–719 (2009)
Acknowledgments
We thank Dustin Tingley, Raymond Hicks, Danella Hafeman, and Adam Glynn for clarifications of the modeling approaches used in their respective papers, to John Antonakis for evocative discussions pertaining to concerns of endogeneity in mediation analysis, and to Julia Adler-Milstein for her invaluable review and edits. We are indebted to the editor and two anonymous reviewers for providing excellent comments which substantially improved the manuscript.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Linden, A., Karlson, K.B. Using mediation analysis to identify causal mechanisms in disease management interventions. Health Serv Outcomes Res Method 13, 86–108 (2013). https://doi.org/10.1007/s10742-013-0106-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10742-013-0106-5