Abstract
Although previous research has studied power in mediation models, the extent to which the inclusion of a mediator will increase power has not been investigated. To address this deficit, in a first study we compared the analytical power values of the mediated effect and the total effect in a single-mediator model, to identify the situations in which the inclusion of one mediator increased statistical power. The results from this first study indicated that including a mediator increased statistical power in small samples with large coefficients and in large samples with small coefficients, and when coefficients were nonzero and equal across models. Next, we identified conditions under which power was greater for the test of the total mediated effect than for the test of the total effect in the parallel two-mediator model. These results indicated that including two mediators increased power in small samples with large coefficients and in large samples with small coefficients, the same pattern of results that had been found in the first study. Finally, we assessed the analytical power for a sequential (three-path) two-mediator model and compared the power to detect the three-path mediated effect to the power to detect both the test of the total effect and the test of the mediated effect for the single-mediator model. The results indicated that the three-path mediated effect had more power than the mediated effect from the single-mediator model and the test of the total effect. Practical implications of these results for researchers are then discussed.
Similar content being viewed by others
References
Albert, J. M., & Nelson, S. (2011). Generalized causal mediation analysis. Biometrics, 67, 1028–1038.
Aroian, L. A. (1944). The probability function of the product of two normally distributed variables. Annals of Mathematical Statistics, 18, 265–271.
Avin, C., Shpitser, I., & Pearl, J. (2005). Identifiability of path-specific effects. In Proceedings of the international joint conference on artificial intelligence. San Francisco, CA: Morgan Kaufman.
Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182.
Cochran, W. G. (1957). Analysis of covariance: Its nature and uses. Biometrics, 13, 261–281.
Cochran, W. G. (1965). The planning of observational studies of human populations (with discussion). Journal of the Royal Statistical Society, Series A, 128, 234–265.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Chicago, IL: Rand McNally.
Cox, D. R. (1960). Regression analysis when there is prior information about supplementary variables. Journal of the Royal Statistical Society: Series B, 22, 172–176.
Cox, D. R. (1992). Causality: Some statistical aspects. Journal of the Royal Statistical Society, 155, 291–301.
Craig, C. C. (1936). On the frequency function of xy. Annals of Mathematical Statistics, 7, 1–15.
Fairchild, A. J., MacKinnon, D. P., Taborga, M. P., & Taylor, A. B. (2009). R 2 effect-size measures for mediation analysis. Behavior Research Methods, 41, 486–498.
Fishbein, M., & Ajzen, I. (1975). Belief attitude, intention and behavior. Reading, MA: Addison-Wesley.
Freedman, L. S., & Schatzkin, A. (1992). Sample size for studying intermediate endpoints within intervention trials or observational studies. American Journal of Epidemiology, 136, 1148–1159.
Fritz, M. S., & MacKinnon, D. P. (2007). Required sample size to detect the mediated effect. Psychological Science, 18, 233–239. doi:10.1111/j.1467-9280.2007.01882.x
Fritz, M. S., Cox, M. G., & MacKinnon, D. P. (2014). Increasing statistical power in mediation models without increasing sample size. Evaluation & the Health Professions. doi:10.1177/0163278713514250
Hill, A. B. (1965). The environment and disease: Association or causation? Proceedings of the Royal Society of Medicine, 58, 295–300.
Hoyle, R. H., & Kenny, D. A. (1999). Sample size, reliability, and tests of statistical mediation. In R. H. Hoyle (Ed.), Statistical strategies for small sample research (pp. 195–222). Thousand Oaks, CA: Sage.
Huck, S. W. (1972). The analysis of covariance: Increased power through reduced variability. Journal of Experimental Education, 41, 42–46.
Imai, K., Keele, L., & Tingley, D. (2010). A general approach to causal mediation analysis. Psychological Methods, 15, 309–334.
Imai, K., & Yamamoto, T. (2013). Identification and sensitivity analysis for multiple causal mechanisms: Revisiting evidence from framing experiments. Political Analysis, 21, 1–31.
Judd, C. M., & Kenny, D. A. (1981). Process analysis: Estimating mediation in treatment evaluations. Evaluation Review, 5, 602–619.
Kenny, D. A., & Judd, C. M. (2014). Power anomalies in testing mediation. Psychological Science, 25, 334–339. doi:10.1177/0956797613502676
Kraemer, H. C., & Thiemann, S. (1989). A strategy to use soft data effectively in randomized controlled clinical trials. Journal of Consulting and Clinical Psychology, 57, 148–154.
Lange, T., Rasmussen, M., & Thygesen, L. C. (2014). Assessing natural direct and indirect effects through multiple pathways. American Journal of Epidemiology, 179, 513–518. doi:10.1093/aje/kwt270
MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. New York, NY: Erlbaum.
MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods, 7, 83–104.
MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research, 39, 99–128.
Maxwell, S. E. (1998). Longitudinal designs in randomized group comparisons: When will intermediate observations increase statistical power? Psychological Methods, 3, 275–290.
McDonald, R. P. (1997). Haldane’s lungs: A case study in path analysis. Multivariate Behavioral Research, 32, 1–38.
Miller, G. A., & Chapman, J. P. (2001). Misunderstanding analysis of covariance. Journal of Abnormal Psychology, 110, 40–48.
Neyman, J., & Pearson, E. S. (1933). The testing of statistical hypotheses in relation to probabilities a priori. Proceedings of the Cambridge Philosophical Society, 24, 492–510.
Prentice, R. L. (1989). Surrogate endpoints in clinical trials: Definition and operational criteria. Statistics in Medicine, 8, 431–440.
Robins, J. M. (2003). Semantics of causal DAG models and the identification of direct and indirect effects. In P. J. Green, N. L. Hjort, & S. Richardson (Eds.), Highly structured stochastic systems (pp. 70–81). Oxford, UK: Oxford University Press.
Rosenbaum, P. R. (2010). Design of observational studies. New York, NY: Springer.
Salthouse, T. A. (1984). Effects of age and skill on typing. Journal of Experimental Psychology: General, 113, 345–371.
Shrout, P. E., & Bolger, N. (2002). Mediation in experimental and nonexperimental studies: New procedures and recommendations. Psychological Methods, 7, 422–445.
Sobel, M. E. (1982). Asymptotic confidence intervals for indirect effects in structural equation models. Sociological Methodology, 13, 290–312.
Taylor, A. B., MacKinnon, D. P., & Tein, J.-Y. (2008). Tests of the three-path mediated effect. Organizational Research Methods, 11, 241–269.
Tekleab, A. G., Bartol, K. M., & Liu, W. (2005). Is it pay levels or pay raises that matter to fairness and turnover? Journal of Organizational Behavior, 26, 899–921.
Thoemmes, F., MacKinnon, D. P., & Reiser, M. R. (2010). Power analysis for complex mediational designs using Monte Carlo methods. Structural Equation Modeling, 17, 510–534.
Tofighi, D., MacKinnon, D. P., & Yoon, M. (2009). Covariances between regression coefficient estimates in a single mediator model. British Journal of Mathematical and Statistical Psychology, 62, 457–484.
VanderWeele, T. J., & Vansteelandt, S. (2009). Conceptual issues concerning mediation, interventions and composition. Statistics and Its Interface, 2, 457–468.
Venter, A., Maxwell, S. E., & Bolig, E. (2002). Power in randomized group comparisons: The value of adding a single intermediate time point to a traditional pretest-posttest design. Psychological Methods, 7, 194–209.
Wang, C., & Xue, X. (2012). Power and sample size calculations for evaluating mediation effects in longitudinal studies. Statistical Methods in Medical Research. doi:10.1177/0962280212465163. Advance online publication.
Wright, S. (1921). Correlation and causation. Journal of Agricultural Research, 20, 557–585.
Wright, S. (1934). The method of path coefficients. Annals of Mathematical Statistics, 5, 161–215.
Yerushalmy, J., & Palmer, C. E. (1959). On the methodology of investigations of etiologic factors in chronic diseases. Journal of Chronic Diseases, 10, 27–40.
Acknowledgments
This research was supported in part by Public Health Service Grant No. DA09757. We thank our colleagues for asking about methods to increase power when sample size is small and fixed, as it is in many research areas.
Author information
Authors and Affiliations
Corresponding author
Appendixes
Appendixes
Appendix 1
For the single-mediator model described in Eqs. 2 and 3, and assuming uncorrelated errors across equations, normally distributed variables, and a linear system of relations (McDonald, 1997), the true covariances among X, M, and Y for the single-mediator model can be used to obtain expected values of the regression coefficients and standard errors of those coefficients, to compute predicted power for the test of a, b, and c at any combination of effect size and sample size (MacKinnon, 2008). The expected \( {\widehat{a}}_T \) coefficient is given in Eq. A.1. The subscript T is used to indicate that this value is the true or theoretical value, given the covariance matrix and sample size.
The true or theoretical (true is again represented by the T in the subscript) variance of the estimator \( \widehat{a} \), \( {\sigma}_{\widehat{a}T}^2 \) (standard error is the square root of the variance), is equal to
In Eq. A.2, σ 2 e3 is the true error variance from Eq. 3, which predicts M from X. An estimator of the true error variance represented in two ways is shown in Eqs. A.3 and A.4.
In Eq. A.3, p is the number of X variables (which is equal to 1 here, for the one-predictor case), and ∑ e 2 is the sum of the squared differences between the predicted and observed M scores. Equation A.4 involves the variances of X and M and the \( \widehat{a} \) coefficient. Similar formulas are used to determine the expected values of \( \widehat{b} \) and \( \widehat{c}^{\prime } \), as is shown in Eqs. A.5 and A.6.
Equations A.7 and A.8 show the true variances of \( {\widehat{b}}_T \) and \( {\widehat{c}}_T^{\prime } \) respectively, where N is the number of participants and \( {\widehat{\sigma}}_{e2T}^2 \) is the expected true error variance from Eq. 2. Equation A.9 shows an estimator of the true error variance for Eq. 2, predicting Y from X and M, and is analogous to the error variance formula in Eq. A.4, but with p = 2 predictors (X and M).
The covariance matrix in Table 5 produces analogous formulas to determine the expected values of \( {\widehat{c}}_T \), \( {\sigma}_{\widehat{c}T}^2 \), and \( {\widehat{\sigma}}_{e1T}^2 \) from Eq. 1, shown in Eqs. A.10, A.11, and A.12, respectively.
Statistical power to detect the mediated effect
Statistical power is determined using these expected values, as is shown in Eqs. A.13, A.14, and A.15, where ϕ t is the probability from the cumulative t distribution. The statistical power of the joint significance test for mediation consists of the power to detect a nonzero a path and the power to detect a nonzero b path. If both paths are statistically significant, the hypothesis of no mediated effect is rejected (MacKinnon et al., 2002).
So, the case in which the power to detect the mediated effect is greater than the power to detect the total effect is given by the following inequality:
All power values were computed analytically with an SAS program available from the authors. These analytical power values were checked empirically in a statistical simulation study for the sample sizes and parameter values described in this article. Note that when both a and b are zero, for the joint significance test of a and b, the Type I error rate is .0025.
Appendix 2: Effect size formulas for the parallel two-mediator model
For a 1:
For a 2:
For b 1:
For b 2:
For c′:
Rights and permissions
About this article
Cite this article
O’Rourke, H.P., MacKinnon, D.P. When the test of mediation is more powerful than the test of the total effect. Behav Res 47, 424–442 (2015). https://doi.org/10.3758/s13428-014-0481-z
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13428-014-0481-z