Abstract
This review of reviews presents an empirically based set of mean effect size distributions for judging the relative impact of the effects of universal mental health promotion and prevention programs for school-age youth (ages 5 through 18) across a range of program targets and types of outcomes. Mean effect size distributions were established by examining the findings from 74 meta-analyses of universal prevention and promotion programs that included more than 1100 controlled outcome studies involving over 490,000 school-age youth. The distributions of mean effect sizes from these meta-analyses indicated considerable variability across program targets and outcomes that differed substantially from Cohen’s (1988, Statistical power analysis for the behavioral sciences (2nd ed.)) widely used set of conventions for assessing if effects are small, medium, or large. These updated mean effect size distributions will provide researchers, practitioners, and funders with more appropriate evidence-based standards for judging the relative effects of universal prevention programs for youth. Limitations in current data and directions for future work are also discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Namely, a meta-analysis might have synthesized findings from universal prevention programs reporting substance use outcomes, regardless of the primary target of the intervention. In this instance, results from a delinquency prevention trial reporting substance use outcomes might have been included in the meta-analysis. In our coding, this meta-analysis would have been categorized as a “substance use program,” unless the meta-analysis authors reported additional subgroup results by actual program target.
We conducted sensitivity analyses (shown in the online Appendix) to assess how the inclusion of dependent mean effect sizes within-meta-analyses influenced the results. Those sensitivity analyses included one mean effect size estimate from each meta-analysis, where we selected the mean effect size based on the largest number of primary studies. Results from those sensitivity analyses are substantively similar to those reported in Table 2.
As shown in the follow-up mean effect size distributions in Appendix B, there was consistent evidence that follow-up mean effects were smaller than immediate posttest mean effects. These findings should be interpreted cautiously, however, given that only 25 of the 74 included meta-analyses reported follow-up effect sizes.
References
Bloom, H. S., Hill, C. J., Black, A. R., & Lipsey, M. W. (2008). Performance trajectories and performance gaps as achievement effect-size benchmarks for educational interventions. Journal of Research on Educational Effectiveness, 1, 289–328. https://doi.org/10.1080/19345740802400072.
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. West Sussex: Wiley. https://doi.org/10.1002/9780470743386.
Bradshaw, C. P., Waasdorp, T. E., & Leaf, P. J. (2012). Effects of school-wide positive behavioral interventions and supports on child behavior problems. Pediatrics, 130, e1136–e1145. https://doi.org/10.1542/peds.2012-0243.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
Cooper, H., & Koenka, A. C. (2012). The overview of reviews: Unique challenges and opportunities when research syntheses are the principal elements of new integrative scholarship. American Psychologist, 67, 446–462. https://doi.org/10.1037/a0027119.
Cox, D. R., & Snell, E. J. (1989). Analysis of binary data (2nd ed.). New York: Chapman & Hall/CRC.
Durlak, J. A. (1995). School-based prevention programs for children and adolescents. Thousand Oaks: Sage. https://doi.org/10.4135/9781483327396.
Durlak, J. A. (1997). Effective prevention programs for children and adolescents. New York: Plenum. https://doi.org/10.1007/978-1-4899-0065-4.
Durlak, J. A. (2009). How to select, calculate, and interpret effect sizes. Journal of Pediatric Psychology, 34, 917–928. https://doi.org/10.1093/jpepsy/jsp004.
Furr-Holden, C. D., Ialongo, N. S., Anthony, J. C., Petras, H., & Kellam, S. G. (2004). Developmentally inspired drug prevention: Middle school outcomes in a school-based randomized prevention trial. Drug and Alcohol Dependence, 73, 149–158. https://doi.org/10.1016/j.drugalcdep.2003.10.002.
Grucza, R. A., Plunk, A. D., Hipp, P. R., Cavazos-Rehg, P., Krauss, M. J., Brownson, R. C., & Bierut, L. J. (2013). Long-term effects of laws governing youth access to tobacco. American journal of Public Health, 103(8), 1493–1499.
Harris, D. N. (2009). Toward policy-relevant benchmarks for interpreting effect sizes: Combining effects with costs. Educational Evaluation and Policy Analysis, 31, 3–29. https://doi.org/10.3102/0162373708327524.
Hill, C. J., Bloom, H. S., Black, A. R., & Lipsey, M. W. (2007). Empirical benchmarks for interpreting effect sizes in research. Child Development Perspectives, 2, 172–177. https://doi.org/10.1111/j.1750-8606.2008.00061.x.
Kumpfer, K. L., & Alvarado, R. (2003). Family-strengthening approaches for the prevention of youth problem behaviors. American Psychologist, 58, 457. https://doi.org/10.1037/0003-066X.58.6-7.457.
Lippke, S., Nigg, C. R., & Maddock, J. E. (2012). Health-promoting and health-risk behaviors: Theory-driven analyses of multiple health behavior change in three international samples. International Journal of Behavioral Medicine, 19, 1–13. https://doi.org/10.1007/s12529-010-9135-4.
Lipsey, M. W., & Cullen, F. T. (2007). The effectiveness of correctional rehabilitation: A review of systematic reviews. Annual Review of Law and Social Science, 3, 297–320. https://doi.org/10.1146/annurev.lawsocsci.3.081806.112833.
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Prisma Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6, e1000097. https://doi.org/10.1371/journal.pmed.1000097.
Peters, L. W., Kok, G., Ten Dam, G. T., Buijs, G. J., & Paulussen, T. G. (2009). Effective elements of school health promotion across behavioral domains: A systematic review of reviews. BMC Public Health, 9, 182. https://doi.org/10.1186/1471-2458-9-182.
Plonsky, L., & Oswald, F. L. (2014). How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64, 878–912. https://doi.org/10.1111/lang.12079.
Sandler, I., Wolchik, S. A., Cruden, G., Mahrer, N. E., Ahn, S., Brincks, A., & Brown, C. H. (2014). Overview of meta-analyses of the prevention of mental health, substance use and conduct problems. Annual Review of Clinical Psychology, 10, 243. https://doi.org/10.1146/annurev-clinpsy-050212-185524.
Substance Abuse & Mental Health Data Archive (SAMHDA). (2016). National Survey on Drug Use and Health 2015 (NSDUH-2016-DS0001). Retrieved from https://www.datafiles.samhsa.gov/study-dataset/national-survey-drug-use-and-health-2015-nsduh-2015-ds0001-nid16894.
Wagenaar, A. C., Salois, M. J., & Komro, K. A. (2009). Effects of beverage alcohol price and tax levels on drinking: A meta-analysis of 1003 estimates from 112 studies. Addiction, 104, 179–190. https://doi.org/10.1111/j.1360-0443.2008.02438.x.
Wiefferink, C. H., Peters, L., Hoekstra, F., Ten Dam, G., Buijs, G. J., & Paulussen, T. G. (2006). Clustering of health-related behaviors and their determinants: Possible consequences for school health interventions. Prevention Science, 7, 127–149. https://doi.org/10.1007/s11121-005-0021-2.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no conflicts of interest. JAD acknowledges that he is an author of several meta-analyses included in this review, but he was not involved in extracting any data from those meta-analyses for the purposes of this review.
Ethical Approval
This article does not contain any studies with human participants performed by any of the authors.
Informed Consent
For this type of study, formal consent is not required.
Electronic Supplementary Material
ESM 1
(DOCX 275 kb)
Rights and permissions
About this article
Cite this article
Tanner-Smith, E.E., Durlak, J.A. & Marx, R.A. Empirically Based Mean Effect Size Distributions for Universal Prevention Programs Targeting School-Aged Youth: A Review of Meta-Analyses. Prev Sci 19, 1091–1101 (2018). https://doi.org/10.1007/s11121-018-0942-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11121-018-0942-1