Quality is Critical for Meaningful Synthesis of Afterschool Program Effects: A Systematic Review and Meta-analysis

Afterschool programs are designed to promote academic achievement and youth development as well as mitigate risk for students who attend. Systematic reviews have reported that afterschool programs are associated with increased academic proficiency, school bonding, prosocial behavior, and decreased problem behavior. However, the majority of meta-analyses that report on these outcomes limit their samples to only rigorously conducted studies, which is not representative of the larger literature base. This systematic review and meta-analysis is designed to extend the literature by meta-analyzing a comprehensive sample of studies and examine the impact of study quality on meta-analytical results. A comprehensive literature and effect size extraction process search yielded 130 effect sizes from 30 studies that examined the effects of afterschool programs on secondary students’ academic achievement and social / behavioral development. The majority of studies were unpublished reports, and study quality in the sample represented a high risk of bias. The results confirm the hypothesis that the inclusion of lower quality studies significantly impacts overall outcomes in comparison to prior meta-analyses that have limited samples. As such, it is imperative that researchers and practitioners increase the rigor of study design across individual program evaluations so that fewer reports will be excluded. This will improve future meta-analytic outcomes by ensuring a more representative sample of the literature base. Only through improved rigor of study design will a more clear understanding of program impacts on secondary students be possible.