Using response ratios for meta-analyzing single-case designs with behavioral outcomes☆
Section snippets
Log response ratios
The LRR effect size is defined based on a simple model for the data from a baseline phase and an intervention phase within a single-case design. Suppose that the baseline phase includes m sessions, with outcome data Y1A, …YmA, and that the intervention phase includes n sessions, with outcome data Y1B, …, YnB. Let us assume that the average level of the outcome is constant within each phase (i.e., lacking any systematic time trend). Let μA denote the mean level of the outcome during the baseline
Preparing LRR estimates for use in meta-analysis
When considering use of LRR effect sizes for synthesizing multiple SCD studies, researchers must address several further issues before carrying out effect size calculations and meta-analysis. This section describes three issues and methods for addressing each, including (1) how to determine whether the LRR is an appropriate effect size metric, (2) how to transform the effect sizes so that their signs (positive or negative) are consistent with the direction of therapeutic improvement for the
Meta-analysis with robust variance estimation
Meta-analysis is a set of statistical techniques for synthesizing results across studies in order to draw generalizations about overall patterns of findings (Borenstein et al., 2009). Meta-analysis can be used to address questions about the overall average magnitude of effects, the degree of consistency or inconsistency (heterogeneity) of results across studies, and characteristics of participants or studies that moderate the magnitude of effect sizes.
In synthesis of between-groups research
Discussion
In this paper, I have demonstrated the use of a recently proposed effect size index, the log response ratio, for meta-analysis of SCDs with behavioral outcome measures. Compared to meta-analysis based on other effect size indices, the proposed methods are distinctive in several respects.
First, development of the LRR was motivated by a realistic model for systematic direct observation procedures (Pustejovsky, 2015), and the index is thus designed to work well with behavioral outcomes. Other
Acknowledgements
This work was supported by Grant R305D160002 from the Institute of Educational Sciences, U.S. Department of Education. The opinions expressed are those of the author and do not represent the views of the Institute or the U.S. Department of Education. The author is grateful to Daniel Maggin, David Rindskopf, Tsuyoshi Yamada, and Kathleen Zimmerman for feedback on draft versions of this paper.
References (60)
- et al.
Efficacy of behavioral interventions for reducing problem behavior in persons with autism: An updated quantitative synthesis of single-subject research
Research in Developmental Disabilities
(2014) - et al.
A generalized least squares regression approach for computing effect sizes in single-case research: Application examples
Journal of School Psychology
(2011) - et al.
Recommendations for choosing single-case data analytical techniques
Behavior Therapy
(2017) - et al.
An improved effect size for single-case research: Nonoverlap of all pairs
Behavior Therapy
(2009) Analysis and meta-analysis of single-case designs: An introduction
Journal of School Psychology
(2014)- et al.
Analysis and meta-analysis of single-case designs with a standardized mean difference statistic: A primer and applications
Journal of School Psychology
(2014) - et al.
Dependent measures and measurement procedures
- et al.
Introduction to meta-analysis
(2009) - et al.
Meta-analysis for single-case research
- et al.
Clinical outcomes of behavioral treatments for pica in children with developmental disabilities
Journal of Autism and Developmental Disorders
(2015)
Statistics and single subject research methodology
Functional assessment-based interventions for students with or at-risk for high-incidence disabilities: Field testing single-case synthesis methods
Remedial and Special Education
Council for Exceptional Children: Standards for evidence-based practices in special education
Teaching Exceptional Children
XYit
Meta-analysis of applied time-series data
The Journal of Applied Behavioral Science
A meta-analytic review of functional communication training across mode of communication, age, and disability
Review Journal of Autism and Developmental Disorders
What are effect sizes and why do we need them?
Child Development Perspectives
The meta-analysis of response ratios in experimental ecology
Ecology
Robust variance estimation in meta-regression with dependent effect size estimates
Research Synthesis Methods
What Works Clearinghouse standards and generalization of single-case design evidence
Journal of Behavioral Education
Synthesizing single-case research to identify evidence-based practices: Some brief reflections
Journal of Behavioral Education
Considerations for the systematic analysis and use of single-case research
Education and Treatment of Children
Irrelevant autocorrelation in least-squares intervention models
Psychological Methods
Identifying autocorrelation generated by various error processes in interrupted time-series regression designs: A comparison of AR1 and portmanteau tests
Educational and Psychological Measurement
Behavioral treatment of self-injury, 1964 to 2000
American Journal of Mental Retardation: AJMR
Single-case intervention research design standards
Remedial and Special Education
Visual analysis of single-case intervention research: Conceptual and methodological issues
An examination of the evidence base for function-based interventions for students with emotional and/or behavioral disorders attending middle and high schools
Exceptional Children
Antecedent social skills interventions for individuals with ASD: What works, for whom, and under what conditions?
Focus on Autism and Other Developmental Disabilities
SAS system for linear mixed models
Cited by (113)
Quantifying potential trade-offs and win-wins between arthropod diversity and yield on cropland under agri-environment schemes–A meta-analysis
2024, Journal of Environmental ManagementA global assessment of the long-term effects of biochar application on crop yield
2024, Current Research in Environmental SustainabilityAssessing differential land use impacts on soil quality: A method based on log-response ratios and polygonal projections
2023, Journal of Environmental ManagementA Meta-Analysis of Mathematics Interventions: Examining the Impacts of Intervention Characteristics
2024, Educational Psychology ReviewThe Engaged Learners Program: Effects on Student Engagement During Small Group Reading Instruction
2024, Behavior Modification
- ☆
An earlier version of this paper was presented at the annual convention of the American Educational Research Association, April 28, 2017 in San Antonio, Texas. Supplementary materials are available at https://osf.io/c3fe9/.