Abstract
In this article, the calculation of effect size measures in single-case research and the use of hierarchical linear models for combining these measures are discussed. Special attention is given to meta-analyses that take into account a possible linear trend in the data. We show that effect size measures that have been proposed for this situation appear to be systematically affected by the duration of the experiment and fail to distinguish between effects on level and slope. To avoid these flaws, we propose to perform a multivariate meta-analysis on the standardized ordinary least squares regression coefficients from the study-specific regression equations describing the response variable.
Article PDF
Similar content being viewed by others
References
Allison, D. B., &Gorman, B. S. (1993). Calculating effect sizes for meta-analysis: The case of the single case.Behaviour Research & Therapy,31, 621–631.
Box, G. E. P., &Jenkins, G. M. (1976).Time series analysis, forecasting and control. San Francisco: Holden-Day.
Bryk, A. S., &Raudenbush, S. W. (1992).Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage.
Bryk, A. S., Raudenbush, S. W., Congdon, R. T., &Seltzer, M. (1988).An introduction to HLM: User’s guide, version 2.0 [Computer program]. Chicago: University of Chicago Press.
Busk, P. L., &Serlin, R. C. (1992). Meta-analysis for single-case research. In T. R. Kratochwill & J. R. Levin (Eds.),Single-case research design and analysis: New directions for psychology and education (pp. 187–212). Hillsdale, NJ: Erlbaum.
Busse, R. T., Kratochwill, T. R., &Elliott, S. N. (1995). Metaanalysis for single-case consultation outcomes: Applications to research and practice.Journal of School Psychology,33, 269–285.
Center, B. A., Skiba, R. J., &Casey, A. (1985–1986). A methodology for the quantitative synthesis of intra-subject design research.Journal of Special Education,19, 387–400.
Cohen, J. (1969).Statistical power analysis for the behavioral sciences. New York: Academic Press.
Faith, M. S., Allison, D. B., &Gorman, B. S. (1997). Meta-analysis of single-case research. In R. D. Franklin, D. B. Allison, & B. S. Gorman (Eds.),Design and analysis of single-case research (pp. 245–277). Mahwah, NJ: Erlbaum.
Gingerich, W. J. (1984). Meta-analysis of applied time-series data.Journal of Applied Behavioral Science,20, 71–79.
Goldstein, H., Rasbash, J., Plewis, I., Draper, D., Browne, W., Yang, M., Woodhouse, G., &Healy, M. (1998).A user’s guide to MLwiN. London: University of London, Multilevel Models Project.
Gorman, B. S., &Allison, D. B. (1997). Statistical alternatives for single-case designs. In R. D. Franklin, D. B. Allison, & B. S. Gorman (Eds.),Design and analysis of single-case research (pp. 159–214). Mahwah, NJ: Erlbaum.
Gottman, J. M. (1981).Time-series analysis: A comprehensive introduction for social scientists. Cambridge: Cambridge University Press.
Hedges, L. V. (1981). Distribution theory for Glass’s estimator of effect size and related estimators.Journal of Educational Statistics,6, 107–128.
Hedges, L. V., &Olkin, I. (1985).Statistical methods for meta-analysis. Orlando, FL: Academic Press.
Hershberger, S. L., Wallace, D. D., Green, S. B., &Marquis, J. G. (1999). Meta-analysis of single-case designs. In R. H. Hoyle (Ed.),Statistical strategies for small sample research (pp. 107–132). London: Sage.
Huitema, B. E. (1985). Autocorrelation in behavioral research: A myth.Behavioral Assessment,1, 107–118.
Huitema, B. E., &McKean, J. W. (2000). Design specification issues in time-series intervention models.Educational & Psychological Measurement,60, 35–58.
Jayaratne, S., Tripodi, T., &Talsma, E. (1988). The comparative analysis and aggregation of single-case data.Journal of Applied Behavioral Science,24, 119–128.
Kalaian, H. A., &Raudenbush, S. W. (1996). A multivariate mixed linear model for meta-analysis.Psychological Methods,1, 227–235.
Kreft, I. G. G., &de Leeuw, J. (1998).Introducing multilevel modelling. London: Sage.
Marlowe, R. H., Madsen, C. H., Jr.,Bowen, C. E., Reardon, R. C., &Logue, P. E. (1978). Severe classroom behavior problems: Teachers or counselors.Journal of Applied Behavioral Analysis,11, 53–66.
Matyas, T. A., &Greenwood, K. M. (1991). Problems in the estimation of autocorrelation in brief time series and some implications for behavioral data.Behavioral Assessment,13, 137–157.
Matyas, T. A., &Greenwood, K. M. (1997). Serial dependency in single-case time series. In R. D. Franklin, D. B. Allison, & B. S. Gorman (Eds.),Design and analysis of single-case research (pp. 215–243). Mahwah, NJ: Erlbaum.
Morris, C. (1983). Parametric empirical Bayes inference, theory and applications.Journal of the American Statistical Association,78, 47–65.
National Research Council (1992).Combining information: Statistical issues and opportunities for research. Washington, DC: National Academy Press.
Onghena, P. (1994).The power of randomization tests for single-case designs. Unpublished doctoral dissertation, Katholieke Universiteit Leuven.
Onghena, P., &Van Damme, G. (1994). SCRT 1.1: Single-case randomization tests.Behavior Research Methods, Instruments, & Computers,26, 369.
Pinheiro, J. C., &Bates, D. M. (2000).Mixed-effects models in S and S-plus. New York: Springer-Verlag.
Rosenthal, R. (1994). Parametric measures of effect size. In H. Cooper & L. V. Hedges (Eds.),The handbook of research synthesis (pp. 231–244). New York: Russell Sage Foundation.
Sheu, C.-F., &Suzuki, S. (2001). Meta-analysis using linear mixed models.Behavior Research Methods, Instruments, & Computers,33, 102–107.
Verbeke, G., &Molenberghs, G. (1997).Linear mixed models in practice: A SAS-oriented approach. New York: Springer-Verlag.
Wang, M. C., &Bushman, B. J. (1999).Integrating results through metaanalytic review using SAS software. Cary, NC: SAS Institute.
White, D. M., Rusch, F. R., Kazdin, A. E., &Hartmann, D. P. (1989). Applications of meta analysis in individual-subject research.Behavioral Assessment,11, 281–296.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Van Den Noortgate, W., Onghena, P. Hierarchical linear models for the quantitative integration of effect sizes in single-case research. Behavior Research Methods, Instruments, & Computers 35, 1–10 (2003). https://doi.org/10.3758/BF03195492
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03195492