Randomization tests for single-case experiments: State of the art, state of the science, and state of the application

https://doi.org/10.1016/j.jcbs.2013.10.002Get rights and content

Highlights

  • A single-case experiment can be used to evaluate the effect of an intervention for a single entity.

  • Schedules to include randomization into the design of single-case experiments are discussed.

  • The use of a free software package is illustrated.

Abstract

A single-case experimental design is a research design that can be used to evaluate the effect of an intervention for a single entity. There are two important schedules to include randomization into the design of single-case experiments: phase designs and alternation designs. We present these two schedules and provide a detailed example for each schedule. For both examples, we illustrate the use of a free software package that assists researchers in designing and analyzing single-case experiments using randomization tests. Furthermore, we discuss several additions (simultaneous and sequential replication designs; meta-analysis of single-case experimental studies) and alternatives (statistical and visual analysis methods).

Section snippets

Single-case experiments

A single-case experiment (SCE) is an experiment that can be used to evaluate the effect of an independent variable for a single entity, for example a single patient, a single therapist–patient dyad, or a single family. An experimental approach is used: the independent variable is manipulated by the experimenter. The dependent variable is measured repeatedly for this entity under different levels of the independent variable. For example, in research on interventions for reducing challenging

Randomization tests for single-case experiments

There are two important schedules to include randomization into the design of SCEs: phase designs and alternation designs. In the former, the moment of phase change is randomly determined. In the latter, the treatment alternation is randomly determined. We now consecutively present these two schedules (2.1 Randomization tests for single-case phase designs, 2.3 Randomization tests for single-case alternation designs), and describe a detailed example for each schedule (2 Randomization tests for

Additions and alternatives

Some important aims of contextual behavioral science (CBS) are understanding the behavior and the promotion of human growth and development. SCE designs can be used by CBS researchers to measure processes of change and behavioral outcomes (cf. Hayes et al., 2012, Vilardaga et al., 2009). As such, CBS researchers can use SCE designs for testing CBS theories and for studying the complexity of human behavior at the level of the individual.

In 3.1 Simultaneous replication designs, 3.2 Meta-analysis

References (111)

  • P. Onghena et al.

    Randomization tests for restricted alternating treatments designs

    Behaviour Research and Therapy

    (1994)
  • R.I. Parker et al.

    Useful effect size interpretations for single case research

    Behavior Therapy

    (2007)
  • T.P. Shakespeare et al.

    Improving interpretation of clinical studies by use of confidence levels, clinical significance curves, and risk-benefit contours

    Lancet

    (2001)
  • D.G. Altman et al.

    The revised CONSORT statement for reporting randomized trials: Explanation and elaboration

    Annals of Internal Medicine

    (2001)
  • American Psychological Association

    Publication manual of the American Psychological Association

    (2010)
  • D.H. Barlow et al.

    Alternating treatments design: One strategy for comparing the effects of two treatments in a single subject

    Journal of Applied Behavior Analysis

    (1979)
  • D.H. Barlow et al.

    The scientist practitioner: research and accountability in clinical and educational settings

    (1985)
  • D.H. Barlow et al.

    Single case experimental designs: Strategies for studying behavior change

    (2009)
  • S.N. Beretvas et al.

    A review of meta-analyses of single-subject experimental designs: Methodological issues and practice

    Evidence-Based Communication Assessment and Intervention

    (2008)
  • N.M. Blampied

    A legacy neglected: Restating the case for single-case research in cognitive-behaviour therapy

    Behaviour Change

    (1999)
  • J.J. Borckardt et al.

    An empirical examination of visual analysis procedures for clinical practice evaluation

    Journal of Social Service Research

    (2004)
  • G.E.P. Box et al.

    Time series analysis: forecasting and control

    (1970)
  • I. Bulté et al.

    An R package for single-case randomization tests

    Behavior Research Methods

    (2008)
  • I. Bulté et al.

    Randomization tests for multiple baseline designs: An extension of the SCRT-R package

    Behavior Research Methods

    (2009)
  • I. Bulté et al.

    When the truth hits you between the eyes. A software tool for the visual analysis of single-case experimental data

    Methodology

    (2012)
  • Bulté, I., Van Den Noortgate, W., & Onghena, P. (May 2010). An R package for the nonparametric meta-analysis of small-n...
  • P.L. Busk et al.

    Statistical analysis in single-case research: Issues, procedures, and recommendations, with applications to multiple behaviors

  • P.L. Busk et al.

    Meta-analysis for single-case research

  • B.A. Center et al.

    A methodology for the quantitative synthesis of intra-subject design research

    Journal of Special Education

    (1985–1986)
  • T.J. Christ

    Experimental control and threats to internal validity of concurrent and nonconcurrent multiple baseline designs

    Psychology in the Schools

    (2007)
  • A. DeProspero et al.

    Inconsistent visual analysis of intrasubject data

    Journal of Applied Behavior Analysis

    (1979)
  • P. Dugard et al.

    Single-case and small-n experimental designs: a practical guide to randomization tests

    (2012)
  • M. Edelson

    Hypothesis and evidence in psychoanalysis

    (1985)
  • E. Edgington et al.

    Randomization tests

    (2007)
  • E.S. Edgington

    Statistical inference from N=1 experiments

    Journal of Psychology

    (1967)
  • E.S. Edgington

    An additive method for combining probability values from independent experiments

    Journal of Psychology

    (1972)
  • E.S. Edgington

    A normal curve method for combining probability values from independent experiments

    Journal of Psychology

    (1972)
  • E.S. Edgington

    Randomization tests for one-subject operant experiments

    Journal of Psychology

    (1975)
  • E.S. Edgington

    Overcoming obstacles to single-subject experimentation

    Journal of Educational Statistics

    (1980)
  • E.S. Edgington

    Randomized single-subject experiments and statistical tests

    Journal of Counseling Psychology

    (1987)
  • J. Ferron et al.

    The functioning of single-case randomization tests with and without random assignment

    Journal of Experimental Education

    (2003)
  • J. Ferron et al.

    Tests for the visual analysis of response-guided multiple-baseline data

    Journal of Experimental Education

    (2006)
  • J. Ferron et al.

    The power of randomization tests for single-case phase designs

    Journal of Experimental Education

    (1996)
  • J. Ferron et al.

    Statistical power of randomization tests used with multiple-baseline designs

    Journal of Experimental Education

    (2002)
  • J. Ferron et al.

    Analyzing single-case data: The power of randomization tests

    Journal of Experimental Education

    (1995)
  • N.A. Gage et al.

    Analysis of effect for single-case design research

    Journal of Applied Sport Psychology

    (2013)
  • J.R. Gentile et al.

    An analysis-of-variance model for the intrasubject replication design

    Journal of Applied Behavior Analysis

    (1972)
  • G.V. Glass et al.

    Design and analysis of time-series experiments

    (1975)
  • J.M. Gottman et al.

    Analysis of interrupted time series analysis

  • Cited by (71)

    • Changed dynamic symptom networks after a self-compassion training in patients with somatic symptom disorder: A multiple single-case pilot project

      2022, Journal of Psychosomatic Research
      Citation Excerpt :

      Because of this design, we cannot rule out that effects found were confounded by history and maturation. A Randomized Controlled Trial (RCT) or a Single-Case Experimental Design (SCED; e.g., by randomization start of the intervention across participants or alternating treatment conditions with a placebo or waiting list condition [35,36]) are considered next steps needed to demonstrate treatment outcomes with more internal validity. An experimental design can be combined with ESM, yielding an overview of multiple single-case pre-to-post treatment changes in means and dynamic networks.

    • Single-Case Experimental Designs: Clinical Research and Practice

      2022, Comprehensive Clinical Psychology, Second Edition
    • EMDR therapy for PTSD symptoms in patients with mild intellectual disability or borderline intellectual functioning and comorbid psychotic disorder: A case series

      2021, Research in Developmental Disabilities
      Citation Excerpt :

      Traditionally, the difference in the mean baseline and intervention scores forms the test statistic in the randomization test. However, since the test does not rely on any distributional assumptions, a variety of descriptive statistics can be used as a test statistic in the randomization test (Heyvaert & Onghena, 2014). In our study, we performed a randomization on the effect-size Cohen’s d.

    View all citing articles on Scopus
    View full text