Abstract
Randomization tests are nonparametric statistical tests that obtain their validity by computationally mimicking the random assignment procedure that was used in the design phase of a study. Because randomization tests do not rely on a random sampling assumption, they can provide a better alternative than parametric statistical tests for analyzing data from single-case designs. In this article, an R package is described for use in designing single-case phase (AB, ABA, and ABAB) and alternation (completely randomized, alternating treatments, and randomized block) experiments, as well as for conducting statistical analyses on data gathered by means of such designs. The R code is presented in a step-by-step way, which at the same time clarifies the rationale behind single-case randomization tests.
Article PDF
Similar content being viewed by others
References
Adams, D. C., & Anthony, C. D. (1996). Using randomization techniques to analyse behavioural data. Animal Behaviour, 51, 733–738.
Adams, D. C., Gurevitch, J., & Rosenberg, M. S. (1997). Resampling tests for meta-analysis of ecological data. Ecology, 78, 1277–1283.
Arndt, S., Cizadlo, T., Andreasen, N. C., Heckel, D., Gold, S., & O’Leary, D. S. (1996). Test for comparing images based on randomization and permutation methods. Journal of Cerebral Blood Flow & Metabolism, 16, 1271–1279.
Avins, A. L., Bent, S., & Neuhaus, J. M. (2005). Use of an embedded N-of-1 trial to improve adherence and increase information from a clinical study. Contemporary Clinical Trials, 26, 397–401.
Backman, C. L., & Harris, S. R. (1999). Case studies, single-subject research, and N of 1 randomized trials: Comparisons and contrasts. American Journal of Physical Medicine & Rehabilitation, 78, 170–176.
Baplu, A. (2005). Rilatine, the smart drug. Een dubbelblind placebogecontroleerd N = 1 experiment bij een adolescent met concentratieproblemen. Unpublished master’s thesis, Katholieke Universiteit Leuven, Belgium.
Barlow, D. H., & Hayes, S. C. (1979). Alternating treatments design: One strategy for comparing the effects of two treatments in a single subject. Journal of Applied Behavior Analysis, 12, 199–210.
Barlow, D. H., & Hersen, M. (1984). Single case experimental designs: Strategies for studying behavior change (2nd ed.). New York: Pergamon.
Besag, J., & Diggle, P. J. (1977). Simple Monte Carlo tests for spatial pattern. Journal of the Royal Statistical Society: Series C, 26, 327–333.
Busse, R. T., Kratochwill, T. R., & Elliott, S. N. (1995). Metaanalysis for single-case consultation outcomes: Applications to research and practice. Journal of School Psychology, 33, 269–285.
Campbell, D. T. (1957). Factors relevant to the validity of experiments in social settings. Psychological Bulletin, 54, 297–312.
Christensen, L. B. (2001). Experimental methodology (8th ed.). Boston: Allyn & Bacon.
Crawley, M. J. (2005). Statistics: An introduction using R. Chichester, U.K.: Wiley.
Crosbie, J. (1993). Interrupted time-series analysis with brief singlesubject data. Journal of Consulting & Clinical Psychology, 61, 966–974.
Dalgaard, P. (2002). Introductory statistics with R. New York: Springer.
Edgington, E. S. (1973). The random-sampling assumption in “Comment on component-randomization tests.“ Psychological Bulletin, 80, 84–85.
Edgington, E. S. (1975). Randomization tests for one-subject operant experiments. Journal of Psychology: Interdisciplinary & Applied, 90, 57–68.
Edgington, E. S. (1980). Validity of randomization tests for one-subject experiments. Journal of Educational Statistics, 5, 235–251.
Edgington, E. S. (1996). Randomized single-subject experimental designs. Behaviour Research & Therapy, 34, 567–574.
Edgington, E. S., & Bland, B. H. (1993). Randomization tests: Application to single-cell and other single-unit neuroscience experiments. Journal of Neuroscience Methods, 47, 169–177.
Edgington, E. S., & Onghena, P. (2007). Randomization tests (4th ed.). Boca Raton, FL: Chapman & Hall/CRC.
Ferron, J., Foster-Johnson, L., & Kromrey, J. D. (2003). The functions of single-case randomization tests with and without random assignment. Journal of Experimental Education, 71, 267–288.
Ferron, J., & Onghena, P. (1996). The power of randomization tests for single-case phase designs. Journal of Experimental Education, 64, 231–239.
Ferron, J., & Sentovich, C. (2002). Statistical power of randomization tests used with multiple-baseline designs. Journal of Experimental Education, 70, 165–178.
Ferron, J., & Ware, W. (1995). Analyzing single-case data: The power of randomization tests. Journal of Experimental Education, 63, 167–178.
Franklin, R. D., Allison, D. B., & Gorman, B. S. (Eds.) (1997). Design and analysis of single-case research. Mahwah, NJ: Erlbaum.
Franklin, R. D., Gorman, B. S., Beasley, T. M., & Allison, D. B. (1997). Graphical display and visual analysis. In R. D. Franklin, D. B. Allison, & B. S. Gorman (Eds.), Design and analysis of single-case research (pp. 119–158). Mahwah, NJ: Erlbaum.
Gentile, J. R., Roden, A. H., & Klein, R. D. (1972). An analysisof-variance model for the intrasubject replication design. Journal of Applied Behavior Analysis, 5, 193–198.
Gorman, B. S., & Allison, D. B. (1997). Statistical alternatives for single-case designs. In R. D. Franklin, D. B. Allison, & B. S. Gorman (Eds.), Design and analysis of single-case research (pp. 159–214). Mahwah, NJ: Erlbaum.
Guyatt, G. H., Keller, J. L., Jaeschke, R., Rosenbloom, D., Adachi, J. D., & Newhouse, M. T. (1990). The n-of-1 randomized controlled trial: Clinical usefulness. Our three-year experience. Annals of Internal Medicine, 112, 293–299.
Guyatt, G., Sackett, D., Adachi, J., Roberts, R., Chong, J., Rosenbloom, D., & Keller, J. (1988). A clinician’s guide for conducting randomized trials in individual patients. Canadian Medical Association Journal, 139, 497–503.
Guyatt, G., Sackett, D., Taylor, D. W., Chong, J., Roberts, R., & Pugsley, S. (1986). Determining optimal therapy—Randomized trials in individual patients. New England Journal of Medicine, 314, 889–892.
Hayes, S. C. (1981). Single case experimental design and empirical clinical practice. Journal of Consulting & Clinical Psychology, 49, 193–211.
Hooton, J. W. (1991). Randomization tests: Statistics for experimenters. Computer Methods & Programs in Biomedicine, 35, 43–51.
Hornik, K. (2007). The R FAQ: Frequently asked questions on R. Retrieved September 24, 2007, from CRAN.R-project.org/doc/FAQ/.
Kazdin, A. E. (1981a). Drawing valid inferences from case studies. Journal of Consulting & Clinical Psychology, 49, 183–192.
Kazdin, A. E. (1981b). External validity and single-case experimentation: Issues and limitations (a response to J. S. Birnbrauer). Analysis & Intervention in Developmental Disabilities, 1, 133–143.
Kazdin, A. E. (1982). Single-case research designs: Methods for clinical and applied settings. New York: Oxford University Press.
Kazdin, A. E. (1984). Statistical analyses for single-case experimental designs. In D. H. Barlow & M. Hersen (Eds.), Single case experimental designs: Strategies for studying behavior change (2nd ed.; pp. 258–324). New York: Pergamon.
Kazdin, A. E. (2003). Research design in clinical psychology (4th ed.). Boston: Allyn & Bacon.
Kelley, K. (2007). Methods for the behavioral, educational, and social sciences: An R package. Behavior Research Methods, 39, 979–984.
Koehler, M. J., & Levin, J. R. (1998). Regulated randomization: A potentially sharper analytical tool for the multiple-baseline design. Psychological Methods, 3, 206–217.
Koehler, M. J., & Levin, J. R. (2000). RegRand: Statistical software for the multiple-baseline design. Behavior Research Methods, Instruments, & Computers, 32, 367–371.
Ludbrook, J. (1994). Advantages of permutation (randomization) tests in clinical and experimental pharmacology and physiology. Clinical & Experimental Pharmacology & Physiology, 21, 673–686.
Lundervold, D. A., & Belwood, M. F. (2000). The best kept secret in counseling: Single-case (N = 1) experimental designs. Journal of Counseling & Development, 78, 92–102.
Maindonald, J., & Braun, J. (2003). Data analysis and graphics using R: An example-based approach. Cambridge: Cambridge University Press.
Maris, E., & Oostenveld, R. (2007). Nonparametric statistical testing of EEG- and MEG-data. Journal of Neuroscience Methods, 164, 177–190.
Maris, E., Schoffelen, J.-M., & Fries, P. (2007). Nonparametric statistical testing of coherence differences. Journal of Neuroscience Methods, 163, 161–175.
Matyas, T. A., & Greenwood, K. M. (1990). Visual analysis of singlecase time series: Effects of variability, serial dependence, and magnitude of intervention effects. Journal of Applied Behavioral Analysis, 23, 341–351.
Moore, D. S., & McCabe, G. P. (2006). Introduction to the practice of statistics (5th ed.). New York: Freeman.
Murray, D. M., Varnell, S. P., & Blitstein, J. L. (2004). Design and analysis of group-randomized trials: A review of recent methodological developments. American Journal of Public Health, 94, 423–432.
Nichols, T. E., & Holmes, A. P. (2001). Nonparametric permutation tests for functional neuroimaging: A primer with examples. Human Brain Mapping, 15, 1–25.
Nikles, C. J., Clavarino, A. M., & Del Mar, C. B. (2005). Using n-of-1 trials as a clinical tool to improve prescribing. British Journal of General Practice, 55, 175–180.
Onghena, P. (1992). Randomization tests for extensions and variations of ABAB single-case experimental designs: A rejoinder. Behavioral Assessment, 14, 153–171.
Onghena, P. (2005). Single case designs. In B. S. Everitt & D. C. Howell (Eds.), Encyclopedia of statistics in behavioral science (Vol. 4; pp. 1850–1854). Chichester, U.K.: Wiley.
Onghena, P., & Edgington, E. S. (1994). Randomization tests for restricted alternating treatments designs. Behaviour Research & Therapy, 32, 783–786.
Onghena, P., & Edgington, E. S. (2005). Customization of pain treatments: Single-case design and analysis. Clinical Journal of Pain, 21, 56–68.
Potvin, C., & Roff, D. A. (1993). Distribution-free and robust statistical methods: Viable alternatives to parametric statistics. Ecology, 74, 1617–1628.
Ramsey, F. L., & Schafer, D. W. (2002). The statistical sleuth: A course in methods of data analysis (2nd ed.). Pacific Grove, CA: Duxbury/Thomson Learning.
Recchia, M., & Rocchetti, M. (1982). The simulated randomization test. Computer Programs in Biomedicine, 15, 111–116.
Robinson, D. H., & Levin, J. R. (1997). Reflections on statistical and substantive significance, with a slice of replication. Educational Researcher, 26, 21–26.
Siegel, S. (1957). Nonparametric statistics. American Statistician, 11, 13–19.
Solow, A. R. (1993). A simple test for change in community structure. Journal of Animal Ecology, 62, 191–193.
Strauss, R. E. (1982). Statistical significance of species clusters in association analysis. Ecology, 63, 634–639.
Todman, J. (2002). Randomization in single-case experimental designs. Advances in Clinical Neuroscience & Rehabilitation, 2, 18–19.
Todman, J. [B.], & Dugard, P. (1999). Accessible randomization tests for single-case and small-n experimental designs in AAC research. Augmentative & Alternative Communication, 15, 69–82.
Todman, J. B., & Dugard, P. (2001). Single-case and small-n experimental designs: A practical guide to randomization tests. Mahwah, NJ: Erlbaum.
Turk, D. C. (2000). Statistical significance and clinical significance are not synonyms! (Editorial). Clinical Journal of Pain, 16, 185–187.
Van den Noortgate, W., & Onghena, P. (2003a). Combining singlecase experimental data using hierarchical linear models. School Psychology Quarterly, 18, 325–346.
Van den Noortgate, W., & Onghena, P. (2003b). Hierarchical linear models for the quantitative integration of effect sizes in single-case research. Behavior Research Methods, Instruments, & Computers, 35, 1–10.
Wegman, A. C. M., van der Windt, D. A. W. M., Bongers, M., Twisk, J. W. R., Stalman, W. A. B., & de Vries, T. P. G. M. (2005). Efficacy of temazepam in frequent users: A series of N-of-1 trials. Family Practice, 22, 152–159.
Wegman, A. C. M., van der Windt, D. A. W. M., de Haan, M., Devillé, W. L. J. M., Fo, C. T. C. A., & de Vries, T. P. G. M. (2003). Switching from NSAIDs to paracetamol: A series of n of 1 trials for individual patients with osteoarthritis. Annals of the Rheumatic Diseases, 62, 1156–1161.
Wilson, J. B. (2007). Priorities in statistics, the sensitive feet of elephants, and don’t transform data. Folia Geobotanica, 42, 161–167.
Zhan, S., & Ottenbacher, K. J. (2001). Single subject research designs for disability research. Disability & Rehabilitation, 23, 1–8.
Author information
Authors and Affiliations
Corresponding author
Additional information
This study was funded by Grant G.0624.07 from the Research Foundation— Flanders (Belgium).
Rights and permissions
About this article
Cite this article
Bulté, I., Onghena, P. An R package for single-case randomization tests. Behavior Research Methods 40, 467–478 (2008). https://doi.org/10.3758/BRM.40.2.467
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BRM.40.2.467