The utility of the PND statistic: A reply to Allison and Gorman

https://doi.org/10.1016/0005-7967(94)90169-4Get rights and content

Abstract

Quantitative synthesis (“meta-analysis”) of single-subject research has rarely been conducted, partly because of a lack of agreement on how study outcomes are to be quantified. This article provides a response to Allison and German (Behaviour Research and Therapy, 31, 621–631, 1993), who listed some problematic characteristics of use of the PND (percent of non-overlapping data) statistic for computing single-subject study outcomes, and recommended a regression-based solution to computation of effect sizes from single-subject research reports. Although Allison and Gorman are generally accurate in pointing out some limitations of the use of the PND statistic, they have been less thorough in identifying its relative strengths. Among these strengths is the fact that the PND statistic and its variations (a) have been shown to be strongly related to qualitative, “expert” ratings, (b) have been successfully employed in at least seven separate integrative reviews, and (c) have produced results which are complementary to more qualitative reviews of the same literature. In contrast, Allison and Gorman did not report results of applications of their procedure and, although their procedure has apparent theoretical support, it may be less useful in synthesizing existing single-subject literature.

References (30)

  • D.B. Allison et al.

    Calculating effect sizes for meta-analysis: The case of the single case

    Behaviour Research and Therapy

    (1993)
  • W.H. Yeaton

    A critique of the effectiveness of applied behavior analysis research

    Advances in Behavior Research and Therapy

    (1982)
  • P.L. Busk et al.

    Meta-analysis for single-case research

  • G. Casto et al.

    The efficacy of early intervention programs: A meta-analysis

    Exceptional Children

    (1986)
  • B.A. Center et al.

    A methodology for the quantitative synthesis of intra-subject design research

    Journal of Special Education

    (1985–1986)
  • A. DeProspero et al.

    Inconsistent visual analysis of variance model for the intrasubject replication design

    Journal of Applied Behavioral Analysis

    (1979)
  • B.E. Huitema

    Auto-correlation in applied behavior analysis: A myth

    Behavioral Assessment

    (1985)
  • R.R. Jones et al.

    Effects of serial dependency on the agreement between visual and statistical inference

    Journal of Applied Behavioral Analysis

    (1978)
  • A.E. Kazdin

    Methodological and interpretive problems of single-case experimental designs

    Journal of Consulting and Clinical Psychology

    (1978)
  • H.C. Kraemer et al.

    A nonparametric technique for meta-analysis effect size calculation

    Psychological Bulletin

    (1982)
  • J.R. Levin

    Single-case research design and analysis: Comments and concerns

  • M.A. Mastropieri et al.

    Early intervention for socially withdrawn children

    Journal of Special Education

    (1985–1986)
  • R.B. Rutherford et al.

    Applied behavior analysis in education: Generalization and maintenance

  • C.L. Salzberg et al.

    Meta-analysis for single-subject research: When does it clarify: When does it obscure?

    Remedial and Special Education

    (1987)
  • R.W. Schlosser et al.

    Effectiveness and treatment validity of interventions addressing self-injurious behavior: From narrative reviews to meta-analyses

  • Cited by (46)

    • A peer-mediated approach to support emergent bilingual preschoolers

      2022, Early Childhood Research Quarterly
      Citation Excerpt :

      Further, this approach provides a more accurate measure of intervention effectiveness in comparison to other parametric measures (e.g., standardized mean difference) that may not adequately account for change when social behaviors are absent at baseline (Ledford, King, Harbin, & Zimmerman, 2018; Pustejovsky et al., 2018). Tau-U provides an extension to traditional approaches, such as percent of nonoverlapping data (PND; Scruggs & Mastropieri, 1994) and nonoverlapping of all pairs analysis (NAP; Parker & Vannest, 2008), as it estimates and corrects for undesirable baseline trends to calculate treatment effectiveness and statistical significance using the nonparametric S sampling distribution (see Parker, Vannest, Davis, & Sauber, 2011 for detailed calculation). As such, Tau-U can be appropriately applied to data that lacks a stable baseline pattern for populations who may be expected to exhibit variable performance (cf. Lee & Cherney, 2018), such as preschool-aged children examined in this study.

    • Meta-analysis of PECS with individuals with ASD: Investigation of targeted versus non-targeted outcomes, participant characteristics, and implementation phase

      2012, Research in Developmental Disabilities
      Citation Excerpt :

      A recent review of meta-analytic procedures found that PND is by far the most widely used effect size in SCR (Maggin et al., 2011). PND is calculated as the percent of “intervention” phase data that exceed (in the intended direction) the single most extreme “baseline” phase data point (Scruggs & Mastropieri, 1994). This metric is similarly interpreted as the percent of intervention phase data that exceeded the single highest baseline or “A” phase data point.

    • Topoi and literary interpretation: The effects of a critical reading and writing intervention on high school students' analytic literary essays

      2011, Contemporary Educational Psychology
      Citation Excerpt :

      PND is not related to an accepted effect size, so it needs its own interpretive guidelines (Parker et al., 2007). Scruggs and Mastropieri (1994) suggest that interventions with a PND > 70 are effective, a PND between 50 and 70 are questionable, and a PND < 50 has no observed effect. The results are organized with respect to the above-mentioned predictions.

    • Combining Nonoverlap and Trend for Single-Case Research: Tau-U

      2011, Behavior Therapy
      Citation Excerpt :

      Ordinary least squares regression analysis (OLS) with a long history of use in large N studies, has shown unequalled flexibility and power when applied to SCR data (Allison & Gorman, 1993; Busk & Serlin, 1992; Parker & Brossart, 2003). However, OLS has been criticized for failing to address the unique constraints of short time series data that are typical in SCR (Parsonson & Baer, 1992; Scruggs & Mastropieri, 1994). OLS is a parametric statistical test, and as such requires a normal score distribution, constant variance, and interval level measurement.

    View all citing articles on Scopus
    View full text