Original article
Development of a Quality Checklist Using Delphi Methods for Prescriptive Clinical Prediction Rules: The QUADCPR

https://doi.org/10.1016/j.jmpt.2009.11.010Get rights and content

Abstract

Objective

Clinical prediction rules (CPRs) are clinician decision-making aids designed to improve the accuracy of a variety of decisions made during patient care. To our knowledge, there are no formally developed consensus-based guidelines designed to provide standards for the creation of CPRs.

Methods

The study used a 3-round Delphi method for consensus of a quality checklist initially developed based on recommendations derived from the literature. The 9 Delphi participants were randomly selected from the authors of peer-reviewed publications of prescriptive CPRs.

Results

During the 3 rounds, the Delphi participants modified the originally derived checklist and, based on a consensus standard, agreed upon a final 23-item checklist, which involved 4 constructs: (1) sample and participants, (2) outcome measures, (3) quality of tests and measures, and (4) statistical assumptions.

Conclusions

Use of the checklist has potential for improving the design and reporting of future prescriptive CPRs.

Section snippets

Development of the CPR Checklist

We developed the CPR checklist following the first 3 procedures outlined by Streiner and Norman36 for development of quality assessment tools. Their process outlines 5 critical steps, as follows: (1) preliminary conceptual decisions; (2) item generation; (3) assessment of face validity; (4) field trials; and 5) generation of a refined instrument, allowing for the generation, refinement, and potential adaptation of a tool, created during a continuous improvement process. For our study, our work

Results

All 9 randomly selected Delphi participants completed rounds I, II, and III (100%). All but 2 participated in clinical practice (mean, 11 years; SD = 5.9), and the group had participated in research for an average of 8.3 years (SD = 3.6). Participant backgrounds included 4 physicians, 2 epidemiologists, and 3 physical therapists and resided in 4 different continents. Most (6) practiced in academic settings, but research interests and practice backgrounds varied between orthopedics (4),

Discussion

The goal of this project was to develop a consensus-based quality checklist for the creation of prescriptive, derivation-based CPRs. The final checklist (QUADCPR) is now available for researchers during the development of prescriptive, derivation-based CPR studies. The final checklist comprises 23 items that encompass 4 disparate constructs (sections), where each item is scored with the scores of “yes,” “no,” or “unclear.”

At present, 2 quality checklists have been used during systematic

Conclusion

The QUADCPR is designed to improve the design and reporting standards for prescriptive, derivation-based CPRs. Future studies should address inter- and intraobserver variability and validation of the checklist or further refinement of items and for scoring to allow for quantitative ratings of quality. In addition, based on the recent perspective involving selection of variables from a randomized controlled trial, future studies should investigate the sensitivity of the use of a derivation phase

Funding Sources and Potential Conflicts of Interest

No funding sources or conflicts of interest were reported for this study.

Practical Applications

  • Most recent rehabilitation-based CPRs have been associated with prescriptive studies.

  • The recent proliferation of CPRs has occurred without guidelines for reporting.

  • Checklists for reporting standards are typically developed using Delphi methodology.

  • This study used a Delphi method to outline appropriate reporting standards for prescriptive CPRs.

  • The checklist (QUADCPR) is designed to be used prospectively during development

References (65)

  • SchmittJC et al.

    The validity of prospective and retrospective global change criterion measures

    Arch Phys Med Rehabil

    (2005)
  • CookC et al.

    Subjective and objective descriptors of clinical lumbar spine instability: a Delphi study

    Man Ther

    (2006)
  • BrehautJC et al.

    Clinical decision rules “in the real world”: how a widely disseminated rule is used in everyday practice

    Acad Emerg Med

    (2005)
  • FlynnT et al.

    A clinical prediction rule for classifying patients with low back pain who demonstrate short-term improvement with spinal manipulation

    Spine

    (2002)
  • ChildsJD et al.

    A clinical prediction rule to identify patients with low back pain most likely to benefit from spinal manipulation: a validation study

    Ann Intern Med

    (2004)
  • HancockMJ et al.

    Independent evaluation of a clinical prediction rule for spinal manipulative therapy: a randomised controlled trial

    Eur Spine J

    (2008)
  • ClelandJA et al.

    Development of a clinical prediction rule for guiding treatment of a subgroup of patients with neck pain: use of thoracic spine manipulation, exercise, and patient education

    Phys Ther

    (2007)
  • Fernandez-de-las-PenasC et al.

    Predictor variables for identifying patients with chronic tension-type headache who are likely to achieve short term success with muscle trigger point therapy

    Cephalgia

    (2008)
  • LesherJD et al.

    Development of a clinical prediction rule for classifying patients with patellofemoral pain syndrome who respond to patellar taping

    J Orthop Sports Phys Ther

    (2006)
  • CurrierLL et al.

    Development of a clinical prediction rule to identify patients with knee pain and clinical evidence of knee osteoarthritis who demonstrate a favorable short-term response to hip mobilization

    Phys Ther

    (2007)
  • IversonCA et al.

    Lumbopelvic manipulation for the treatment of patients with patellofemoral pain syndrome: development of a clinical prediction rule

    J Orthop Sports Phys Ther

    (2008)
  • VicenzinoB et al.

    Development of a clinical prediction rule to identify initial responders to mobilization with movement and exercise for lateral epicondylalgia

    Man Ther

    (2008)
  • VicenzinoB et al.

    A clinical prediction rule for identifying patients with patellofemoral pain

    Br J Sports Med

    (2008)
  • AlghamdiAA et al.

    Development and validation of Transfusion Risk Understanding Scoring Tool (TRUST) to stratify cardiac surgery patients according to their blood transfusion needs

    Transfusion

    (2006)
  • Van BelleA et al.

    Christopher Study Investigators. Effectiveness of managing suspected pulmonary embolism using an algorithm combining clinical probability, d-dimer testing, and computed tomography

    JAMA

    (2006)
  • DunningJ et al.

    A validated rule for predicting patients who require prolonged ventilation post cardiac surgery

    Eur J Cardiothorac Surg

    (2003)
  • FischerJE et al.

    Use of simple heuristics to target macrolide prescription in children with community-acquired pneumonia

    Arch Pediatr Adolesc Med

    (2002)
  • ChildsJD et al.

    Development and application of clinical prediction rules to improve decision making in physical therapist practice

    Phys Ther

    (2006)
  • HierDB et al.

    Deriving clinical prediction rules from stroke outcome research

    Stroke

    (1991)
  • WassonJH et al.

    Clinical prediction rules: applications and methodological standards

    New Engl J Med

    (1985)
  • SmidtN et al.

    Reproducibility of the STARD checklist: an instrument to assess the quality of reporting of diagnostic accuracy studies

    BMC Med Res Methodol

    (2006)
  • AltmanDG et al.

    Statistics notes: concealing treatment allocation in randomised trials

    BMJ

    (2001)
  • Cited by (22)

    • Existing validated clinical prediction rules for predicting response to physiotherapy interventions for musculoskeletal conditions have limited clinical value: A systematic review

      2021, Journal of Clinical Epidemiology
      Citation Excerpt :

      We conducted a two-stage process for quality appraisal, separately assessing original derivation studies and validation studies. As there is currently no systematic framework for appraising prescriptive CPRs [12], derivation studies were assessed based on criteria guided by the PROBAST tool for prognostic and diagnostic CPRs, as well as published recommendations relating to prescriptive CPRs in rehabilitation practice [6,12,18,24]. Validation studies were assessed using nine criteria for studies with control groups outlined by the Cochrane Effective Practice and Organisation of Care group [25].

    • Design and validation of a tool for the evaluation of the quality of Cardiopulmonary Resuscitation: SIEVCA-CPR 2.0®

      2018, Intensive and Critical Care Nursing
      Citation Excerpt :

      Finally, the participant group was formed by 11 experts. The Delphi method usually recommends nine expert opinions to obtain data saturation (Cook et al., 2010; García Valdés and Suárez Marín, 2013). The stages in which the Delphi method was developed were as follows (Fig. 1):

    • Validation and impact analysis of prognostic clinical prediction rules for low back pain is needed: A systematic review

      2015, Journal of Clinical Epidemiology
      Citation Excerpt :

      The interobserver reliability of a patient's status on a CPR is a potential threat to its validity [39,40] and ideally should be evaluated and reported similar to accepted standards for single-item tests [89]. Finally, during all phases of a CPR's development, the reporting of uncertainty intervals for outcome prevalence, CPR accuracy (e.g., sensitivity, specificity, +LR, −LR), and posterior probabilities would enable a more meaningful interpretation of a study's findings [27,36,42]. There are limitations of this study that need to be acknowledged.

    View all citing articles on Scopus
    View full text