Abstract
Formal statistical methods for analyzing clinical trial data are widely accepted by the medical community. Unfortunately, the interpretation and reporting of trial results from the perspective of clinical importance has not received similar emphasis. This imbalance promotes the historical tendency to consider clinical trial results that are statistically significant as also clinically important, and conversely, those with statistically insignificant results as being clinically unimportant. In this paper, we review the present state of knowledge in the determination of the clinical importance of study results. This work also provides a simple, systematic method for determining the clinical importance of study results. It uses the relationship between the point estimate of the treatment effect (with its associated confidence interval) and the estimate of the smallest treatment effect that would lead to a change in a patient’s management. The possible benefits of this approach include enabling clinicians to more easily interpret the results of clinical trials from a clinical perspective, and promoting a more rational approach to the design of prospective clinical trials.
Similar content being viewed by others
References
Jaeschke R, Singer J, Guyatt GH. Measurement of health status: ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10:407–15.
Atrial Fibrillation Investigators. Risk factors for stroke and efficacy of antithrombotic therapy in atrial fibrillation. Arch Intern Med. 1994;154:1449–57.
EAFT (European Atrial Fibrillation Study) Group. Secondary prevention in non-rheumatic atrial fibrillation after transient ischemic attack or minor stroke. Lancet. 1993;342:1255–62.
Albers GW, Dalen JE, Laupacis A, Manning WJ, Petersen P, Singer DE. Antithrombotic therapy in atrial fibrillation. Chest. 2001;119: 194S-206S.
Gage BF, Waterman AD, Shannon W, Boechler M, Rich M, Radford MJ. Validation of clinical classification schemes for predicting stroke. JAMA. 2001;285:2864–70.
Hart RG, Benavente O, McBride R, Pearce LA. Antithrombotic therapy to prevent stroke in patients with atrial fibrillation: a meta-analysis. Ann Intern Med. 1999;131:492–501.
Atrial Fibrillation Investigators. The efficacy of aspirin in patients with atrial fibrillation. Arch Intern Med. 1997;157:1237–40.
Segal JB, McNamara RL, Miller MR, et al. Prevention of thromboembolism in atrial fibrillation. A meta-analysis of trials of anticoagulants and antiplatelet drugs. J Gen Intern Med. 2000; 15:56–67.
Young MJ, Bresnitz EA, Strom BL. Sample size nomograms for interpreting negative clinical studies. Ann Intern Med. 1983;99:248–51.
Goodman SN, Berlin JA. The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results. Ann Intern Med. 1994;121:200–6.
Feinstein AR. Clinical Biostatistics. Saint Louis: CV Mosby Company; 1977:333.
Naylor CD, Llewellyn-Thomas HA. Can there be a more patient-centred approach to determining clinically important effect sizes for randomized treatment trials. J Clin Epidemiol. 1994;47:787–95.
Pauker SG, Kassirer JP. The threshold approach to clinical decision making. N Engl J Med. 1980;302:1109–17.
Bellamy N, Carrette S, Ford PM, et al. Osteoarthritis antirheumatic drug trials III. Setting the delta for clinical trials—results of a consensus development (Delphi) exercise. J Rheumatol. 1992;20:557–60.
Gorelick PB, Born GV, D’Agostino RB, Hanley DF Jr, Moye L, Pepine CJ. Therapeutic benefit. Aspirin revisited in light of the introduction of clopidogrel. Stroke. 1999;30:1716–21.
Albers GW, Amarenco P, Easton JD, Sacco RL, Teal P. Antithrombotic and thrombolytic therapy for ischemic stroke. Chest. 2001; 119:300S-20S.
Llewellyn-Thomas HA, Thiel EC, Clark RM. Patients versus surrogates: whose opinion counts on ethics review panels? Clin Res. 1989;37:501–5.
Llewellyn-Thomas HA, McGreal MJ, Thiel EC, Fine S, Erlichman C. Patients’ willingness to enter clinical trials: measuring the association with perceived benefit and preference for decision participation. Soc Sci Med. 1991;32:35–42.
Man-Son-Hing M, Laupacis A, O’Connor A, et al. Warfarin for atrial fibrillation: the patient’s perspective. Arch Intern Med. 1996;156:1841–8.
Detsky AS. Using economic analysis to determine the resource consequences of choices made in planning clinical trials. J Chronic Dis. 1985;38:753–65.
van Walraven C, Mahon JL, Moher D, Bohn C, Laupacis A. Surveying physicians to determine the minimal important difference: implications for sample-size calculations. J Clin Epidemiol. 1999;52:717–23.
Chan K, Man-Son-Hing M, Molnar FJ, Laupacis A. How well is the clinical importance of study results reported. An assessment of randomized controlled trials. CMAJ. 2001;165:1197–202.
Kosinski M, Zhao SZ, Dedhiya S, Osterhaus JT, Ware JE Jr. Determining minimally important changes in generic and disease-specific health-related quality of life questionnaires in clinical trials of rheumatoid arthritis. Arthritis Rheum. 2000;43:1478–87.
Wells G, Beaton D, Shea B, et al. Minimal clinically important differences: review of methods. J Rheumatol. 2001;28:406–12.
Redelmeier DA, Guyatt GH, Goldstein RS. Assessing the minimal important difference in symptoms: a comparison of two techniques. J Clin Epidemiol. 1996;49:1215–9.
Redelmeier DA, Lorig JP. Assessing the clinical importance of statistical significance: illustration in rheumatology. Arch Intern Med. 1993;153:1337–42.
Wells GA, Tugwell P, Kraag GR, Baker P, Groh J, Redelmeier D. Minimum important difference between patients with rheumatoid arthritis: the patient’s perspective. J Rheumatol. 1993;20:557–60.
Juniper EF, Guyatt GH, Willan A, Griffith LE. Determining a minimal important change in a disease-specific quality of life questionnaire. J Clin Epidemiol. 1994;47:81–7.
Cohen J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988:19–27.
Jacobson NS, Roberts LJ, Berns SB, McGlinchey JB. Methods for defining and determining the clinical significance of treatment effects: description, application, and alternatives. J Consul Clin Psychol. 1999;67:300–307.
Kendall PC, Marrs-Garcia A, Nath SR, Sheldrick RC. Normative comparisons for the evaluation of clinical significance. J Consult Clin Psychol. 1999;67:285–99.
Anonymous. Significance of significance. N Engl J Med. 1968;278:1232–3.
Melton AW. Editorial. J Exp Psychol. 1962;64:553–7.
Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology. A Basic Science for Clinical Medicine, 2nd ed. Boston: Little, Brown and Company; 1991.
Freiman J, Chalmers TC, Smith H, Kaubler R. The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. N Engl J Med. 1978;299:690–4.
Cook DJ, Guyatt GL, Laupacis A, Sackett DL, Goldberg RJ. Clinical recommendations using levels of evidence for antithrombotic agents. Chest. 1995;108:227S-30S.
Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ. Users’ guide to the medical literature IX. A method for grading health care recommendations. JAMA. 1995;274:1800–4.
Moher D, Schulz KF, Altman DG, for the CONSORT Group. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. Ann Intern Med. 2001;134:657–62.
Mahon J, Laupacis A, Donner A, Wood T. Randomised study of n of 1 trials versus standard practice. BMJ. 1996;312:1069–74.
Detsky AS, Sackett DL. When was a ‘negative’ clinical trial big enough. Arch Intern Med. 1985;145:709–12.
Jones B, Jarvis P, Lewis JA, Ebbutt AF. Trials to assess equivalence: importance of rigorous methods. BMJ. 1996;313:36–9.
Dunnett CW, Gent M. Significance testing to establish equivalence between treatments, with special reference to data in the form of 2 × 2 tables. Biometrics. 1977;33:593–602.
Pitt B, Zannad F, Remme WJ, et al. The effect of spironolactone on morbidity and mortality in patients with severe heart failure. N Engl J Med. 1999;341:709–17.
Naylor MD, Broten D, Campbell R, et al. Comprehensive discharge planning and home follow-up of hospitalized elders. JAMA. 1999; 281:613–20.
Dalby DM, Sellors JW, Fraser FD, Fraser C, van Ineveld C, Howard M. Effect of preventive home visits by a nurse on the outcomes of frail elderly people in the community: a randomized controlled trial. CMAJ. 2000;62:497–500.
Silverstein FE, Graham DY, Senior JR, et al. Misoprostol reduces serous gastrointestinal complications in patients with rheumatoid arthritis receiving nonsteroidal anti-inflammatory drugs. Ann Intern Med. 1995;123:241–9.
Dale JJ, Ruckley CV, Harper DR, Gibson B, Nelson EA, Prescott RJ. Randomised, double blind placebo controlled trial of pentoxifylline in the treatment of venous leg ulcers. BMJ. 1999;319:875–8.
GISSI-Prevenzione Investigators. Dietary supplementation with n-3 polyunsaturated fatty acids and vitamin E after myocardial infarction: results of the GISSI-Prevenzione trial. Lancet. 1999; 354:447–55.
Hulley S, Grady D, Bush T, et al. Randomized trial of estrogen for secondary prevention of coronary heart disease in postmenopausal women. Heart and Estrogen/progestin Replacement Study (HERS) Research Group. JAMA. 1998;280:605–13.
Pocock SJ. When to stop a clinical trial. BMJ. 1992;305:235–40.
Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC. Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med. 1992;327:248–54.
Guyatt GH, Sinclair J, Cook DJ, Glasziou P. Users’ guide to the medical literature XVI. How to use a treatment recommendation. JAMA. 1999;281:1836–43.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Man-Son-Hing, M., Laupacis, A., O’Rourke, K. et al. Determination of the clinical importance of study results. J GEN INTERN MED 17, 469–476 (2002). https://doi.org/10.1046/j.1525-1497.2002.11111.x
Issue Date:
DOI: https://doi.org/10.1046/j.1525-1497.2002.11111.x