Skip to main content
Log in

Determination of the clinical importance of study results

A review

  • Review
  • Published:
Journal of General Internal Medicine Aims and scope Submit manuscript

Abstract

Formal statistical methods for analyzing clinical trial data are widely accepted by the medical community. Unfortunately, the interpretation and reporting of trial results from the perspective of clinical importance has not received similar emphasis. This imbalance promotes the historical tendency to consider clinical trial results that are statistically significant as also clinically important, and conversely, those with statistically insignificant results as being clinically unimportant. In this paper, we review the present state of knowledge in the determination of the clinical importance of study results. This work also provides a simple, systematic method for determining the clinical importance of study results. It uses the relationship between the point estimate of the treatment effect (with its associated confidence interval) and the estimate of the smallest treatment effect that would lead to a change in a patient’s management. The possible benefits of this approach include enabling clinicians to more easily interpret the results of clinical trials from a clinical perspective, and promoting a more rational approach to the design of prospective clinical trials.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Jaeschke R, Singer J, Guyatt GH. Measurement of health status: ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10:407–15.

    Article  PubMed  CAS  Google Scholar 

  2. Atrial Fibrillation Investigators. Risk factors for stroke and efficacy of antithrombotic therapy in atrial fibrillation. Arch Intern Med. 1994;154:1449–57.

    Article  Google Scholar 

  3. EAFT (European Atrial Fibrillation Study) Group. Secondary prevention in non-rheumatic atrial fibrillation after transient ischemic attack or minor stroke. Lancet. 1993;342:1255–62.

    Google Scholar 

  4. Albers GW, Dalen JE, Laupacis A, Manning WJ, Petersen P, Singer DE. Antithrombotic therapy in atrial fibrillation. Chest. 2001;119: 194S-206S.

    Article  PubMed  CAS  Google Scholar 

  5. Gage BF, Waterman AD, Shannon W, Boechler M, Rich M, Radford MJ. Validation of clinical classification schemes for predicting stroke. JAMA. 2001;285:2864–70.

    Article  PubMed  CAS  Google Scholar 

  6. Hart RG, Benavente O, McBride R, Pearce LA. Antithrombotic therapy to prevent stroke in patients with atrial fibrillation: a meta-analysis. Ann Intern Med. 1999;131:492–501.

    PubMed  CAS  Google Scholar 

  7. Atrial Fibrillation Investigators. The efficacy of aspirin in patients with atrial fibrillation. Arch Intern Med. 1997;157:1237–40.

    Article  Google Scholar 

  8. Segal JB, McNamara RL, Miller MR, et al. Prevention of thromboembolism in atrial fibrillation. A meta-analysis of trials of anticoagulants and antiplatelet drugs. J Gen Intern Med. 2000; 15:56–67.

    Article  PubMed  CAS  Google Scholar 

  9. Young MJ, Bresnitz EA, Strom BL. Sample size nomograms for interpreting negative clinical studies. Ann Intern Med. 1983;99:248–51.

    PubMed  CAS  Google Scholar 

  10. Goodman SN, Berlin JA. The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results. Ann Intern Med. 1994;121:200–6.

    PubMed  CAS  Google Scholar 

  11. Feinstein AR. Clinical Biostatistics. Saint Louis: CV Mosby Company; 1977:333.

    Google Scholar 

  12. Naylor CD, Llewellyn-Thomas HA. Can there be a more patient-centred approach to determining clinically important effect sizes for randomized treatment trials. J Clin Epidemiol. 1994;47:787–95.

    Article  PubMed  CAS  Google Scholar 

  13. Pauker SG, Kassirer JP. The threshold approach to clinical decision making. N Engl J Med. 1980;302:1109–17.

    Article  PubMed  CAS  Google Scholar 

  14. Bellamy N, Carrette S, Ford PM, et al. Osteoarthritis antirheumatic drug trials III. Setting the delta for clinical trials—results of a consensus development (Delphi) exercise. J Rheumatol. 1992;20:557–60.

    Google Scholar 

  15. Gorelick PB, Born GV, D’Agostino RB, Hanley DF Jr, Moye L, Pepine CJ. Therapeutic benefit. Aspirin revisited in light of the introduction of clopidogrel. Stroke. 1999;30:1716–21.

    PubMed  CAS  Google Scholar 

  16. Albers GW, Amarenco P, Easton JD, Sacco RL, Teal P. Antithrombotic and thrombolytic therapy for ischemic stroke. Chest. 2001; 119:300S-20S.

    Article  PubMed  CAS  Google Scholar 

  17. Llewellyn-Thomas HA, Thiel EC, Clark RM. Patients versus surrogates: whose opinion counts on ethics review panels? Clin Res. 1989;37:501–5.

    PubMed  CAS  Google Scholar 

  18. Llewellyn-Thomas HA, McGreal MJ, Thiel EC, Fine S, Erlichman C. Patients’ willingness to enter clinical trials: measuring the association with perceived benefit and preference for decision participation. Soc Sci Med. 1991;32:35–42.

    Article  PubMed  CAS  Google Scholar 

  19. Man-Son-Hing M, Laupacis A, O’Connor A, et al. Warfarin for atrial fibrillation: the patient’s perspective. Arch Intern Med. 1996;156:1841–8.

    Article  PubMed  CAS  Google Scholar 

  20. Detsky AS. Using economic analysis to determine the resource consequences of choices made in planning clinical trials. J Chronic Dis. 1985;38:753–65.

    Article  PubMed  CAS  Google Scholar 

  21. van Walraven C, Mahon JL, Moher D, Bohn C, Laupacis A. Surveying physicians to determine the minimal important difference: implications for sample-size calculations. J Clin Epidemiol. 1999;52:717–23.

    Article  PubMed  Google Scholar 

  22. Chan K, Man-Son-Hing M, Molnar FJ, Laupacis A. How well is the clinical importance of study results reported. An assessment of randomized controlled trials. CMAJ. 2001;165:1197–202.

    PubMed  CAS  Google Scholar 

  23. Kosinski M, Zhao SZ, Dedhiya S, Osterhaus JT, Ware JE Jr. Determining minimally important changes in generic and disease-specific health-related quality of life questionnaires in clinical trials of rheumatoid arthritis. Arthritis Rheum. 2000;43:1478–87.

    Article  PubMed  CAS  Google Scholar 

  24. Wells G, Beaton D, Shea B, et al. Minimal clinically important differences: review of methods. J Rheumatol. 2001;28:406–12.

    PubMed  CAS  Google Scholar 

  25. Redelmeier DA, Guyatt GH, Goldstein RS. Assessing the minimal important difference in symptoms: a comparison of two techniques. J Clin Epidemiol. 1996;49:1215–9.

    Article  PubMed  CAS  Google Scholar 

  26. Redelmeier DA, Lorig JP. Assessing the clinical importance of statistical significance: illustration in rheumatology. Arch Intern Med. 1993;153:1337–42.

    Article  PubMed  CAS  Google Scholar 

  27. Wells GA, Tugwell P, Kraag GR, Baker P, Groh J, Redelmeier D. Minimum important difference between patients with rheumatoid arthritis: the patient’s perspective. J Rheumatol. 1993;20:557–60.

    PubMed  CAS  Google Scholar 

  28. Juniper EF, Guyatt GH, Willan A, Griffith LE. Determining a minimal important change in a disease-specific quality of life questionnaire. J Clin Epidemiol. 1994;47:81–7.

    Article  PubMed  CAS  Google Scholar 

  29. Cohen J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988:19–27.

    Google Scholar 

  30. Jacobson NS, Roberts LJ, Berns SB, McGlinchey JB. Methods for defining and determining the clinical significance of treatment effects: description, application, and alternatives. J Consul Clin Psychol. 1999;67:300–307.

    Article  CAS  Google Scholar 

  31. Kendall PC, Marrs-Garcia A, Nath SR, Sheldrick RC. Normative comparisons for the evaluation of clinical significance. J Consult Clin Psychol. 1999;67:285–99.

    Article  PubMed  CAS  Google Scholar 

  32. Anonymous. Significance of significance. N Engl J Med. 1968;278:1232–3.

  33. Melton AW. Editorial. J Exp Psychol. 1962;64:553–7.

    Article  Google Scholar 

  34. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology. A Basic Science for Clinical Medicine, 2nd ed. Boston: Little, Brown and Company; 1991.

    Google Scholar 

  35. Freiman J, Chalmers TC, Smith H, Kaubler R. The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. N Engl J Med. 1978;299:690–4.

    Article  PubMed  CAS  Google Scholar 

  36. Cook DJ, Guyatt GL, Laupacis A, Sackett DL, Goldberg RJ. Clinical recommendations using levels of evidence for antithrombotic agents. Chest. 1995;108:227S-30S.

    PubMed  CAS  Google Scholar 

  37. Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ. Users’ guide to the medical literature IX. A method for grading health care recommendations. JAMA. 1995;274:1800–4.

    Article  PubMed  CAS  Google Scholar 

  38. Moher D, Schulz KF, Altman DG, for the CONSORT Group. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. Ann Intern Med. 2001;134:657–62.

    PubMed  CAS  Google Scholar 

  39. Mahon J, Laupacis A, Donner A, Wood T. Randomised study of n of 1 trials versus standard practice. BMJ. 1996;312:1069–74.

    PubMed  CAS  Google Scholar 

  40. Detsky AS, Sackett DL. When was a ‘negative’ clinical trial big enough. Arch Intern Med. 1985;145:709–12.

    Article  PubMed  CAS  Google Scholar 

  41. Jones B, Jarvis P, Lewis JA, Ebbutt AF. Trials to assess equivalence: importance of rigorous methods. BMJ. 1996;313:36–9.

    PubMed  CAS  Google Scholar 

  42. Dunnett CW, Gent M. Significance testing to establish equivalence between treatments, with special reference to data in the form of 2 × 2 tables. Biometrics. 1977;33:593–602.

    Article  PubMed  CAS  Google Scholar 

  43. Pitt B, Zannad F, Remme WJ, et al. The effect of spironolactone on morbidity and mortality in patients with severe heart failure. N Engl J Med. 1999;341:709–17.

    Article  PubMed  CAS  Google Scholar 

  44. Naylor MD, Broten D, Campbell R, et al. Comprehensive discharge planning and home follow-up of hospitalized elders. JAMA. 1999; 281:613–20.

    Article  PubMed  CAS  Google Scholar 

  45. Dalby DM, Sellors JW, Fraser FD, Fraser C, van Ineveld C, Howard M. Effect of preventive home visits by a nurse on the outcomes of frail elderly people in the community: a randomized controlled trial. CMAJ. 2000;62:497–500.

    Google Scholar 

  46. Silverstein FE, Graham DY, Senior JR, et al. Misoprostol reduces serous gastrointestinal complications in patients with rheumatoid arthritis receiving nonsteroidal anti-inflammatory drugs. Ann Intern Med. 1995;123:241–9.

    PubMed  CAS  Google Scholar 

  47. Dale JJ, Ruckley CV, Harper DR, Gibson B, Nelson EA, Prescott RJ. Randomised, double blind placebo controlled trial of pentoxifylline in the treatment of venous leg ulcers. BMJ. 1999;319:875–8.

    PubMed  CAS  Google Scholar 

  48. GISSI-Prevenzione Investigators. Dietary supplementation with n-3 polyunsaturated fatty acids and vitamin E after myocardial infarction: results of the GISSI-Prevenzione trial. Lancet. 1999; 354:447–55.

    Article  Google Scholar 

  49. Hulley S, Grady D, Bush T, et al. Randomized trial of estrogen for secondary prevention of coronary heart disease in postmenopausal women. Heart and Estrogen/progestin Replacement Study (HERS) Research Group. JAMA. 1998;280:605–13.

    Article  PubMed  CAS  Google Scholar 

  50. Pocock SJ. When to stop a clinical trial. BMJ. 1992;305:235–40.

    Article  PubMed  CAS  Google Scholar 

  51. Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC. Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med. 1992;327:248–54.

    Article  PubMed  CAS  Google Scholar 

  52. Guyatt GH, Sinclair J, Cook DJ, Glasziou P. Users’ guide to the medical literature XVI. How to use a treatment recommendation. JAMA. 1999;281:1836–43.

    Article  PubMed  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Malcolm Man-Son-Hing MD, MSc.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Man-Son-Hing, M., Laupacis, A., O’Rourke, K. et al. Determination of the clinical importance of study results. J GEN INTERN MED 17, 469–476 (2002). https://doi.org/10.1046/j.1525-1497.2002.11111.x

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1046/j.1525-1497.2002.11111.x

Key Words

Navigation