Probability, proof, and clinical significance

 

 Permissions and Reprints

A study reports that treatment A “provided significantly better pain relief” than treatment B. How do you know if the effect is real or due to chance? Assuming that the difference between treatment groups on the outcome is statistically significant, does this mean that your patients will have a clinically significant improvement? This article explores these questions as well as provides some additional points to consider when critically appraising and conducting research.

When a difference in an outcome (eg, pain) between exposures (eg, treatment groups) is observed, one needs to consider whether the effect is truly due to the exposure or if alternate explanations are possible. In other words, in order to evaluate the validity of a research study, factors that might distort the true association and/or influence its interpretation need to be carefully considered. This means evaluating the role of bias and considering the study’s statistical precision.

Bias relates to systematic sources of error which need to be considered. (Bias will be discussed in more detail in future issues). By contrast, evaluation of statistical precision involves consideration of random error within the study, random error being the part of the study that cannot be predicted, ie, that part attributable to chance. In addition, one needs to consider whether a clinically meaningful improvement is represented.