Likely change indexes do not always index likely change; moreover, there is no need for them

Excerpt

Peipert et al. [1] introduced the concept of “likely change index” (LCI) as a kind of reliable change index (RCI) [2] but with relaxed significance criteria. The RCI was defined as the change score, divided by √2 times the standard error of measurement (SEM). Essentially, the RCI represents a significance test of an individual change score by determining the probability (p) of observing that change score if in reality there were no change (the null hypothesis H₀). If the RCI exceeds the critical value 1.96, p is smaller than 0.05, H₀ is rejected and the change score is assumed to represent real change (the alternative hypothesis H₁). To get a change score threshold to classify patients as reliably changed or not, the authors used a transformation of the RCI, the coefficient of repeatability (CR), calculated as critical value * √2 * SEM. The disadvantage of the “CR at the 95% confidence” is well known: it tends to misclassify many patients who feel they have changed and indeed may truly have changed, just because the amount of measurement error in the change score prevents reliably distinguishing real change from chance fluctuation at smaller change scores. Because the authors felt that, for some applications, it may be acceptable to be less certain that a change threshold is differentiable from measurement error, they introduced “relaxed” CRs at reduced confidence levels: 68%, based on the critical value 0.994, and 50%, based on the critical value 0.674, referring to these CRs as LCIs. The expression “likely change index” seems to suggest that real change is “likely” instead of “reliably” as with the RCI. It should be noted, however, that the proposed LCIs imply significance testing at p-values of 0.32 and 0.50, respectively, instead of 0.05 (which seems a bit awkward). …