When better is the enemy of good: two cautionary tales of conceptual validity versus parsimony in clinical psychometric research

This paper presents an empirical challenge to the assumption that an item-response theory analysis always yields a better measure of a clinical construct. We summarize results from two measurement development studies that showed that such an analysis lost important content reflecting the conceptual model (“conceptual validity”). The cost of parsimony may thus be too high. Conceptual models that form the foundation of QOL measurement reflect the patient’s experience. This experience may include concepts and items that are psychometrically “redundant” but capture distinct features of the concept. Good measurement is likely a balance between relying on IRT’s quantitative metrics and recognizing the importance of conceptual validity and clinical utility.