On the misuse of manifest variables in the detection of measurement bias

Abstract

Measurement invariance (lack of bias) of a manifest variableY with respect to a latent variableW is defined as invariance of the conditional distribution ofY givenW over selected subpopulations. Invariance is commonly assessed by studying subpopulation differences in the conditional distribution ofY given a manifest variableZ, chosen to substitute forW. A unified treatment of conditions that may allow the detection of measurement bias using statistical procedures involving only observed or manifest variables is presented. Theorems are provided that give conditions for measurement invariance, and for invariance of the conditional distribution ofY givenZ. Additional theorems and examples explore the Bayes sufficiency ofZ, stochastic ordering inW, local independence ofY andZ, exponential families, and the reliability ofZ. It is shown that when Bayes sufficiency ofZ fails, the two forms of invariance will often not be equivalent in practice. Bayes sufficiency holds under Rasch model assumptions, and in long tests under certain conditions. It is concluded that bias detection procedures that rely strictly on observed variables are not in general diagnostic of measurement bias, or the lack of bias.