Multiple indicators: Internal consistency or no necessary relationship?

Conclusions

The preceding discussion demonstrates the importance of having an explicit measurement model before analyzing measures. It is not valid to make any blanket statement on whether or not indicators should correlate until we know what type of indicators they are. If they are effect-indicators that have “well-behaved” errors and are positive measures of a single latent variable, then the internal-consistency view is appropriate and positive correlations of the indicators should occur. If cause-indicators are used then the NNR view is correct; indicator intercorrelations may be positive, negative, or zero. Finally, in general MIMIC models, cause-indicators have NNR while effect-indicators should be positively related under the assumptions of the model. In general, a cause- or an effect-indicator may have any type of relation.

Given the dominance of the internal-consistency perspective these simple results have serious implications. The empirical practice of factor-analyzing items to determine which measures “hang together” makes little sense if some of the indicators are cause-indicators. Similarly, computing “item-total” (cf. Nunnally, 1978, pp. 279–287) correlations as a means to select items for an index is not valid if cause-indicators are present. It seems quite possible that a number of items (or indicators) have not been used in research because of their low or negative correlation with other indicators designed to measure the same concept. If some of these are cause-indicators, researchers may have unknowingly removed valid measures.

On the other hand, these findings are not an excuse to include any indicators of interest in a measure. Ideally, the researcher should decide in advance which are effect- and which are cause-indicators. On the basis of the assumed measurement model, the expected associations may be predicted and tested.

In sum, the advice of Blalock seems particularly appropriate: “One should be especially on guard against procedures that supposedly permit one to appraise the ‘validity’ of an indicator on the basis of magnitudes of correlation coefficients, without the benefit of a specific theoretical model” (Namboodiri et al., 1975, p. 600).