Should I allow my confirmatory factors to correlate during factor score extraction? Implications for the applied researcher

Abstract

With complex models becoming increasingly popular in the social sciences, many researchers have begun using latent variable modeling in multiple-steps, saving, estimating, or otherwise extracting factor scores from one confirmatory factor analysis (CFA) for use in a second inferential analysis. With two or more factors identified in a CFA, there exist few practical guidelines as to how researchers should proceed. In Study 1, we examine two common practices when CFAs have two or more factors: Fitting separate CFAs or allowing them to correlate in the model used for extraction. We provide a simulation study to demonstrate the bias introduced in each of the two approaches. In Study 2, we demonstrate that the between-factor correlation bias can be mitigated through the use of a different estimator; using ten Berge estimation shows near zero bias on the critical correlations between factors. Finally, we demonstrate this with an example dataset.

Appendices

Appendix 1

1.1 Data generation

The data generation model was based on Eq. 4 (data model) and Eq. 5 (implied covariance form), where $y_{i}$ is the p × 1 observed data vector for the ith observation and p = number of indicators, i.e. 8; ${\Sigma }$ is the implied covariance matrix for y; $\mu$ is the p × 1 intercept vector fixed to 0; ${\Lambda }$ is the p × m matrix of factor loadings (fixed), where m = number of factors (i.e. 2), $\eta_{i}$ is the m × 1 vector of “true” factor scores for the ith observation following a multivariate normal distribution, MVN $\left( {0,{\Phi }} \right)$, and $\epsilon_{i}$ is the p × 1 residual vector for the ith observation following a multivariate normal distribution, MVN $\left( {0,{\Psi }} \right)$. Note that ${\Phi }$ is a m × m matrix which represents the covariance matrix between the factors, and ${\Psi }$ is a p × p diagonal matrix representing the error covariance matrix, where the error variances are set up as such that the variance of y is 1.

$$y_{i} = \mu + \Lambda \eta_{i} + \upepsilon_{i}$$

(4)

$$\Sigma = \Lambda \Phi \Lambda^{^{\prime}} + \Psi$$

(5)

In addition, we also introduced an external variable z that correlated at 0.5 with factor 1 only. It is assumed that z is normally distributed with a variance of 1. The covariance matrix between factors and variable z is therefore

$$\Phi_{{\left( {m + 1} \right) \times \left( {m + 1} \right)}}^{*} = \left[ {\begin{array}{*{20}c} \Phi &\upgamma \\ {\upgamma ^{\prime } } & 1 \\ \end{array} } \right],$$

where ${\upgamma }^{\prime }$ = [0.5, 0, …, 0] with dimension of 1 × m.

For each replication, data were generated in the following steps: (1) a series of true factor scores (γ) as well as variables (z) were drawn randomly from the multivariate normal distribution MVN(0, ${\Phi }^{*}$), and a series of residuals $\upepsilon$were drawn from MVN(0, ${\Psi }$); and (2) the observed data matrix Y of dimension p x n (n = sample size) was calculated based on Eq. 4.

Appendix 2

2.1 Code to calculate ten Berge factor scores

Appendix 3

Means and standard deviations of between-factor correlation estimates used to convert to the bias estimates presented in Table 3.

True corr	n	Orthogonal extraction			Correlated extraction
		High	High mixed	Mixed	High	High mixed	Mixed
0	100	0 (0.10)	0 (0.10)	0 (0.10)	0 (0.13)	0 (0.14)	0 (0.15)
	250	0 (0.06)	0 (0.06)	0 (0.06)	0 (0.08)	0 (0.09)	0 (0.09)
	500	0 (0.04)	0 (0.04)	0 (0.04)	0 (0.06)	0 (0.06)	0 (0.07)
0.3	100	0.25 (0.09)	0.25 (0.09)	0.23 (0.10)	0.33 (0.12)	0.34 (0.13)	0.34 (0.14)
	250	0.27 (0.06)	0.25 (0.06)	0.23 (0.06)	0.34 (0.07)	0.34 (0.09)	0.35 (0.09)
	500	0.26 (0.04)	0.25 (0.04)	0.23 (0.04)	0.34 (0.05)	0.34 (0.06)	0.35 (0.06)
0.5	100	0.43 (0.08)	0.40 (0.09)	0.38 (0.09)	0.55 (0.10)	0.56 (0.11)	0.57 (0.13)
	250	0.44 (0.05)	0.41 (0.05)	0.39 (0.05)	0.55 (0.06)	0.57 (0.07)	0.59 (0.07)
	500	0.44 (0.04)	0.42 (0.04)	0.39 (0.04)	0.56 (0.04)	0.57 (0.05)	0.59 (0.05)
0.8	100	0.70 (0.05)	0.65 (0.06)	0.61 (0.07)	0.87 (0.05)	0.88 (0.06)	0.88 (0.06)
	250	0.70 (0.03)	0.66 (0.04)	0.63 (0.04)	0.87 (0.03)	0.88 (0.03)	0.89 (0.04)
	500	0.70 (0.02)	0.66 (0.02)	0.63 (0.03)	0.87 (0.02)	0.88 (0.02)	0.89 (0.03)

1. Standard deviations are in parentheses. For High = all factor loadings set to 0.80; HM = for factor 1, the loadings are all 0.80, and for factor 2 the loadings are 0.40, 0.40, 0.80, 0.80; Mixed = for factor 1 and factor 2, the loadings are 0.40, 0.40, 0.80, 0.80

Appendix 4

Mean estimates and standard deviations for the correlation between the extracted factor and an external variable (z), this information is presented as bias estimates in Table 4.

True Corr	n	Orthogonal extraction			Correlated extraction
		High	High mixed	Mixed	High	High mixed	Mixed
0	100	0.47 (0.08)	0.47 (0.08)	0.44 (0.08)	0.47 (0.08)	0.47 (0.08)	0.44 (0.08)
	250	0.47 (0.05)	0.46 (0.05)	0.44 (0.05)	0.47 (0.05)	0.46 (0.05)	0.44 (0.05)
	500	0.47 (0.03)	0.47 (0.04)	0.44 (0.04)	0.47 (0.03)	0.47 (0.04)	0.44 (0.04)
0.3	100	0.47 (0.08)	0.47 (0.08)	0.44 (0.08)	0.46 (0.08)	0.46 (0.08)	0.43 (0.08)
	250	0.46 (0.05)	0.46 (0.05)	0.44 (0.05)	0.46 (0.05)	0.46 (0.05)	0.44 (0.05)
	500	0.47 (0.04)	0.47 (0.04)	0.44 (0.04)	0.46 (0.04)	0.46 (0.04)	0.44 (0.04)
0.5	100	0.47 (0.08)	0.47 (0.08)	0.44 (0.08)	0.45 (0.08)	0.45 (0.08)	0.42 (0.09)
	250	0.47 (0.05)	0.47 (0.05)	0.44 (0.05)	0.45 (0.05)	0.45 (0.05)	0.42 (0.05)
	500	0.47 (0.04)	0.47 (0.04)	0.45 (0.04)	0.45 (0.04)	0.45 (0.04)	0.42 (0.04)
0.8	100	0.46 (0.08)	0.46 (0.08)	0.44 (0.08)	0.40 (0.09)	0.41 (0.09)	0.36 (0.10)
	250	0.47 (0.05)	0.47 (0.05)	0.44 (0.05)	0.40 (0.05)	0.41 (0.05)	0.36 (0.06)
	500	0.47 (0.04)	0.47 (0.03)	0.44 (0.04)	0.40 (0.04)	0.41 (0.04)	0.36 (0.04)

1. Standard deviations are in parentheses. The correlation between the factor and the external variable was set to 0.50 for all conditions