Skip to main content

Advertisement

SpringerLink
Log in

Empirical Correction to the Likelihood Ratio Statistic for Structural Equation Modeling with Many Variables

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

Survey data typically contain many variables. Structural equation modeling (SEM) is commonly used in analyzing such data. The most widely used statistic for evaluating the adequacy of a SEM model is T ML, a slight modification to the likelihood ratio statistic. Under normality assumption, T ML approximately follows a chi-square distribution when the number of observations (N) is large and the number of items or variables (p) is small. However, in practice, p can be rather large while N is always limited due to not having enough participants. Even with a relatively large N, empirical results show that T ML rejects the correct model too often when p is not too small. Various corrections to T ML have been proposed, but they are mostly heuristic. Following the principle of the Bartlett correction, this paper proposes an empirical approach to correct T ML so that the mean of the resulting statistic approximately equals the degrees of freedom of the nominal chi-square distribution. Results show that empirically corrected statistics follow the nominal chi-square distribution much more closely than previously proposed corrections to T ML, and they control type I errors reasonably well whenever N≥max(50,2p). The formulations of the empirically corrected statistics are further used to predict type I errors of T ML as reported in the literature, and they perform well.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1.
Figure 2.
Figure 3.
Figure 4.
Figure 5.
Figure 6.
Figure 7.

Similar content being viewed by others

Notes

  1. If the model possesses a hierarchical factor structure, m will be the number of factors at the first level only.

  2. In our experience, there exist systematic differences between converged and non-converged replications as well as between samples resulting in proper and improper solutions. This was also noted by Yuan and Hayashi (2003) in the context of bootstrap simulation.

  3. The convergence is defined as, within 300 iterations, the maximum difference for all elements of θ between two consecutive iterations is smaller than 0.0001.

  4. We also obtained the results of including p 2 as a predictor in (5). But the corresponding T MLe did not perform substantially better with respect to BIC. Actually, Bartlett correction for EFA also does not include the p 2 term.

References

  • Amemiya, Y., & Anderson, T.W. (1990). Asymptotic chi-square tests for a large class of factor analysis models. The Annals of Statistics, 18, 1453–1463.

    Article  Google Scholar 

  • Anderson, J.C., & Gerbing, D.W. (1984). The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood confirmatory factor analysis. Psychometrika, 49, 155–173.

    Article  Google Scholar 

  • Anderson, J.C., & Gerbing, D.W. (1988). Structural equation modeling in practice: a review and recommended two-step approach. Psychological Bulletin, 103, 411–423.

    Article  Google Scholar 

  • Barndorff-Nielsen, O.E., & Hall, P. (1988). On the level-error after Bartlett adjustment of the likelihood ratio statistic. Biometrika, 75, 374–378.

    Article  Google Scholar 

  • Bartlett, M.S. (1937). Properties of sufficiency and statistical tests. Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences, 160, 268–282.

    Article  Google Scholar 

  • Bartlett, M.S. (1951). The effect of standardization on a χ 2 approximation in factor analysis. Biometrika, 38, 337–344.

    Google Scholar 

  • Bartlett, M.S. (1954). A note on the multiplying factors in various χ 2 approximations. Journal of the Royal Statistical Society, Series B, 16, 296–298.

    Google Scholar 

  • Bentler, P.M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246.

    Article  PubMed  Google Scholar 

  • Bentler, P.M. (2008). EQS 6 structural equations program manual. Encino, CA: Multivariate Software.

  • Bentler, P.M., & Bonett, D.G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–606.

    Article  Google Scholar 

  • Bentler, P.M., & Yuan, K.-H. (1999). Structural equation modeling with small samples: test statistics. Multivariate Behavioral Research, 34, 181–197.

    Article  Google Scholar 

  • Beran, R. (1988). Prepivoting test statistics: a bootstrap view of asymptotic refinements. Journal of the American Statistical Association, 83, 687–697.

    Article  Google Scholar 

  • Bollen, K.A., & Stine, R. (1993). Bootstrapping goodness of fit measures in structural equation models. In K.A. Bollen & J.S. Long (Eds.), Testing structural equation models (pp. 111–135). Newbury Park: Sage.

    Google Scholar 

  • Boomsma, A. (1982). The robustness of LISREL against small sample sizes in factor analysis models. In K.G. Jöreskog & H. Wold (Eds.), Systems under indirect observation: causality, structure, prediction (Part I) (pp. 149–173). Amsterdam: North-Holland.

    Google Scholar 

  • Box, G.E.P. (1949). A general distribution theory for a class of likelihood criteria. Biometrika, 36, 317–346.

    Article  PubMed  Google Scholar 

  • Browne, M.W., MacCallum, R.C., Kim, C.-T., Andersen, B.L., & Glaser, R. (2002). When fit indices and residuals are incompatible. Psychological Methods, 7, 403–421.

    Article  PubMed Central  PubMed  Google Scholar 

  • Casella, G., & Berger, R.L. (2002). Statistical inference (2nd ed.). Pacific Grove: Duxbury Press.

    Google Scholar 

  • Cox, D.R., & Hinkley, D.V. (1974). Theoretical statistics. Boca Raton: Chapman and Hall.

    Book  Google Scholar 

  • Curran, P.J., West, S.G., & Finch, J.F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1, 16–29.

    Article  Google Scholar 

  • Efron, B., & Tibshirani, R.J. (1993). An introduction to the bootstrap. New York: Chapman & Hall.

    Book  Google Scholar 

  • Evans, M., Hastings, N., & Peacock, B. (2000). Statistical distributions (3rd ed.). New York: Wiley.

    Google Scholar 

  • Fan, X., Thompson, B., & Wang, L. (1999). Effects of sample size, estimation methods, and model specification on structural equation modeling fit indexes. Structural Equation Modeling, 6, 56–83.

    Article  Google Scholar 

  • Fouladi, R.T. (2000). Performance of modified test statistics in covariance and correlation structure analysis under conditions of multivariate nonnormality. Structural Equation Modeling, 7, 356–410.

    Article  Google Scholar 

  • Fujikoshi, Y. (2000). Transformations with improved chi-squared approximations. Journal of Multivariate Analysis, 72, 249–263.

    Article  Google Scholar 

  • Geweke, J.F., & Singleton, K.J. (1980). Interpreting the likelihood ratio statistic in factor models when sample size is small. Journal of the American Statistical Association, 75, 133–137.

    Article  Google Scholar 

  • Hall, P. (1992). The bootstrap and Edgeworth expansion. New York: Springer.

    Book  Google Scholar 

  • Herzog, W., & Boomsma, A. (2009). Small-sample robust estimators of noncentrality-based and incremental model fit. Structural Equation Modeling, 16, 1–27.

    Article  Google Scholar 

  • Herzog, W., Boomsma, A., & Reinecke, S. (2007). The model-size effect on traditional and modified tests of covariance structures. Structural Equation Modeling, 14, 361–390.

    Article  Google Scholar 

  • Holzinger, K.J., & Swineford, F. (1939). Supplementary educational monographs: Vol. 48. A study in factor analysis: the stability of a bi-factor solution. Chicago: University of Chicago Press.

    Google Scholar 

  • Hu, L.T., Bentler, P.M., & Kano, Y. (1992). Can test statistics in covariance structure analysis be trusted? Psychological Bulletin, 112, 351–362.

    Article  PubMed  Google Scholar 

  • Ichikawa, M., & Konishi, S. (1995). Application of the bootstrap methods in factor analysis. Psychometrika, 60, 77–93.

    Article  Google Scholar 

  • Jackson, D.L. (2001). Sample size and number of parameter estimates in maximum likelihood confirmatory factor analysis: a Monte Carlo investigation. Structural Equation Modeling, 8, 205–223.

    Article  Google Scholar 

  • Jöreskog, K.G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183–202.

    Article  Google Scholar 

  • Kano, Y. (1992). Robust statistics for test-of-independence and related structural models. Statistics & Probability Letters, 15, 21–26.

    Article  Google Scholar 

  • Lawley, D.N. (1956). A general method for approximating to the distribution of likelihood ratio criteria. Biometrika, 43, 295–303.

    Article  Google Scholar 

  • Lawley, D.N., & Maxwell, A.E. (1971). Factor analysis as a statistical method (2nd ed.). New York: Elsevier.

    Google Scholar 

  • MacCallum, R.C., & Austin, J.T. (2000). Applications of structural equation modeling in psychological research. Annual Review of Psychology, 51, 201–226.

    Article  PubMed  Google Scholar 

  • Mardia, K.V. (1970). Measure of multivariate skewness and kurtosis with applications. Biometrika, 57, 519–530.

    Article  Google Scholar 

  • McDonald, R.P., & Marsh, H.W. (1990). Choosing a multivariate model: noncentrality and goodness of fit. Psychological Bulletin, 107, 247–255.

    Article  Google Scholar 

  • Mooijaart, A., & Bentler, P.M. (1991). Robustness of normal theory statistics in structural equation models. Statistica Neerlandica, 45, 159–171.

    Article  Google Scholar 

  • Moshagen, M. (2012). The model size effect in SEM: inflated goodness-of-fit statistics are due to the size of the covariance matrix. Structural Equation Modeling, 19, 86–98.

    Article  Google Scholar 

  • Nevitt, J., & Hancock, G.R. (2004). Evaluating small sample approaches for model test statistics in structural equation modeling. Multivariate Behavioral Research, 39, 439–478.

    Article  Google Scholar 

  • Ogasawara, H. (2010). Asymptotic expansions of the null distributions of discrepancy functions for general covariance structures under nonnormality. American Journal of Mathematical and Management Sciences, 30, 385–422.

    Article  Google Scholar 

  • Okada, K., Hoshino, T., & Shigemasu, K. (2007). Bartlett correction of test statistics in structural equation modeling. Japanese Journal of Educational Psychology, 55, 382–392 (in Japanese).

    Article  Google Scholar 

  • Satorra, A., & Bentler, P.M. (1990). Model conditions for asymptotic robustness in the analysis of linear relations. Computational Statistics & Data Analysis, 10, 235–249.

    Article  Google Scholar 

  • Satorra, A., & Bentler, P.M. (1994). Corrections to test statistics and standard errors in covariance structure analysis. In A. von Eye & C.C. Clogg (Eds.), Latent variables analysis: applications for developmental research (pp. 399–419). Thousand Oaks: Sage.

    Google Scholar 

  • Schott, J.R. (2007). A test for the equality of covariance matrices when the dimension is large relative to the sample size. Computational Statistics & Data Analysis, 51, 6535–6542.

    Article  Google Scholar 

  • Sörbom, D. (1989). Model modification. Psychometrika, 54, 371–384.

    Article  Google Scholar 

  • Srivastava, M.S., & Yanagihara, H. (2010). Testing the equality of several covariance matrices with fewer observations than the dimension. Journal of Multivariate Analysis, 101, 1319–1329.

    Article  Google Scholar 

  • Swain, A.J. (1975). Analysis of parametric structures for variance matrices. Doctoral dissertation, University of Adelaide, Australia.

  • Wakaki, H., Eguchi, S., & Fujikoshi, Y. (1990). A class of tests for a general covariance structure. Journal of Multivariate Analysis, 32, 313–325.

    Article  Google Scholar 

  • White, H. (1982). Maximum likelihood estimation of misspecified models. Econometrica, 50, 1–25.

    Article  Google Scholar 

  • Yuan, K.-H. (2005). Fit indices versus test statistics. Multivariate Behavioral Research, 40, 115–148.

    Article  Google Scholar 

  • Yuan, K.-H., & Bentler, P.M. (1998). Normal theory based test statistics in structural equation modeling. British Journal of Mathematical & Statistical Psychology, 51, 289–309.

    Article  Google Scholar 

  • Yuan, K.-H., & Bentler, P.M. (1999a). On normal theory and associated test statistics in covariance structure analysis under two classes of nonnormal distributions. Statistica Sinica, 9, 831–853.

    Google Scholar 

  • Yuan, K.-H., & Bentler, P.M. (1999b). F-tests for mean and covariance structure analysis. Journal of Educational and Behavioral Statistics, 24, 225–243.

    Article  Google Scholar 

  • Yuan, K.-H., & Chan, W. (2002). Fitting structural equation models using estimating equations: a model segregation approach. British Journal of Mathematical & Statistical Psychology, 55, 41–62.

    Article  Google Scholar 

  • Yuan, K.-H., & Chan, W. (2008). Structural equation modeling with near singular covariance matrices. Computational Statistics & Data Analysis, 52, 4842–4858.

    Article  Google Scholar 

  • Yuan, K.-H., & Hayashi, K. (2003). Bootstrap approach to inference and power analysis based on three statistics for covariance structure models. British Journal of Mathematical & Statistical Psychology, 56, 93–110.

    Article  Google Scholar 

  • Yung, Y.-F., & Bentler, P.M. (1996). Bootstrapping techniques in analysis of mean and covariance structures. In G.A. Marcoulides & R.E. Schumacker (Eds.), Advanced structural equation modeling: techniques and issues (pp. 195–226). Hillsdale: Erlbaum.

    Google Scholar 

  • Yung, Y.-F., & Chan, W. (1999). Statistical analyses using bootstrapping: concepts and implementation. In R.H. Hoyle (Ed.), Statistical strategies for small sample research (pp. 81–105). Thousand Oaks: Sage.

    Google Scholar 

Download references

Acknowledgements

The research was supported in part by a grant from the National Natural Science Foundation of China (31271116), and in part by a grant from the Ministry of Education, Science, Sports, and Culture of Japan (22650058).

We would like to thank Drs. Peter Bentler, Yutaka Kano, and Haruhiko Ogasawara for comments on earlier versions of this manuscript.

Author information

Authors and Affiliations

  1. Department of Psychology, University of Notre Dame, Notre Dame, IN, 46556, USA

    Ke-Hai Yuan, Yubin Tian & Hirokazu Yanagihara

Authors
  1. Ke-Hai Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yubin Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hirokazu Yanagihara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ke-Hai Yuan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yuan, KH., Tian, Y. & Yanagihara, H. Empirical Correction to the Likelihood Ratio Statistic for Structural Equation Modeling with Many Variables. Psychometrika 80, 379–405 (2015). https://doi.org/10.1007/s11336-013-9386-5

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-013-9386-5

Key words

Search

Navigation