Applications of latent-variable models in educational psychology: The need for methodological-substantive synergies

https://doi.org/10.1016/j.cedpsych.2006.10.008Get rights and content

Abstract

As emphasized in the call for papers by Jonna Kulikowich and Gregory Hancock, the primary goal of this special issue of Contemporary Educational Psychology is to assemble a collection of illustrative empirical studies in educational psychology that utilize one or more state-of-the-art latent variable modeling procedures. Distinguishing these articles from those in leading measurement and statistical journals, Kulikowich and Hancock encouraged the submission of manuscripts from authors (or teams) with strong backgrounds in both latent variable methodology and substantive issues in educational psychology. In this respect, they sought to provide an outlet for articles representing a synergy between sophisticated methodology and meaningful substantive issues. Here we briefly review why we think that this is such an important undertaking; consider some of the problems that such undertakings must address; emphasize a construct validity approach to the interpretations based on such studies; and highlight some of the key issues in the particular articles in this special issue.

Section snippets

Methodological-substantive synergies

As quantitative educational psychologists, we live in exciting times. We are armed with a bevy of new and evolving quantitative tools to address a range of substantive and policy related questions, with statistical power and flexibility that was previously unimaginable. However, this power comes at a cost to the educational psychology research discipline. In order to remain current as researchers—or even informed consumers of this research—we must become conversant with an ever-increasing range

Brief history of the growth of the application of latent variable models

Historically, SEM procedures can be traced to Sewell Wright’s work in biology, Otis Duncan’s work in sociology, and early psychometric research by Joreskog, McDonald, and others, providing confirmatory approaches to factor analysis and a means to formally test the goodness of fit of a priori measurement and structural models based on latent variables inferred from multiple indictors (Hancock & Mueller, 2006; also see Wolfle, 1999, Wolfle, 2003). In particular, SEM approaches have combined tests

Construct validity approach to evaluating latent variable models: a multiple perspective approach

Particularly when educational psychology researchers are working on the cutting edge of both methodological and substantive areas of research, it is important to critically evaluate interpretations of the results from a construct validity approach. Here we briefly outline how we have applied this approach in the design and interpretation of results from our own research.

All constructs in educational psychology are hypothetical constructs—unobservable, theoretical abstractions—inferred

Selected issues in the application of latent variable models

In this section we briefly review some of the critical issues that face methodological researchers in the application of latent variable models emphasized in our own research programs. Because of length restrictions we cannot do justice to any of these topics, and there are many other issues that could—perhaps should—have been included.

Articles in the special issue of contemporary educational psychology

Because of space limitations we can only highlight selected features of the studies included in this special issue. Our comments are necessarily idiosyncratic to our own methodological and substantive interests, and are not intended to represent a comprehensive review of the articles. By way of introduction we note that the collection of articles—as identified in the original call for papers—represents an important contribution to methodological-substantive synergies in educational

Summary

Educational psychology research is experiencing an exciting period, stimulated in part by an explosion of new and evolving latent variable approaches. However, there have not been sufficient demonstrations of the usefulness of these new techniques in substantively meaningful studies. Particularly in an applied area of research like educational psychology, it is important to demonstrate the applicability of new statistical tools in a range of applications that demonstrate their superiority over

References (49)

  • L.S. Aiken et al.

    Graduate training in statistics, methodology, and measurement in psychology: A survey of PhD programs in North America

    American Psychologist

    (1990)
  • L.M. Collins et al.

    A comparison of inclusive and restrictive strategies in modern missing data procedures

    Psychological Methods

    (2001)
  • De Fraine, B. Van Damme, J., & Onghena, P. (2006). A longitudinal analysis of gender differences in academic...
  • A.J. Elliot

    Approach and avoidance motivation and achievement goals

    Educational Psychologist

    (1999)
  • C.K. Enders

    Analyzing structural equation models with missing data

  • S.J. Finney et al.

    Nonnormal and categorical data in structural equation modeling

  • J.W. Graham et al.

    Multiple imputation in multivariate research

  • S.L. Hershberger

    The growth of structural equation modeling: 1994–2001

    Structural Equation Modeling: A Multidisciplinary Journal

    (2003)
  • S.L. Hershberger

    The problem of equivalent structural models

  • R.H. Hoyle et al.

    Writing about structural equation models

  • L. Hu et al.

    Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification

    Psychological Methods

    (1998)
  • L. Hu et al.

    Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives

    Structural Equation Modeling: A Multidisciplinary Journal

    (1999)
  • S.R. Hutchinson et al.

    A review of methodological characteristics of research published in key journals in higher education: Implications for graduate research training

    Research in Higher Education

    (2004)
  • R.J.A. Little et al.

    Statistical analysis with missing data

    (1987)
  • H.W. Marsh

    The measurement of physical self-concept: A construct validation approach

  • Marsh, H. W. (in press). Application of confirmatory factor analysis and structural equation modeling in sport/exercise...
  • H.W. Marsh et al.

    Causal ordering of academic self-concept and achievement: Reanalysis of a pioneering study and revised recommendations

    Educational Psychologist

    (1999)
  • H.W. Marsh et al.

    Reciprocal effects of self-concept and performance from a multidimensional perspective: Beyond seductive pleasure and unidimensional perspectives

    Perspectives on Psychological Science

    (2006)
  • H.W. Marsh et al.

    Evaluation of the Big-Two-Factor Theory of academic motivation orientations: An evaluation of jingle-jangle fallacies

    Multivariate Behavioral Research

    (2003)
  • H.W. Marsh et al.

    Why multicollinearity matters: A re-examination of relations between self-efficacy, self-concept and achievement

    Journal of Educational Psychology

    (2004)
  • H.W. Marsh et al.

    Latent-variable models of multitrait-multimethod data

  • H.W. Marsh et al.

    Assessing goodness of fit: Is parsimony always desirable?

    Journal of Experimental Education

    (1996)
  • H.W. Marsh et al.

    Is parsimony always desirable: Response to Hoyle, Sivo & Willson, Markus, Mulaik, Tweedledee, Tweedledum, the Cheshire Cat, and others

    Journal of Experimental Education

    (1998)
  • Cited by (181)

    • Joint trajectories of extrinsic motivation and competence beliefs: A parallel-process growth mixture modeling approach

      2021, Contemporary Educational Psychology
      Citation Excerpt :

      Therefore, this work can add to existing knowledge about heterogeneity in students’ motivation trajectories in current literature, thereby providing a more nuanced view of students’ developmental motivation processes. Our investigation may be one of the representative demonstrations of substantive-methodological synergy—pursuing both cutting-edge methodological tools and substantively important issues (Marsh & Hau, 2007)—as parallel-process GMM allows us to provide new, important knowledge about heterogeneous developmental patterns of multiple motivational resources from a sample of students. To provide a more integrated understanding of motivational development, we used parallel-process GMM and identified multiple latent subgroups of growth patterns describing extrinsic motivation covaried with competence beliefs from grades 8 to 11, using a nationally representative sample of secondary school students in South Korea.

    View all citing articles on Scopus

    We note that authors of articles in this special issue were, in some cases, asked to revise their manuscripts in relation to our comments on their original draft papers. Because of this iterative process some of our comments in relation to these particular articles might be more relevant to earlier versions of the articles than the final published version. Even when this is the case, however, it is still instructive to present the original concern as a general issue that might be relevant to other researchers pursuing methodological-substantive synergies.

    View full text