Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Multistudy Report

Multimethod Assessement of Time-Stable and Time-Variable Interindividual Differences

Introduction of a New Multitrait-Multimethod Latent State-Trait IRT Model

Published Online:July 06, 2020https://doi.org/10.1027/1015-5759/a000577

Abstract

Abstract. The dynamic development of interindividual differences and the temporal interplay between different personality constructs are of major interest to many researchers in the field of personality psychology. Furthermore, the collection of multiple rater-perspectives complementing classical self-report measures in psychological assessment is increasingly applied also in longitudinal research. Nevertheless, models to analyze longitudinal multitrait-multimethod (MTMM) data are scarce. A new Latent State-Trait (LST) Graded Response Model for the analysis of longitudinal MTMM data with ordered categorical response variables is introduced. The model combines advantages of LST theory and MTMM models for different types of raters (interchangeable and structurally different) with an Item Response Theory (IRT) approach. The model allows researchers to analyze the stability and variability of personality constructs, discriminant and convergent validity, as well as rater effects on the item-level. Model application and interpretation are illustrated using subjective well-being data of young adults. Results of an extensive simulation study indicate that the model can be accurately estimated with Bayesian statistics with at least 3 measurement occasions and more than 250 target persons rated by at least 5 interchangeable raters under moderate degrees of convergent validity.

References

  • Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105. https://doi.org/10.1037/h0046016 First citation in articleCrossref, Google Scholar

  • Carretero-Dios, H., Eid, M., & Ruch, W. (2011). Analyzing multitrait-mulitmethod data with multilevel confirmatory factor analysis: An application to the validation of the state-trait cheerfulness inventory. Journal of Research in Personality, 45, 153–164. https://doi.org/10.1016/j.jrp.2010.12.007 First citation in articleCrossref, Google Scholar

  • Cole, D. A., Martin, N. C., & Steiger, J. H. (2005). Empirical and conceptual problems with longitudinal trait-state models: Introducing a trait-state-occasion model. Psychological Methods, 10, 3–20. https://doi.org/10.1037/1082-989X.10.1.3 First citation in articleCrossref, Google Scholar

  • Courvoisier, D. S., Nussbeck, F. W., Eid, M., Geiser, C., & Cole, D. A. (2008). Analyzing the convergent and discriminant validity of states and traits: Development and applications of multimethod latent state-trait models. Psychological Assessment, 20, 270–280. https://doi.org/10.1037/a0012812 First citation in articleCrossref, Google Scholar

  • Crayen, C., Geiser, C., Scheithauer, H., & Eid, M. (2011). Evaluating interventions with multimethod data: A structural equation modeling approach. Structural Equation Modeling, 18, 497–524. https://doi.org/10.1080/10705511.2011.607068 First citation in articleCrossref, Google Scholar

  • Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The Satisfaction with Life Scale. Journal of Personality Assessment, 49, 71–75. https://doi.org/10.1207/s15327752jpa4901_13 First citation in articleCrossref, Google Scholar

  • Eid, M. (1996). Longitudinal confirmatory factor analysis for polytomous item responses: Model definition and model selection on the basis of stochastic measurement theory. Methods of Psychological Research Online, 1, 65–85. First citation in articleGoogle Scholar

  • Eid, M. (2000). A multitrait-multimethod model with minimal assumptions. Psychometrika, 65, 241–261. https://doi.org/10.1007/BF02294377 First citation in articleCrossref, Google Scholar

  • Eid, M., & Diener, E. (2004). Global judgments of subjective well-being: Situational variability and long-term stability. Social Indicators Research, 65, 245–277. https://doi.org/10.1023/B:SOCI.0000003801.89195.bc First citation in articleCrossref, Google Scholar

  • Eid, M., Geiser, C., & Koch, T. (2016). Measuring method effects from traditional to design-oriented approaches. Current Directions in Psychological Science, 25, 275–280. https://doi.org/10.1177/0963721416649624 First citation in articleCrossref, Google Scholar

  • Eid, M., & Hoffmann, L. (1998). Measuring variability and change with an item response model for polytomous variables. Journal of Educational and Behavioral Statistics, 23, 193–215. https://doi.org/10.3102/10769986023003193 First citation in articleCrossref, Google Scholar

  • Eid, M., Holtmann, J., Santangelo, P., & Ebner-Priemer, U. (2017). On the definition of latent-state-trait models with autoregressive effects: Insights from LST-R theory. European Journal of Psychological Assessment, 33, 285–295. https://doi.org/10.1027/1015-5759/a000435 First citation in articleLink, Google Scholar

  • Eid, M., & Langeheine, R. (2003). Separating stable from variable individuals in longitudinal studies by mixture distribution models. Measurement: Interdisciplinary Research and Perspectives, 1, 179–206. https://doi.org/10.1207/S15366359MEA0103_01 First citation in articleCrossref, Google Scholar

  • Eid, M., Lischetzke, T., Nussbeck, F., & Trierweiler, L. I. (2003). Separating trait effects from trait-specific method effects in multitrait-multimethod models: A multiple-indicator CT-C(M-1) model. Psychological Methods, 8, 38–60. https://doi.org/10.1037/1082-989X.8.1.38 First citation in articleCrossref, Google Scholar

  • Eid, M., Nussbeck, F. W., Geiser, C., Cole, D. A., Gollwitzer, M., & Lischetzke, T. (2008). Structural equation modeling of multitrait-multimethod data: Different models for different types of methods. Psychological Methods, 13, 230–253. https://doi.org/10.1037/a0013219 First citation in articleCrossref, Google Scholar

  • Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9, 466–491. https://doi.org/10.1037/1082-989X.9.4.466 First citation in articleCrossref, Google Scholar

  • Geiser, C., Eid, M., & Nussbeck, F. W. (2008). On the meaning of the latent variables in the CT-C(M-1) model: A comment on Maydeu-Olivares and Coffman (2006). Psychological Methods, 13, 49–57. https://doi.org/10.1037/1082-989X.13.1.49 First citation in articleCrossref, Google Scholar

  • Geiser, C., Eid, M., West, S. G., Lischetzke, T., & Nussbeck, F. W. (2012). A comparison of method effects in two confirmatory factor models for structurally different methods. Structural Equation Modeling, 19, 409–436. https://doi.org/10.1080/10705511.2012.687658 First citation in articleCrossref, Google Scholar

  • Geiser, C., & Lockhart, G. (2012). A comparison of four approaches to account for method effects in latent state-trait analyses. Psychological Methods, 17, 255–283. https://doi.org/10.1037/a0026977 First citation in articleCrossref, Google Scholar

  • Glaesmer, H., Grande, G., Braehler, E., & Roth, M. (2011). The German version of the Satisfaction with Life Scale (SWLS). European Journal of Psychological Assessment, 27, 127–132. https://doi.org/10.1027/1015-5759/a000058 First citation in articleLink, Google Scholar

  • Hamaker, E. L., Nesselroade, J. R., & Molenaar, P. C. (2007). The integrated trait–state model. Journal of Research in Personality, 41, 295–315. https://doi.org/10.1016/j.jrp.2006.04.003 First citation in articleCrossref, Google Scholar

  • Holtmann, J. (2017). Multilevel graded response models for analyzing longitudinal multitrait-multimethod data (Unpublished doctoral dissertation, Freie Universität Berlin, Germany). Retrieved from https://refubium.fu-berlin.de/handle/fub188/6457 First citation in articleGoogle Scholar

  • Holtmann, J., Koch, T., Bohn, J., & Eid, M. (2017). Bayesian analysis of longitudinal mulitrait-multimethod data with ordinal response variables. British Journal of Mathematical and Statistical Psychology, 70, 42–80. https://doi.org/10.1111/bmsp.12081 First citation in articleCrossref, Google Scholar

  • Holtmann, J., Koch, T., Lochner, K., & Eid, M. (2016). A comparison of ML, WLSMV, and Bayesian methods for multilevel structural equation models in small samples: A simulation study. Multivariate Behavioral Research, 51, 661–680. https://doi.org/10.1080/00273171.2016.1208074 First citation in articleCrossref, Google Scholar

  • Jeon, M., & Rijmen, F. (2014). Recent developments in maximum likelihood estimation of MTMM models for categorical data. Frontiers in Psychology, 5, 269. https://doi.org/10.3389/fpsyg.2014.00269 First citation in articleCrossref, Google Scholar

  • Kenny, D. A. (1995). The multitrait-multimethod matrix: Design, analysis, and conceptual issues. In P. E. ShroutS. T. FiskeEds., Personality research, methods, and theory: A festschrift honoring Donald W. Fiske (pp. 111–124). Hillsdale, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Koch, T., Eid, M., & Lochner, K. (2018). Multitrait-multimethod-analysis: The psychometric foundation of CFA-MTMM models. In P. IrwingT. BoothD. HughesEds., The Wiley handbook of psychometric testing (pp. 781–846). London, UK: Wiley. First citation in articleGoogle Scholar

  • Koch, T., Schultze, M., Holtmann, J., Geiser, C., & Eid, M. (2017). A multimethod latent state-trait model for structurally different and interchangeable methods. Psychometrika, 82, 17–47. https://doi.org/10.1007/s11336-016-9541-x First citation in articleCrossref, Google Scholar

  • Liang, X., & Yang, Y. (2014). An evaluation of WLSMV and Bayesian methods for confirmatory factor analysis with categorical indicators. International Journal of Quantitative Research in Education, 2, 17–38. https://doi.org/10.1504/IJQRE.2014.060972 First citation in articleCrossref, Google Scholar

  • Lischetzke, T., Eid, M., & Nussbeck, F. W. (2002, September). Unterschiedliche Definitionen von Methodeneffekten in MTMM Modellen und ihre Implikationen für die Analyse der Validität [Different definitions of method effects in MTMM models and their implications for validity analysis]. Paper presented at the 43rd meeting of the German Psychological Association, Berlin, Germany. First citation in articleGoogle Scholar

  • Luhmann, M., Bohn, J., Holtmann, J., Koch, T., & Eid, M. (2016). I’m lonely, can’t you tell? Convergent validity of self- and informant ratings of loneliness. Journal of Research in Personality, 61, 50–60. https://doi.org/10.1016/j.jrp.2016.02.002 First citation in articleCrossref, Google Scholar

  • Lyubomirsky, S., & Lepper, H. S. (1999). A measure of subjective happiness: Preliminary reliability and construct validation. Social Indicators Research, 46, 137–155. https://doi.org/10.1023/A:1006824100041 First citation in articleCrossref, Google Scholar

  • McNeish, D. (2019). Two-level dynamic structural equation models with small samples. Structural Equation Modeling, 26, 948–966. https://doi.org/10.1080/10705511.2019.1578657 First citation in articleCrossref, Google Scholar

  • Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525–543. https://doi.org/10.1007/BF02294825 First citation in articleCrossref, Google Scholar

  • Muthén, B. O., & Asparouhov, T. (2002). Latent variable analysis with categorical outcomes: Multiple-Group and growth modeling in Mplus, (Mplus Web Notes Nos. 4, Version 5). Retrieved from https://www.statmodel.com/download/webnotes/CatMGLong.pdf First citation in articleGoogle Scholar

  • Muthén, B. O., & Asparouhov, T. (2012). Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychological Methods, 17, 313–335. https://doi.org/10.1037/a0026802 First citation in articleCrossref, Google Scholar

  • Muthén, L. K., & Muthén, B. O. (1998–2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthén & Muthén. First citation in articleGoogle Scholar

  • Nussbeck, F. W., Eid, M., & Lischetzke, T. (2006). Analysing multitrait–multimethod data with structural equation models for ordinal variables applying the WLSMV estimator: What sample size is needed for valid results? British Journal of Mathematical and Statistical Psychology, 59, 195–213. https://doi.org/10.1348/000711005X67490 First citation in articleCrossref, Google Scholar

  • Pavot, W., & Diener, E. (1993). Review of the Satisfaction with Life Scale. Psychological Assessment, 5, 164–172. https://doi.org/10.1037/1040-3590.5.2.164 First citation in articleCrossref, Google Scholar

  • Pohl, S., & Steyer, R. (2010). Modeling common traits and method effects in multitrait-multimethod analysis. Multivariate Behavioral Research, 45, 45–72. https://doi.org/10.1080/00273170903504729 First citation in articleCrossref, Google Scholar

  • Pohl, S., Steyer, R., & Kraus, K. (2008). Modelling method effects as individual causal effects. Journal of the Royal Statistical Society: Series A, 171, 41–63. https://doi.org/10.1111/j.1467-985X.2007.00517.x First citation in articleGoogle Scholar

  • Rammstedt, B., & Beierlein, C. (2015). Can’t we make it any shorter? Journal of Individual Differences, 35, 212–220. https://doi.org/10.1027/1614-0001/a000141 First citation in articleLink, Google Scholar

  • Rhemtulla, M., Brosseau-Liard, P. E., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17, 354–373. https://doi.org/10.1037/a0029315 First citation in articleCrossref, Google Scholar

  • Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika, 34, 1–97. https://doi.org/10.1007/BF03372160 First citation in articleCrossref, Google Scholar

  • Schermelleh-Engel, K., Keith, N., Moosbrugger, H., & Hodapp, V. (2004). Decomposing person and occasion-specific effects: An extension of latent state-trait (LST) theory to hierarchical LST models. Psychological Methods, 9, 198–219. https://doi.org/10.1037/1082-989X.9.2.198 First citation in articleCrossref, Google Scholar

  • Scherpenzeel, A., & Saris, W. (2007). Multitrait-multimethod models for longitudinal research. In K. van MontfortJ. OudA. SatorraEds., Longitudinal models in the behavioral and related sciences (pp. 381–401). Mahwah, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Schneider, L., & Schimmack, U. (2009). Self-informant agreement in well-being ratings: A meta-analysis. Social Indicators Research, 94, 363–376. https://doi.org/10.1007/s11205-009-9440-y First citation in articleCrossref, Google Scholar

  • Steyer, R. (1989). Models of classical psychometric test theory as stochastic measurement models: Representation, uniqueness, meaningfulness, identifiability, and testability. Methodika, 3, 25–60. First citation in articleGoogle Scholar

  • Steyer, R., Ferring, D., & Schmitt, M. J. (1992). States and traits in psychological assessment. European Journal of Psychological Assessment, 8, 79–98. First citation in articleGoogle Scholar

  • Steyer, R., Mayer, A., Geiser, C., & Cole, D. A. (2015). A theory of states and traits – revised. Annual Review of Clinical Psychology, 11, 71–98. https://doi.org/10.1146/annurev-clinpsy-032813-153719 First citation in articleCrossref, Google Scholar

  • Steyer, R., & Nagel, W. (2017). Probability and conditional expectation: Fundamentals for the empirical sciences. Chichester, UK: Wiley. First citation in articleGoogle Scholar

  • Steyer, R., Schmitt, M., & Eid, M. (1999). Latent state-trait theory and research in personality and individual differences. European Journal of Personality, 13, 389–408. https://doi.org/10.1002/(SICI)1099-0984(199909/10)13:5<389::AID-PER361>3.0.CO;2-A First citation in articleCrossref, Google Scholar

  • Swami, V., Stieger, S., Voracek, M., Dressler, S. G., Eisma, L., & Furnham, A. (2009). Psychometric evaluation of the Tagalog and German Subjective Happiness Scales and a cross-cultural comparison. Social Indicators Research, 93, 393–406. https://doi.org/10.1007/s11205-008-9331-7 First citation in articleCrossref, Google Scholar

  • Takane, Y., & De Leeuw, J. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52, 393–408. https://doi.org/10.1007/BF02294363 First citation in articleCrossref, Google Scholar

  • Tov, W. (2018). Well-being concepts and components. In E. DienerS. OishiL. TayEds., Handbook of well-being (pp. 1–15). Salt Lake City, UT: DEF. Retrieved from https://nobascholar.com/ First citation in articleGoogle Scholar