Hostname: page-component-848d4c4894-x5gtn Total loading time: 0 Render date: 2024-05-20T02:41:05.965Z Has data issue: false hasContentIssue false
  • English
  • Français

A call for renewed attention to construct validity and measurement in psychopathology research

Published online by Cambridge University Press:  21 October 2022

Elizabeth P. Hayden
Elizabeth P. Hayden*
Affiliation:
Department of Psychology, Western University, London, ON N6A 5C2, Canada
*
Author for correspondence: Elizabeth P. Hayden, E-mail: ehayden@uwo.ca
Get access Rights & Permissions [Opens in a new window]

Abstract

Psychopathologists have failed to make significant progress toward understanding the causes of psychopathology. Despite the foundational importance of construct validity and measurement to our field, insufficient attention is paid to these concerns in the assessment of psychopathology vulnerabilities prior to their implementation in causal models. I review the current state of construct validity and measurement in psychopathology research, highlighting the lack of consensus regarding how we should define and measure vulnerability constructs. The limited capacity of open science practices to address these definitional and measurement challenges is discussed. Recommendations for progress are made, including the need for consensus agreement on (1) working definitions and (2) measures of vulnerability constructs. Other recommendations include (3) the need to incentivize ‘pre-clinical’ descriptive work focused on measurement development, (4) the formation of open-access databases designed to facilitate measurement evaluation and development, and (5) increased exploration of the use of novel technologies to facilitate the collection of high-quality measures of vulnerability.

Keywords

Construct validityetiologymeasurementpsychiatric diagnosis
Type
Commentary
Information
Psychological Medicine , Volume 52 , Issue 14 , October 2022 , pp. 2930 - 2936
Check for updates
Copyright
Copyright © The Author(s), 2022. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: Joint Committee on Standards for Educational and Psychological Testing.Google Scholar
American Psychiatric Association. (1980). Diagnostic and statistical manual of mental disorders (3rd ed.). Washington, DC: American Psychiatric Publishing.Google Scholar
Berenbaum, H. (2013). Classification and psychopathology research. Journal of Abnormal Psychology, 122, 894–901.CrossRefGoogle ScholarPubMed
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept of validity. Psychological Review, 111(4), 1061–1071.CrossRefGoogle ScholarPubMed
Bullock, J. G., Green, D. P., & Ha, S. E. (2010). Yes, but what's the mechanism? (Don't expect an easy answer). Journal of Personality and Social Psychology, 98(4), 550–558.CrossRefGoogle ScholarPubMed
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105.CrossRefGoogle ScholarPubMed
Campos, J. J., Frankel, C. B., & Camras, L. (2004). On the nature of emotion regulation. Child Development, 75(2), 377–394.CrossRefGoogle ScholarPubMed
Cole, P. M., Martin, S. E., & Dennis, T. A. (2004). Emotion regulation as a scientific construct: Methodological challenges and directions for child development research. Child Development, 75(2), 317–333.CrossRefGoogle ScholarPubMed
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302.CrossRefGoogle ScholarPubMed
Durbin, C. E., & Wilson, S. (2012). Convergent validity of and bias in maternal reports of child emotion. Psychological Assessment, 24(3), 647–660.CrossRefGoogle ScholarPubMed
Epstein, S. (1983). Aggregation and beyond: Some basic issues on the prediction of behavior. Journal of Personality, 51(3), 360–392.CrossRefGoogle ScholarPubMed
Evans, T. C., & Britton, J. C. (2018). Improving the psychometric properties of dot-probe attention measures using response-based computation. Journal of Behavior Therapy and Experimental Psychiatry, 60, 95–103.CrossRefGoogle ScholarPubMed
Feighner, J. P., Robins, E., Guze, S. B., Woodruff, R. A. Jr., Winokur, G., & Munoz, R. (1972). Diagnostic criteria for use in psychiatric research. Archives of General Psychiatry, 26(1), 57–63.CrossRefGoogle ScholarPubMed
Flake, J. K., Davidson, I. J., Wong, O., & Pek, J. (2022). Construct validity and the validity of replication studies: A systematic review. The American Psychologist, 77(4), 576–588.CrossRefGoogle ScholarPubMed
Flake, J. K., & Fried, E. I. (2020). Measurement schmeasurement: Questionable measurement practices and how to avoid them. Advances in Methods and Practices in Psychological Science, 3(4), 456–465.CrossRefGoogle Scholar
Flake, J. K., Pek, J., & Hehman, E. (2017). Construct validation in social and personality research: Current practice and recommendations. Social Psychological and Personality Science, 8(4), 370–378.CrossRefGoogle Scholar
Fried, E. I., Flake, J. K., & Robinaugh, D. J. (2022). Revisiting the theoretical and methodological foundations of depression measurement. Nature Reviews Psychology, 1, 1–11.CrossRefGoogle Scholar
Harkness, K. L., & Monroe, S. M. (2016). The assessment and measurement of adult life stress: Basic premises, operational principles, and design requirements. Journal of Abnormal Psychology, 125(5), 727–745.CrossRefGoogle ScholarPubMed
Hayden, E. P., & Durbin, C. E. (2019). Development and psychopathology. In Ollendick, T. H., White, S. W., & White, B. A. (Eds.), The Oxford handbook of clinical child and adolescent psychology (pp. 31–41). New York, NY: Oxford University Press.Google Scholar
Hayden, E. P., & Harkness, K. L. (2020). Stress and mental health: Epilogue. In Harkness, K. L. & Hayden, E. P. (Eds.), The Oxford handbook of stress and mental health (pp. 725–730). New York, NY: Oxford University Press.Google Scholar
Howard, S. J., & Melhuish, E. (2017). An early years toolbox for assessing early executive function, language, self-regulation, and social development: Validity, reliability, and preliminary norms. Journal of Psychoeducational Assessment, 35(3), 255–275.CrossRefGoogle ScholarPubMed
Hyman, S. E. (2021). Psychiatric disorders: Grounded in human biology but not natural kinds. Perspectives in Biology and Medicine, 64, 6–28.CrossRefGoogle Scholar
Iddiols, B. C., Daoust, A. R., Mullen, J. N., & Hayden, E. P. (2022). Automated coding of children's emotional language predicts their internalizing symptoms and cortisol stress reactivity. [Poster Presentation]. Society for Research in Psychopathology Annual Meeting (8–11 September). Philadelphia, Pennsylvania.Google Scholar
Insel, T., Cuthbert, B., Garvey, M., Heinssen, R., Pine, D. S., Quinn, K., … Wang, P. (2010). Research domain criteria (RDoC): Toward a new classification framework for research on mental disorders. The American Journal of Psychiatry, 167(7), 748–751.CrossRefGoogle Scholar
Kendler, K. S. (2016). The phenomenology of major depression and the representativeness and nature of DSM criteria. The American Journal of Psychiatry, 173(8), 771–780.CrossRefGoogle ScholarPubMed
Kendler, K. S. (2017). DSM disorders and their criteria: How should they inter-relate? Psychological Medicine, 47(12), 2054–2060.CrossRefGoogle ScholarPubMed
Klein Haneveld, E., Molenaar, D., de Vogel, V., Smid, W., & Kamphuis, J. H. (2022). Do we hold males and females to the same standard? A measurement invariance study on the psychopathy checklist-revised. Journal of Personality Assessment, 104(3), 368–379.CrossRefGoogle Scholar
Kotov, R., Krueger, R. F., Watson, D., Achenbach, T. M., Althoff, R. R., Bagby, R. M., … Zimmerman, M. (2017). The hierarchical taxonomy of psychopathology (HiTOP): A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology, 126, 454–477.CrossRefGoogle ScholarPubMed
Lilienfeld, S. O., Smith, S. F., & Watts, A. L. (2014). Issues in diagnosis: Conceptual issues and controversies. In Craighead, E., Miklowitz, J., & Craighead, L. W. (Eds.), Psychopathology: History, diagnosis, and empirical foundations (pp. 1–35). New York: Wiley.Google Scholar
Lilienfeld, S. O., & Treadway, M. T. (2016). Clashing diagnostic approaches: DSM-ICD versus RDoC. Annual Review of Clinical Psychology, 12, 435–463.CrossRefGoogle ScholarPubMed
Loevinger, J. (1957). Objective tests as instruments of psychological theory. Psychological Reports, 3, 635–694.Google Scholar
Markus, K. A., & Borsboom, D. (2013). Frontiers of test validity theory: Measurement, causation, and meaning. Multivariate Applications Series, 341. Retrieved from https://psycnet.apa.org/fulltext/2013-15050-000.pdf.CrossRefGoogle Scholar
Messick, S. (1987). Validity. ETS Research Report Series, 1987(2), i–208.Google Scholar
Olino, T. M., McMakin, D. L., & Forbes, E. E. (2018). Toward an empirical multidimensional structure of anhedonia, reward sensitivity, and positive emotionality: An exploratory factor analytic study. Assessment, 25(6), 679–690.CrossRefGoogle ScholarPubMed
Parsons, S., Kruijt, A.-W., & Fox, E. (2019). Psychological science needs a standard practice of reporting the reliability of cognitive-behavioral measurements. Advances in Methods and Practices in Psychological Science, 2(4), 378–395.CrossRefGoogle Scholar
Peng, Y., Knotts, J. D., Taylor, C. T., Craske, M. G., Stein, M. B., Bookheimer, S., … Paulus, M. P. (2021). Failure to identify robust latent variables of positive or negative valence processing across units of analysis. Biological Psychiatry. Cognitive Neuroscience and Neuroimaging, 6(5), 518–526.CrossRefGoogle ScholarPubMed
Robinson, C. C., Mandleco, B., Olsen, S. F., & Hart, C. H. (1995). Authoritative, authoritarian, and permissive parenting practices: Development of a new measure. Psychological Reports, 77(3), 819–830.CrossRefGoogle Scholar
Roefs, A., Fried, E. I., Kindt, M., Martijn, C., Elzinga, B., Evers, A. W. M., … Jansen, A. (2022). A new science of mental disorders: Using personalised, transdiagnostic, dynamical systems to understand, model, diagnose and treat psychopathology. Behaviour Research and Therapy, 153, 104096.CrossRefGoogle ScholarPubMed
Sechrest, L. (2005). Validity of measures is no simple matter. Health Services Research, 40(5 Pt 2), 1584–1604.CrossRefGoogle ScholarPubMed
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366.CrossRefGoogle ScholarPubMed
Spitzer, R. L. (1983). Psychiatric diagnosis: Are clinicians still necessary? Comprehensive Psychiatry, 24(5), 399–411.CrossRefGoogle ScholarPubMed
Strauss, M. E., & Smith, G. T. (2009). Construct validity: Advances in theory and methodology. Annual Review of Clinical Psychology, 5, 1–25.CrossRefGoogle ScholarPubMed
Świątkowski, W., & Dompnier, B. (2017). Replicability crisis in social psychology: Looking at the past to find new pathways for the future. International Review of Social Psychology, 30(1), 111.CrossRefGoogle Scholar
Thompson, W. H., Wright, J., & Bissett, P. G. (2020). Open exploration. eLife, 9. https://doi.org/10.7554/eLife.52157.CrossRefGoogle ScholarPubMed
Westfall, J., & Yarkoni, T. (2016). Statistically controlling for confounding constructs is harder than you think. PLoS One, 11(3), e0152719.CrossRefGoogle ScholarPubMed