Abstract
The previous chapter focused on the need for latent constructs in the study of affective characteristics and the array of scaling and measurement issues that need to be addressed in order to justify their treatment as mathematical structures in affective instruments. In this chapter, we introduce the most fundamental concept in instrument development: validity. While validity has no single agreed definition in the larger scientific community, in the area of instrument development it does refer to a very specific evidence gathering process for inferences made about latent quantities of latent constructs. As a result, Chapter 3 can be seen as an extension of the latent construct operationism started in Chapter 2. This evidence gathering process is a recurring one with elements discussed also in Chapters 4, 5, and 6. The test content validity discussed in this chapter is part of a non-statistical judgmental process done by content experts before it is present to respondents. As a result, the chapter is devoted to the methods involved in helping instrument developers translate their conceptual understanding of the affective characteristic into a well-functioning operational construct. This chapter introduces the researchers to the most commonly employed approaches to the process of establishing validity through the test content
“Validity is simple; Validation can be difficult”
M. Kane 2009, p. 49
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
It should be noted that there is a re-emerging school of thought that suggests that validity is in fact a property of instruments (Borsboom et al. 2003, 2004). Proponents of this approach to validity argue that there is a “scientific realism” to the unobserved latent constructs that affective instruments are designed to measure (Hood 2009). In contrast, supporters of the “inferential” approach to validity advocated by Messick (1989) make no strong assumptions on the psychological reality of the constructs under investigation.
- 2.
An example of a content validation form from Baslanti and McCoach′s work on the CSAS appears in Appendix A.
References
American Educational Research Association (AERA), American Psychological Association (APA) and National Council on Measurement in Education (NCME). (1999). The standards for educational and psychological testing. Washington, DC: American Educational Research Association.
Beck, C. T., & Gable, R. K. (2001). Ensuring content validity: An illustration of the process. Journal of Nursing Measurement, 9(2), 201–215.
Beck, C. T. (1992). The lived experience of postpartum depression: A phenomenological study. Nursing Research, 41, 166–170.
Beck, C. T. (1993). Teetering on the edge: A substantive theory of postpartum depression. Nursing Research, 42, 42–48.
Beck, C. T. (1996). Postpartum depressed mothers’ experiences interacting with their children. Nursing Research, 45, 98–104.
Beck, C. T., & Gable, R. K. (2000). Postpartum depression screening scale: Development and psychometric testing. Nursing Research, 49, 272–282.
Beck, C. H., & Gable, R. K. (2002). Postpartum depression screening scale. Los Angeles: Western Psychological Services.
Borsboom, D., Mellenbergh, G. J., & Van Heerden, J. (2003). Validity and truth. In H. Yanai, A. Okada, K. Shingemasu, Y. Kano, & J. J. Meulman (Eds.), New developments in psychometrics: Proceedings of the international psychometrics society 2001 (pp. 321–328). Tokyo: Springer.
Borsboom, D., Mellenbergh, G. J., & Van Heerden, J. (2004). The concept of validity. Psychological Review, 111, 1061–1071.
Clark, L. A., & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309–319.
Colaizzi, P. (1978). Psychological research as the phenomenologist views it. In R. Valle & M. King (Eds.), Existential phenomenological alternative for psychology (pp. 48–71). New York: Oxford University Press.
Cronbach, L. J. (1971). Test validation. In R. L. Thomdike (Ed.), Educational measurement (2nd ed., pp. 443–507). Washington, DC: American Council on Education.
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302.
Haynes, S. N., Richard, D. C., & Kubany, E. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7(3), 238–247.
Hood, S. B. (2009). Validity in psychological testing and scientific realism. Theory and Psychology, 19(4), 451–473.
Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112(3), 527–535.
Kane, M. T. (2009). Validating the interpretations and uses of test scores. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions, and applications (pp. 39–64). Charlotte: Information Age Publishing.
Loevinger, J. (1957). Objective tests as instruments of psychological theory. Psychological Reports, 3, 635–694.
McKenzie, J. F., Wood, M. L., Kotecki, J. E., Clark, J. K., & Brey, R. A. (1999). Establishing content validity: Using qualitative and quantitative steps. American Journal of Health Behavior, 23, 311–318.
Messick, S. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher, 18(2), 5–11.
Messick, S. (1995). Validity of psychological assessment: Validation of inferences as scientific inquiry into score meaning. American Psychologist, 50, 741–749.
Popham, W. J. (1978). Criterion-reference measurement. Englewood Cliffs: Prentice Hall.
Sireci, S. G. (2009). Packing and unpacking sources of validity evidence: History repeats itself again. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions, and applications (pp. 19–37). Charlotte: Information Age Publishing.
Vogt, D. S., King, D. W., & King, L. A. (2004). Focus groups in psychological assessment: Enhancing content validity by consulting members of the target population. Psychological Assessment, 16(3), 231–243.
Waltz, C., Strickland, O., & Lenz, E. (1991). Measurement in nursing research (2nd ed.). Philadelphia: F. A. Davis.
Wilson, H. S., & Hutchinson, S. A. (1991). Triangulation of qualitative methods: Heideggerian hermeneutics and grounded theory. Qualitative Health Research, 1(2), 263–276.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2013 Springer Science+Business Media New York
About this chapter
Cite this chapter
McCoach, D.B., Gable, R.K., Madura, J.P. (2013). Evidence Based on Test Content. In: Instrument Development in the Affective Domain. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-7135-6_3
Download citation
DOI: https://doi.org/10.1007/978-1-4614-7135-6_3
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-7134-9
Online ISBN: 978-1-4614-7135-6
eBook Packages: Behavioral ScienceBehavioral Science and Psychology (R0)