Skip to main content
SpringerLink
Log in

Validation Practices in Counseling: Major Journals, Mattering Instruments, and the Kuder Occupational Interest Survey (KOIS)

  • Chapter
  • First Online:
Validity and Validation in Social, Behavioral, and Health Sciences

Part of the book series: Social Indicators Research Series ((SINS,volume 54))

Abstract

Validity of measurement is an important concept in counseling. We present the results of three studies synthesizing the practices of validation in counseling, with an eye towards comparing the validation practices against the Standards for Educational and Psychological Testing (AERA, APA, NCME, Standards for educational and psychological testing. American Educational Research Association, Washington, DC, 1999). In Study 1, validation papers published in four major counseling journals in North America in 2009–2011 were synthesized. In Study 2, papers addressing the validation practices on the construct of mattering were summarized. In Study 3, the validity evidence of the Kuder Occupational Interest Survey (KOIS) was reviewed. Together, the three studies provide a broad lens into the validation practices in counseling. The results showed that validation practices were not in line with the current, modern view of validity. Implications for validation research, practice, and pedagogy/curriculum development in the counseling profession are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Note: The following notations are used to denote specific references

    • *1 indicates references from Study 1; additionally, references marked with # indicate studies included in the review

    • *2 indicates references from Study 2; additionally, references marked with a circle o indicate studies included in the review

    • *3 indicates references from Study 3; additionally, references marked with a square ⎕ indicate studies included in the review

References

Note: The following notations are used to denote specific references

  • *1 indicates references from Study 1; additionally, references marked with # indicate studies included in the review

  • *2 indicates references from Study 2; additionally, references marked with a circle o indicate studies included in the review

  • *3 indicates references from Study 3; additionally, references marked with a square ⎕ indicate studies included in the review

  • American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.*1 *2 *3

    Google Scholar 

  • Amundson, N. E. (1993). Mattering: A foundation for employment counseling and training. Journal of Employment Counseling, 30, 146–152.*2

    Google Scholar 

  • Anastasi, A. (1986). Evolving concepts of test validation. Annual Review of Psychology, 37, 1–15.

    Article  Google Scholar 

  • Auewarakul, C., Downing, S. M., Jaturatamrong, U., & Praditsuwan, R. (2005). Sources of validity evidence for an internal medicine student evaluation system: An evaluative study of assessment methods. Medical Education, 39, 276–283.*2 *3

    Google Scholar 

  • Bailey, T. K. M., Chung, Y. B., Williams, W. S., Singh, A. A., & Terrell, H. K. (2011). Development and validation of the Internalized Racial Oppression Scale for Black individuals. Journal of Counseling Psychology, 58(4), 481–493.*1 #

    Google Scholar 

  • Barofsky, I., Meadows, K., & McColl, E. (2003). Cognitive aspects of survey methodology and quality of life assessment: Summary of meeting. Quality of Life Research, 12, 281–282.*2

    Google Scholar 

  • Beckman, T. J., Cook, D. A., & Mandrekar, J. N. (2005). What is the validity evidence for assessments of clinical teaching. Journal of General Internal Medicine, 20, 1159–1164.*2 *3

    Google Scholar 

  • Bolt, D. M., & Rounds, J. (2000). Advances in psychometric theory and methods. In S. D. Brown & R. W. Lent (Eds.), Handbook of counseling psychology (3rd ed., pp. 140–176). Hoboken: Wiley.*1

    Google Scholar 

  • Brewster, M. E., & Moradi, B. (2010). Perceived experiences of anti-bisexual prejudice: Instrument development and evaluation. Journal of Counseling Psychology, 57(4), 451–468.*1 #

    Google Scholar 

  • Chan, E. K. H., & Zumbo, B. D. (2012, April). When validity theory meets validation practice: Research syntheses of validity evidence reported in seven areas. Symposium to be conducted at the Annual Convention of the American Educational Research Association, Vancouver, BC, Canada.*2

    Google Scholar 

  • Chan, E. K. H., Darmawanti, I., Mulyana, O. P., & Zumbo, B. D. (2011). Validity evidence and validation practice in papers published in Value in Health (1998–2010). Value in Health, 14, A151.*2

    Google Scholar 

  • Cizek, G. J., Rosenberg, S. L., & Koons, H. H. (2008). Sources of validity evidence for educational and psychological tests. Educational and Psychological Measurement, 68, 397–412.*1 *2 *3

    Google Scholar 

  • Cizek, G. J., Bowen, D., & Church, K. (2010). Sources of validity evidence for educational and psychological tests: A follow-up study. Educational and Psychological Measurement, 70, 732–743.*1 *2 *3

    Google Scholar 

  • Cobo, E., Cortés, J., Ribera, J. M., Cardellach, F., Selva-O’Callaghan, A., Kostov, B., et al. (2011). Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: Masked randomized trial. British Medical Journal, 343, d6783.

    Google Scholar 

  • Connolly, K. M., & Myers, J. E. (2002). Wellness and mattering: The role of holistic factors in job satisfaction. Journal of Employment Counseling, 40, 152–159.*2

    Google Scholar 

  • Corbiere, M., & Amundson, N. (2007). Perceptions of the ways of mattering by people with mental illness. The Career Development Quarterly, 56, 141–149.*2 o

    Google Scholar 

  • Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302.

    Article  Google Scholar 

  • Del Prado, A. M., & Church, A. T. (2010). Development and validation of the Enculturation Scale for Filipino Americans. Journal of Counseling Psychology, 57(4), 469–483.*1 #

    Google Scholar 

  • Denker, E. R., & Tittle, C. K. (1976). “ Reasonableness” of KOIS results for re-entry women: Implications for test validity. Educational and Psychological Measurement, 36(2), 495–500.*3 ⎕

    Google Scholar 

  • Diamond, E. E., & Zytowski, D. G. (2000). The kuder occupational interest survey. In C. E. Watkins & V. L. Campbell (Eds.), Testing and assessment in counselling practice (pp. 263–294). Mahwah: Lawrence Erlbaum Associates.

    Google Scholar 

  • Dixon Rayle, A. L. (2005). Adolescent gender differences in mattering and wellness. Journal of Adolescence, 28, 753–763.*2

    Google Scholar 

  • Dixon, A. L, Scheidegger, C., & McWhirter, J. J. (2009).The adolescent mattering experience: Gender variations in perceived mattering, anxiety, and depression. Journal of Counseling & Development, 87, 302–318.*2

    Google Scholar 

  • Einarsdóttir, S., Rounds, J., & Su, R. (2010). Holland in Iceland revisited: An emic approach to evaluating US vocational interest models. Journal of Counseling Psychology, 57(3), 361–367.*1 #

    Google Scholar 

  • Elliott, G. C. (2009). Family matters: The importance of mattering to family in adolescence. Chichester: Wiley-Blackwell.*2

    Google Scholar 

  • Elliott, G. C, Kao, S., & Grant, A. M. (2004). Mattering: Empirical validation of a social-psychological construct. Self and Identity, 3, 339–354.*2 o

    Google Scholar 

  • Elliott, G. C, Colangelo, M. R., & Gelles, R. J. (2005). Mattering and suicide ideation: Establishing and elaborating a relationship. Social Psychology Quarterly, 68, 223–238.*2

    Google Scholar 

  • Fan, J., Meng, H., Gao, X., Lopez, F. J., & Liu, C. (2010). Validation of a US Adult Social Self-Efficacy Inventory in Chinese Populations 1Ψ7. The Counseling Psychologist, 38(4), 473–496.*1 #

    Google Scholar 

  • France, M., & Finney, S. J. (2009). What matters in the measurement of mattering? A construct validity study. Measurement and Evaluation in Counseling and Development, 42, 104–120.*2 o

    Google Scholar 

  • France, M., & Finney, S. J. (2010). Conceptualization and utility of university mattering: A construct validity study. Measurement and Evaluation in Counseling and Development, 43, 48–65.*2 o

    Google Scholar 

  • Friedlander, M. L., Friedman, M. L., Miller, M. J., Ellis, M. V., Friedlander, L. K., & Mikhaylov, V. G. (2010). Introducing a brief measure of cultural and religious identification in American Jewish identity. Journal of Counseling Psychology, 57(3), 345–360.*1 #

    Google Scholar 

  • Gadermann, A. M., Guhn, M., & Zumbo, B. D. (2011). Investigating the substantive aspect of construct validity for the Satisfaction with Life Scale Adapted for Children: A focus on cognitive processes. Social Indicators Research, 100, 37–60.*1 *2

    Google Scholar 

  • Guirguis, L. M., & Chewning, B. A. (2008). Development of a measure to assess pharmacy students’ beliefs about monitoring chronic diseases. Research in Social and Administrative Pharmacy, 4, 402–416.*2 o

    Google Scholar 

  • Hansen, C. J., & Zytowski, D. G. (1979). The Kuder Occupational Interest inventory as a moderator of its predictive validity. Educational and Psychological Measurement, 39(1), 107–118.*3 ⎕

    Google Scholar 

  • Hubley, A. M., & Zumbo, B. D. (2011). Validity and the consequences of test interpretation and use. Social Indicators Research, 103, 219–230.*1 *2 *3

    Google Scholar 

  • Hubley, A. M., & Zumbo, B. D. (2013). Psychometric characteristics of assessment procedures: An overview. In K. F. Geisinger (Ed.), APA handbook of testing and assessment in psychology, Volume 1 (pp. 3–19). Washington, DC: American Psychological Association Press.*1 *2

    Google Scholar 

  • James, W. (1890). Principles of psychology. New York: Holt.*2

    Google Scholar 

  • Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 17–64). Westport: American Council on Education/Praeger.*1 *2 *3

    Google Scholar 

  • Kuder, F., Diamond, E. E., & Zytowski, D. G. (1998). Differentiation as fundamental validity for criterion-group scaled interest inventories. Educational and Psychological Measurement, 58(1), 38–41.*3 ⎕

    Google Scholar 

  • Lee, D. G., & Park, H. J. (2011). Cross-cultural validity of the frost multidimensional perfectionism scale in Korea. The Counseling Psychologist, 39(2), 320–345.*1 #

    Google Scholar 

  • Locke, B. D., Buzolitz, J. S., Lei, P. W., Boswell, J. F., McAleavey, A. A., Sevig, T. D., Dowis, J.D., & Hayes, J. A. (2011). Development of the Counseling Center Assessment of Psychological Symptoms-62 (CCAPS-62). Journal of Counseling Psychology, 58(1), 97–109.*1 #

    Google Scholar 

  • Loevinger, J. (1957). Objective tests as instruments of psychological theory. Psychological Reports, 3, 635–694.*2

    Google Scholar 

  • Mak, L., & Marshall, S. K. (2004). Perceived mattering in young adults’ romantic relationships. Journal of Social and Personal Relationships, 21, 469–486.*2 o

    Google Scholar 

  • Marcus, F. M. (1991). Mattering: Its measurement and theoretical significance. Unpublished manuscript.*2

    Google Scholar 

  • Marshall, S. K. (2001). Do I matter? Construct validation of adolescents’ perceived mattering to parents and friends. Journal of Adolescence, 24, 473–490.*2 o

    Google Scholar 

  • Meier, S. T., & Davis, S. R. (1990). Trends in reporting psychometric properties of scales used in counseling psychology research. Journal of Counseling Psychology, 37, 113–115.*1

    Google Scholar 

  • Mercer, S. H., Zeigler-Hill, V., Wallace, M., & Hayes, D. M. (2011). Development and initial validation of the Inventory of Microaggressions Against Black Individuals. Journal of Counseling Psychology, 58(4), 457–469.*1 #

    Google Scholar 

  • Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement, (3rd ed., pp. 13–103). New York: American Council on Education and Macmillan.*1 *2 *3

    Google Scholar 

  • Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749.*1 *2 *3

    Google Scholar 

  • Miller, M. J. (2010). Testing a bilinear domain-specific model of acculturation and enculturation across generational status. Journal of Counseling Psychology, 57(2), 179–186.*1 #

    Google Scholar 

  • Miller, M. J., & Sendrowitz, K. (2011). Counseling psychology trainees’ social justice interest and commitment. Journal of Counseling Psychology, 58(2), 159–169.*1 #

    Google Scholar 

  • Mohr, J. J., & Kendra, M. S. (2011). Revision and extension of a multidimensional measure of sexual minority identity: The Lesbian, Gay, and Bisexual Identity Scale. Journal of Counseling Psychology, 58(2), 234–245.*1 #

    Google Scholar 

  • Myers, J. E., & Bechtel, A. (2004). Stress, wellness, and mattering among cadets at West Point: Factors affecting a fit and healthy force. Military Medicine, 169, 475–482.*2

    Google Scholar 

  • Nadal, K. L. (2011). The Racial and Ethnic Microaggressions Scale (REMS): Construction, reliability, and validity. Journal of Counseling Psychology, 58(4), 470–480.*1 #

    Google Scholar 

  • Nafziger, D. H., & Helms, S. T. (1974). Cluster analyses of interest inventory scales as tests of Holland’s occupational classification. Journal of Applied Psychology, 59(3), 344–353.*3 ⎕

    Google Scholar 

  • Nugent, F. A., & Jones, K. D. (2005). Introduction to the profession of counseling (4th ed.). Hoboken: Wiley.*1

    Google Scholar 

  • Pinterits, E. J., Poteat, V. P., & Spanierman, L. B. (2009). The White Privilege Attitudes Scale: Development and initial validation. Journal of Counseling Psychology, 56(3), 417–429.*1 #

    Google Scholar 

  • Powers, A., Myers, J., Tingle, L., & Powers, J. (2004). Wellness, perceived stress, mattering, and marital satisfaction among first year medical residents and their spouses: Implications for education and counseling. The Family Journal, 12, 26–36.*2

    Google Scholar 

  • Reeve, B. B., Willis, G., Shariff-Marco, S. N., Breen, N., Williams, D. R., & Gee, G. C., et al. (2011). Comparing cognitive interviewing and psychometric methods to evaluate a racial/ethnic discrimination scale. Field Methods, 23, 397–419.*2

    Google Scholar 

  • Rosenberg, M. (1989). Society and the adolescent self-image (rev. ed.). Middletown: Wesleyan University Press.*2

    Google Scholar 

  • Rosenberg, M., & McCullough, B. C. (1981). Mattering: Inferred significance and mental health among adolescents. Research in Community Mental Health, 2, 163–182.*2

    Google Scholar 

  • Rottinghaus, P. J., Coon, K. L., Gaffey, A. R., & Zytowski, D. G. (2007). Thirty-year stability and predictive validity of vocational interests. Journal of Career Assessment, 15(1), 5–22.*3 ⎕

    Google Scholar 

  • Salahuddin, N. M., & O’Brien, K. M. (2011). Challenges and resilience in the lives of urban, multiracial adults: An instrument development study. Journal of Counseling Psychology, 58(4), 494–507.*1 #

    Google Scholar 

  • Savickas, M. L., Taber, B. J., & Spokane, A. R. (2002). Convergent and discriminant validity of five interest inventories. Journal of Vocational Behavior, 61(1), 139–184.*3 ⎕

    Google Scholar 

  • Schlossberg, N. K. (1990). The mattering scales for adult students in postsecondary education. Center for Adult Learning and Educational Credentials. Washington, DC: American Council on Education.

    Google Scholar 

  • Schwartz, S. J., & Finley, G. E. (2010). Troubled ruminations about parents: Conceptualization and validation with emerging adults. Journal of Counseling & Development, 88(1), 80–91.*1 #

    Google Scholar 

  • Sifford, A., Ng, K. M., & Wang, C. (2009). Further validation of the Psychosocial Costs of Racism to Whites Scale on a sample of university students in the southeastern United States. Journal of Counseling Psychology, 56(4), 585–589.*1 #

    Google Scholar 

  • Strong, E. K. (1943). Vocational interests of men and women. Stanford: Stanford University Press.*3

    Google Scholar 

  • Tovar, E., Simon, M. A., & Lee, H. B. (2009). Development and validation of the college mattering inventory with diverse urban college students. Measurement and Evaluation in Counseling and Development, 42, 154–178.*2 o

    Google Scholar 

  • Vogel, D. L., Wade, N. G., & Ascheman, P. L. (2009). Measuring perceptions of stigmatization by others for seeking psychological help: Reliability and validity of a new stigma scale with college students. Journal of Counseling Psychology, 56(2), 301–308.*1 #

    Google Scholar 

  • Wang, K. T., Yuen, M., & Slaney, R. B. (2009). Perfectionism, depression, loneliness, and life satisfaction a study of high school Students in Hong Kong. The Counseling Psychologist, 37(2), 249–274.*1 #

    Google Scholar 

  • Wei, M., Alvarez, A. N., Ku, T. Y., Russell, D. W., & Bonett, D. G. (2010). Development and validation of a Coping with Discrimination Scale: Factor structure, reliability, and validity. Journal of Counseling Psychology, 57(3), 328–344.*1 #

    Google Scholar 

  • Yoo, H. C., Burrola, K. S., & Steger, M. F. (2010). A preliminary report on a new measure: Internalization of the Model Minority Myth Measure (IM-4) and its psychological correlates among Asian American college students. Journal of Counseling Psychology, 57(1), 114–127.*1 #

    Google Scholar 

  • Zumbo, B. D. (2007). Validity: Foundational issues and statistical methodology. In C. R. Rao & S. Sinharay (Eds.), Psychometrics, Handbook of Statistics (Vol. 26, pp. 45–79). Amsterdam: Elsevier Science B.V.*1 *2 *3

    Google Scholar 

  • Zumbo, B. D. (2009). Validity as contextualized and pragmatic explanation, and its implications for validation practice. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions and applications (pp. 65–82). Charlotte: Information Age Publishing.*1 *2 *3

    Google Scholar 

  • Zytowski, D. G. (1972a). A concurrent test of accuracy-of-classification for the Strong Vocational Interest and Kuder Occupational Interest Survey. Journal of Vocational Behavior, 2(3), 245–250.*3 ⎕

    Google Scholar 

  • Zytowski, D. G. (1972b). Equivalence of the Kuder Occupational Interest Survey and the Strong Vocational Interest Blank revisited. Journal of Applied Psychology, 56(2), 184–185.*3 ⎕

    Google Scholar 

  • Zytowski, D. G. (1976). Predictive validity of the Kuder Occupational Interest Survey: A 12-to 19-year follow-up. Journal of Counseling Psychology, 23(3), 221–233.*3 ⎕

    Google Scholar 

  • Zytowski, D. G. (1992). Three generations: The continuing evolution of Frederic Kuder’s interest inventories. Journal of Counselling Development, 71(2), 245–248.*3 ⎕

    Google Scholar 

  • Zytowski, D. G., & Laing, J. (1978). Validity of other-gender-normed scales on the Kuder Occupational Interest Survey. Journal of Counseling Psychology, 25(3), 205–209.*3 ⎕

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Measurement, Evaluation, and Research Methodology (MERM) Program, Department of Educational and Counseling Psychology, and Special Education, The University of British Columbia, 2125 Main Mall, Vancouver, BC, V6T 1Z4, Canada

    Eric K. H. Chan Ph.D., Bruno D. Zumbo Ph.D. & Neelam Ark

  2. Counseling Psychology Program, Department of Educational and Counseling Psychology, and Special Education, The University of British Columbia, 2125 Main Mall, Vancouver, BC, V6T 1Z4, Canada

    David W. Munro, Alexander H. S. Huang & Roya Vojdanijahromi

Authors
  1. Eric K. H. Chan Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David W. Munro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander H. S. Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bruno D. Zumbo Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Roya Vojdanijahromi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Neelam Ark
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eric K. H. Chan Ph.D. .

Editor information

Editors and Affiliations

  1. Measurement, Evaluation, and Research Methodology (MERM) Program, Department of Educational and Counseling, Psychology, and Special Education, The University of British Columbia, Vancouver, British Columbia, Canada

    Bruno D. Zumbo

  2. Measurement, Evaluation, and Research Methodology (MERM) Program Department of Educational and Counseling, Psychology, and Special Education, The University of British Columbia, Vancouver, British Columbia, Canada

    Eric K.H. Chan

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Chan, E.K.H., Munro, D.W., Huang, A.H.S., Zumbo, B.D., Vojdanijahromi, R., Ark, N. (2014). Validation Practices in Counseling: Major Journals, Mattering Instruments, and the Kuder Occupational Interest Survey (KOIS). In: Zumbo, B., Chan, E. (eds) Validity and Validation in Social, Behavioral, and Health Sciences. Social Indicators Research Series, vol 54. Springer, Cham. https://doi.org/10.1007/978-3-319-07794-9_5

Download citation

Publish with us

Policies and ethics

Search

Navigation