Skip to main content
Log in

Designing and Evaluating Standard-Setting Procedures for Licensure and Certification Tests

  • Published:
Advances in Health Sciences Education Aims and scope Submit manuscript

Abstract

Passing scores for licensure and certification tests are justified by showing that decisions based on the passing score achieve the purposes of the credentialing program while avoiding any serious negative consequences. The standard should be high enough to provide adequate protection for the public, and not so high as to unnecessarily restrict the supply of qualified practitioners or to exclude competent candidates from practicing. This paper begins by examining the intended outcomes of licensure and certification programs and by outlining the interpretive argument that is typically used for written credentialing examinations. Criteria are then developed for evaluating standard-setting methods in terms of how well they serve the goals of protecting the public, maintaining an adequate supply of practitioners, and protecting the rights of candidates. Finally, the criteria are used to evaluate the Angoff method. This analysis identifies two potential sources of bias in the Angoff method, and suggests ways to control these weaknesses.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational Measurement, 2nd edn., pp. 508–600. American Council on Education, Washington, DC.

    Google Scholar 

  • Berk, R. (1986). A consumer's guide to setting performance standards on criterion-referenced tests. Review of Educational Research 56: 137–172.

    Article  Google Scholar 

  • Busch, M. I. and Jaeger, R. M. (1990). Influence of type of judge, normative information, and discussion on standards recommended for the National Teacher Examination. Journal of Educational Measurement 27: 145–163.

    Article  Google Scholar 

  • Clauser, B. and Clyman, S. (1994). A contrasting groups approach to standard setting for performance assessments of clinical skills. Academic Medicine, RIME Supplement 69(10): S42-S44.

    Google Scholar 

  • Clauser, B. E. and Nungester, R. J. (1997). Setting standards on performance assessments of physician's clinical skills using contrasting groups and receiver operating characteristic curves. Evaluation and the Health Professions 20: 215–238.

    Google Scholar 

  • Ebel, R. L. (1972). Essentials of Educational Measurement. Prentice-Hall, Englewood Cliffs, NJ.

    Google Scholar 

  • Harker, J. and Cope, R. (1988). The Effects of Several Variables on Judgementally-Obtained Cut Scores. Paper presented at the annual meeting of the National Council on measurement in Education, New Orleans.

  • Jaeger, R. M. (1982). An iterative structured judgment process for establishing standards on competency tests: Theory and application. Educational Evaluation and Policy Analysis 4: 461–476.

    Article  Google Scholar 

  • Jaeger, R. M. (1989). Certification of Student Competence. In R. L. Linn (Ed.), Educational Measurement, 3rd edn., pp. 485–514. American Council on Education and Macmillan, New York.

    Google Scholar 

  • Kane, M. (1992). An argument-based approach to validity. Psychological Bulletin 112: 527–535.

    Article  Google Scholar 

  • Kane, M. (1994). Validating the performance standards associated with passing scores. Review of Educational Research 64: 425–461.

    Article  Google Scholar 

  • Kane, M. (1995). Examinee-centered vs. task-centered standard setting. In Joint Conference on Standard Setting for Large-scale Assessments, Proceedings, Vol. II, pp. 119–141.

    Google Scholar 

  • Kane, M. and Wilson, J. (1984). Errors of measurement and standard setting in mastery testing. Applied Psychological Measurement 8: 107–115.

    Google Scholar 

  • LaDuca, A. (1994). Validation of professional licensure examinations: Professions theory, test design, and construct validity. Evaluations and the Health Professions 17(2): 178–197.

    Google Scholar 

  • Livingston, S. A. and Zieky, M. J. (1982). Passing Scores: A Manual for Setting Standards of Performance on Educational and Occupational Tests. Educational Testing Service, Princeton, NJ.

    Google Scholar 

  • McLaughlin, D. H. (1993). Validity of the 1992 NAEP Achievement-Level-Setting Process. In Setting Performance Standards for Student Achievement: Background Studies. The National Academy of Education, Stanford, CA.

    Google Scholar 

  • Mehrens, W. A. (1995). Methodological issues in standard setting for educational exams. In Joint Conference on Standard Setting for Large-Scale Assessments, Proceedings, Volume II. US Government Printing Office.

  • Nedelsky, L. (1954). Absolute grading standards for objective tests. Educational and Psychological Measurement 14: 3–19.

    Google Scholar 

  • Norcini, J. and Shea, J. (1997). The credibility and comparability of standards. Applied Measurement in Education 10: 39–59.

    Article  Google Scholar 

  • Norcini, J. (1994). Research on standards for professional licensure and certification examinations. Evaluations and the Health Professions 17(2): 160–177.

    Google Scholar 

  • Norcini, J., Shea, J. and Ping J. (1988). A note on the application of multiple matrix sampling to standard setting. Journal of Educational Measurement 25(2): 159–164.

    Article  Google Scholar 

  • Norcini, J., Shea, J. and Kanya, D. (1988). The effect of various factors on standard setting. Journal of Educational Measurement 25: 7–65.

    Google Scholar 

  • Plake, B. (1996). Setting Performance Standards for Professional Licensure and Certification. Paper presented at workshop on NAEP Achievement Levels: Setting Consensus Goals for Academic Achievement, Washington, DC.

  • Shepard, L. (1984) Setting performance standards. In R. A. Berk (Ed.), A Guide to Criterion-Referenced Test Construction. Johns Hopkins Press, Baltimore.

    Google Scholar 

  • Shepard, L., Glaser, R., Linn, R. and Bohrnstedt, G. (1993). Setting Performance Standards for Student Achievement. National Academy of Education, Stanford University, Stanford, CA.

    Google Scholar 

  • Sireci, S. A. and Biskin, B. M. (1992). Measurement practices in national licensing examination programs: A survey. CLEAR Exam Review 3: 21–35.

    Google Scholar 

  • Tamblyn, R. (1994). Is the public being protected? Suboptimal medical practice through training programs and credentialing examinations. Evaluations and the Health Professions 17(2): 198–221.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kane, M., Crooks, T. & Cohen, A. Designing and Evaluating Standard-Setting Procedures for Licensure and Certification Tests. Adv Health Sci Educ Theory Pract 4, 195–207 (1999). https://doi.org/10.1023/A:1009849528247

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1009849528247

Navigation