Skip to main content
Log in

A new procedure for detection of crossing DIF

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

The purpose of this paper is to present a hypothesis testing and estimation procedure, Crossing SIBTEST, for detecting crossing DIF. Crossing DIF exists when the difference in the probabilities of a correct answer for the two examinee groups changes signs as ability level is varied. In item response theory terms, crossing DIF is indicated by two crossing item characteristic curves. Our new procedure, denoted as Crossing SIBTEST, first estimates the matching subtest score at which crossing occurs using least squares regression analysis. A Crossing SIBTEST statistic then is used to test the hypothesis of crossing DIF. The performance of Crossing SIBTEST is evaluated in this study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bennett, R. E., Rock, D. A., & Kaplan, B. A. (1987). SAT differential item performance for nine handicapped groups.Journal of Educational Measurement, 24(1), 56–64.

    Google Scholar 

  • Bock, R. D. (1975).Multivariate statistical methods. New York: McGraw-Hill.

    Google Scholar 

  • Dorans, N. J., & Kulick, E. (1986). Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the scholastic aptitude test.Journal of Educational Measurement, 23, 355–368.

    Google Scholar 

  • Douglas, J., Roussos, L., & Stout, W. F. (in press). Item bundle DIF hypothesis testing: identifying suspect bundles and assessing their DIF.Journal of Educational Measurement.

  • Drasgow, F. (1987). A study of measurement bias of two standard psychological tests.Journal of Applied Psychology, 72, 19–30.

    Google Scholar 

  • Edgington, E. S. (1987).Randomization tests. New York, NY: Maecel Dekker.

    Google Scholar 

  • Ellis, B. (1989). Differential item functioning: Implications for test translations.Journal of Applied Psychology, 74(6), 912–921.

    Google Scholar 

  • Hambleton, R. K., & Rogers, H. J. (1989). Detecting potentially biased test items: Comparison of IRT area and Mantel-Haenszel methods.Applied Measurement in Education, 2(4), 313–334.

    Google Scholar 

  • Holland, P. W., & Thayer, D. T. (1988). Differential item functioning and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.),Test validity (pp. 129–145). Hillsadle, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Li, H., & Stout, W. (1993, June). A new procedure for detection of crossing bias/DIF. Paper presented at the 58th Annual Meeting of Psychometric Society, Berkeley, CA.

  • Li, H., & Stout, W. (1994, April). Detecting crossing item bias/DIF: Comparison of logistic regression and crossing SIBTEST procedures. Paper presented at the 1994 AERA Annual Meeting, New Orleans, LA.

  • Mellenbergh, G. J. (1982). Contingency table models for assessing item bias.Journal of Educational Statistics, 7(2), 105–118.

    Google Scholar 

  • Mislevy, R. J., & Bock, R. D. (1984). Item operating characteristics of the Armed Services Aptitute Battery (ASVAB), Form 8A (Tech. Rep. N00014-83-C-0283). Washington, DC: Office of Naval Research.

    Google Scholar 

  • Nandakumar, R. (1992). Simultaneous DIF amplification and cancellation: Shealy-Stout's test for DIF.Journal of Educational Measurement, 30, 293–311.

    Google Scholar 

  • Oshima, T., & Miller, D. (1991, April). Multidimensionality and item bias in item response theory. Presented at 1991 AERA Annual Meeting, Chicago, IL.

  • Ramsay, J. O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation.Psychometrika, 56, 611–630.

    Google Scholar 

  • Ramsay, J. O. (1993). TESTGRAF: A program for the graphical analysis of multiple choice test and questionnaire data. TESTGRAF user's guide.

  • Roussos, L., & Stout, W. F. (1991). BIASDEM: A program for simulating models of unidirectional and crossing DIF. Unpublished manuscript.

  • Roussos, L., & Stout, W. F. (in press). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel Type I error performance.Journal of Educational Measurement.

  • Shealy, R., & Stout, W. (1993a). An item response theory model for test bias. In P. W. Holland & H. Wainer (Eds.),Differential item functioning (pp. 197–239). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Shealy, R., & Stout, W. (1993b). A model-based standardization approach that separate true bias/DIF from group ability differences and detects test bias/DIF as well as item bias/DIF.Psychometrika, 58, 159–194.

    Google Scholar 

  • Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures.Journal of Educational Measurement, 27(4), 361–370.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Additional information

This research was partially supported by a grant from the Law School Admission Council and by National Science Foundation Mathematics Grant NSF-DMS-94-04327. The research reported here is collaborative in every respect and the order of authorship is alphabetical. The authors thank Jeff Douglas and Louis Roussos for their useful comments and discussions.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Li, HH., Stout, W. A new procedure for detection of crossing DIF. Psychometrika 61, 647–677 (1996). https://doi.org/10.1007/BF02294041

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02294041

Key words

Navigation