Abstract
The purpose of this paper is to present a hypothesis testing and estimation procedure, Crossing SIBTEST, for detecting crossing DIF. Crossing DIF exists when the difference in the probabilities of a correct answer for the two examinee groups changes signs as ability level is varied. In item response theory terms, crossing DIF is indicated by two crossing item characteristic curves. Our new procedure, denoted as Crossing SIBTEST, first estimates the matching subtest score at which crossing occurs using least squares regression analysis. A Crossing SIBTEST statistic then is used to test the hypothesis of crossing DIF. The performance of Crossing SIBTEST is evaluated in this study.
Similar content being viewed by others
References
Bennett, R. E., Rock, D. A., & Kaplan, B. A. (1987). SAT differential item performance for nine handicapped groups.Journal of Educational Measurement, 24(1), 56–64.
Bock, R. D. (1975).Multivariate statistical methods. New York: McGraw-Hill.
Dorans, N. J., & Kulick, E. (1986). Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the scholastic aptitude test.Journal of Educational Measurement, 23, 355–368.
Douglas, J., Roussos, L., & Stout, W. F. (in press). Item bundle DIF hypothesis testing: identifying suspect bundles and assessing their DIF.Journal of Educational Measurement.
Drasgow, F. (1987). A study of measurement bias of two standard psychological tests.Journal of Applied Psychology, 72, 19–30.
Edgington, E. S. (1987).Randomization tests. New York, NY: Maecel Dekker.
Ellis, B. (1989). Differential item functioning: Implications for test translations.Journal of Applied Psychology, 74(6), 912–921.
Hambleton, R. K., & Rogers, H. J. (1989). Detecting potentially biased test items: Comparison of IRT area and Mantel-Haenszel methods.Applied Measurement in Education, 2(4), 313–334.
Holland, P. W., & Thayer, D. T. (1988). Differential item functioning and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.),Test validity (pp. 129–145). Hillsadle, NJ: Lawrence Erlbaum.
Li, H., & Stout, W. (1993, June). A new procedure for detection of crossing bias/DIF. Paper presented at the 58th Annual Meeting of Psychometric Society, Berkeley, CA.
Li, H., & Stout, W. (1994, April). Detecting crossing item bias/DIF: Comparison of logistic regression and crossing SIBTEST procedures. Paper presented at the 1994 AERA Annual Meeting, New Orleans, LA.
Mellenbergh, G. J. (1982). Contingency table models for assessing item bias.Journal of Educational Statistics, 7(2), 105–118.
Mislevy, R. J., & Bock, R. D. (1984). Item operating characteristics of the Armed Services Aptitute Battery (ASVAB), Form 8A (Tech. Rep. N00014-83-C-0283). Washington, DC: Office of Naval Research.
Nandakumar, R. (1992). Simultaneous DIF amplification and cancellation: Shealy-Stout's test for DIF.Journal of Educational Measurement, 30, 293–311.
Oshima, T., & Miller, D. (1991, April). Multidimensionality and item bias in item response theory. Presented at 1991 AERA Annual Meeting, Chicago, IL.
Ramsay, J. O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation.Psychometrika, 56, 611–630.
Ramsay, J. O. (1993). TESTGRAF: A program for the graphical analysis of multiple choice test and questionnaire data. TESTGRAF user's guide.
Roussos, L., & Stout, W. F. (1991). BIASDEM: A program for simulating models of unidirectional and crossing DIF. Unpublished manuscript.
Roussos, L., & Stout, W. F. (in press). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel Type I error performance.Journal of Educational Measurement.
Shealy, R., & Stout, W. (1993a). An item response theory model for test bias. In P. W. Holland & H. Wainer (Eds.),Differential item functioning (pp. 197–239). Hillsdale, NJ: Lawrence Erlbaum.
Shealy, R., & Stout, W. (1993b). A model-based standardization approach that separate true bias/DIF from group ability differences and detects test bias/DIF as well as item bias/DIF.Psychometrika, 58, 159–194.
Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures.Journal of Educational Measurement, 27(4), 361–370.
Author information
Authors and Affiliations
Additional information
This research was partially supported by a grant from the Law School Admission Council and by National Science Foundation Mathematics Grant NSF-DMS-94-04327. The research reported here is collaborative in every respect and the order of authorship is alphabetical. The authors thank Jeff Douglas and Louis Roussos for their useful comments and discussions.
Rights and permissions
About this article
Cite this article
Li, HH., Stout, W. A new procedure for detection of crossing DIF. Psychometrika 61, 647–677 (1996). https://doi.org/10.1007/BF02294041
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02294041