Skip to main content
SpringerLink
Log in

Differential Item Functioning: its Multidimensional Model and Resulting Sibtest Detection Procedure

  • Published:
Behaviormetrika Aims and scope Submit manuscript

Abstract

A multidimensional model of differential item functioning (DIF) developed by Shealy and Stout is presented. It explains how individual items combine to produce differential test functioning (DTF) at the test level. Recent developments based on this approach, including development of the DIF/DTF detection procedure SIBTEST, are surveyed. The Shealy-Stout model not only offers insight into how DIF can occur, but suggests methods for investigating the root causes of DIF, which are useful to both substantive and statistical researchers of DIF. This new modeling paradigm offers the possibility of the proactive reduction of DIF in tests at the item manufacturing stage. SIBTEST is a nonparametric procedure that both tests for and estimates the amount of DIF in an item or set of items while controlling inflated Type I error by using a regression correction technique. It is shown to perform as well as and in many realistic situations better than other popular DIF assessment procedures. Recent modifications to the procedure demonstrate that it can be an effective tool for examining both crossing DIF and, through the use of kernel smoothing, local DIF. The procedure has been extended for use with tests containing polytomous items as well as tests that are intentionally multidimensional. Many real and simulated data analyses are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ackerman, T. (1992). A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective. Journal of Educational Measurement, 29, 67–91.

    Article  Google Scholar 

  • Allen, N.L., and Donoghue, J.R. (1994). DIF Analysis based on complex samples of dichotomous and poly torn ous items. Presented at the Annual Meeting of the American Educational Research Association.

    Google Scholar 

  • Camilli, G., Shepard, L.A. (1994). Methods for identifying biased test items. Sage: Thousand Oaks, CA.

    Google Scholar 

  • Chang, Hua, Mazzeo, John, and Roussos, Louis. (1995). Detecting DIF for polytomously scored items: An adaptation of the SIBTEST procedure. Accepted by the Journal of Educational Measurement

    Google Scholar 

  • Dorans, N.J., and Kulick, E.M. (1986). Demonstrating the utility of the standardization approach to assessing differential item performance on the Scholastic Aptitude Test. Journal of Educational Measurement, 23, 355–368.

    Article  Google Scholar 

  • Dorans, N. J., & Schmitt, A.P. (1991). Constructed response and differential response functioning: A pragmatic approach (ETS Research Report No. 91-47). Princeton, NJ: Educational Testing Service.

    Google Scholar 

  • Dorans, NJ. & Holland, P.W. (1993). DIF detection and description: Mantel-Haenszel and Standardization. In. P.W. Holland and H. Wainer (Eds.) Differential Item Functioning (pp. 35–66). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Douglas, Jeff, Roussos, Louis, and Stout, William, (1995a). Item bundle DIF hypothesis testing: Identifying suspect bundles and assessing their DIF. Accepted by the Journal of Educational Measurement.

    Google Scholar 

  • Douglas, Jeff, Roussos, Louis, and Stout, William. (1995b). Refinement of the Shealy-Stout multidimensional IRT model of DIF with practical implications. Manuscript to be submitted.

    Google Scholar 

  • Douglas, Jeff, Stout, William, and DiBello, Lou. (1995). A kernel smoothed version of SIBTEST with applications to local DIF inference and function estimation. Accepted for publication by Journal of Educational Statistics.

    Google Scholar 

  • Holland, P.W., and Thayer, D.T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer and II.I Braun (Eds.), Test Validity (pp. 129–145). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Jain, A.K., & Dubes, R.C. (1988). Algorithms for clustering data. Englewood Cliffs, NJ: Prentice Hall.

    MATH  Google Scholar 

  • Kok, F. (1988). Item bias and test multidimensionality. In R. Langeheine and J. Rost (Eds.), Latent trait and latent class models (pp. 263–274). New York: Plenum.

    Chapter  Google Scholar 

  • Li, Hsin-hung and Stout, William. (1995). A new procedure for detection of crossing DIF. Accepted for publication by Psychometrika.

    MATH  Google Scholar 

  • Li, Hsin-hung, Nandakumar, Ratna, and Stout, William. (1995). Use of SIBTEST to do DIF when the matching subtest is intentionally multidimensional. Paper presented at Annual National Council of Measurement on Educational Meeting, San Francisco.

    Google Scholar 

  • Mantel, N., and Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719–748.

    Google Scholar 

  • Mantel, N. (1963). Chi-square tests with one degree of freedom.: Extensions of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58, 690–700.

    MathSciNet  MATH  Google Scholar 

  • Masters, G.N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149–174.

    Article  Google Scholar 

  • Mellenbergh, G. J. (1982). Contingency table models for assessing item bias. Journal of Educational Statistics, 7(2), 105–118.

    Article  Google Scholar 

  • Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement 16, 159–176.

    Article  Google Scholar 

  • Nandakumar, Ratna. (1993). Simultaneous DIF amplification and cancellation; Shealy-Stout’s test for DIF. Journal of Educational Measurement, 30, pp. 293–311.

    Article  Google Scholar 

  • Oshima, T.C. & Miller, M.D. (1992). Multidimensionality and item bias in item response theory. Applied Psychological Measurement 16, 237–248.

    Article  Google Scholar 

  • Ramsay, J.O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56.

    Article  Google Scholar 

  • Ramsay, J.O. (1993). TESTGRAF; A program for the graphical analysis of multiple choice and questionnaire data. Unpublished user’s guide to TESTGRAF.

    Google Scholar 

  • Reckase, M.D. & McKinley, R.L. (1983, April). The definition of difficulty and discrimination for multidimensional item response theory models. Paper presented at the annual meeting of the American Educational Research Association, Montreal.

    Google Scholar 

  • Roussos, Louis and Stout, William. (1995a). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel Type 1 error performance. 1–20. To appear in Journal of Educational Measurement

    Google Scholar 

  • Roussos, Louis and Stout, William (1995b). DIF from the multidimensional perspective. Invited paper to appear in Applied Psychological Measurement

    Google Scholar 

  • Roussos, Louis, Stout, William, and Marden, John. (1994). Analysis of the multidimensional simple structure of standardized tests using DIMTEST with hierarchical cluster analysis. 1–34. Submitted for publication.

    Google Scholar 

  • Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph, No. 17.

    Google Scholar 

  • Samejima, F. (1972). A general model for free-response data. Psychometric Monograph, No. 18.

    Google Scholar 

  • Shealy, Robin, and Stout, William. (1993a). An item response theory model for test bias and differential test functioning. In P. Holland & II. Wainer (Eds.), Differential Item Functioning (pp. 197–240). Hillsdale, NJ: Earlbaum.

    MATH  Google Scholar 

  • Shealy, Robin, and Stout, William. (1993b). A model- based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DIF as well as item bias/DIF. Psychometrika, 58, 159–194.

    Article  Google Scholar 

  • Stout, William. (1987). A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52, 589–617.

    Article  MathSciNet  Google Scholar 

  • Stout, William and Roussos, Louis. (1995). SIBTEST Users Manual. 1–68.

    Google Scholar 

  • Stout, William, Douglas, J., Kim, H.R., Roussos, L., and Zhang, J. (1995). Conditional covariance based nonparametric multidimensional assessment. Invited paper to appear to in Applied Psychological Measurement.

    Google Scholar 

  • Swaminathan, H., and Rogers, H.J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361–370.

    Article  Google Scholar 

  • Wainer, H., Sireci, S.G., and Thissen, D. (1991). Differential testlet functioning. Journal of Educational Measurement, 28, 197–220.

    Article  Google Scholar 

  • Zwick, R. (1990). When do item response function and Mantel-Haenszel definitions of differential item functioning coincide?, Journal of Educational Statistics, 15, 185–197.

    Article  Google Scholar 

  • Zwick, R., Donoghue, J. and Grima, A. (1993). Assessment of differential item functioning for performance tasks. Journal of Educational Measurement, 30, 233–251.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, University of Illinois at Urbana-Champaign, Psychology Building, 603 E. Daniel, Champaign, IL, 61820, USA

    Daniel Bolt

  2. Department of Statistics, University of Illinois at Urbana-Champaign, 101 Illini Hall, 725 S. Wright St., Champaign, IL, 61820, USA

    William Stout

Authors
  1. Daniel Bolt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. William Stout
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to William Stout.

About this article

Cite this article

Bolt, D., Stout, W. Differential Item Functioning: its Multidimensional Model and Resulting Sibtest Detection Procedure. Behaviormetrika 23, 67–95 (1996). https://doi.org/10.2333/bhmk.23.67

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.2333/bhmk.23.67

Key Words and Phrases

Search

Navigation