Skip to main content
Log in

Multidimensional adaptive testing

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

Maximum likelihood and Bayesian procedures for item selection and scoring of multidimensional adaptive tests are presented. A demonstration using simulated response data illustrates that multidimensional adaptive testing (MAT) can provide equal or higher reliabilities with about one-third fewer items than are required by one-dimensional adaptive testing (OAT). Furthermore, holding test-length constant across the MAT and OAT approaches, substantial improvements in reliability can be obtained from multidimensional assessment. A number of issues relating to the operational use of multidimensional adaptive testing are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ackerman, T. A. (1989). Unidimensional IRT calibration of compensatory and non-compensatory multidimensional items.Applied Psychological Measurement, 13, 113–127.

    Google Scholar 

  • Ackerman, T. A. (1991). The use of unidimensional parameter estimates of multidimensional items in adaptive testing.Applied Psychological Measurement, 15, 13–24.

    Google Scholar 

  • Anderson, T. W. (1984).An introduction to multivariate statistical analysis (2nd ed.). New York: John Wiley & Sons.

    Google Scholar 

  • Ansley, T. N., & Forsyth, R. A. (1985). An examination of the characteristics of unidimensional IRT parameter estimates derived from two-dimensional data.Applied Psychological Measurement, 9, 37–48.

    Google Scholar 

  • Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick (Eds.),Statistical theories of mental test scores (pp. 397–479). Reading, MA: Addison-Wesley.

    Google Scholar 

  • Bloxom, B., & Vale, C. D. (1987, June).Multidimensional adaptive testing: An approximate procedure for updating. Paper presented at the meeting of the Psychometric Society, Montreal.

  • Bock, R. D., Gibbons, R., & Muraki, E. (1988). Full-information item factor analysis.Applied Psychological Measurement, 12, 261–280.

    Google Scholar 

  • Bollen, K. A. (1989).Structural equations with latent variables. New York: John Wiley & Sons.

    Google Scholar 

  • Carlson, J. E. (1987).Multidimensional item response theory estimation: A computer program (Research Report ONR 87-2). Iowa City, IA: The American College Testing Program.

    Google Scholar 

  • Drasgow, F., Levine, M. V., & McLaughlin, M. E. (1991). Appropriateness measurement for some multidimensional test batteries.Applied Psychological Measurement, 15, 171–191.

    Google Scholar 

  • Drasgow, F., & Parsons, C. K. (1983). Application of unidimensional item response theory models to multidimensional data.Applied Psychological Measurement, 7, 189–199.

    Google Scholar 

  • Folk, V. G., & Green, B. F. (1989). Adaptive estimation when the unidimensionality assumption of IRT is violated.Applied Psychological Measurement, 13, 373–389.

    Google Scholar 

  • Fraser, C. (1988).NOHARM II. A Fortran program for fitting unidimensional and multidimensional normal ogive models of latent trait theory. Armidale, Australia: The University of New England, Center for Behavioral Studies.

    Google Scholar 

  • Harrison, D. A. (1986). Robustness of IRT parameter estimation to violations of the unidimensionality assumption.Journal of Educational Statistics, 11, 91–115.

    Google Scholar 

  • Hattie, J. (1981).Decision criteria for determining unidimensionality. Unpublished doctoral dissertation, University of Toronto, Canada.

    Google Scholar 

  • Hendrickson, A. E., & White, P. O. (1964). PROMAX: A quick method for rotation to oblique simple structure.British Journal of Mathematical and Statistical Psychology, 17, 65–70.

    Google Scholar 

  • IMSL (1991).International Mathematical and Statistical Libraries (Stat/Library), User's Manual, Houston, TX: Author.

    Google Scholar 

  • Kaiser, H. F. (1958). The varimax criterion for analytic rotation in factor analysis.Psychometrika, 23, 187–200.

    Google Scholar 

  • Lord, F. M. (1980).Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • McDonald, R. P. (1985). Unidimensional and multidimensional models for item response theory. In D. J. Weiss (Ed.),Proceedings of the 1982 Computerized Adaptive Testing Conference (pp. 127–148). Minneapolis: University of Minnesota, Department of Psychology, Psychometrics Methods Program.

    Google Scholar 

  • McKinley, R. L. (1989).Confirmatory analysis of test structure using multidimensional item response theory (Report No. RR-89-31). Princeton, NJ: Educational Testing Service.

    Google Scholar 

  • McKinley, R. L., & Reckase, M. D. (1983). MAXLOG: A computer program for the estimation of the parameters of a multidimensional logistic model.Behavior Research Methods & Instrumentation, 15, 389–390.

    Google Scholar 

  • Miller, T., Reckase, M. D., Spray, J. A., Luecht, R., & Davey, T. (in press).Multidimensional Item Response Theory.

  • Mislevy, R. J. (1984). Estimating latent distributions.Psychometrika, 49, 359–381.

    Google Scholar 

  • Mislevy, R. J. (1989).PC-BILOG: Item analysis and test scoring with binary logistic models. Mooresville, IN: Scientific Software.

    Google Scholar 

  • Moreno, K. E., & Segall, D. O. (1993). CAT-ASVAB Precision.Proceedings of the 34th Annual Conference of the Military Testing Association. San Diego: Navy Personnel Research and Development Center.

    Google Scholar 

  • Muthén, B. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators.Psychometrika, 49, 115–132.

    Google Scholar 

  • Owen, R. J. (1975). A Bayesian sequential procedure for quantal response in the context of adaptive mental testing.Journal of the American Statistical Association, 70, 351–356.

    Google Scholar 

  • Prestwood, J. S., Vale, C. D., Massey, R. H., & Welsh, J. R. (1985).Armed Services Vocational Aptitude Battery: Development of an adaptive item pool (Technical Report 85-19). San Antonio, TX: Air Force Human Resources Laboratory.

    Google Scholar 

  • Reckase, M. D. (1979). Unifactor latent trait models applied to multifactor tests: Results and implications.Journal of Educational Statistics, 4, 207–230.

    Google Scholar 

  • Reckase, M. D., Ackerman, T. A., & Carlson, J. E. (1988). Building a unidimensional test using multidimensional items.Journal of Educational Measurement, 25, 193–203.

    Google Scholar 

  • Searle, S. R. (1982).Matrix algebra useful for statistics. New York: John Wiley & Sons.

    Google Scholar 

  • Segall, D. O., Moreno, K. E., & Hetter, R. D. (1987).ACAP item pools: Analysis and recommendations. Unpublished manuscript, Navy Personnel Research and Development Center, San Diego.

    Google Scholar 

  • Sympson, J. B., & Hetter, R. D. (1985, October).Controlling item exposure rates in computerized adaptive tests. Paper presented at the 27th Annual meeting of the Military Testing Association, San Diego, CA.

  • Tam, S. S. (1992).A comparison of methods for adaptive estimation of a multidimensional trait. Unpublished doctoral dissertation, Columbia University.

  • Wainer, H. W., Dorans, N. J., Flaugher, R., Green, B. F., Mislevy, R. J., Steinberg, L., & Thissen, D. (1990).Computerized adaptive testing: A primer. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Way, W. D., Ansley, T. N., & Forsyth, R. A. (1988). The comparative effects of compensatory and noncompensatory two-dimensional data on unidimensional IRT estimation.Applied Psychological Measurement, 12, 239–252.

    Google Scholar 

  • Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model.Applied Psychological Measurement, 8, 125–145.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Additional information

The work reported in this paper was sponsored by the Office of Naval Research. The author wishes to thank the three anonymous reviewers for their useful comments on an earlier version of this manuscript. The opinions expressed in this article are those of the Author, are not official and do not necessarily reflect the views of the Navy Department.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Segall, D.O. Multidimensional adaptive testing. Psychometrika 61, 331–354 (1996). https://doi.org/10.1007/BF02294343

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02294343

Key words

Navigation