Analyzing the Structure of Test Data

Abstract

One of the common uses of MIRT is the analysis of the structure of the item response data that results from the administration of a set of test items to a sample of examinees. This type of analysis can be done in either an exploratory or confirmatory way. The exploratory analysis of item response data is used when either there is no clear hypothesis for the structure of the item response data, or when an unconstrained solution is seen as a strong test of the hypothesized structure. Confirmatory analyses require a clear hypothesis for the structure of the item response data. That is, there must be hypotheses about the number of coordinate dimensions needed to model the data and the relationship of the item characteristic surface to the coordinate axes. The types of confirmatory analyses that are typically done specify the relationship of the direction best measured by a test item (the direction of maximum discrimination) with the coordinate axes. It is also possible to have hypotheses about the difficulty of test items, but such hypotheses are seldom checked with the MIRT models. The difficulty parameters are typically left as parameters to be estimated without constraints.

In many cases, what is labeled as an exploratory analysis also has a confirmatory analysis component because the number of coordinate dimensions is selected prior to estimating the item and person-parameters. The resulting measures of fit of the model to the data are a test of a hypothesis about the number of dimensions. Because the number of coordinate dimensions is a critical component of both exploratory and confirmatory analyses, this issue will be addressed first. After consideration of the number-of-dimensions problem, procedures are described for determining the structure of item response data using exploratory and confirmatory procedures.