Tree-based latent variable model for assessing differential item functioning in patient-reported outcome measures: a simulation study

The validity of patient-reported outcome measures (PROMs) can be threatened by heterogeneity in how patients with the same underlying health status interpret and respond to questions about their health, a phenomenon known as differential item functioning (DIF). Although tree-based latent variable models, such as the partial credit model based on recursive partitioning (PCTree), are used to test for DIF, their performance has not been comprehensively investigated. We evaluated the statistical properties of PCTree to test for DIF in polytomous-scored PROM items.

Computer simulations for a variety of data-analytic conditions were used to evaluate the performance of PCTree with and without Bonferroni adjustments. The performance of this model was assessed using Type I error and statistical power rates. The robustness of PCTree with respect to control of the familywise Type I error rate was evaluated using Bradley’s liberal criterion; error rates within the interval of 2.5–7.5% for α = 5.0% indicate the method is robust.

Using Bradley’s criterion, PCTree with a Bonferroni correction provided good control of familywise Type I error rate across all simulation conditions. The average statistical power rate of PCTree with Bonferroni correction was at least 80% when N \(\ge\) 500. The average statistical power rate of the PCTree decreased as the number of explanatory variables not associated with DIF increased.

PCTree is promising for evaluating DIF in potentially heterogeneous populations. We provide recommendations about data analytic conditions for using the model.