Empirical Bayes Versus Standard Mantel-Haenszel Statistics for Detecting Differential Item Functioning Under Small Sample Conditions

In this study, the authors assess several strategies created on the basis of the Mantel-Haenszel (MH) procedure for conducting differential item functioning (DIF) analysis with small samples. One of the analytical strategies is a loss function (LF) that uses empirical Bayes Mantel-Haenszel estimators, whereas the other strategies use the classical MH statistics (the MH chisquare statistic using high levels of significance [0.20] or empirical criteria on the basis of the magnitude of the MH-delta estimator). The authors conducted a series of computer simulations in which they manipulated different types of tests, sample sizes, and ability distributions. The results show that the loss function does not offer advantages, in terms of power and Type I error rate, over other analytical strategies that use the classical MH statistics.