Examining of Internal Consistency Coefficients in Mixed-Format Tests in Different Simulation Conditions

The present study aims to evaluate how the reliabilities computed using α, Stratified α, Angoff-Feldt, and Feldt-Raju estimators may differ when sample size (500, 1000, and 2000) and item type ratio of dichotomous to polytomous items (2:1; 1:1, 1:2) included in the scale are varied. Research Methods: In this study, Cronbach’s α, Stratified α, Angoff-Feldt, and Feldt-Raju reliability coefficients were estimated on simulated datasets (sample sizes 500, 1000, 2000) and the number of dichotomous versus polytomous item ratios (2:1, 1:1, 1:2). Findings: In the simulation conditions of this research, in all sample size conditions, estimated Angoff-Feldt, and Feldt-Raju reliability coefficients were higher when the number of dichotomous items in the item-type ratio was higher than that of polytomous items. This was also the case for the estimated α and Stratified α reliability coefficients when the item-type ratio was reversed. While all different reliability estimators gave similar results in the large samples (n≥1000), there were some differences in reliability estimates depending on the item-type ratio in the small samples (n=500). Implications for Research and Practice: In the light of the findings and conclusions obtained in this study, it may be advisable to use α and Stratified α for mixed-type scales when the number of polytomously scored items in the scale is higher than that of the dichotomously scored items. On the other hand, the coefficients Angoff-Feldt and Feldt-Raju are recommended when the number of items scored dichotomously is higher.