Measuring effect size: a robust heteroscedastic approach for two or more groups

Motivated by involvement in an intervention study, the paper proposes a robust, heteroscedastic generalization of what is popularly known as Cohen’s d. The approach has the additional advantage of being readily extended to situations where the goal is to compare more than two groups. The method arises quite naturally from a regression perspective in conjunction with a robust version of explanatory power. Moreover, it provides a single numeric summary of how the groups compare in contrast to other strategies aimed at dealing with heteroscedasticity. Kulinskaya and Staudte [16] studied a heteroscedastic measure of effect size similar to the one proposed here, but their measure of effect size depends on the sample sizes making it difficult for applied researchers to interpret the results. The approach used here is based on a generalization of Cohen’s d that obviates the issue of unequal sample sizes. Simulations and illustrations demonstrate that the new measure of effect size can make a practical difference regarding the conclusions reached.