Exponential Distance Statistics to Detect the Effects of Population Subdivision

Statistical tests are needed to determine whether spatial structure has had a significant effect on the genetic differentiation of subpopulations. Here we introduce a new family of statistics based on a sum of an exponential function of the distances between individuals, which can be used with any genetic distance (e.g., nucleotide differences, number of nonshared alleles, or separation on a phylogenetic tree). The power of the tests to detect genetic differentiation in Wright–Fisher island models and stepping stone models was calculated for various sample sizes, rates of migration and mutation, and definitions of spatial neighborhoods. We found that our new test was in some cases more powerful than the K*s statistic of Hudson et al. (Mol. Biol. Evol. 9, 138–151, 1992), but in all cases was slightly less powerful than both a traditional χ2 test without lumping of rare haplotypes and the Snn test of Hudson (Genetics 155, 2011–2014, 2000). However, when we applied our new tests to three data sets, we found in some cases highly significant results that were missed by the other tests.