Genetic associations in large versus small studies: an empirical assessment

Background Advances in human genetics could help us to assess prognosis on an individual basis and to optimise the management of complex diseases. However, different studies on the same genetic association sometimes have discrepant results. Our aim was to assess how often large studies arrive at different conclusions than smaller studies, and whether this situation arises more frequently when findings of first published studies disagree with those of subsequent research. Methods We examined the results of 55 meta-analyses (579 study comparisons) of genetic associations and tested whether the magnitude of the genetic effect differs in large versus smaller studies. Findings We noted significant between-study heterogeneity in 26 (47%) meta-analyses. The magnitude of the genetic effect differed significantly in large versus smaller studies in ten (18%), 20 (36%), and 21 (38%) meta-analyses with tests of rank correlation, regression on SE, and regression on inverse of variance, respectively. The largest studies generally yielded more conservative results than the complete meta-analyses, which included all studies (p=0•005). In 14 (26%) meta-analyses the proposed association was significantly stronger in the first studies than in subsequent research. Only in nine (16%) meta-analyses was the genetic association significant and replicated without hints of heterogeneity or bias. There was little concordance in first versus subsequent discrepancies, and large versus small discrepancies. Interpretation Genuine heterogeneity and bias could affect the results of genetic association studies. Genetic risk factors for complex diseases should be assessed cautiously and, if possible, using large scale evidence.