Estimation of the number of true null hypotheses when conducting a multiple testing

Application of the standard hypothesis test with no adjustment for the multiple testing leads to a large number of false discoveries. The most convenient error measure in multiple testing is False Discovery Rate, FDR. However, calculation of FDR requires good estimation of the number of true null hypotheses, n0, (or equivalently, of π0 = n0/n, where n denotes the number of all tested hypotheses). Estimation of π0 is a non-trivial problem and it can be done under several assumptions about input data. In our study, several approaches to the estimation of π0 values are compared for the different data models (independent features, block-correlated data and mixture models). The presented results give evidence that general dependence of the features leads to a very doubtful estimation of their significance.