The minimum coverage probability of confidence intervals in regression after a preliminary F test

Consider a linear regression model with regression parameter β = ( β 1 , … , β p ) and independent normal errors. Suppose the parameter of interest is θ = a T β , where a is specified. Define the s-dimensional parameter vector τ = C T β − t , where C and t are specified. Suppose that we carry out a preliminary F test of the null hypothesis H 0 : τ = 0 against the alternative hypothesis H 1 : τ ≠ 0 . It is common statistical practice to then construct a confidence interval for θ with nominal coverage 1 − α , using the same data, based on the assumption that the selected model had been given to us a priori (as the true model). We call this the naive 1 − α confidence interval for θ . This assumption is false and it may lead to this confidence interval having minimum coverage probability far below 1 − α , making it completely inadequate. We provide a new elegant method for computing the minimum coverage probability of this naive confidence interval, that works well irrespective of how large s is. A very important practical application of this method is to the analysis of covariance. In this context, τ can be defined so that H0 expresses the hypothesis of “parallelism”. Applied statisticians commonly recommend carrying out a preliminary F test of this hypothesis. We illustrate the application of our method with a real-life analysis of covariance data set and a preliminary F test for “parallelism”. We show that the naive 0.95 confidence interval has minimum coverage probability 0.0846, showing that it is completely inadequate.