Resampling Permutations in Regression without Second Moments

For given observations we consider the resampling distribution obtained by permuting residuals versus thedistribution of errors conditional on their order statistics. We observe thatwith high probabilityboth are approximately equal oncontrast vectorsafter suitable normalization. No moment/distribution assumptions on the underlying errors other than exchangeability are required. These results remain valid if we consider reduced regression models derived from the original one by removing data for which the corresponding residuals take extreme values. We obtain general conditions under which confidence regions produced by such methods are accurate. In the absence of finite second moments randomness of the limiting bootstrap distributions has been considered as a failure of the traditional bootstrap and no practical meaning has been given to the phenomenon. For our method this type of randomness is still present in the limit but it has clear probabilistic interpretation as a conditional distribution which can be used, e.g., to obtain conditional confidence sets. We study this phenomenon in detail for the case of independent errors with distribution from the domain of attraction of stable laws.