Unbiased Learning for Hierarchical Models

It is known that overfitting occurs when a conventional statistical learning method such as maximum likelihood estimation, maximum a posteriori estimation or Bayesian estimation is applied to hierarchical models. This paper gives an explanation why overfitting occurs and propose an appropriate learning framework unbiased learning for hierarchical models. The method suggest to train the hyperparameters based on unbiased likelihood which is estimated by an appropriate information criterion. Therefore, it can say that the unbiased learning is a generalization of hyperparameters selection. Unbiased learning with several information criteria is tested by computer simulations.