An asymptotic property of model selection criteria

Author

Yang, Yuhong ; Barron, Andrew R.

Author_Institution

Dept. of Stat., Iowa State Univ., Ames, IA, USA

Volume

44

Issue

1

fYear

1998

fDate

1/1/1998 12:00:00 AM

Firstpage

95

Lastpage

116

Abstract

Probability models are estimated by use of penalized log-likelihood criteria related to Akaike (1973) information criterion (AIC) and minimum description length (MDL). The accuracies of the density estimators are shown to be related to the tradeoff between three terms: the accuracy of approximation, the model dimension, and the descriptive complexity of the model classes. The asymptotic risk is determined under conditions on the penalty term, and is shown to be minimax optimal for some cases. As an application, we show that the optimal rate of convergence is simultaneously achieved for log-densities in Sobolev spaces W₂^s(U) without knowing the smoothness parameter s and norm parameter U in advance. Applications to neural network models and sparse density function estimation are also provided

Keywords

convergence of numerical methods; information theory; maximum likelihood estimation; minimax techniques; neural nets; probability; Akaike information criterion; Sobolev spaces; approximation; asymptotic property; asymptotic risk; density estimators accuracy; descriptive complexity; log-densities; minimax optimal penalty term; minimum description length; model classes; model dimension; model selection criteria; neural network models; norm parameter; optimal convergence rate; penalized log-likelihood criteria; probability models; smoothness parameter; sparse density function estimation; Convergence; Density functional theory; Entropy; Extraterrestrial measurements; Maximum likelihood estimation; Minimax techniques; Neural networks; Parametric statistics; Polynomials; Spline;

fLanguage

English

Journal_Title

Information Theory, IEEE Transactions on

Publisher

ieee

ISSN

0018-9448

Type

jour

DOI

10.1109/18.650993

Filename

650993