On stochastic complexity estimation: a decision-theoretic approach

The concept of stochastic complexity developed by Rissanen(1989) leads to consistent probability density estimators. These density estimators are defined to achieve the best compromise between likelihood and simplicity, namely, the stochastic complexity based on the observed sample. In this paper, a density estimation-based complexity decision rule is proposed which uses the quality of these estimators to estimate the corresponding unknown element of the true probability density. In the development, we introduce a loss function which includes the total variation of the squared distance of the characteristic functions to evaluate the performance of the density decision rule. The resulting complexity density decision procedure is shown to be admissible, to achieve the minimum expected risk, and to form a minimal complete class