Title :
Asymptotically minimax regret by Bayes mixtures for non-exponential families
Author :
Takeuchi, Jun ; Barron, Andrew R.
Author_Institution :
Dept. of Inf., Kyushu Univ., Fukuoka, Japan
Abstract :
We study the problems of data compression, gambling and prediction of a sequence xn = x1x2...xn from an alphabet X, in terms of regret with respect to various families of probability distributions. It is known that the regret of the Bayes mixture with respect to a general exponential families asymptotically achieves the minimax value when variants of Jeffreys prior are used, under the condition that the maximum likelihood estimate is in the interior of the parameter space. We discuss a modification of Jeffreys prior which has measure outside the given family of densities, to achieve minimax regret with respect to non-exponential type families, e.g. curved exponential families and mixture families. These results also provide characterization of Rissanen´s stochastic complexity for those classes.
Keywords :
Bayes methods; data compression; maximum likelihood estimation; minimax techniques; sequences; statistical distributions; stochastic processes; Bayes mixtures; Jeffreys prior; Rissanen´s stochastic complexity; alphabet; asymptotically minimax regret; data compression; gambling; maximum likelihood estimation; nonexponential families; prediction; probability distributions; sequence; Complexity theory; Context; Data compression; Educational institutions; Encoding; Information geometry; Maximum likelihood estimation;
Conference_Titel :
Information Theory Workshop (ITW), 2013 IEEE
Conference_Location :
Sevilla
Print_ISBN :
978-1-4799-1321-3
DOI :
10.1109/ITW.2013.6691254