The asymptotics of posterior entropy and error probability for Bayesian estimation

We consider the Bayesian parameter estimation problem where the value of a finitary parameter X should be decided on the basis of i.i.d. sample Yⁿ of size n. In this context, the amount of missing information on X after observing Yⁿ may be evaluated by the posterior entropy, which is often called the equivocation or the conditional entropy, of X given Yⁿ, while it is well known that the minimum possible probability of error in estimating X is achieved by the maximum a posteriori probability (MAP) estimator. In this work, the focus is on the asymptotic relation between the posterior entropy and the MAP error probability as the sample size n becomes sufficiently large. It is shown that if the sample size n is large enough, the posterior entropy as well as the MAP error probability decay with n to zero at the identical exponential rate, and that the maximum achievable exponent for this decay is determined by the minimum Chernoff information over all the possible pairs of distinct parameter values