on the optimal K-term approximation of a sparse parameter vector MMSE estimate

This paper considers approximations of marginalization sums that arise in Bayesian inference problems. Optimal approximations of such marginalization sums, using a fixed number of terms, are analyzed for a simple model. The model under study is motivated by recent studies of linear regression problems with sparse parameter vectors, and of the problem of discriminating signal-plus-noise samples from noise-only samples. It is shown that for the model under study, if only one term is retained in the marginalization sum, then this term should be the one with the largest a posteriori probability. By contrast, if more than one (but not all) terms are to be retained, then these should generally not be the ones corresponding to the components with largest a posteriori probabilities.