On Bayesian decision-making and approximation of probability densities

An approximation of a true, unknown, posterior probability density (pd) representing some real state-space system is presented as Bayesian decision-making among a set of possible descriptions (models). The decision problem is carefully defined on its basic elements and it is shown how it leads to the use of the Kullback-Leibler (KL) divergence [11] for evaluating a loss of information between the unknown posterior pd and its approximation. The resulting algorithm is derived on a general level, allowing specific algorithms to be designed according to a selected class of the probability distributions. A concrete example of the algorithm is proposed for the Gaussian case. An experiment is performed assuming that none of the possible descriptions is precisely identical with the unknown system.