The Exchange Monte Carlo Method for Bayesian Learning in Singular Learning Machines

A lot of singular learning machines such as neural networks, normal mixtures, Bayesian networks and hidden Markov models are widely used in practical information systems. In these learning machines, it was clarified that the Bayesian learning provides the better generalization performance than the maximum likelihood method. However, it needs huge computational costs to realize the Bayesian posterior distribution by the conventional Monte Carlo method. In this paper, we propose that the exchange Monte Carlo method is appropriate for the Bayesian learning of singular learning machines, and experimentally show that it attains the better posterior distribution than the conventional Monte Carlo method.