Online Bayesian Learning With Natural Sequential Prior Distribution

Online Bayesian learning has been successfully applied to online learning for multilayer perceptrons and radial basis functions. In online Bayesian learning, typically, the conventional transition model has been used. Although the conventional transition model is based on the squared norm of the difference between the current parameter vector and the previous parameter vector, the transition model does not adequately consider the difference between the current observation model and the previous observation model. To adequately consider this difference between the observation models, we propose a natural sequential prior. The proposed transition model uses a Fisher information matrix to consider the difference between the observation models more naturally. For validation, the proposed transition model is applied to an online learning problem for a three-layer perceptron.