Particle filters for neural network training

The generic particle filter has been applied with success to neural network training, but the proposal distribution chosen by the generic particle filter does not incorporate the latest observations which can deteriorate the performance of the algorithm. In this paper, we propose to use the iterated extended Kalman filter to generate proposal distribution in particle filtering framework. The iterated extended Kalman filter can make efficient use of the latest observation, and the generated proposal distribution can approximate the posterior distribution of neural network weights much better and improve the performance of particle filter. The experimental results show that the proposed particle filter outperforms the generic particle filter and the EKPF in neural network training.