Radial basis function regression using trans-dimensional sequential Monte Carlo

We consider the general problem of sampling from a sequence of distributions that is defined on a union of sub-spaces. We will illustrate the general approach on the problem of sequential radial basis function (RBF) regression where the number of kernels is variable and unknown. Our approach, which we term trans-dimensional sequential Monte Carlo (TD-SMC), is based on a generalisation of importance sampling to spaces of variable dimension. In the spirit of P. Del Moral and A. Doucet (2002) we augment the target parameter space at the current time step with an auxiliary space corresponding to the parameters at the previous time step. This facilitates the design of efficient proposal distributions, which can then be formulated as moves from the auxiliary parameter space to the target parameter space, lending our algorithm its sequential character. These proposals are very general, and may include within model moves to update parameters, and trans-dimensional birth or death moves to add or remove parameters when appropriate. From this perspective our approach is reminiscent of the reversible jump Markov Chain Monte Carlo (RJ-MCMC) algorithm [P.J. Green, 1995].