On convergence and accuracy of state-space approximations of squared exponential covariance functions

In this paper we study the accuracy and convergence of state-space approximations of Gaussian processes (GPs) with squared exponential (SE) covariance functions. This kind of approximations is important in construction of Kalman filtering and smoothing based GP regression algorithms, which have a linear (as opposed to conventional cubic) computational complexity in the number of training samples. We start by deriving general conditions for a spectral density approximation to give a uniform convergence of the mean and covariance functions. We then show that the previously proposed reciprocal Taylor series approximation gives such uniform convergence. We then derive new approximations based on Padé approximants of the exponential function as well as approximations inspired by the central limit theorem, and prove their uniform convergence. Finally, we compare accuracy of the different approximations numerically.