Generalization ability of a class of empirical risk minimization algorithms and the support vector regression method

In this paper, the generalization ability of empirical risk minimization algorithms is investigated in the context of distribution-free probably approximately correct (PAC) learning. We identify a class of empirical risk minimization algorithms that are PAC, and show that the generic version of the support vector regression method belongs to the class for any given Mercer kernel. Moreover, it is shown that a regularized approximation of the generic support vector method is PAC to any given accuracy when the regularization parameter is sufficiently large. The generalization ability of the usual support vector regression method is deduced from these results.