Nearest neighbor rules PAC-approximate feedforward networks

The problem of function estimation using feedforward neural networks based on an independently and identically generated sample is addressed. The feedforward networks with a single hidden layer of 1/(1+e ^-γz)-units and bounded parameters are considered. It is shown that given a sufficiently large sample, a nearest neighbor rule approximates the best neural network such that the expected error is arbitrarily bounded with an arbitrary high probability. The result is extendible to other neural networks where the hidden units satisfy a suitable Lipschitz condition. A result of practical interest is that the problem of computing a neural network that approximates (in the above sense) the best possible one is computationally difficult, whereas a nearest neighbor rule is linear-time computable in terms of the sample size