Characterization of Degree of Approximation for Neural Networks with One Hidden Layer

There have been various studies on approximation ability of feedforward neural networks (FNNs). Most of the existing studies are, however, only concerned with density or upper bound estimation on how a function can be approximated by an FNN, and consequently, the essential approximation ability of an FNN can not been revealed. In this paper, by establishing both upper and lower bound estimations on degree of approximation, the essential approximation ability of a class of FNNs is clarified in terms of the modulus of smoothness of functions to be approximated. The involved FNNs can not only approximate any continuous functions arbitrarily well, but also provide an explicit lower bound on number of hidden units required. By making use of approximation tools, it is shown that when the functions to be approximated are Lipschitzian, the approximation speed of the FNNs is determined by modulus of smoothness of the functions