How to Measure the Essential Approximation Capability of a FNN

In this paper, we firstly review the recent work on approximation properties of feedforward neural networks (FNN). We summarize the state-of-the-art results and explain their impact and significance. For feedforward neural networks, it is revealed the essential order of their approximation. It is proven that for any continuous function defined on a compact set of R^d, there exist three layer of FNNs with fixed number of hidden neurons that attain the essential order. Under certain assumption on the FNNs, the ideal upper bound and lower bound estimations on approximation precision of the FNNs are provided. The obtained results not only characterize the intrinsic property of approximation of the FNNs, but also uncover the implicit relationship between the precision (speed) and the number of hidden neurons of the FNNs.