Statistical properties of artificial neural networks

Convergence properties of empirically estimated neural networks are examined. In this theory, an appropriate size feedforward network is automatically determined from the data. The networks studied include two- and three-layer networks with an increasing number of simple sigmoidal nodes, multiple-layer polynomial networks, and networks with certain fixed structures but an increasing complexity in each unit. Each of these classes of networks is dense in the space of continuous functions on compact subsets of d-dimensional Euclidean space, with respect to the topology of uniform convergence. It is shown how, with the use of an appropriate complexity regularization criterion, the statistical risk of network estimators converges to zero as the sample size increases. Bounds on the rate of convergence are given in terms of an index of the approximation capability of the class of networks