Neural networks as function approximators: teaching a neural network to multiply

Artificial neural networks (ANNs) were first proposed, by Hecht-Nieisen (1987), as multivariate function approximators based on Kolmogorov´s theorem. Since then, several researchers have proven that multilayer ANNs, with an arbitrary squashing function in the hidden layer, can approximate any multivariate function to any degree of accuracy. Based on these results, researchers have attempted to train backpropagation networks to realize arbitrary functions. Although their results are encouraging, this technique has many shortcomings and may lead to an inappropriate response by the network. In this paper, the authors present an alternative neural network architecture, based on cascaded univariate function approximators, which can be trained to multiply two real numbers and may be used to realize arbitrary multivariate function mappings