On practical constraints of approximation using neural networks on current digital computers

Goal of this paper is to highlight the most common problems and constraints which accompany the implementation of artificial neural networks on current digital computers. We focus on feed-forward multilayer neural networks, i.e. multilayer perceptrons, in role of function approximators. Multiple constraints of approximation by neural networks are discussed within the paper, taking into account research from the previous two decades. We address the issues of structural construction of feed-forward neural networks, learning and data pretreatment. Conclusions stated by universal approximation theorem cannot be blindly applied to implementations on real hardware without considering the limitations such as finite accuracy of floating point operations and data type overflow issues. This fact is emphasized in the paper.