Generalized networks for complex function modeling

A generalized neural network architecture and learning algorithm are proposed that are capable of implementing a wide variety of neural and statistical function estimation paradigms, including basis functions, splines, polynomial neural networks, multilayer perceptrons, recurrent networks, and others. The discussion begins with a description of a generic nodal element that can perform a number of user-defined linear and nonlinear transformations. These nodal elements are combined into networks using an information-theoretic approach that reduces excess network complexity. Finally, an iterative Gauss-Newton training algorithm is developed, and it is shown how this algorithm maybe used to optimize the network for a variety of loss functions. The intent is to provide insight into both neural and statistical modeling by exploring the relationships between existing paradigms and by providing a technique that allows the best aspects of existing paradigms to be combined into novel function estimation strategies