A connectionist learning with high-order functional networks and its internal representation

A novel architecture for supervised neural network learning is proposed. The necessary conditions of the network architecture for learning the structures of continuous mappings are obtained. The novel network architecture comprises high-order functional networks with some high-order functional units as input units. High-order functional networks trained with back-propagation can generalize and infer the highly nonlinear structures of the continuous mappings. The internal representation capability of the high-order functional networks is analyzed. Nonlinear mappings can be characterized by the features of their extrema and curvatures. The combination of the high order functional input units and the hidden units makes it possible to realize and learn a proper internal representation of the networks for extracting these features of the continuous mappings. On the basis of these internal representation capabilities, a methodology for determining the network architecture and parameters is proposed