Extensions to projection pursuit learning networks with parametric smoothers

A neural network which can grow its own structure on the training attracts a lot of research. Projection pursuit learning networks (PPLNs) and cascaded correlation learning networks (CCLNs) are two such neural networks. Unlike a CCLN where cascaded connections from the existing hidden units to the new candidate hidden unit are required to establish high-order nonlinearity in approximating the residual error, a PPLN approximates the high-order nonlinearity by using trainable nonlinear unit activation functions (e.g., hermite polynomials). To relax the necessity of predefined smoothness of nonlinearity in a PPLN, we propose in this paper a new learning network, called a pooling projection pursuit network (PPPN), which alleviates the the critical requirement of adequate order pre-selection without suffering the regression performance degradation as often encountered in a previously proposed cascaded projection pursuit network