Learning in feedforward networks with nonsmooth functions: an I ∞ example

The authors consider the problem of learning in networks where some or all of the functions involved are not smooth. Examples of such networks are those whose neural transfer functions are piecewise-linear and those whose error function is defined in terms of the I_∞ norm. The authors draw upon some results from the field of nonsmooth optimization (NSO) to present an algorithm for the nonsmooth case. They demonstrate the viability of using NSO for training networks in cases that standard procedures, with their implicit smoothness assumption, would find difficult or impossible. The motivation for this work arose out of the fact that it has been possible to show that an error function based on the I_∞ norm overcomes the difficulties which can occur when using backpropagation´s I₂ norm