Chaos in discrete learning systems

Departing from the customary view of the sigmoid thresholding functions as smooth nonlinearities introduced into multilayer perceptron networks to enable continuously differentiable gradient descents toward optimal solutions minimizing some error norm, here a different more fundamental point of view is advanced: the intrinsic local dynamics of the network become that of the quadratic map of the chaos theory. This new insight aids understanding of important supervised learning algorithms such as the widely used Backpropagation scheme.