Can artificial neural networks discover useful regularities?

Argues that the success of backpropagation artificial neural networks (BP ANNs) depends on input representations which recode input parameters in terms of low-order input statistics. We provide an analysis of how BP ANNs can take advantage of two types of low-order statistical effects to learn a given problem. First, BP ANNs utilise the spurious low-order statistics associated with almost any `natural´ problem. These statistics provide partial information about the underlying mapping, and therefore do not, in general, permit accurate generalisation. Second, it as always possible to hand-craft inputs so that a parameter which was originally coded by relations between input components is coded by a single input component. Such a recoding enables ANNs to perform tasks on which they would otherwise fail. We conjecture that many `natural´ problems represent sparse codings of some underlying relational mapping. If true, this suggests that the ability of BP ANNs to solve such problems is more apparent than real, because BP ANNs rely upon informative, but essentially spurious, correlations between input and output variables