An empirical study of the time complexity of various error functions with conjugate gradient backpropagation

Describes an empirical comparison of the scaling behaviour of six error functions, on a conjugate gradient form of backpropagation. The authors classify the functions according to the limit behaviours of their respective error signals, as the target value and the actual output value approach opposite extremes. These limit behaviours are zero limit, finite limit, and infinite limit. Despite such a wide divergence in their limit behaviours, the authors find that all six error functions exhibit a median run-time order of approximately O(N⁴) on the N-2-N encoder. This result indicates that, while some factors affecting the scaling behaviour of standard and conjugate gradient backpropagation have been previously identified (such as saturation), other factors remain unidentified