Optimally distributed computation in augmented networks

The concept is introduced of `optimally distributed computation´ in feedforward neural networks via regularisation of weight saliency. By constraining the relative importance of the parameters, computation can be distributed thinly and evenly throughout the network. It is proposed that this will have beneficial effects on fault-tolerance performance and generalisation ability in augmented network architectures. These theoretical predictions are verified by simulation experiments on two problems; one artificial and the other a `real-world´ task. Regularisation terms are presented for distributing neural computation optimally