Circuit complexity for neural computation

One common model for artificial neural networks is the threshold circuit multilayer feedforward network of linear threshold gates. The authors provide a rigorous analysis of this model via a circuit complexity theoretic approach by focusing on the basic computational properties of threshold circuits with binary inputs. The model of threshold circuits is shown to be computationally more powerful than the conventional model of AND-OR logic circuits. In particular, the best known results on the depth of threshold circuits are presented in implementing common arithmetic functions such as multiplication, division, and sorting. The issues of depth-size tradeoffs are investigated by demonstrating that a small increase in the depth can significantly decrease the size required in the threshold circuit for the class of symmetric functions