The effect of stochastic interconnects in artificial neural network classification

Assuming that each neural state is in some sense uncorrelated with the others, each neuron represents a computational degree of freedom available to the network. The number of degrees of freedom can be artificially increased through the use of neurons in a hidden layer, the states of which can be almost any nonlinear combination of the stimulus neural states. Such nonlinearities are generated with stochastically chosen interconnects between the input and hidden neural layers with a sigmoidal nonlinearity at each hidden neuron. The hidden-to-output interconnects are chosen to be a (trainable) projection matrix whose values are a function of the stochastically chosen interconnects and the training data. Preliminary simulations of such networks show an approach to fixed generalization boundaries as the number of hidden neurons becomes larger.<>