On the capacity and generalisation ability of more realistic neurons

The effect on the storage capacity and generalisation ability of neurons with temporal, stochastic, and nonlinear features is analyzed using statistical methods. It is shown that addition of temporal processing powers allows for an increase of storage capacity and does not lead to any degradation of generalization powers. Stochasticity at synapses allows for capacity analysis and also leads to a Gibb´s distribution on weight space with the temperature determined by the square of the postsynaptic efficacy multiplied by the variance of the presynaptic distribution. It is argued that postsynaptic plasticity would generate better generalization powers. In addition, the authors discuss a simplified model of the integrating probabilistic RAM by extending their analysis of stochasticity to the case of a binary threshold neuron with nonlinear synaptic inputs