Improving the performance of symmetric diffusion networks via biologically inspired constraints

Symmetric diffusion networks (SDNs) are a class of networks based upon the principles of continuous, stochastic, adaptive, and interactive processing. SDNs are basically a continuous form of a Boltzmann machine trained with the contrastive Hebbian learning algorithm. Thus, one advantage SDNs have over standard backpropagation networks is that they can learn continuous probability distributions, that is, they can learn multiple distinct outputs for a single input. However, SDNs are difficult to train, especially on large training sets. In order to improve network learning performance, neurophysiologically inspired constraints were systematically imposed upon the networks. Results indicate that the application of such constraints dramatically increases the performance of SDNs in terms of their rate of learning, and in terms of their learning appropriate internal representations