Optimizing the hard address distribution for sparse distributed memories

A N-bit sparse distributed memory (SDM) is realized by sampling the 2N address space and building a hard location for each address in the sample. In the basic SDM model the hard addresses are chosen from a uniform distribution, which is optimal only if the data stored are distributed randomly and uniformly. In this paper the authors study the problem of optimizing the hard address distribution for sparse distributed memories for nonuniform data distributions using the information given by a representative sample data set. The authors report experiments on both the basic model proposed by Kanerva as well as on variations using the hyperplane and selected coordinate models. For the optimization a simple sampling scheme and Kohonen self-organizing maps were used. The generalization performance of the basic model was compared to the optimized ones. The empirical results support the theoretical expectations that the performance of SDMs can be significantly improved by optimizing the hard address distribution