Second order associative memory models with threshold logics - eigen mode selections

The capacity of an order-d associative memory model is O(N^d /logN) where N is the memory size in bit. The exponential growth of the capacity with respect to d gives higher order models (d > 1) a significant advantage over the Hopfield network, whose capacity is limited to O(N/logN). In particular, a second order correlation memory (d = 2) has attractive properties: a small implementation cost of O(N²), a small number of spurious states, and the presence of a diagonalization form. Due to these properties, it is of both practical and scientific interests to investigate biological feasibility of such network. One disadvantage of higher order associative memory is that it cannot be implemented with simple threshold neurons or McCulloch-Pitts neurons, thus a direct implementation of its computational mechanism on a biological substrate is questionable. In this paper, we propose two approximation models of a second order associative memory using threshold logics. Both are two-layered and employ eigenvalue decomposition of the correlation tensor. The first model uses a winner-take-all mechanism and the second uses a multiple selection mechanism. Extensive numerical simulations demonstrate effectiveness of the proposed models.