Characterization of a higher-order associative memory that uses product units

The characteristics of a novel 3-layer feedforward neural network that can be used as a higher-order associative memory is studied. The network structure consists of a hidden layer that contains product units for which each input is raised to a power determined by a trainable weight. The operation of the network consists of three steps: 1) preprocess the prescribed associative vectors and determine the principal connection weights (i.e., the first phase learning); 2) estimate the required number of product units and connections based on the results from (1); and 3) train the network using the backpropagation algorithm until satisfactory recall accuracy is achieved. The use of this two-phase learning is shown to enable us to achieve: 1) learning without requiring long training time; and 2) major reduction in the number of connection weights. Various interesting characteristics of the network, including input noise tolerance and fault tolerance, can be seen in this network