Extraction of features and attention areas in hierarchical neural networks

A new method of automatic extraction of features and attention areas from input data is proposed in this paper using a hierarchical neural network. Human-beings can distinguish useful information from those in ambiguous and noisy environments using attention functionality. It is said that human-beings may recognize information roughly at first, and pay attention to the detailed features to confirm what the information really means. The main difficulty in realizing the attention function in the computational model is to determine the part to which the system pays attention. To determine the attention areas automatically, the proposed model consists of three hierarchical neural networks, associative memory layer, middle layer and symbol layer. The associative memory plays a great role for finding out the features of the input pattern. The middle layer corresponds to the feature extracting layer. The units in each layer connect to the units in other layers. The connection weights between the layers are modified through a Hebbian learning rule. The connection weights between the input and the middle layers represent rough features of the input patterns, while those between the symbol and the input layers represent the attention areas. In the proposed model, these features and attention areas can be extracted automatically without the designers´ knowledge