Structured models from structured data: emergence of modular information processing within one sheet of neurons

We investigate how structured information processing within a neural net can emerge as a result of unsupervised learning from data. Our model consists of input neurons and hidden neurons which are recurrently connected and which represent the thalamus and the cortex, respectively. On the basis of a maximum likelihood framework the task is to generate given input data using the code of the hidden units. Hidden neurons are fully connected allowing for different roles to play within the unfolding time-dynamics of this data generation process. One parameter which is related to the sparsity of neuronal activation varies across the hidden neurons. As a result of training the net captures the structure of the data generation process. The results imply that the division of the cortex into laterally and hierarchically organized areas can evolve to a certain degree as an adaptation to the environment