Superimposing memory by dynamic and spatial changing synaptic weights

A novel neural network model is presented for incremental learning tasks where networks are required to learn new knowledge without forgetting the old. An essential core of the proposed neural network structure is its dynamic and spatial changing weights (DSCWs). A learning scheme is developed for the formulation of the dynamic changing weights, while a structural adaptation is formulated by the spatial changing (growing) connecting weights. As the new. synaptic connections are formed, a new network structure is superimposed on the previous structure. In this superimposition, to avoid disturbing the past knowledge due to the creation of new connections, a restoration mechanism is introduced by using the DSCWs. The usefulness of the proposed model is demonstrated by using pattern classification and system identification tasks.