Incremental class learning-an approach to longlife and scalable learning

Incremental class learning (ICL) presents a possible solution to the catastrophic interference problem and provides a framework for the development of scalable learning systems. With respect to multi-class classification problems, the ICL approach can be summarized as follows. Initially the system focuses on one category. After it learns this category, it tries to identify a compact subset of features (nodes) in the hidden layers, that are crucial for the recognition of this category. The system then freezes these crucial nodes (features) by fixing their incoming weights. As a result, these features cannot be obliterated in subsequent learning. Moreover, these frozen features are available during subsequent learning and can be shared among a number of categories. Finally, as more categories are learned, the set of features gradually stabilizes and learning a new category requires less effort. We present results of applying the ICL approach to the handwritten digit recognition problem.