The Incremental Risk Functional: Basics of a Novel Incremental Learning Approach

Incremental learning gets increasing attention in research and practice as it has the advantages of continuous adaptation and handling big data with a low computation and memory demand at the same time. Several approaches have been proposed recently for online learning, but only few work has been done to regard the influence of the approximation structure. Hence, we introduce the incremental risk functional which directly incorporates knowledge about the approximation structure into its parameter update. Exemplary, we apply this approach to regression estimation through linear-in-parameter approximators. We show that the resulting learning algorithm converges and changes the global functional behavior only as little as necessary with every learning step, thus resulting in a stable incremental learning approach.