Complex dynamics in learning systems

To describe the dynamics of learning an input–output relation from a set of examples, the evolution of an appropriate choice of macroscopic dynamical variables have to be found. Recent progress in on-line learning only addresses the often unrealistic case of an infinite training set. For restricted training sets, previous studies have so far been limited to asymptotic dynamics or simple learning rules. Using the cavity method and diagrammatic techniques, we introduce a new framework to model batch learning of restricted sets of examples, widely applicable to any learning cost function, and fully taking into account the temporal correlations introduced by the recycling of the examples.