Stochastic Meta Descent in online kernel methods

Learning system is a method to approximate an underlying function from a finite observation data. Since batch learning has a disadvantage in dealing with large data set, online learning is proposed to prevent the computational expensive. Iterative method called Stochastic Gradient Descent (SGD) is applied to solve for the underlying function on reproducing kernel Hilbert spaces (RKHSs). To use SGD in time-varying environment, a learning rate is adjusted by Stochastic Meta Descent (SMD). The simulation results show that SMD can follow shifting and switching target function whereas the size of model can be restricted using sparse solution.