Accelerating Local Convergence of the Information Theory-Based Algorithm for Principal Component Analysis

This work presents an adaptive principal component analysis (PCA) algorithm that equips the information theory-based algorithm with the momentum mechanism, which can be used to speedup the convergence and to stablize the weight trajectory. Issues such as equilibria, local stability, and convergence improvement are addressed. The theoretical analysis is largely facilitated by the ordinary differential equation (ODE) approach which characterizes the averaged convergence behavior of a nonlinear dynamic system. A demonstrative example is given to show the possible merits of this scheme. It is noteworthy to point out that the scope of this research is confined to local convergence improvement, whereas global convergence acceleration is a different issue that we have not covered.