Title :
On relative convergence properties of principal component analysis algorithms
Author :
Chatterjee, Chanchal ; Roychowdhury, Vwani P. ; Chong, Edwin K P
Author_Institution :
GDE Syst. Inc., San Diego, CA, USA
fDate :
3/1/1998 12:00:00 AM
Abstract :
We investigate the convergence properties of two different stochastic approximation algorithms for principal component analysis, and analytically explain some commonly observed experimental results. In our analysis, we use the theory of stochastic approximation, and in particular the results of Fabian (1968), to explore the asymptotic mean square errors (AMSEs) of the algorithms. This study reveals the conditions under which the algorithms produce smaller AMSEs, and also the conditions under which one algorithm has a smaller AMSE than the other. Experimental study with multidimensional Gaussian data corroborate our analytical findings. We next explore the convergence rates of the two algorithms. Our experiments and an analytical explanation reveals the conditions under which the algorithms converge faster to the solution, and also the conditions under which one algorithm converges faster than the other
Keywords :
convergence of numerical methods; eigenvalues and eigenfunctions; error analysis; least mean squares methods; mathematics computing; statistical analysis; adaptive eigendecomposition; asymptotic mean square errors; eigenvectors; multidimensional Gaussian data; principal component analysis; relative convergence; stochastic approximation; Algorithm design and analysis; Approximation algorithms; Convergence; Data analysis; Eigenvalues and eigenfunctions; Hebbian theory; Mean square error methods; Multidimensional systems; Principal component analysis; Stochastic processes;
Journal_Title :
Neural Networks, IEEE Transactions on
