Title :
Trace Ratio Problem Revisited
Author :
Jia, Yangqing ; Nie, Feiping ; Zhang, Changshui
Author_Institution :
Dept. of Autom., Tsinghua Univ., Beijing
fDate :
4/1/2009 12:00:00 AM
Abstract :
Dimensionality reduction is an important issue in many machine learning and pattern recognition applications, and the trace ratio (TR) problem is an optimization problem involved in many dimensionality reduction algorithms. Conventionally, the solution is approximated via generalized eigenvalue decomposition due to the difficulty of the original problem. However, prior works have indicated that it is more reasonable to solve it directly than via the conventional way. In this brief, we propose a theoretical overview of the global optimum solution to the TR problem via the equivalent trace difference problem. Eigenvalue perturbation theory is introduced to derive an efficient algorithm based on the Newton-Raphson method. Theoretical issues on the convergence and efficiency of our algorithm compared with prior literature are proposed, and are further supported by extensive empirical results.
Keywords :
Newton-Raphson method; approximation theory; convergence of numerical methods; data reduction; eigenvalues and eigenfunctions; learning (artificial intelligence); optimisation; perturbation theory; Newton-Raphson method; convergence method; data dimensionality reduction; eigenvalue perturbation theory; generalized eigenvalue decomposition approximation; machine learning; optimization problem; pattern recognition application; trace ratio problem; Dimensionality reduction; Newton–Raphson method; eigenvalue perturbation; trace ratio (TR);
Journal_Title :
Neural Networks, IEEE Transactions on
DOI :
10.1109/TNN.2009.2015760