Convergence of Gradient Descent for Low-Rank Matrix Approximation

Author

Pitaval, Renaud-Alexandre ; Wei Dai ; Tirkkonen, Olav

Author_Institution

Dept. of Commun. & Networking, Aalto Univ., Aalto, Finland

Volume

61

Issue

8

fYear

2015

fDate

Aug. 2015

Firstpage

4451

Lastpage

4457

Abstract

This paper provides a proof of global convergence of gradient search for low-rank matrix approximation. Such approximations have recently been of interest for large-scale problems, as well as for dictionary learning for sparse signal representations and matrix completion. The proof is based on the interpretation of the problem as an optimization on the Grassmann manifold and Fubiny-Study distance on this space.

Keywords

approximation theory; convergence of numerical methods; gradient methods; learning (artificial intelligence); matrix algebra; search problems; signal representation; Fubiny-Study distance; Grassmann manifold; dictionary learning; global gradient descent convergence; large-scale problems; low-rank matrix approximation; matrix completion; sparse signal representations; Approximation algorithms; Approximation methods; Convergence; Hafnium; Manifolds; Optimized production technology; Sparse matrices; Dimensionality reduction; Grassmann manifold; gradient descent; low-rank matrix; optimization;

fLanguage

English

Journal_Title

Information Theory, IEEE Transactions on

Publisher

ieee

ISSN

0018-9448

Type

jour

DOI

10.1109/TIT.2015.2448695

Filename

7131516