Momentum algorithms in neural networks and the applications in numerical algebra

Author

Zhang, Naimin

Author_Institution

Sch. of Math. & Inf. Sci., Wenzhou Univ., Wenzhou, China

fYear

2011

fDate

8-10 Aug. 2011

Firstpage

2192

Lastpage

2195

Abstract

Momentum algorithms in neural networks and the applications for solving linear systems are discussed. The sufficient and necessary conditions for the convergence of the stationary iteration with momentum acceleration are obtained. Furthermore, the optimal momentum factor which minimizes the spectral radius of the associated momentum acceleration iteration matrix is also obtained. Some numerical results have demonstrated the effectiveness of the stationary iterations with momentum acceleration.

Keywords

convergence of numerical methods; iterative methods; linear systems; mathematics computing; matrix algebra; neural nets; associated momentum acceleration iteration matrix; linear system solving; momentum algorithm; neural network; numerical algebra; optimal momentum factor; spectral radius; stationary iteration; Acceleration; Convergence; Eigenvalues and eigenfunctions; Gradient methods; Jacobian matrices; Linear systems; Neural networks; Neural networks; convergence; momentum; numerical algebra;

fLanguage

English

Publisher

ieee

Conference_Titel

Artificial Intelligence, Management Science and Electronic Commerce (AIMSEC), 2011 2nd International Conference on

Conference_Location

Deng Leng

Print_ISBN

978-1-4577-0535-9

Type

conf

DOI

10.1109/AIMSEC.2011.6011205

Filename

6011205