Design and analysis of successive approximation algorythms via bilinear control theory

Gradient techniques which use a weighting matrix to accelerate convergence are shown to be bilinear in the neighborhood of the extremum, while the gradient method itself behaves as a linear system. An investigation of their reachable sets at the extremum provides a basis for explaining the improved performance experienced in practice. Using concepts from bilinear control theory, two new algorithms are designed which achieve further improvements in performance.