Homotopy continuation method for neural networks

This paper proposes the use of homotopy continuation method for training the neural networks. Homotopy methods have been demonstrated to be superior to gradient descent methods in many applications such as nonlinear optimization problems, mathematical analysis, and more recently, signal processing. The power of these methods lies in their ability to provide globally convergent solutions, and under certain conditions, exhaustive solutions. The approach, consists of formulating a homotopy function which defines a family of paths from known solutions of a simple problem to the desired solutions of a more complex problem. The theoretical development of homotopy continuation methods is briefly described