The controlled conjugate gradient type trajectory-following neural net for minimization of nonconvex functions.

This paper presents a unified way to design neural networks characterized as second-order ordinary differential equations (ODE), the trajectories of which can converge to the global minimum of nonconvex scalar functions. These neural networks, sometimes also called continuous-time algorithms, are interpreted as closed-loop control systems and the state feedback design is based on control Lyapunov functions. The focus is on a new family of continuous-time versions of conjugate gradient method, named controlled conjugate gradient (CCG) nets, that generalize heavy ball with friction (HBF) nets. For nonconvex functions, the goal of these nets is to produce trajectories that start from an arbitrary initial point and can escape from local minima, thereby increasing the chances of converging to the global minimum. Several numerical examples are given, using benchmark problems, showing that this escape and subsequent convergence to the global minimum occurs.