The surrogate gradient algorithm for Lagrangian relaxation method

A key step of Lagrangian relaxation is to optimize the dual function, and the subgradient method is frequently used when the dual function is nondifferentiable. However, the subgradient method requires minimizing all the subproblems to obtain the subgradient direction, and for problems of large size this may be very time consuming. To overcome this difficulty, the “interleaved subgradient method” minimizes only one subproblem to obtain a direction. Numerical results show that the interleaved subgradient method converges faster than the subgradient method, though algorithm convergence was not established. In this paper, the “surrogate subgradient method” is constructed, where a direction can be obtained without minimizing all the subproblems. In fact, only near optimization of one subproblem is necessary to obtain a proper surrogate subgradient direction. The convergence of the algorithm is proved, where the interleaved subgradient method can be viewed as a special case of this general method. Compared with methods which take efforts to find a better direction, the surrogate gradient method saves efforts in obtaining a direction and thus provides a different approach which is especially powerful for large size problems