Shifted limited-memory DFP systems

Quasi-Newton methods are optimization techniques suitable for large data-generated problems that are subject to errors and uncertainty since these methods only use first-order information, do not require storing large matrices, and are easily scalable to large problems. Generally speaking, because they make use of multiple updates to Hessian approximations, these methods enjoy faster convergence rates on nonconvex problems than other first-order methods. In this paper, we consider Davidon-Fletcher-Powell (DFP) quasi-Newton matrices. We derive a compact formulation of DFP updates and propose a method to solve shifted DFP quasi-Newton systems of the form (B ; σI)x = y where M_DFP = B^-1 is the DFP matrix approximation to the inverse Hessian and σ > 0 is a positive scalar. This paper extends work done in [1] and [2] on general quasi-Newton matrices (and, in particular, L-BFGS matrices) to DFP matrices. Finally, we generalize this method to solve systems of the form (B;G)x = y, where G is a symmetric positive-definite matrix.