Efficient optimization methods for maximum likelihood parameter estimation

Recent research in the area of numerical optimization has led to development of the efficient algorithms based on Update Methods and Model Trust Region Techniques. The update methods are a class of iterative schemes that avoids expensive evaluations of (approximate) Hessians, yet retains the rapid convergence properties of Newton-like methods that require second-derivative (Hessian) information. Model Trust Region techniques have recently been proposed as a way of avoiding costly step-length calculations that are required by standard iterative methods based on approximate Newton methods. The purpose of this paper is to describe briefly the most sucessful of the update methods and to show how it, or more conventional methods such as scoring, can be combined with a model trust region technique to produce numerical algorithms that are ideally suited to maximum likelihood (ML) parameter estimation. Specific properties of these algorithms that are important for ML parameter estimation include a fast (superlinear) rate of convergence together with the ability to handle parameter constraints easily and efficiently.