Direct search computational methods for maximum likelihood parameter estimation

Though the theory of the maximum likelihood method for parameter estimation in dynamic systems is well developed, its application to complex systems has been limited by the unavailability of fast and reliable computational algorithms to maximize the likelihood function. Gradient-based algorithms have been mostly used for this purpose until now. A summary of such techniques was given by Gupta and Mehra (1974). Recent experience with algorithms which do not explicitly compute the gradients of the innovations or the likelihood function indicates that such algorithms offer potential benefits over gradient-based algorithms. This paper surveys direct search optimization methods and compares them to the algorithms requiring a direct computation of the gradients. An example problem is presented to show the class of problems for which such methods are likely to be useful.