Optimization Using a Modified Second-Order Approach With Evolutionary Enhancement

An optimization algorithm is presented which effectively combines the desirable characteristics of both gradient descent and evolutionary computation into a single robust algorithm. The method uses a population-based gradient approximation which allows it to recognize surface behavior on both large and small scales. By adjusting the population radius between iterations, the algorithm is able to escape local minima by shifting its focus onto global trends rather than local behavior. The algorithm is compared experimentally with existing methods over a set of relevant test cases, and each method is ranked on the basis of both reliability and rate of convergence. For each case, the algorithm is shown to outperform other methods in terms of both measures of performance, truly making it the best of both worlds.