Efficiently minimizing expensive cost functions with a hybrid evolutionary algorithm using clustering and a derivative-free optimizer: preliminary results

A novel hybrid algorithm is presented to efficiently locate the global minimum of a function where each function evaluation is expensive and no expression is available for the function nor its derivatives. The hybrid employs an evolutionary algorithm, a density cluster analysis algorithm and a derivate-free optimizer in a multi-level hierarchical structure. The hybrid algorithm utilizes information generated during the minimization to reduce the number of function evaluations and to improve its domain exploration. The hybrid was compared to an evolutionary algorithm and to a multi-start derivative-free optimizer since both are candidate algorithms to handle this global minimization problem. The algorithms were tested using a small and a large domain. Test results showed that while the evolutionary algorithm did not progress much after an initial phase the hybrid maintained a high rate of minimization throughout and accordingly provided a final result which was on average O(10⁶) more accurate for the small domain and O(10⁸) more accurate for the large domain. Furthermore, the number of function evaluations required by the multi-start derivative-free optimizer was affected by the initial random population and accordingly by an increase in the domain size. In contrast the hybrid was not affected since it employed the explorative evolutionary algorithm phase and thus was able to locate better starting nodes in a larger domain.