Descent Search with Mean Direction Evolution Strategies Based on GPU with CUDA

In this paper, we first present a hybrid optimization method of Covariance Matrix Adaptation Evolution Strategy, (called MDDS-C-CMA-ES), which is based on Cholesky decomposition (Cholesky-CMA-ES) and local descent search with mean direction. Then we design a parallel version of the method based on the GPU with C-CUDA to solve the problem of large dimensionality. The main advantage of the MDDS-C-CMA-ES method is that every individual is locally searched with the direction that point to the mean vector before being used to calculate a new mean vector. The algorithm can effectively accelerate the convergence speed of the CMA-ES. And the parallel algorithm on the GPU can significantly reduce the computation time further. In order to test the performance we present two experiments. First, we use the serial algorithm to optimize some classical benchmark functions. The results show our method has better performance than NES^[2] and CMA-ES. Then we test the performance of the parallel version on some benchmark functions with 1000-dimension and 1500-dimension. The results show the algorithm obtains a 68x speedup in case of 1000-dimension and 90× speedup in case of 1500 dimension.