Designing the Boltzmann estimation of multivariate normal distribution: Issues, goals and solutions

This paper introduces a new Estimation of Distribution Algorithm (EDA) based on the multivariate Boltzmann distribution. In this work, the design variables and the energy function of the Boltzmann distribution are continuous. Note that since the population has finite size, it can only approximate a continuous Boltzmann distribution with some error. In order to tackle this issue, the parameter estimators for the mean vector and covariance matrix of a Multivariate Normal Density that approximate the Boltzmann density, are derived by minimizing the Kullback-Leibler divergence. The algorithm introduced here uses one energy function for the mean estimator and another for the covariance matrix estimator. The first function places the probability mass around the most promising regions by assigning larger weights to individuals with higher fitness. However, the second function orients the covariance matrix along improving directions by assigning larger weights to individuals with lower fitness. Our proposal combines the conveniences of linear weights with a simple annealing schedule to regulate the exploration and exploitation of the search process. The resulting algorithm is named the Boltzmann Estimation of Multivariate Normal Algorithm (BEMNA). By applying the developed formulae the BEMNA is capable of adapting the structure of a density model to the promisory search directions. BEMNA is tested with a benchmark of 16 functions and contrasted with the AMaLGaM algorithm, a state of the art EDA. Statistical tests of the experimental data show the competitiveness of the proposed algorithm.