Sequential Sampling for Noisy Optimisation with CMA-ES

This paper proposes a novel sequential sampling scheme to allocate samples to individuals in order to maximally inform the selection step in Covariance Matrix Adaptation Evolution Strategies (CMA-ES) for noisy function optimisation. More specifically, we adopt the well-known Knowledge Gradient (KG) method to minimise the Kullback-Leibler divergence (relative entropy) between the distribution used for generating the next offspring population based on the mu selected individuals, and the distribution based on the true mu best individuals that would have been chosen in the absence of noise. Empirical tests demonstrate the benefit of integrating sequential sampling into CMA-ES, and that the proposed KG technique specifically adapted to the needs of CMA-ES indeed outperforms a more straightforward application of KG.