Local function approximation in evolutionary algorithms for the optimization of costly functions

We develop an approach for the optimization of continuous costly functions that uses a space-filling experimental design and local function approximation to reduce the number of function evaluations in an evolutionary algorithm. Our approach is to estimate the objective function value of an offspring by fitting a function approximation model over the k nearest previously evaluated points, where k = (d + 1)(d + 2)/2 and d is the dimension of the problem. The estimated function values are used to screen offspring to identify the most promising ones for function evaluation. To fit function approximation models, a symmetric Latin hypercube design (SLHD) is used to determine initial points for function evaluation. We compared the performance of an evolution strategy (ES) with local quadratic approximation, an ES with local cubic radial basis function (RBF) interpolation, an ES whose initial parent population comes from an SLHD, and a conventional ES. These algorithms were applied to a twelve-dimensional (12-D) groundwater bioremediation problem involving a complex nonlinear finite-element simulation model. The performances of these algorithms were also compared on the Dixon-Szego test functions and on the ten-dimensional (10-D) Rastrigin and Ackley test functions. All comparisons involve analysis of variance (ANOVA) and the computation of simultaneous confidence intervals. The results indicate that ES algorithms with local approximation were significantly better than conventional ES algorithms and ES algorithms initialized by SLHDs on all Dixon-Szego test functions except for Goldstein-Price. However, for the more difficult 10-D and 12-D functions, only the cubic RBF approach was successful in improving the performance of an ES. Moreover, the results also suggest that the cubic RBF approach is superior to the quadratic approximation approach on all test functions and the difference in performance is statistically significant for all test functions with dimension d greater than or equal to 4.