org.apache.commons.math3.optimization.direct

Class CMAESOptimizer

• All Implemented Interfaces:
BaseMultivariateOptimizer<MultivariateFunction>, BaseMultivariateSimpleBoundsOptimizer<MultivariateFunction>, BaseOptimizer<PointValuePair>, MultivariateOptimizer

```public class CMAESOptimizer
extends BaseAbstractMultivariateSimpleBoundsOptimizer<MultivariateFunction>
implements MultivariateOptimizer```

An implementation of the active Covariance Matrix Adaptation Evolution Strategy (CMA-ES) for non-linear, non-convex, non-smooth, global function minimization. The CMA-Evolution Strategy (CMA-ES) is a reliable stochastic optimization method which should be applied if derivative-based methods, e.g. quasi-Newton BFGS or conjugate gradient, fail due to a rugged search landscape (e.g. noise, local optima, outlier, etc.) of the objective function. Like a quasi-Newton method, the CMA-ES learns and applies a variable metric on the underlying search space. Unlike a quasi-Newton method, the CMA-ES neither estimates nor uses gradients, making it considerably more reliable in terms of finding a good, or even close to optimal, solution.

In general, on smooth objective functions the CMA-ES is roughly ten times slower than BFGS (counting objective function evaluations, no gradients provided). For up to $N=10$ variables also the derivative-free simplex direct search method (Nelder and Mead) can be faster, but it is far less reliable than CMA-ES.

The CMA-ES is particularly well suited for non-separable and/or badly conditioned problems. To observe the advantage of CMA compared to a conventional evolution strategy, it will usually take about $30 N$ function evaluations. On difficult problems the complete optimization (a single run) is expected to take roughly between $30 N$ and $300 N2$ function evaluations.

This implementation is translated and adapted from the Matlab version of the CMA-ES algorithm as implemented in module `cmaes.m` version 3.51.

Since:
3.0
Version:
\$Id\$
• Field Summary

Fields
Modifier and Type Field and Description
`static int` `DEFAULT_CHECKFEASABLECOUNT`
Default value for `checkFeasableCount`: 0.
`static int` `DEFAULT_DIAGONALONLY`
Default value for `diagonalOnly`: 0.
`static boolean` `DEFAULT_ISACTIVECMA`
Default value for `isActiveCMA`: true.
`static int` `DEFAULT_MAXITERATIONS`
Default value for `maxIterations`: 30000.
`static RandomGenerator` `DEFAULT_RANDOMGENERATOR`
Default value for `random`.
`static double` `DEFAULT_STOPFITNESS`
Default value for `stopFitness`: 0.0.
• Fields inherited from class org.apache.commons.math3.optimization.direct.BaseAbstractMultivariateOptimizer

`evaluations`
• Constructor Summary

Constructors
Constructor and Description
`CMAESOptimizer()`
Default constructor, uses default parameters
`CMAESOptimizer(int lambda)`
```CMAESOptimizer(int lambda, double[] inputSigma)```
```CMAESOptimizer(int lambda, double[] inputSigma, int maxIterations, double stopFitness, boolean isActiveCMA, int diagonalOnly, int checkFeasableCount, RandomGenerator random, boolean generateStatistics)```
```CMAESOptimizer(int lambda, double[] inputSigma, int maxIterations, double stopFitness, boolean isActiveCMA, int diagonalOnly, int checkFeasableCount, RandomGenerator random, boolean generateStatistics, ConvergenceChecker<PointValuePair> checker)```
• Method Summary

Methods
Modifier and Type Method and Description
`protected PointValuePair` `doOptimize()`
Perform the bulk of the optimization algorithm.
`List<RealMatrix>` `getStatisticsDHistory()`
`List<Double>` `getStatisticsFitnessHistory()`
`List<RealMatrix>` `getStatisticsMeanHistory()`
`List<Double>` `getStatisticsSigmaHistory()`
• Methods inherited from class org.apache.commons.math3.optimization.direct.BaseAbstractMultivariateSimpleBoundsOptimizer

`getLowerBound, getUpperBound, optimize, optimize`
• Methods inherited from class org.apache.commons.math3.optimization.direct.BaseAbstractMultivariateOptimizer

`computeObjectiveValue, getConvergenceChecker, getEvaluations, getGoalType, getMaxEvaluations, getStartPoint`
• Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`
• Methods inherited from interface org.apache.commons.math3.optimization.BaseMultivariateOptimizer

`optimize`
• Methods inherited from interface org.apache.commons.math3.optimization.BaseOptimizer

`getConvergenceChecker, getEvaluations, getMaxEvaluations`
• Constructor Detail

• CMAESOptimizer

`public CMAESOptimizer()`
Default constructor, uses default parameters
• CMAESOptimizer

`public CMAESOptimizer(int lambda)`
Parameters:
`lambda` - Population size.
• CMAESOptimizer

```public CMAESOptimizer(int lambda,
double[] inputSigma)```
Parameters:
`lambda` - Population size.
`inputSigma` - Initial search volume; sigma of offspring objective variables.
• CMAESOptimizer

```public CMAESOptimizer(int lambda,
double[] inputSigma,
int maxIterations,
double stopFitness,
boolean isActiveCMA,
int diagonalOnly,
int checkFeasableCount,
RandomGenerator random,
boolean generateStatistics)```
Parameters:
`lambda` - Population size.
`inputSigma` - Initial search volume; sigma of offspring objective variables.
`maxIterations` - Maximal number of iterations.
`stopFitness` - Whether to stop if objective function value is smaller than `stopFitness`.
`isActiveCMA` - Chooses the covariance matrix update method.
`diagonalOnly` - Number of initial iterations, where the covariance matrix remains diagonal.
`checkFeasableCount` - Determines how often new random objective variables are generated in case they are out of bounds.
`random` - Random generator.
`generateStatistics` - Whether statistic data is collected.
• CMAESOptimizer

```public CMAESOptimizer(int lambda,
double[] inputSigma,
int maxIterations,
double stopFitness,
boolean isActiveCMA,
int diagonalOnly,
int checkFeasableCount,
RandomGenerator random,
boolean generateStatistics,
ConvergenceChecker<PointValuePair> checker)```
Parameters:
`lambda` - Population size.
`inputSigma` - Initial search volume; sigma of offspring objective variables.
`maxIterations` - Maximal number of iterations.
`stopFitness` - Whether to stop if objective function value is smaller than `stopFitness`.
`isActiveCMA` - Chooses the covariance matrix update method.
`diagonalOnly` - Number of initial iterations, where the covariance matrix remains diagonal.
`checkFeasableCount` - Determines how often new random objective variables are generated in case they are out of bounds.
`random` - Random generator.
`generateStatistics` - Whether statistic data is collected.
`checker` - Convergence checker.
• Method Detail

• getStatisticsSigmaHistory

`public List<Double> getStatisticsSigmaHistory()`
Returns:
History of sigma values.
• getStatisticsMeanHistory

`public List<RealMatrix> getStatisticsMeanHistory()`
Returns:
History of mean matrix.
• getStatisticsFitnessHistory

`public List<Double> getStatisticsFitnessHistory()`
Returns:
History of fitness values.
• getStatisticsDHistory

`public List<RealMatrix> getStatisticsDHistory()`
Returns:
History of D matrix.
• doOptimize

`protected PointValuePair doOptimize()`
Perform the bulk of the optimization algorithm.
Specified by:
`doOptimize` in class `BaseAbstractMultivariateOptimizer<MultivariateFunction>`
Returns:
the point/value pair giving the optimal value for the objective function.