org.apache.commons.math.optimization.direct
Class CMAESOptimizer

java.lang.Object
  org.apache.commons.math.optimization.direct.BaseAbstractScalarOptimizer<MultivariateRealFunction>
      org.apache.commons.math.optimization.direct.CMAESOptimizer

All Implemented Interfaces:

BaseMultivariateRealOptimizer<MultivariateRealFunction>, BaseOptimizer<RealPointValuePair>, MultivariateRealOptimizer

public class CMAESOptimizer
extends BaseAbstractScalarOptimizer<MultivariateRealFunction>
implements MultivariateRealOptimizer

An implementation of the active Covariance Matrix Adaptation Evolution Strategy (CMA-ES) for non-linear, non-convex, non-smooth, global function minimization. The CMA-Evolution Strategy (CMA-ES) is a reliable stochastic optimization method which should be applied if derivative-based methods, e.g. quasi-Newton BFGS or conjugate gradient, fail due to a rugged search landscape (e.g. noise, local optima, outlier, etc.) of the objective function. Like a quasi-Newton method, the CMA-ES learns and applies a variable metric on the underlying search space. Unlike a quasi-Newton method, the CMA-ES neither estimates nor uses gradients, making it considerably more reliable in terms of finding a good, or even close to optimal, solution.

In general, on smooth objective functions the CMA-ES is roughly ten times slower than BFGS (counting objective function evaluations, no gradients provided). For up to $N=10$ variables also the derivative-free simplex direct search method (Nelder and Mead) can be faster, but it is far less reliable than CMA-ES.

The CMA-ES is particularly well suited for non-separable and/or badly conditioned problems. To observe the advantage of CMA compared to a conventional evolution strategy, it will usually take about $30\; N$ function evaluations. On difficult problems the complete optimization (a single run) is expected to take roughly between $30\; N$ and $300\; N2$ function evaluations.

This implementation is translated and adapted from the Matlab version of the CMA-ES algorithm as implemented in module cmaes.m version 3.51.

For more information, please refer to the following links:

• Matlab code
• Introduction to CMA-ES
• Wikipedia

Since:

3.0

Version:

$Id$

Field Summary

static int	DEFAULT_CHECKFEASABLECOUNT Default value for checkFeasableCount: 0.
static int	DEFAULT_DIAGONALONLY Default value for diagonalOnly: 0.
static boolean	DEFAULT_ISACTIVECMA Default value for isActiveCMA: true.
static int	DEFAULT_MAXITERATIONS Default value for maxIterations: 30000.
static RandomGenerator	DEFAULT_RANDOMGENERATOR Default value for random.
static double	DEFAULT_STOPFITNESS Default value for stopfitness: 0.0.

Fields inherited from class org.apache.commons.math.optimization.direct.BaseAbstractScalarOptimizer

evaluations

Constructor Summary

CMAESOptimizer()
Default constructor, uses default parameters

CMAESOptimizer(int lambda)

CMAESOptimizer(int lambda, double[] inputSigma, double[][] boundaries)

CMAESOptimizer(int lambda, double[] inputSigma, double[][] boundaries, int maxIterations, double stopfitness, boolean isActiveCMA, int diagonalOnly, int checkFeasableCount, RandomGenerator random, boolean generateStatistics)

Method Summary

protected RealPointValuePair	doOptimize() Perform the bulk of the optimization algorithm.
java.util.List<RealMatrix>	getStatisticsDHistory()
java.util.List<java.lang.Double>	getStatisticsFitnessHistory()
java.util.List<RealMatrix>	getStatisticsMeanHistory()
java.util.List<java.lang.Double>	getStatisticsSigmaHistory()

Methods inherited from class org.apache.commons.math.optimization.direct.BaseAbstractScalarOptimizer

computeObjectiveValue, getConvergenceChecker, getEvaluations, getGoalType, getMaxEvaluations, getStartPoint, optimize, setConvergenceChecker

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.commons.math.optimization.BaseMultivariateRealOptimizer

optimize

Methods inherited from interface org.apache.commons.math.optimization.BaseOptimizer

getConvergenceChecker, getEvaluations, getMaxEvaluations, setConvergenceChecker

Field Detail

DEFAULT_CHECKFEASABLECOUNT

public static final int DEFAULT_CHECKFEASABLECOUNT

Default value for checkFeasableCount: 0.

See Also:

Constant Field Values

DEFAULT_STOPFITNESS

public static final double DEFAULT_STOPFITNESS

Default value for stopfitness: 0.0.

See Also:

Constant Field Values

DEFAULT_ISACTIVECMA

public static final boolean DEFAULT_ISACTIVECMA

Default value for isActiveCMA: true.

See Also:

Constant Field Values

DEFAULT_MAXITERATIONS

public static final int DEFAULT_MAXITERATIONS

Default value for maxIterations: 30000.

See Also:

Constant Field Values

DEFAULT_DIAGONALONLY

public static final int DEFAULT_DIAGONALONLY

Default value for diagonalOnly: 0.

See Also:

Constant Field Values

DEFAULT_RANDOMGENERATOR

public static final RandomGenerator DEFAULT_RANDOMGENERATOR

Default value for random.

Constructor Detail

CMAESOptimizer

public CMAESOptimizer()

Default constructor, uses default parameters

CMAESOptimizer

public CMAESOptimizer(int lambda)

Parameters:

lambda - Population size.

CMAESOptimizer

public CMAESOptimizer(int lambda,
                      double[] inputSigma,
                      double[][] boundaries)

Parameters:

lambda - Population size.

inputSigma - Initial search volume - sigma of offspring objective variables.

boundaries - Boundaries for objective variables.

CMAESOptimizer

public CMAESOptimizer(int lambda,
                      double[] inputSigma,
                      double[][] boundaries,
                      int maxIterations,
                      double stopfitness,
                      boolean isActiveCMA,
                      int diagonalOnly,
                      int checkFeasableCount,
                      RandomGenerator random,
                      boolean generateStatistics)

Parameters:

lambda - Population size.

inputSigma - Initial search volume - sigma of offspring objective variables.

boundaries - Boundaries for objective variables.

maxIterations - Maximal number of iterations.

stopfitness - stop if objective function value < stopfitness.

isActiveCMA - Chooses the covariance matrix update method.

diagonalOnly - Number of initial iterations, where the covariance matrix remains diagonal.

checkFeasableCount - Determines how often new. random objective variables are generated in case they are out of bounds.

random - Used random generator.

generateStatistics - Indicates whether statistic data is collected.

Method Detail

getStatisticsSigmaHistory

public java.util.List<java.lang.Double> getStatisticsSigmaHistory()

Returns:

History of sigma values.

getStatisticsMeanHistory

public java.util.List<RealMatrix> getStatisticsMeanHistory()

Returns:

History of mean matrix.

getStatisticsFitnessHistory

public java.util.List<java.lang.Double> getStatisticsFitnessHistory()

Returns:

History of fitness values.

getStatisticsDHistory

public java.util.List<RealMatrix> getStatisticsDHistory()

Returns:

History of D matrix.

doOptimize

protected RealPointValuePair doOptimize()

Perform the bulk of the optimization algorithm.

Specified by:

doOptimize in class BaseAbstractScalarOptimizer<MultivariateRealFunction>

Returns:

the point/value pair giving the optimal value for the objective function.