Optimizers

Choosing an optimizer

All optimizers minimize. Scalar optimizer result objects contain a best point x, objective value y, evaluation and iteration counts, and an integer stop code. ModeResult instead contains its population matrices plus iteration and stop state; MAP-Elites retains its results in Archive.

Parallel evaluation: ask/tell and retry

fcmaes-core offers two independent parallelism boundaries, both intended to improve the result returned after a fixed user-visible wait:

1. retry runs complete optimizer instances concurrently. It is especially useful for multimodal problems and for adapting otherwise serial external optimizers.
2. Ask/tell separates candidate generation from evaluation. Evaluate one population concurrently, preserve its order, then tell the scores back to the same optimizer. This is especially useful when objective calls are expensive.

Thus every population-based optimizer in the core has direct ask/tell or an equivalent batch interface. Dual Annealing is not population-based and is the only listed optimizer without one. The external cmaes 0.2.2 crate used in the GTOP retry benchmark likewise has no public ask/tell boundary, although it does offer internal Rayon run_parallel() and next_parallel() methods.

For native CPU evaluation, parallel_batch preserves candidate order:

#![allow(unused)]
fn main() {
use fcmaes_core::parallel_batch;

let xs = optimizer.ask();
let ys = parallel_batch(&xs, 16, |x| expensive_objective(x));
optimizer.tell(&ys);
}

The same split permits GPU batches, asynchronous services, or external simulator pools, as long as results are restored to ask order before tell. When combining retry and population evaluation, budget threads explicitly: for example, four retry lanes with four evaluation workers each on 16 physical cores. Using 16 lanes each with 16 inner workers normally oversubscribes the machine and can make wall time worse.

Shared objective and bounds

An ordinary synchronized scalar closure implements Objective automatically:

#![allow(unused)]
fn main() {
let objective = |x: &[f64]| x.iter().map(|value| value * value).sum::<f64>();
}

Most stateful optimizers receive a Fitness object:

#![allow(unused)]
fn main() {
use fcmaes_core::Fitness;

let mut fitness = Fitness::bounded(4, 1, &[-5.0; 4], &[5.0; 4]);
fitness.set_normalize(true);
}

Fitness::new also supports unbounded problems when both bound vectors are empty. Retry and BiteOpt require finite box bounds.

Differential Evolution

De implements DE/best/1 plus temporal locality, age-based population replacement, oscillating mutation/crossover parameters, optional sampling around a guess, and optional integer-coordinate mutation.

Key DeParams defaults:

Pass empty guess and sigma slices for uniform initialization. With a guess, provide a same-dimensional sigma vector. The optional integer mask must match the problem dimension.

Active CMA-ES

Cmaes implements the active covariance update. A guess is required, and input_sigma is either one value (broadcast to all dimensions) or one value per coordinate.

Key CmaesParams defaults:

Cmaes::optimize(objective, workers) controls population-evaluation parallelism. Use 1 when outer retry already supplies parallelism.

CR-FM-NES

Crfmnes evaluates complete populations through a batch closure:

#![allow(unused)]
fn main() {
let result = optimizer.optimize_batch(|rows| {
    rows.iter().map(|x| objective(x)).collect()
});
}

Defaults include population 32, 100,000 evaluations, penalty coefficient 1e5, and constraint-violation handling enabled. ask_pop returns decoded, in-bound rows; tell_pop consumes one value per row.

PGPE

Pgpe uses mirrored sampling, ranking, and ADAM updates for the distribution center. Defaults include population 32, ranking enabled, center learning rate 0.15, standard-deviation learning rate 0.1, and 100,000 evaluations.

The batch and ask/tell interfaces mirror CR-FM-NES. The optimizer tracks the true objective best even when ranking is used for updates.

The neural-controller policy-search tutorial uses these interfaces to compare PGPE and CR-FM-NES on the same 118-parameter fixed-topology policy, rollout budget, scenario schedule, and disjoint validation protocol.

Dual Annealing

Use the free function:

#![allow(unused)]
fn main() {
use fcmaes_core::{optimize_da, DaParams};

let params = DaParams {
    max_evaluations: 50_000,
    use_local_search: true,
    seed: 1,
    ..Default::default()
};
let result = optimize_da(
    &objective,
    &[0.0; 4],
    vec![-5.0; 4],
    vec![5.0; 4],
    &params,
);
}

The optional local search is a self-contained bounded projected L-BFGS implementation using finite-difference gradients. Empty lower and upper vectors select the unbounded path.

BiteOpt

The Rust implementation includes the adaptive selector tree, primary generators, diverging populations, spherical and sequential Nelder-Mead secondary optimizers, dynamic population sizing, deep multi-population mode, and delayed-feedback ask/tell.

Key BiteParams defaults:

One-shot use:

#![allow(unused)]
fn main() {
use fcmaes_core::{optimize_bite, BiteParams};

let params = BiteParams {
    max_evaluations: 100_000,
    seed: 1,
    ..Default::default()
};
let result = optimize_bite(
    &objective,
    &[-5.0; 4],
    &[5.0; 4],
    None,
    &params,
    1,
);
}

The final argument is deep-mode population depth. Values at most one select a plain run; the validated maximum is 36. Bounds must be finite and strict.

Ask/tell freezes the selector decisions associated with every candidate until feedback arrives. Do not call tell without a pending batch or with the wrong number of costs. Non-finite costs are converted to a large finite rejection value. Batching changes the feedback timing and can trade some convergence quality for wall-clock throughput on expensive objectives.

MODE

Mode is a bounded ask/tell optimizer for nobj minimized objectives followed by ncon constraints. A constraint is feasible when its value is at most zero. Mode::try_new validates the objective width, finite strict bounds, population size, probabilities, and optional integer mask.

#![allow(unused)]
fn main() {
use fcmaes_core::{Fitness, Mode, ModeParams};

let fitness = Fitness::bounded(4, 2, &[0.0; 4], &[1.0; 4]);
let parameters = ModeParams {
    popsize: 64,
    nsga_update: true,
    seed: 1,
    ..Default::default()
};
let mut mode = Mode::try_new(fitness, 2, 0, None, &parameters)?;
let xs = mode.ask();
let ys: Vec<Vec<f64>> = xs
    .iter()
    .map(|x| vec![x[0], x.iter().map(|v| (v - 1.0).powi(2)).sum()])
    .collect();
mode.tell(&ys);
Ok::<(), &'static str>(())
}

NSGA-II updates use normalized crowding distance across every objective. Odd population sizes are supported. Non-finite told values are treated as rejected candidates. The DE update requires at least four population members.

Key ModeParams defaults:

The final population need not hold every non-dominated point a run evaluated. For two objectives, insert each feasible told row into a separate ParetoArchive2D to keep that exact front and its hypervolume; see the Pareto archive guide and cargo run --release -p fcmaes-core --example pareto_mode.

CVT-MAP-Elites and Diversifier

Archive::try_new builds a CVT behavior archive from descriptor bounds. K-means++ initialization and Lloyd assignment use Rayon for the dominant distance kernels. Seed parent solutions with seed_uniform, evaluate an initial population, then call map_elites; diversify uses niche improvement as the objective seen by CMA-ES.

Expensive native objectives can implement QdBatchFitness and use map_elites_batch / diversify_batch. Archive::update_evaluated separates parallel evaluation from ordered archive mutation. The general parallel_batch helper uses the same cached, explicitly sized Rayon pools as population fitness evaluation.

For two descriptor dimensions, samples_per_niche = 0 selects a rectangular grid with O(1) niche lookup. Positive values select k-means CVT centers and a nearest-center scan; use that path when irregular Voronoi niches are required.

The archive minimizes fitness. It exposes occupancy, best fitness, descriptors, counts, and a Python-compatible QD score. For all-positive fitness the score is the sum of reciprocals; when any negative value exists it is the sum of negated negative elites.

Key driver defaults:

Evaluation limits and reproducibility

Population optimizers check budgets at generation boundaries, so actual evaluation totals can overshoot the configured limit by part of a population. Use reported evaluations for accounting.

The retry root seed spawns one persistent, independent PCG stream per worker; each retry draws its optimizer seed from its worker’s stream. A single-worker run is exactly repeatable. In a multi-worker run, scheduling can change how many retries each worker stream consumes, so repeated results need not be bit-identical. Rust is also not expected to reproduce historical C++ or Python streams exactly; port parity is statistical.