Choosing an optimization tool? Start with the shared solver-selection and agent guide or the optimization ecosystem overview. Use fcmaes for Python black-box objectives, parallel retry, Pareto search and quality diversity. The guide also routes to native Rust optimization, LP/MIP, smooth nonlinear solvers and durable MCP jobs. This Python package does not include an MCP server. experiment-craft runs reproducible campaigns of those jobs (parameter sweeps, dependent steps, budgets and reports); it drives the Rust workers, not this package, and is in early development.
fcmaes extends scipy optimize with additional optimization methods, fast C++/Eigen-based implementations, and a coordinated parallel retry mechanism. It supports multi-threaded use of several gradient-free algorithms partly implemented in C++. It is designed for real-world problems, including multi-objective and constrained optimization. The main algorithms are available in both Python and C++. They support parallel fitness evaluation as well as parallel retry strategies.
The documentation book combines the API references with many practical tutorials and examples that show how to use fcmaes in detail.
Detailed benchmark results are available in the book’s performance chapter.
Features
-
Built for optimization problems that are difficult to solve and benefit from modern many-core CPUs.
-
Supports parallel fitness evaluation and several parallel retry strategies.
-
Scales well with the number of available CPU cores.
-
Keeps algorithm overhead low, even in high-dimensional problems.
-
Includes the multi-objective and constrained optimizer MODE. It combines ideas from Differential Evolution and NSGA-II and supports parallel function evaluation. It also uses enhanced multiple constraint ranking, which improves constraint handling in engineering design problems.
-
Supports quality-diversity methods, including CVT-map-elites, a CMA-ES emitter, and the new "diversifier" meta-algorithm based on CVT-map-elites archives.
-
Offers a selection of efficient single-objective optimizers.
-
Provides an ask-tell interface for CMA-ES, CR-FM-NES, DE, jSO, MODE, and PGPE.
Update: Native Rust implementation of fcmaes
See fcmaes-rust. This repo is an alternative both for Rust and Python users as it provides a PyO3 API. See python example.
Update (2.0.2): Ask Tell interface for BiteOpt
The native BiteOpt backend now also provides an ask/tell interface through bitecpp.Bite_C.
This makes it possible to parallelize a single BiteOpt optimization run:
generate a batch of candidate points with ask(), evaluate them externally in parallel, and feed the resulting objective values back with tell().
This is most useful for expensive objective functions where wall-clock time is dominated by the simulation or model evaluation. In that setting, evaluating several candidates concurrently can reduce elapsed optimization time even if the optimizer itself needs a few more function evaluations.
There is an important trade-off: BiteOpt normally updates its internal selection strategy immediately after each evaluation.
In ask/tell mode these updates are delayed until tell(), so a whole batch is generated from a frozen optimizer state.
That delayed feedback introduces a convergence penalty compared to the ordinary sequential BiteOpt loop.
In other words, ask/tell usually improves parallel throughput, but may reduce convergence efficiency measured in evaluations.
So the rule of thumb is:
-
For cheap objective functions, prefer the standard sequential BiteOpt run or parallel retry.
-
For expensive objective functions, BiteOpt ask/tell can be a very practical way to parallelize one optimization run across several cores or workers.
Update: Split-Brain Automated Design of Stochastic Biochemical Circuits
autoresearch-circuit applies the same split-brain pattern to biochemical circuit design, inspired by the CircuiTree paper. An LLM proposes circuit topology — which genes regulate which. fcmaes tunes the kinetic rates against a noisy stochastic simulator. A validation stage stress-tests the result with knockouts, knockdowns, and parameter jitter. Structure and numbers stay separated, each handled by the right tool.
Update: The Split-Brain Pattern For Agentic Search
autoresearch-trading applies Karpathys autoresearch-idea to trading algorithm optimization. LLMs are good at structure. Classical optimizers are good at searching numbers. So I built a system that separates the two. I think this pattern goes beyond trading. Anywhere you have a simulator and a mix of logic + tunable numbers, this approach may be useful.
Update: Open Source AI Optimization Assistant
See Optimization Assistant How to set up, configure and use an open source AI optimization assistant specialized for fcmaes.
Update: A real case in aging research
See Moran Process Optimize a Moran Process. Check out if you have to optimize parameters for an expensive simulation. Includes a detailed comparison with Optuna.
Update: How to compute π-groups for dimensional analysis
See BuckinghamExamples Reveal the underlying structure of physical models by using evolutionary algorithms to fit continuous powers that maximize data collapse R2 under variance constraints.
Update: How to compute weighted spherical t-designs
See Spherical t-design Shows how to compute weighted spherical t-designs to optimize laser fusion reactor designs.
Update: Interaction between AI code generation and optimization
LLMs can help to generate code implementing a trading strategy. It can even propose ways to optimize the final return. prophet_opt.py shows:
-
The LLM prompts used to generate the strategy back-testing code.
-
How to identify the parameters to optimize using the AI.
-
How the parameter optimization process can be automated efficiently utilizing trading simulations executed in parallel.
Changes in version 2.2.0:
The full list, with earlier versions, is in CHANGELOG.md.
-
New jSO (
fcmaes.jso,fcmaes.jsocpp): adaptive differential evolution with a hard evaluation cap, ask/tell and retry support, and the sequenceoptimizer.jso_cma, which starts CMA-ES from the jSO incumbent. In coordinated retry (advretry)de_cmastays the better choice. -
CMA-ES option
repair_z(defaultFalse, the update of 2.1.0):Truelets the evolution paths and the covariance follow samples that the bound repair moved onto the bounds. Try it when runs stop early with the step-size stop or when optima lie on or outside the bounds. -
PGPE: the result is the best evaluated point and its objective value, ADAM starts from the normalized
x0, andstop_fitnesscompares objective values. PGPE results with bounds other than[-1, 1]differ from 2.1.0. -
C++ optimizers: a seeded run no longer depends on the runs before it in the same process, and
cmaescpp.minimize/decpp.minimizeno longer hang when called from two threads or from inside their own objective. Withworkers > 1they no longer read freed memory (crashes on Windows). -
CMA-ES: a sample told at exactly the mean no longer puts NaN into the covariance;
ACMA_C.tell(ys, xs)afterask()updates fromxs;Cma_cppwithsdevs=Noneuses 0.1 instead of NaN step sizes, so sequences such asde_cma(…).minimize(fun, bounds)run their CMA-ES stage. -
Tutorials: figures and claims that disagreed are corrected, section levels are fixed and broken links repaired; the READMEs link the algorithm portfolio of fcmaes-rust and experiment-craft.
Changes in version 2.1.0:
-
MODE (
modecpp,mode): SBX can expand beyond the parents, polynomial mutation no longer pins variables at their lower bound, levels are true non-dominated fronts, crowding uses all objectives, and a point is feasible only if it violates no constraint. MODE results change; it no longer collapses on ZDT2/ZDT4. -
New
fcmaes.pareto.ParetoArchive2D: the exact two-objective front of all evaluations with incremental hypervolume (derived from BiteOpt’sbitehv2.h). -
moretry: satisfied constraints no longer enter the weighted norm, which penalized points for lying deep inside the feasible region and made results depend on the constraint weights; only violations count.value_limitsmay list only the objectives of a constrained problem (before, every result was dropped). -
from fcmaes import *works again (fcmaes.allnamed missing modules). -
Tutorials and examples use the loguru
loggerof version 1.6.3 on; thelogger=logger()arguments of earlier versions are gone. -
The API reference pages cover
fcmaes.pareto, the nanobind bindings andexamples/pareto_mode.py;examples.alllists the example modules. -
New
examples/external_simulator.py: runs an external simulator program as the objective, with a process group per run, timeouts, a failure ledger shared across worker processes, seeds for common random numbers and holdout validation, and CPU-quota-aware parallelism. -
AGENTS.mdandai-context.mdguide coding agents; the READMEs link the shared solver-selection guide of the five fcmaes and craft repositories. -
Releases: a tag must match the version in
pyproject.toml, and CI installs the source distribution and runs the tests on it.
Changes from version 2.0.0:
-
Switch from ctypes C++ binding to nanobind.
-
MacOS fully supported - including ARM.
-
Improved documentation and tutorials
-
Improved type annotations
-
Improved test coverage
Changes from version 1.6.3:
-
Logging now based on loguru. All examples are adapted.
-
New dependency: loguru.
numbais optional and mainly used as a speedup for MAP-Elites and some examples. -
New tutorial related to the GECCO 2023 Space Optimization Competition: ESAChallenge.
-
You can define an initial population as guess for multi objective optimization.
Changes from version 1.4.0:
-
Pure Python versions of the algorithms are now usable also for parallel retry. Pure Python features: Algorithms: CMA-ES, CR-FM-NES, DE, MODE (multiple objective), Map-Elites+Diversifier (quality diversity). All Python algorithms support an ask-tell interface and parallel function evaluation. Additionally parallel retry / advanced retry (smart boundary management) are supported for these algorithms.
-
Python version > 3.7 required, 3.6 is no longer supported.
-
PEP 0484 compatible type hints useful for IDEs like PyCharm.
-
Most algorithms now support an unified ask/tell interface: cmaes, cmaescpp, crfmnes, crfmnescpp, de, decpp, mode, modecpp, pgpecpp. This is useful for monitoring and parallel fitness evaluation.
-
Added support for Quality Diversity [QD]: MAP-Elites with additional CMA-ES emitter, new meta-algorithm Diversifier, a generalized variant of CMA-ME, "drill down" for specified niches and bidirectional archive <→ store transfer between the QD-archive and the smart boundary management meta algorithm (advretry). All QD algorithms support parallel optimization utilizing all CPU-cores and statistics for solutions associated to a specific niche: mean, stdev, maximum, minimum and count.
Derivative free optimization of machine learning models often have several thousand decision variables and require GPU/TPU based parallelization both of the fitness evaluation and the optimization algorithm. CR-FM-NES, PGPE and the QD-Diversifier applied to CR-FM-NES (CR-FM-NES-ME) are excellent choices in this domain. Since fcmaes has a different focus (parallel optimizations and parallel fitness evaluations) we contributed these algorithms to EvoJax which utilizes JAX for GPU/TPU execution.
Optimization algorithms
The algorithm portfolio of fcmaes-rust records how the scalar algorithms were selected: which are in, which were benchmarked and left out, and the measurements behind each decision. The algorithms it shares with this package are BiteOpt, DE, jSO, CMA-ES, CR-FM-NES, PGPE and Dual Annealing.
To utilize modern many-core processors all single-objective algorithms should be used with the parallel retry for cheap fitness functions, otherwise use parallel function evaluation.
-
MO-DE: A new multi-objective optimization algorithm merging concepts from differential evolution and NSGA. Implemented both in Python and in C++. Provides an ask/tell interface and supports constraints and parallel function evaluation. Can also be applied to single-objective problems with constraints. Supports mixed integer problems (see CFD for details)
-
CVT-map-elites/CMA: A new Python implementation of CVT-map-elites including a CMA-ES emitter providing low algorithm overhead and excellent multi-core scaling even for fast fitness functions. Enables "drill down" for specific selected niches. See mapelites.py and Map-Elites.
-
Diversifier: A new Python meta-algorithm based on CVT-map-elites archives generalizing ideas from CMA-ME to other wrapped algorithms. See diversifier.py and Quality Diversity.
-
BiteOpt algorithm from Aleksey Vaneev BiteOpt. Only a C++ version is provided. If your problem is single objective and if you have no clue what algorithm to apply, try this first. Works well with almost all problems. For constraints you have to use weighted penalties.
-
Differential Evolution: Implemented both in Python and in C++. Additional concepts implemented are temporal locality, stochastic reinitialization of individuals based on their age and oscillating CR/F parameters. Provides an ask/tell interface and supports parallel function evaluation. Supports mixed integer problems (see CFD for details)
-
jSO: Adaptive differential evolution with success-history memory and a shrinking population.
fcmaes.jsois the Python implementation andfcmaes.jsocppuses the C++ backend. Both honor a hard jSO evaluation budget. Useoptimizer.jso_cmato refine the jSO incumbent with CMA-ES. See the jSO tutorial. -
CMA-ES: Implemented both in Python and in C++. Provides an ask/tell interface and supports parallel function evaluation. Good option for low number of decision variables (< 500).
-
CR-FM-NES: Fast Moving Natural Evolution Strategy for High-Dimensional Problems, see https://arxiv.org/abs/2201.11422. Derived from https://github.com/nomuramasahir0/crfmnes . Implemented both in Python and in C++. Both implementations provide parallel function evaluation and an ask/tell interface. Good option for high number of decision variables (> 100).
-
PGPE Parameter Exploring Policy Gradients, see http://mediatum.ub.tum.de/doc/1099128/631352.pdf . Implemented in C++. Provides parallel function evaluation and an ask/tell interface. Good option for very high number of decision variables (> 1000) and for machine learning tasks. An equivalent Python implementation can be found at pgpe.py, use this on GPUs/TPUs.
-
Wrapper for cmaes which provides different CMA-ES variants implemented in Python like separable CMA-ES and CMA-ES with Margin (see https://arxiv.org/abs/2205.13482) which improves support for mixed integer problems. The wrapper additionally supports parallel function evaluation.
-
Dual Annealing: Eigen based implementation in C++. Use the scipy implementation if you prefer a pure Python variant or need more configuration options.
-
Expressions: There are two operators for constructing expressions over optimization algorithms: Sequence and random choice. Not only the single objective algorithms above, but also scipy and NLopt optimization methods and custom algorithms can be used for defining algorithm expressions.
Installation
Linux
-
pip install fcmaes -
Recommended Python environment: miniforge Python 12.
Wheels are built by the release CI. If no matching wheel is available for your
platform or Python version, pip falls back to a source build.
Windows
-
pip install fcmaes -
Install C++ runtime libraries https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads
-
Recommended Python environment: miniforge Python 12.
For parallel fitness function evaluation use the native Python optimizers or the ask/tell interface of the C++ ones. Python multiprocessing works better on Linux. To get optimal scaling from parallel retry and parallel function evaluation you may still use:
-
Linux subsystem for Windows WSL.
The Linux subsystem can read/write NTFS, so you can do your development on a NTFS partition. Just the Python call is routed to Linux. If performance of the fitness function is an issue and you don’t want to use the Linux subsystem for Windows, think about using the fcmaes java port: fcmaes-java.
MacOS
-
pip install fcmaes
The nanobind-based native backend now supports macOS as part of the wheel build pipeline.
If no matching wheel is available for your Python version or architecture, pip
falls back to a source build.
From source
Building from source needs:
-
Python 3.10 or newer and
pip. The build backend (scikit-build-core), nanobind and, if missing, CMake are installed bypipitself. -
A C++17 compiler: GCC or Clang on Linux, Xcode’s Clang on macOS, Visual Studio 2019 or newer (MSVC) on Windows.
-
Eigen 3.4 or newer, either installed on the system (for example
apt install libeigen3-devorbrew install eigen) or fetched during the build withFCMAES_FETCH_EIGEN=ONas below. -
fcmaes depends on no other repository of ours; the PCG random number generator and LBFGSpp headers are bundled.
git clone https://github.com/dietmarwo/fast-cma-es
cd fast-cma-es
python -m venv .venv && . .venv/bin/activate # Windows: .venv\Scripts\activate
python -m pip install . --config-settings=cmake.define.FCMAES_FETCH_EIGEN=ON
python -m pip install pytest # for the tests
cd .. && python -m pytest fast-cma-es/tests -q # run the tests outside the checkout
pip install fcmaes --no-binary fcmaes builds the published source
distribution the same way. RELEASING.md
describes how releases are built and published.
Usage
Usage is similar to scipy.optimize.minimize.
For parallel retry use:
from fcmaes import retry
ret = retry.minimize(fun, bounds)
The retry logs mean and standard deviation of the results, so it can be used to test and compare optimization algorithms: You may choose different algorithms for the retry:
from fcmaes.optimizer import Bite_cpp, De_cpp, Cma_cpp, Sequence
ret = retry.minimize(fun, bounds, optimizer=Bite_cpp(100000))
ret = retry.minimize(fun, bounds, optimizer=De_cpp(100000))
ret = retry.minimize(fun, bounds, optimizer=Cma_cpp(100000))
ret = retry.minimize(fun, bounds, optimizer=Sequence([De_cpp(50000), Cma_cpp(50000)]))
Here tutorials for more details.
Dependencies
Runtime:
-
numpy: https://github.com/numpy/numpy, version >= 1.20
-
scipy: https://github.com/scipy/scipy, version >= 1.8
-
scikit-learn: https://github.com/scikit-learn/scikit-learn (for CVT-Map-Elites), version >= 1.1
Compile-time dependencies for source builds:
-
Eigen https://gitlab.com/libeigen/eigen (version >= 3.4.0 is required for CMA).
-
PCG Random Number Generation https://github.com/imneme/pcg-cpp - used in all C++ optimization algorithms.
-
LBFGSpp: https://github.com/yixuan/LBFGSpp/tree/master/include - used for dual annealing local optimization.
Prebuilt wheels are built by CI for Linux, Windows, and macOS.
Optional dependencies:
Example dependencies:
-
pykep: pykep. Install with 'pip install pykep'.
Documentation
The GitHub Pages workflow builds this README, all AsciiDoc tutorials, and the three maintained HTML API references as one navigable site:
Build and validate the same site locally with Asciidoctor 2.0.23 or newer:
gem install asciidoctor --version 2.0.23 --no-document
python scripts/build_docs.py
The generated site is written to target/site. AsciiDoc remains the canonical
source; no Markdown conversion is required.
For AI agents, ai-context.md describes how to choose,
configure and validate the optimizers, and AGENTS.md is the
short entry point for coding agents. To choose between fcmaes and the sibling
LP/MIP, NLP and job-API projects, start at the
optimization-jobs entry point.
Packaging and release
-
Pull requests and branch pushes build wheels on Linux, Windows, and macOS.
-
A manual GitHub Actions run with
publish_target=testpypipublishes to TestPyPI. -
Tags matching
v*publish to PyPI; the tag must equalvplus the version inpyproject.toml. -
CI also installs the source distribution and runs the tests on it.
-
CI releases use Trusted Publishing, so no long-lived PyPI API token is required in GitHub Actions.
Citing
@misc{fcmaes2025,
author = {Dietmar Wolz},
title = {fcmaes - A Python-3 derivative-free optimization library},
note = {Python/C++ source code, with description and examples},
year = {2025},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {Available at \url{https://github.com/dietmarwo/fast-cma-es}},
}
Third-party integration
OptunaHub’s FastCmaesSampler, by Jacob Pfeil, connects fcmaes CMA-ES to Optuna. Its published example and verified Optuna version are maintained there. This demonstrates an external integration; it is not a comparative benchmark of this ecosystem.