fcmaes - a Python 3 gradient-free optimization library

fcmaes can give you a real advantage in optimization competitions such as ESA’s optimization challenges, especially if you use its parallelization features well. Here is a list of the ranks achieved by fcmaes:

For most challenges, the fcmaes result is either the best or within 1 % of the best. All competition code was written in Python, although fcmaes and or-tools wrap C++ code for speed. All 15 challenges are still open. You can submit solutions, and your result will appear on the leaderboard.

Engineering design problems often combine multiple competing objectives with many constraints. Here we use the Mazda real-world car structure design benchmark. The task is to optimize three car models at the same time, reduce their weight, increase the number of shared thicknesses of structural parts, and satisfy 54 constraints. In 2017 there was a competition related to this problem, documented in the Report of Evolutionary Computation Competition 2017. Even so, many of the ideas developed there have still not found their way into open source optimization libraries.

We applied modecpp.py for about one hour on one AMD 5950x CPU under Linux, using the de/rand/1 strategy (nsga_update=False, pareto_update=False, ints=[True]*dim) with population size 256. We chose the best result from two runs executed in parallel, each using 16 threads (workers=16). Together, both runs performed about 8200 function evaluations per second.

The resulting Pareto front has hypervolume 0.4074:

The "reference" NSGA-II solution provided with the benchmark has hypervolume 0.1456:

The reference solution was computed with a limited budget. Still, NSGA-II scales much worse than fcmaes-MoDe when using enhanced multiple constraint ranking. This example shows that fcmaes also works well on constrained multi-objective engineering problems, not only on single-objective benchmarks.

Remark: This section was initially created in 2022 and updated in 2024. What changed:

fcmaes provides fast parallel example solvers for the real-world space flight design problems from GTOP and for the F-8_aircraft problem based on differential equations. At GTOPX you can find implementations of the corresponding objective functions in several programming languages. The solution times in the tables below were measured on Linux with an AMD 9950x CPU using 32 parallel processes.

Here, stopVal is the threshold value that determines success, and mean time includes failed runs. Execute benchmark_gtop.py to reproduce these results. The same optimization algorithm was used for all problems, with the same parameters for both the optimizer and the coordinated retry / boundary management.

Because of its well-known complexity, ESA’s 26-dimensional Messenger full problem is often used as a reference in the literature. See, for instance, the MXHCP paper.

fcmaes solves this problem in less than half an hour using an AMD 9950x.

The problem models a multi-gravity-assist interplanetary mission from Earth to Mercury. In 2009, the first good solution (6.9 km/s) was submitted. It took more than five years to reach 1.959 km/s, and three more years until 2017 to find the optimum 1.958 km/s. The picture below shows the progress of the broader research community since 2009:

The next picture shows 100 coordinated retry runs:

In 79 of these 100 runs, the result was better than 2 km/s:

About 2.1*10^6 function evaluations per second were performed. This shows how well the algorithm scales when using all 16 cores / 32 threads. For a larger setup, fcmaesray shows how fcmaes coordinated retry performs on a 5-node cluster with 96 CPU cores.