Native Rust GTOP benchmark results

This report collects the recorded GTOP tables in one Markdown document. All three tests use relaxed stopping values of approximately 1.005 * absolute_best for positive objectives and the corresponding 0.5% relaxation toward zero for negative objectives. The raw TSV files preserve every individual experiment.

The problem definitions and putative best solutions originate from ESA’s Global Trajectory Optimisation Problems database. The unusually difficult Tandem case is documented separately on ESA’s TandEM problem page, including its 18-variable box bounds and selectable fly-by sequences.

Coordinated retry

Messenger Full remains excluded because its full 100-run measurement is too expensive for this test. Tandem was measured separately with the same configuration and is included here.

Raw data: benchmark_gtop_100_raw.tsv and benchmark_gtop_tandem_100_raw.tsv. The separate benchmark_gtop_tandem_100_metadata.json records the slow-run configuration and total invocation time.

Configuration:

• 32 native retry threads and 100 independent experiments per problem
• 1,500 initial evaluations per retry
• advanced maximum evaluation factor 50
• diversity checkpoint interval 100 retries
• optimizer sequence: 40% Differential Evolution, then 60% active CMA-ES
• per-problem retry caps and value limits from examples/src/benchmark_gtop.rs

The coordinated test has an adaptive budget, not the 240,000-evaluation allowance used by the basic-retry and external-library comparison. A retry’s limit grows linearly from 1,500 to 75,000 evaluations as the run ID approaches the per-problem retry cap. The ramp’s mean is 38,250 evaluations, so each exact theoretical ceiling is 38,250 × retry cap. The actual recorded means are:

Every run consumed less than its theoretical ceiling. The 15 Tandem failures are included in its mean and sdev. Across all 100 Tandem runs, the mean final optimum was -1488.503885 (population sdev 21.850556); the best was -1500.468716.

The original Python/C++ fcmaes Performance report records 81% Tandem success, 166.92 s mean wall time, and 147.87 s wall-time sdev on the same AMD 9950X class of CPU with 32 parallel Python processes. The native Rust run records 85%, 40.21 s, and 39.11 s with 32 native retry threads. The close success rates are consistent with the implementations using the same adaptive search policy; the wall times also reflect their different process and thread execution models.

The substantially larger actual budgets explain why coordinated retry has much better success rates than the 240,000-evaluation comparison. This table is a quality reference, not an equal-budget speed comparison.

BiteOpt basic retry

This benchmark includes Tandem, excludes Messenger Full, and uses 24 independent BiteOpt retries of at most 10,000 evaluations on 24 native worker threads for every experiment.

Raw data: benchmark_biteopt_gtop_rust_100_raw.tsv.

DE→CMA basic retry

This test has the same 100 experiments, 24 workers, 24 retries, and 10,000 evaluations per retry as the BiteOpt test. Each retry assigns 4,000 evaluations to Differential Evolution and 6,000 to active CMA-ES.

The 240,000 value is the requested budget. DE and CMA-ES complete their current population batch, so mean actual work can exceed it slightly; the largest mean overshoot above is 755 evaluations (0.31%). Values below the request reflect early target stops or optimizer termination; the success column distinguishes target attainment.

Raw data: benchmark_de_cma_gtop_rust_100_raw.tsv.

External CMA-ES through generic retry

The dependency-isolated GTOP CMA-ES retry experiment adds an external single-threaded optimizer to this benchmark family. Its primary comparison gives one serial CMA-ES restart lane and 16 lanes coordinated through fcmaes_core::retry the same four-second wall allowance. Protective CMA-ES terminations trigger immediate deterministic restarts, so both arms consume the allowance rather than silently returning early. Across 100 paired seeds, best-objective mean/sdev and paired retry wins are the main outcomes; measured wall mean/sdev audits fairness.

The retry arm deliberately spends more aggregate CPU during the same user wait. That is the practical value being measured: converting the otherwise idle physical cores of a modern processor into a better solution distribution. The coordinated DE→CMA table above remains the full fcmaes system-level reference because it intentionally uses adaptive budgets and search coordination.

In the 100-pair equal-wall result, retry improves the objective mean on all seven non-Messenger-Full cases and wins 86–96 paired runs. It reduces objective sdev on five cases; SAGAS and Tandem instead have better means but larger spreads. Cassini1 target success improves from 38/100 to 100/100. Both arms remain at 0/100 target success on the other six cases, so the result is not presented as a four-second solution of the GTOP suite. Serial and retry wall means stay within 4.000109–4.000523 s, while measured active cores are approximately 1 and 16.

The paired win count is mainly a scheduler check. The retry arm completed 14.75–15.00 times as many independent starts; under iid restarts with no information sharing, that ratio predicts a 93.7%–93.8% win probability. The observed win/tie/loss counts are consistent with that null. The mean and sdev columns quantify the more informative change in the returned distribution.

The original SAGAS block found a physically impossible negative time-to-50-AU value. A nonnegative travel-time guard was added to the port and the complete SAGAS block was repeated with the same seeds. The final bundle links the correction provenance and uses only the corrected rows.

The secondary five-pair scaling pilot uses 40 retries of 2,000 evaluations and reaches 11.18×–12.23× mean fixed-work speedup at 16 workers across Cassini1, Rosetta, and Tandem. Every one of the 60 worker-count pairs passes the same-work audit. These runs deliberately do not reach the GTOP targets; they demonstrate multicore scheduling, not optimizer quality. It is not the main result.

Statistics and environment

Means and standard deviations include every experiment, including failures. Standard deviations use the population definition (ddof=0), matching NumPy’s default.

• CPU: AMD Ryzen 9 9950X, 16 physical cores / 32 logical CPUs
• Compiler: rustc 1.93.0 for the original six-problem run and rustc 1.97.1 for the later Tandem run; Cargo release profile
• OS: Linux 6.8.0-136-generic x86_64
• Date: 2026-07-22 for the original run; 2026-07-23 for Tandem

Rust workers are native threads. Each top-level experiment is run sequentially and has exclusive use of its configured retry workers.

Reproduce

From the repository root:

cargo run --release -p fcmaes-examples --bin benchmark-gtop -- \
  --runs 100 --workers 32 --seed 1 \
  --raw-output benchmarks/benchmark_gtop_100_raw.tsv

python3 benchmarks/run_coordinated_tandem.py

cargo run --release -p fcmaes-examples --bin benchmark-biteopt-gtop -- \
  --algo biteopt --runs 100 --workers 24 --retries 24 \
  --evaluations 10000 --seed 1 \
  --raw-output benchmarks/benchmark_biteopt_gtop_rust_100_raw.tsv

cargo run --release -p fcmaes-examples --bin benchmark-biteopt-gtop -- \
  --algo de_cma --runs 100 --workers 24 --retries 24 \
  --evaluations 10000 --seed 1 \
  --raw-output benchmarks/benchmark_de_cma_gtop_rust_100_raw.tsv

Each binary prints the corresponding Markdown table. --table-output PATH can also write that table to a chosen .md file. The Tandem driver records each experiment in a resumable shard before combining the final raw file. For coordinated retry, --include-slow adds Tandem and Messenger Full; --problem NAME runs one case explicitly.