Frozen foundations benchmark specification

This file is the comparison contract. All objectives are minimized, all decision bounds are closed, and an optimizer receives the same requested evaluation count as its controls. Checked-in conformance artifacts contain full-precision values; Markdown rounds only for display. The publication preset is an artifact-size label, not a claim of statistical benchmarking.

Single-objective suites

The classic functions use their conventional unshifted, unrotated forms. Dimensions 2, 10, and 40 are conformance cases; publication uses dimension 10. Each function’s bounds and known solution are defined in src/suites/classic.rs. The baseline is the first 31 points of the seeded uniform-random control, not the box center (which is the exact optimum of several unshifted functions).

Lennard-Jones scaling family

The reduced-unit potential is 4 Σ(r^-12-r^-6) over unordered atom pairs. Study sizes are N=13, 38, 55, 75, and 98. The free encoding has 3N coordinates. The fixed-frame encoding places atom 0 at the origin, atom 1 on positive x, and atom 2 in the xy half-plane with positive y, leaving 3N-6 coordinates. Degenerate anchors are errors.

Every Cartesian coordinate is bounded by a cube of half-width 1.5 N^(1/3). Initial candidates are rejection samples in a sphere of radius 0.8 N^(1/3) with pair separation at least r_min = 0.75. Every arm, including random, uses this compact generator. Optimizer candidates crossing the box are projected and counted.

For squared pair distance s < s_min = 0.75², the physical pair term is continued as

q(s) = V(s_min) + V'(s_min)(s-s_min)
       + 0.5 |V'(s_min)|/s_min (s-s_min)².

This is finite, C1 at the boundary, and monotonically worse toward s=0. Guarded pairs are counted. Exact coincidence has zero directional gradient, which is unavoidable for a differentiable finite radial guard; the separated initializer prevents it.

The primary budget is pair traversals. Energy-only evaluation and one combined value/gradient evaluation each cost one traversal. Objective calls, gradient calls, pair time, wall time, overlap pairs, and projections are separate fields. The four derivative-free arms use four single-threaded basic retries; the outer case layer owns parallelism. argmin L-BFGS multistart and fixed basin hopping are mandatory reference arms. Their optional dependency is enabled by gradient-reference and never enters fcmaes-core.

The population policies are algorithm-specific and explicit rather than matched: DE uses its core default 15d, CMA-ES uses floor(4 + 3 ln(d)), and CR-FM-NES uses 16. Each retry receives one quarter of the arm budget. A disjoint three-seed sensitivity grid over normalized CR-FM-NES sigma {0.05, 0.15, 0.50} and population {16, 32, 64} selected sigma=0.05, population 16 before the ten-seed primary rerun. The sensitivity table is diagnostic and cannot support a general optimizer ranking.

The source-cited putative-minimum target energies are:

Supplying --reference-directory audits separately obtained Cambridge point files named by atom count. Hashes and evaluator residuals are recorded without redistributing the coordinates. Exact success is best_energy <= target + 1e-3; target-relative gaps and attainment within 1%, 5%, and 10% are secondary diagnostics.

Publication uses ten seeds and 20,000 full pair-loop traversals per arm. Smoke uses two seeds and 600, so it cannot support rankings. The pair_terms_evaluated field multiplies traversals by N(N-1)/2, making scaling across atom counts explicit.

The QD pilot uses N=38 fixed-frame candidates, normalized radius of gyration in [0.25,0.75], mean coordination in [0,12] at cutoff 1.35, and a 12×12 grid. Its holdout adds deterministic independent N(0,0.01) coordinate perturbations. Gates are clipping ≤5%, absolute Spearman correlation ≤0.90, mean coverage ≥8%, fine holdout retention ≥60%, 6×6 retention ≥75%, and cutoff ±0.01 retention ≥60%. A failed gate writes a QD skip manifest.

Multi-objective suites

ZDT uses 30 variables except ZDT4, which uses 10. DTLZ uses three objectives with conventional k: 5 for DTLZ1, 10 for DTLZ2–6, and 20 for DTLZ7. The publication campaign reports ZDT1–4 and ZDT6 plus DTLZ1–7. It never mixes training points into the analytic reference set.

The MODE protocol fixes three choices:

• Population update: population 64 with nsga_update=true, MODE’s default NSGA-II-style update. nsga_update=false selects MODE’s supported DE update, but that alternative is outside this campaign. No Foundations table is a DE-versus-NSGA-II comparison. Lessons L4–L6 also retain the default.
• Evaluation mode: analytic populations run sequentially with evaluation_workers=1. At this cost, dispatching 64 suite calls to threads would mostly measure scheduler overhead.
• Parallelism check: --workers is used only by lesson L3’s schedule-independence test. Costly applications can evaluate each ordered ask batch through fcmaes_core::parallel_batch before telling MODE.

Objective-space normalization is fixed per problem and uses the extrema of its deterministic analytic reference set. DTLZ1 therefore uses [0, 0, 0] and [0.5, 0.5, 0.5]; DTLZ7 uses the extrema of its deterministic disconnected oversample. Every arm records the exact extrema it used in mo/indicators.csv.

Indicators

• Primary hypervolume uses one shared reference per problem, computed from the union of every published normalized arm/checkpoint front. Each coordinate is the union maximum plus 10% of max(observed range, 1). The rule is frozen; the resulting coordinates are recorded in every row. This metric compares arms within a run and is not comparable across problems or campaigns.
• Fixed-box hypervolume at [1.1; m] after analytic-front normalization is a secondary cross-run field. It is computed on the complete front only when every point is inside. Otherwise its value is null, its kind is not-applicable-outside-reference, and the outside count is recorded.
• Exact hypervolume is used through four objectives. Higher-dimensional estimates are labeled Monte Carlo and record seed, samples, and standard error.
• IGD, IGD+, GD, and GD+ are arithmetic means of nearest Euclidean or modified Euclidean distances, rather than the alternative root-mean-square variant.
• Additive epsilon is max_r min_a max_j(a_j-r_j) for minimized objectives.
• Exact duplicates collapse before hypervolume. Dominated unique points are removed. Both counts are reported.
• Non-finite points and mismatched dimensions are errors. Points outside the primary reference are errors. No front is clipped or filtered for either hypervolume convention.

ZDT3 reference points come only from its five nondominated intervals. DTLZ5 and DTLZ6 use their one-dimensional degenerate manifold for m > 2. DTLZ7’s disconnected reference set is a deterministic Pareto filter of a Halton oversample. Reference generation is seed-free.

Fairness and replay

Every table contains initial, random, and optimizer rows. The fairness and replay rules are explicit:

• random receives exactly the optimizer’s requested budget.
• initial consumes only its declared population and is never described as an optimizer result.
• Requested-budget equality does not imply identical actual counts. In the checked artifacts, scalar DE completes its current population batch and records 4,006–4,029 evaluations against random’s exact 4,000. MODE and its random control both record exactly 4,096.
• Every retained decision is evaluated again before indicators are written. The artifacts record the recheck count and maximum absolute discrepancy; rechecks do not consume optimizer budget.

The same-evaluator replay can detect nondeterminism or bookkeeping corruption, not independent model error. Machine-readable artifacts also record requested and actual evaluations, seed, workers, dimensions, normalization, both reference conventions, and wall time.