Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Controlled active CMA-ES implementation diagnostic

These are paired wall-deadline measurements of fcmaes-core and cmaes 0.2.2. Both use active (negative-weight) CMA-ES, the same normalized reflected objective, explicit population size, initial mean, and sigma. This is an implementation diagnostic, not a general CMA-ES performance benchmark: its easy analytic functions isolate overhead, scaling, and stopping behavior rather than recommending CMA-ES for those functions. Matching numeric seeds label pairs but do not create matching random streams. A run may exceed its deadline by one generation.

Bundle summary

Raw rowsComplete pairsObjective callsActive wallProcess CPU
79203960249635434555.327 h43.731 h
LibraryInternal-stop rowsAll rows
fcmaes-core03960
cmaes24343960

Paired final-objective outcomes at the longest endpoint (10000 ms):

Armfcmaes-core winscmaes winsTiesPairs
a9678266440
b8490266440
c7580285440

Detailed results

The table reports medians over available seeds. Wins compare final objective values within a relative 1e-10 tie band. Residual ns/eval is shown only for serial Arm A and subtracts the separately measured shared-objective calibration; it is diagnostic, not a library-internal profiler.

ProblemCostArmDeadlinePairsWins fc/cma/tieInternal stops fc/cmaMedian best fcMedian best cmaMedian eval/s fcMedian eval/s cmaAllocated cores fcAllocated cores cmaResidual ns/eval fcResidual ns/eval cma
cassini10 nsa100 ms2010/10/00/201.570902e11.519210e12702892713701.000.32371356
cassini10 nsa1000 ms2011/9/00/201.417267e11.470643e12725342731311.000.04340332
cassini10 nsa10000 ms2013/7/00/201.567472e11.671256e12750402709011.000.00307362
cassini10 nsb100 ms2010/10/00/201.570902e11.519210e137739334308914.863.48——
cassini10 nsb1000 ms2011/9/00/201.417267e11.470643e136778034572614.850.44——
cassini10 nsb10000 ms2013/7/00/201.567472e11.671256e137026034520314.850.03——
cassini10 nsc100 ms209/11/00/201.099648e18.149953e03774339250298915.996.46——
cassini10 nsc1000 ms2011/9/00/207.762970e06.960041e03900340241019016.000.66——
cassini10 nsc10000 ms209/11/00/201.099648e11.099648e13871653239951516.000.07——
ellipsoid1000 nsa100 ms200/20/00/01.324602e69.537787e428211917331.001.00345369990
ellipsoid1000 nsa1000 ms200/20/00/05.024374e31.840431e228207934311.001.00345429792
ellipsoid1000 nsa10000 ms200/0/200/206.085251e-231.502134e-2031653959361.000.35306829513
ellipsoid1000 nsb100 ms200/20/00/01.376752e61.278446e526902817991.793.38——
ellipsoid1000 nsb1000 ms200/20/00/05.462756e32.705426e227157860981.703.18——
ellipsoid1000 nsb10000 ms200/0/200/206.377486e-231.502134e-2029953877591.781.22——
ellipsoid1000 nsc100 ms200/20/00/01.108834e69.752471e4393671127080415.9415.91——
ellipsoid1000 nsc1000 ms200/20/00/04.772216e31.992702e2395235130133715.9915.99——
ellipsoid1000 nsc10000 ms200/0/200/205.027433e-237.375883e-22439884130025116.006.52——
ellipsoid1001000 nsa100 ms200/20/00/01.599246e61.211162e528180903951.001.003445010026
ellipsoid1001000 nsa1000 ms200/20/00/04.982679e32.031136e228017922661.001.00346569801
ellipsoid1001000 nsa10000 ms200/0/200/205.573322e-238.771284e-2131380947601.000.36308319516
ellipsoid1001000 nsb100 ms200/20/00/01.703138e61.475023e526976830101.783.34——
ellipsoid1001000 nsb1000 ms200/20/00/05.609066e32.646340e227072855321.733.14——
ellipsoid1001000 nsb10000 ms200/0/200/205.820596e-238.771284e-2130010884381.771.19——
ellipsoid1001000 nsc100 ms200/20/00/01.234806e61.006984e5382835119970215.9315.89——
ellipsoid1001000 nsc1000 ms200/20/00/04.876275e32.088383e2395817129982515.9915.99——
ellipsoid1001000 nsc10000 ms200/0/200/205.452474e-239.500913e-22439407129191216.006.54——
ellipsoid100100000 nsa100 ms2012/8/00/05.064770e65.078403e6752791231.001.00328019564
ellipsoid100100000 nsa1000 ms202/18/00/01.537194e51.138606e5746991061.001.00338419773
ellipsoid100100000 nsa10000 ms2012/8/00/01.724218e22.018932e2750791131.001.00331609684
ellipsoid100100000 nsb100 ms200/20/00/02.374564e67.675128e520587414093.596.24——
ellipsoid100100000 nsb1000 ms200/20/00/01.082403e42.706133e320409420003.546.19——
ellipsoid100100000 nsb10000 ms200/20/00/204.322223e-61.550447e-2021842427273.695.02——
ellipsoid100100000 nsc100 ms207/13/00/03.753626e63.396196e611352414176715.8215.89——
ellipsoid100100000 nsc1000 ms200/20/00/01.049715e57.452990e411685914419915.9815.98——
ellipsoid100100000 nsc10000 ms209/11/00/01.288818e21.324399e211759114484516.0016.00——
rastrigin100 nsa100 ms2011/9/00/201.343192e11.293446e1142701818221571.000.14568416
rastrigin100 nsa1000 ms2010/8/20/201.243697e11.094454e1139709318398171.000.02584411
rastrigin100 nsa10000 ms209/10/10/201.243698e11.293446e1145918918393021.000.00553411
rastrigin100 nsb100 ms2011/9/00/181.343192e11.293446e142861244698714.508.13——
rastrigin100 nsb1000 ms2010/8/20/201.243697e11.094454e143534743622314.450.97——
rastrigin100 nsb10000 ms209/10/10/201.243698e11.293446e143952442752814.440.09——
rastrigin100 nsc100 ms2011/8/10/204.974795e05.969752e0159813761130218715.994.09——
rastrigin100 nsc1000 ms2010/6/40/204.974795e04.974795e0166079831173703216.000.42——
rastrigin100 nsc10000 ms207/10/30/205.472272e05.472272e0173091381133232516.000.04——
rastrigin101000 nsa100 ms2012/8/00/201.193950e11.293446e16109006905651.000.37582393
rastrigin101000 nsa1000 ms2010/9/10/201.243698e11.392942e16321946930181.000.04527388
rastrigin101000 nsa10000 ms2011/7/20/201.044706e11.193950e16165886917741.000.00567391
rastrigin101000 nsb100 ms2012/8/00/191.193950e11.293446e138368941006714.658.60——
rastrigin101000 nsb1000 ms2010/9/10/201.243698e11.392942e140710544089914.690.96——
rastrigin101000 nsb10000 ms2011/7/20/201.044706e11.193950e140078741512514.650.09——
rastrigin101000 nsc100 ms209/8/30/204.974795e04.974793e08768829603373315.997.89——
rastrigin101000 nsc1000 ms208/12/00/205.969754e04.974795e08902804591570816.000.78——
rastrigin101000 nsc10000 ms2010/10/00/204.974795e04.974795e08895940599283316.000.08——
rastrigin10100000 nsa100 ms2013/7/00/02.147252e13.206089e1993299501.001.00633456
rastrigin10100000 nsa1000 ms2012/8/00/01.243698e11.243698e1993699541.001.00591412
rastrigin10100000 nsa10000 ms2011/8/10/201.343193e11.343194e1993799541.000.24580411
rastrigin10100000 nsb100 ms2012/8/00/01.343193e11.392941e1754537499011.3611.20——
rastrigin10100000 nsb1000 ms2012/8/00/201.243698e11.243698e1768637613911.304.69——
rastrigin10100000 nsb10000 ms2011/8/10/201.343193e11.343194e1772857772711.260.34——
rastrigin10100000 nsc100 ms208/12/00/08.789651e07.158243e015829815854915.9715.97——
rastrigin10100000 nsc1000 ms207/9/40/05.969752e05.472272e015861815884715.9915.99——
rastrigin10100000 nsc10000 ms2010/8/20/204.974795e05.969752e015873310379616.004.41——
rastrigin400 nsa100 ms209/11/00/07.511923e17.462181e12033633561291.001.0045352425
rastrigin400 nsa1000 ms2014/6/00/36.964701e18.059150e12108633779801.001.0043602263
rastrigin400 nsa10000 ms2013/7/00/206.914954e17.760661e12142863828031.000.1642842230
rastrigin400 nsb100 ms209/11/00/07.511923e17.462181e11496772198965.507.26——
rastrigin400 nsb1000 ms2014/6/00/06.964701e18.059150e11549502379955.337.10——
rastrigin400 nsb10000 ms2013/7/00/206.914954e17.760661e11617282565855.021.63——
rastrigin400 nsc100 ms2012/8/00/04.775799e15.024539e12698475469688015.9715.93——
rastrigin400 nsc1000 ms207/13/00/175.074287e14.825547e12975406507503916.0015.53——
rastrigin400 nsc10000 ms208/12/00/205.074286e14.427563e12994555366830916.002.97——
rastrigin401000 nsa100 ms2015/5/00/06.765712e18.158646e11812522895171.001.0044622398
rastrigin401000 nsa1000 ms206/13/10/48.656124e17.810414e11856143017191.001.0043322259
rastrigin401000 nsa10000 ms209/11/00/207.163690e17.462181e11872243101911.000.2342862168
rastrigin401000 nsb100 ms2015/5/00/06.765712e18.158646e11483532140905.617.41——
rastrigin401000 nsb1000 ms206/13/10/38.656124e17.810414e11541472314035.477.12——
rastrigin401000 nsb10000 ms209/11/00/207.163690e17.462181e11587372382525.142.17——
rastrigin401000 nsc100 ms208/11/10/04.875295e14.974790e12588796402962515.9915.96——
rastrigin401000 nsc1000 ms2011/9/00/144.875295e14.875296e12660348423852116.0015.89——
rastrigin401000 nsc10000 ms2012/8/00/204.875296e14.974791e12681940305061916.003.41——
rastrigin40100000 nsa100 ms209/11/00/03.699582e23.619340e2957397561.001.0043642408
rastrigin40100000 nsa1000 ms209/11/00/07.412429e17.661168e1958197651.001.0042732311
rastrigin40100000 nsa10000 ms2011/9/00/07.611419e17.462180e1958197691.001.0042792267
rastrigin40100000 nsb100 ms209/11/00/07.661165e17.412433e1732958560510.2411.80——
rastrigin40100000 nsb1000 ms209/11/00/07.412429e17.661168e1735298883310.2511.84——
rastrigin40100000 nsb10000 ms2011/9/00/197.611419e17.462180e1740969008110.248.47——
rastrigin40100000 nsc100 ms2010/10/00/03.307478e23.301660e215145515464815.9415.92——
rastrigin40100000 nsc1000 ms207/13/00/04.974790e14.576807e115231315545215.9915.99——
rastrigin40100000 nsc10000 ms2010/10/00/04.726050e15.074286e115251715571316.0016.00——
rosenbrock100 nsa100 ms201/2/170/203.159388e-281.519400e-23155351220195361.000.04574426
rosenbrock100 nsa1000 ms202/1/170/202.248254e-281.763676e-23146988820315871.000.00611423
rosenbrock100 nsa10000 ms201/0/190/208.085824e-291.695842e-23149636120121861.000.00599427
rosenbrock100 nsb100 ms201/2/170/204.646391e-281.519400e-2343288342553014.392.38——
rosenbrock100 nsb1000 ms202/1/170/202.252198e-281.763676e-2343143441057514.350.26——
rosenbrock100 nsb10000 ms201/0/190/201.206957e-281.695842e-2343066341766314.300.03——
rosenbrock100 nsc100 ms200/0/200/201.522502e-283.526697e-24171840591071194415.991.22——
rosenbrock100 nsc1000 ms200/0/200/200.000000e05.363685e-2417100193979028516.000.12——
rosenbrock100 nsc10000 ms200/0/200/200.000000e05.209533e-2417104564944628716.000.01——
rosenbrock101000 nsa100 ms201/2/170/203.975859e-281.364954e-236189396837351.000.12572419
rosenbrock101000 nsa1000 ms202/2/160/201.684218e-281.409886e-236176786826541.000.01576422
rosenbrock101000 nsa10000 ms202/2/160/201.995818e-281.655112e-236092626830481.000.00598421
rosenbrock101000 nsb100 ms201/2/170/204.579338e-281.364954e-2338452138357714.792.77——
rosenbrock101000 nsb1000 ms202/2/160/202.335028e-281.409886e-2339340738112914.670.28——
rosenbrock101000 nsb10000 ms202/2/160/201.995818e-281.655112e-2338897938400114.590.03——
rosenbrock101000 nsc100 ms200/0/200/201.621109e-284.229442e-247851410404820615.992.70——
rosenbrock101000 nsc1000 ms200/0/200/200.000000e04.804196e-247921142398635016.000.27——
rosenbrock101000 nsc10000 ms200/0/200/200.000000e05.835709e-247763736590654116.000.03——
rosenbrock10100000 nsa100 ms209/11/00/07.682553e07.883253e0993099481.001.00661478
rosenbrock10100000 nsa1000 ms201/2/170/186.500214e-281.896318e-23993799521.000.79588430
rosenbrock10100000 nsa10000 ms201/2/170/203.171221e-281.459882e-23993899521.000.08571435
rosenbrock10100000 nsb100 ms200/1/190/58.773137e-222.183126e-22755267520211.3311.21——
rosenbrock10100000 nsb1000 ms201/2/170/203.561707e-281.896318e-23766117621411.271.15——
rosenbrock10100000 nsb10000 ms201/2/170/201.747327e-281.459882e-23765227603911.260.11——
rosenbrock10100000 nsc100 ms2010/10/00/05.346727e05.602502e015727615830715.8915.96——
rosenbrock10100000 nsc1000 ms200/0/200/203.786532e-285.215371e-2415846013125315.9912.66——
rosenbrock10100000 nsc10000 ms200/0/200/201.585610e-284.690566e-241585566850116.001.38——
rosenbrock400 nsa100 ms200/20/00/02.249071e11.104097e12214093929821.001.0043782406
rosenbrock400 nsa1000 ms201/2/170/201.537135e-261.448309e-252324383996631.000.1841632363
rosenbrock400 nsa10000 ms201/2/170/201.211138e-261.648420e-252357813998051.000.0241022362
rosenbrock400 nsb100 ms202/18/00/02.755111e12.360387e11499972128295.657.40——
rosenbrock400 nsb1000 ms201/2/170/201.583244e-261.448309e-251677782436285.012.05——
rosenbrock400 nsb10000 ms201/2/170/201.211138e-261.648420e-251676692470375.260.20——
rosenbrock400 nsc100 ms200/20/00/02.153261e11.183745e13117189536629615.9915.95——
rosenbrock400 nsc1000 ms200/0/200/201.260403e-268.553027e-263250114335182916.003.71——
rosenbrock400 nsc10000 ms200/0/200/209.318419e-278.566477e-263352776348670716.000.37——
rosenbrock401000 nsa100 ms200/20/00/02.503701e11.789026e11844952911151.001.0043822397
rosenbrock401000 nsa1000 ms201/0/190/201.590659e-261.389515e-251922822957981.000.2541632343
rosenbrock401000 nsa10000 ms200/3/170/201.231767e-261.277860e-251949132941141.000.0240922362
rosenbrock401000 nsb100 ms202/18/00/02.712830e12.295237e11546132136905.597.36——
rosenbrock401000 nsb1000 ms201/0/190/201.624186e-261.389515e-251594772394775.552.25——
rosenbrock401000 nsb10000 ms200/3/170/201.249043e-261.277860e-251663802530235.440.20——
rosenbrock401000 nsc100 ms200/20/00/02.419971e11.783019e12560328396972615.9915.96——
rosenbrock401000 nsc1000 ms200/0/200/201.192915e-268.286510e-262785655284034116.004.59——
rosenbrock401000 nsc10000 ms200/0/200/209.362596e-278.259098e-262851316273987516.000.46——
rosenbrock40100000 nsa100 ms209/11/00/05.184376e24.831804e2958097601.001.0043312405
rosenbrock40100000 nsa1000 ms207/13/00/03.231953e13.216240e1958597671.001.0042732334
rosenbrock40100000 nsa10000 ms202/2/160/172.172523e-261.584881e-25960097681.000.7441142328
rosenbrock40100000 nsb100 ms208/12/00/03.444010e13.328130e1731908618310.2211.84——
rosenbrock40100000 nsb1000 ms201/2/170/182.723542e-261.746783e-25744218859210.379.68——
rosenbrock40100000 nsb10000 ms202/2/160/201.307103e-261.584881e-25759918871710.400.96——
rosenbrock40100000 nsc100 ms2012/8/00/02.780853e22.950155e215123715441815.9215.91——
rosenbrock40100000 nsc1000 ms2011/9/00/02.962583e12.972031e115241515551815.9915.99——
rosenbrock40100000 nsc10000 ms200/0/200/201.581508e-268.590892e-2615279512021016.0012.27——
sphere100 nsa100 ms200/0/200/200.000000e00.000000e0148642221492691.000.21593386
sphere100 nsa1000 ms200/0/200/200.000000e00.000000e0142417921470601.000.02623386
sphere100 nsa10000 ms200/0/200/200.000000e00.000000e0139684121359241.000.00636389
sphere100 nsb100 ms200/0/200/110.000000e00.000000e042702344157214.3113.28——
sphere100 nsb1000 ms200/0/200/200.000000e00.000000e043769245170614.271.39——
sphere100 nsb10000 ms200/0/200/200.000000e00.000000e040964343648214.210.14——
sphere100 nsc100 ms200/0/200/200.000000e00.000000e0161967201227859915.996.54——
sphere100 nsc1000 ms200/0/200/200.000000e00.000000e0163978561298487516.000.65——
sphere100 nsc10000 ms200/0/200/200.000000e00.000000e0161336241249016716.000.07——
sphere101000 nsa100 ms200/0/200/200.000000e00.000000e06068826935961.000.64592386
sphere101000 nsa1000 ms200/0/200/200.000000e00.000000e06055616934161.000.07596387
sphere101000 nsa10000 ms200/0/200/200.000000e00.000000e05954076968301.000.01624380
sphere101000 nsb100 ms200/0/200/110.000000e00.000000e038800942344514.5813.96——
sphere101000 nsb1000 ms200/0/200/200.000000e00.000000e038219341576614.671.52——
sphere101000 nsb10000 ms200/0/200/200.000000e00.000000e039014043302214.570.16——
sphere101000 nsc100 ms200/0/200/200.000000e00.000000e07709846702796015.9913.73——
sphere101000 nsc1000 ms200/0/200/200.000000e00.000000e07768164671955016.001.38——
sphere101000 nsc10000 ms200/0/200/200.000000e00.000000e07621611669348116.000.14——
sphere10100000 nsa100 ms2010/10/00/05.081594e-68.171095e-6993499491.001.00614460
sphere10100000 nsa1000 ms200/0/200/10.000000e00.000000e0993599531.001.00598415
sphere10100000 nsa10000 ms200/0/200/200.000000e00.000000e0993499551.000.48605394
sphere10100000 nsb100 ms200/0/200/00.000000e00.000000e0750787497311.3311.19——
sphere10100000 nsb1000 ms200/0/200/200.000000e00.000000e0765617708311.286.77——
sphere10100000 nsb10000 ms200/0/200/200.000000e00.000000e0763897757411.280.67——
sphere10100000 nsc100 ms2015/5/00/08.366297e-71.490632e-615734715830415.9015.97——
sphere10100000 nsc1000 ms200/0/200/60.000000e00.000000e015840515863515.9915.99——
sphere10100000 nsc10000 ms200/0/200/200.000000e00.000000e015849612837716.007.29——
sphere1000 nsa100 ms200/20/00/01.542927e05.915987e-727686958711.001.003586010170
sphere1000 nsa1000 ms200/0/200/158.375716e-236.790515e-2727295962731.000.333637710127
sphere1000 nsa10000 ms200/0/200/175.328755e-286.244623e-2728105973851.000.033532010008
sphere1000 nsb100 ms200/20/00/02.103431e09.578082e-626003826771.733.24——
sphere1000 nsb1000 ms200/0/200/151.978444e-216.790515e-2725716822321.681.25——
sphere1000 nsb10000 ms200/0/200/175.328755e-286.244623e-2726480856881.640.11——
sphere1000 nsc100 ms200/20/00/01.463292e02.005969e-6374336127388015.9415.90——
sphere1000 nsc1000 ms200/0/200/202.683294e-212.366583e-3038786972987115.998.22——
sphere1000 nsc10000 ms200/0/200/200.000000e02.366583e-3039374742271116.004.44——
sphere1001000 nsa100 ms200/20/00/01.354593e02.527173e-627989899891.001.003468210066
sphere1001000 nsa1000 ms200/0/200/165.043379e-236.661930e-2727463907321.000.35353669975
sphere1001000 nsa10000 ms200/0/200/138.365870e-286.943554e-2727481912141.000.04353429917
sphere1001000 nsb100 ms200/20/00/01.848524e01.024064e-526762827241.783.28——
sphere1001000 nsb1000 ms200/0/200/163.818507e-226.661930e-2726442822991.661.28——
sphere1001000 nsb10000 ms200/0/200/138.365870e-286.943554e-2726402839961.670.13——
sphere1001000 nsc100 ms200/20/00/01.663087e06.153456e-6382138121407515.9415.91——
sphere1001000 nsc1000 ms200/0/200/204.757963e-212.366583e-3038601971778815.998.76——
sphere1001000 nsc10000 ms200/0/200/200.000000e02.366583e-3038746740015916.004.47——
sphere100100000 nsa100 ms203/17/00/01.039374e28.035319e1754091281.001.00325759507
sphere100100000 nsa1000 ms200/20/00/07.755509e-51.405297e-6744891021.001.00342149825
sphere100100000 nsa10000 ms200/0/200/161.120577e-275.790239e-27743190961.000.35345269897
sphere100100000 nsb100 ms200/20/00/06.382395e07.498890e-220644416643.576.20——
sphere100100000 nsb1000 ms200/0/200/162.667843e-166.685991e-2720089417003.494.69——
sphere100100000 nsb10000 ms200/0/200/169.123176e-285.790239e-2720143418923.460.47——
sphere100100000 nsc100 ms201/19/00/08.384273e15.784852e111423014254315.8415.85——
sphere100100000 nsc1000 ms200/20/00/05.597461e-51.007702e-611622914414115.9815.98——
sphere100100000 nsc10000 ms200/0/200/200.000000e02.366583e-301161387396316.008.18——

Interpretation boundary

The three arms isolate different implementation choices:

  • Arm A is the closest same-family implementation comparison.
  • Arm B also compares each library’s population-evaluation path.
  • Arm C gives both implementations the same independent-multistart architecture.

None of these rows compares fcmaes coordinated DE→CMA retry with cmaes BIPOP. That is a different-algorithm system comparison covered by the broader optimizer benchmark.

The analytic controls do not show that CMA-ES is the right solver for them. For sufficiently costly application objectives, the measured implementation overhead also becomes irrelevant. Smoke and pilot presets validate the harness; they are not publication evidence.