Dev Builds » 20230425-0619

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 09:59:25 1204769 3334 1390 313 1631 +116.4 +/- 5.34 1 51 515 1070 30 +256.83 +/- 15.02
ncm-dbt-02 09:52:08 1232347 3294 1383 299 1612 +118.75 +/- 5.43 3 45 499 1065 35 +262.67 +/- 15.26
ncm-dbt-03 09:59:08 1232697 3346 1451 269 1626 +128.26 +/- 5.31 1 35 467 1121 49 +286.66 +/- 15.77
ncm-dbt-04 09:56:10 1227214 3350 1395 296 1659 +118.35 +/- 5.48 1 49 523 1054 48 +256.48 +/- 14.9
ncm-dbt-05 09:52:28 1224945 3320 1369 322 1629 +113.43 +/- 5.44 1 59 522 1048 30 +248.11 +/- 14.92
ncm-dbt-06 09:58:19 1227827 3356 1405 306 1645 +118.13 +/- 5.3 0 48 517 1079 34 +260.31 +/- 14.99
20000 8393 1805 9802 +118.88 +/- 2.2 7 287 3043 6437 226 +261.6 +/- 6.17

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
185396 ncm-dbt-02 1221536 294 128 30 136 +120.41 +/- 17.82 0 3 47 93 4 +263.1 +/- 50.27
185395 ncm-dbt-05 1213314 320 132 41 147 +101.6 +/- 20.11 0 15 42 100 3 +214.85 +/- 51.62
185394 ncm-dbt-01 1178530 334 144 40 150 +111.9 +/- 16.18 0 4 57 104 2 +246.74 +/- 45.46
185393 ncm-dbt-03 1251187 346 147 26 173 +126.85 +/- 17.44 1 3 49 114 6 +282.01 +/- 49.39
185392 ncm-dbt-04 1222601 350 142 33 175 +111.92 +/- 18.19 0 9 54 106 6 +234.68 +/- 46.83
185391 ncm-dbt-06 1232370 356 159 32 165 +129.64 +/- 16.77 0 4 50 117 7 +284.32 +/- 48.88
185384 ncm-dbt-02 1239657 500 202 53 245 +106.78 +/- 14.52 2 7 85 152 4 +234.38 +/- 37.19
185383 ncm-dbt-05 1232277 500 212 44 244 +121.45 +/- 13.84 0 8 71 166 5 +270.57 +/- 40.83
185382 ncm-dbt-01 1183423 500 203 51 246 +109.07 +/- 14.05 0 12 76 160 2 +240.82 +/- 39.35
185381 ncm-dbt-03 1223463 500 215 50 235 +119.11 +/- 14.19 0 7 79 156 8 +256.44 +/- 38.62
185380 ncm-dbt-04 1212601 500 203 43 254 +115.22 +/- 13.72 0 6 84 154 6 +249.64 +/- 37.33
185379 ncm-dbt-06 1216285 500 206 44 250 +116.77 +/- 13.36 0 5 83 157 5 +256.44 +/- 37.51
185372 ncm-dbt-02 1236154 500 214 40 246 +126.17 +/- 13.95 0 9 63 173 5 +285.49 +/- 43.31
185371 ncm-dbt-05 1236224 500 195 51 254 +102.97 +/- 14.02 0 11 87 149 3 +221.9 +/- 36.76
185370 ncm-dbt-03 1234996 500 217 38 245 +130.14 +/- 13.88 0 4 73 163 10 +285.49 +/- 40.16
185369 ncm-dbt-04 1225403 500 201 44 255 +112.91 +/- 14.05 1 9 74 164 2 +254.16 +/- 39.94
185368 ncm-dbt-01 1229733 500 207 41 252 +119.89 +/- 13.51 0 8 71 168 3 +270.57 +/- 40.83
185367 ncm-dbt-06 1232364 500 206 38 256 +121.45 +/- 14.17 0 10 67 168 5 +270.57 +/- 41.96
185360 ncm-dbt-02 1213361 500 210 46 244 +118.33 +/- 14.03 0 7 79 157 7 +256.44 +/- 38.62
185359 ncm-dbt-05 1218112 500 205 40 255 +119.11 +/- 13.69 0 7 76 162 5 +263.42 +/- 39.41
185357 ncm-dbt-04 1234844 500 213 46 241 +120.67 +/- 14.35 0 5 84 150 11 +254.16 +/- 37.27
185356 ncm-dbt-03 1240136 500 211 41 248 +123.02 +/- 12.74 0 5 72 171 2 +282.94 +/- 40.5
185355 ncm-dbt-01 1188722 500 216 38 246 +129.35 +/- 12.41 0 2 72 172 4 +298.62 +/- 40.29
185353 ncm-dbt-06 1233980 500 211 53 236 +113.68 +/- 13.89 0 9 78 159 4 +249.64 +/- 38.9
185348 ncm-dbt-02 1238506 500 205 41 254 +118.33 +/- 13.52 0 5 82 157 6 +258.75 +/- 37.76
185345 ncm-dbt-05 1217880 500 209 49 242 +115.22 +/- 13.2 0 3 90 151 6 +249.64 +/- 35.69
185343 ncm-dbt-04 1213692 500 201 37 262 +118.33 +/- 13.7 0 4 86 152 8 +254.16 +/- 36.71
185340 ncm-dbt-06 1221474 500 208 45 247 +117.55 +/- 13.18 0 5 81 160 4 +261.07 +/- 38.02
185339 ncm-dbt-01 1214823 500 206 36 258 +123.02 +/- 13.65 0 7 71 167 5 +275.45 +/- 40.84
185338 ncm-dbt-03 1211892 500 229 35 236 +142.25 +/- 12.98 0 2 61 178 9 +330.23 +/- 44.08
185331 ncm-dbt-02 1238270 500 216 49 235 +120.67 +/- 13.84 0 8 72 165 5 +268.17 +/- 40.54
185329 ncm-dbt-05 1230113 500 208 52 240 +112.14 +/- 14.22 1 7 82 155 5 +245.2 +/- 37.88
185328 ncm-dbt-04 1252673 500 221 56 223 +119.11 +/- 14.36 0 9 74 160 7 +258.75 +/- 39.96
185327 ncm-dbt-01 1229772 500 213 52 235 +116.0 +/- 14.21 1 6 80 157 6 +254.16 +/- 38.36
185326 ncm-dbt-03 1244916 500 211 44 245 +120.67 +/- 13.32 0 7 72 168 3 +273.0 +/- 40.54
185325 ncm-dbt-06 1241062 500 201 49 250 +109.07 +/- 14.05 0 10 82 154 4 +236.51 +/- 37.9
185318 ncm-dbt-05 1226695 500 208 45 247 +117.55 +/- 13.53 0 8 74 165 3 +263.42 +/- 39.97
185317 ncm-dbt-02 1238951 500 208 40 252 +121.46 +/- 13.84 1 6 71 168 4 +275.45 +/- 40.84
185316 ncm-dbt-04 1228688 500 214 37 249 +128.55 +/- 14.08 0 7 67 168 8 +285.49 +/- 42.08
185315 ncm-dbt-06 1217259 500 214 45 241 +122.24 +/- 13.3 0 5 76 164 5 +273.0 +/- 39.35
185314 ncm-dbt-01 1208386 500 201 55 244 +104.49 +/- 14.96 0 12 88 142 8 +215.85 +/- 36.55
185313 ncm-dbt-03 1222290 500 221 35 244 +135.76 +/- 14.47 0 7 61 171 11 +301.33 +/- 44.13

Commit

Commit ID 41f50b2c83a0ba36a2b9c507c1783e57c9b13485
Author Linmiao Xu
Date 2023-04-25 06:19:00 UTC
Update default net to nn-e1fb1ade4432.nnue Created by retraining nn-dabb1ed23026.nnue with a dataset composed of: * The previous best dataset (nn-1ceb1a57d117.nnue dataset) * Adding de-duplicated T80 data from feb2023 and the last 10 days of jan2023, filtered with v6-dd Initially trained with the same options as the recent master net (nn-1ceb1a57d117.nnue). Around epoch 890, training was manually stopped and max epoch increased to 1000. ``` python3 easy_train.py \ --experiment-name leela96-dfrc99-T60novdec-v2-T80augsep-v6-T80junjuloctnovjanfebT79aprmayT78jantosepT77dec-v6dd \ --training-dataset /data/leela96-dfrc99-T60novdec-v2-T80augsep-v6-T80junjuloctnovjanfebT79aprmayT78jantosepT77dec-v6dd.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes \ --start-from-engine-test-net True \ --early-fen-skipping 30 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --max_epoch 900 \ --lr 4.375e-4 \ --gamma 0.995 \ --tui False \ --gpus "0," \ --seed $RANDOM ``` The same v6-dd filtering and binpack minimizer was used for preparing the recent nn-1ceb1a57d117.nnue dataset. ``` python3 interleave_binpacks.py \ leela96-filt-v2.binpack \ dfrc99-filt-v2.binpack \ T60-nov2021-12tb7p-eval-filt-v2.binpack \ T60-dec2021-12tb7p-eval-filt-v2.binpack \ filt-v6/test80-aug2022-16tb7p-filter-v6.min-mar2023.binpack \ filt-v6/test80-sep2022-16tb7p-filter-v6.min-mar2023.binpack \ filt-v6-dd/test80-jun2022-16tb7p-filter-v6-dd.min-mar2023.binpack \ filt-v6-dd/test80-jul2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test80-oct2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test80-nov2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test80-jan2022-3of3-16tb7p-filter-v6-dd.min-mar2023.binpack \ filt-v6-dd/test80-feb2023-16tb7p-filter-v6-dd.min-mar2023.binpack \ filt-v6-dd/test79-apr2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test79-may2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test78-jantomay2022-16tb7p-filter-v6-dd.min-mar2023.binpack \ filt-v6-dd/test78-juntosep2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test77-dec2021-16tb7p-filter-v6-dd.binpack \ /data/leela96-dfrc99-T60novdec-v2-T80augsep-v6-T80junjuloctnovjanfebT79aprmayT78jantosepT77dec-v6dd.binpack ``` Links for downloading the training data components can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move: nn-epoch919.nnue : 2.6 +/- 2.8 Passed STC vs. nn-dabb1ed23026.nnue https://tests.stockfishchess.org/tests/view/644420df94ff3db5625f2af5 LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 125960 W: 33898 L: 33464 D: 58598 Ptnml(0-2): 351, 13920, 34021, 14320, 368 Passed LTC vs. nn-1ceb1a57d117.nnue https://tests.stockfishchess.org/tests/view/64469f128d30316529b3dc46 LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 24544 W: 6817 L: 6542 D: 11185 Ptnml(0-2): 8, 2252, 7488, 2505, 19 closes https://github.com/official-stockfish/Stockfish/pull/4546 bench 3714847
Copyright 2011–2024 Next Chess Move LLC