Dev Builds » 20230227-2107

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 04:59:20 1123893 1666 682 159 825 +112.88 +/- 7.52 0 25 275 518 15 +246.22 +/- 20.56
ncm-dbt-02 04:57:08 1237739 1632 674 150 808 +115.64 +/- 7.86 0 29 254 513 20 +250.57 +/- 21.42
ncm-dbt-03 05:01:13 1250005 1664 683 164 817 +112.1 +/- 7.6 0 26 278 511 17 +242.66 +/- 20.44
ncm-dbt-04 05:00:17 1233511 1684 699 148 837 +118.02 +/- 7.5 0 22 267 533 20 +257.96 +/- 20.86
ncm-dbt-05 04:59:17 1226707 1674 702 189 783 +110.01 +/- 7.39 0 26 282 519 10 +241.34 +/- 20.29
ncm-dbt-06 05:00:45 1220591 1680 687 164 829 +111.87 +/- 7.26 0 25 274 534 7 +248.69 +/- 20.6
ncm-et-3 06:21:33 1301315 1654 665 176 813 +105.88 +/- 7.89 0 36 283 491 17 +225.35 +/- 20.28
ncm-et-4 06:20:30 1308008 1650 666 176 808 +106.38 +/- 7.75 0 30 292 486 17 +226.69 +/- 19.94
ncm-et-9 06:24:27 1289545 1682 681 174 827 +108.08 +/- 7.68 1 28 292 503 17 +232.17 +/- 19.94
ncm-et-10 06:23:13 1299052 1678 702 167 809 +114.77 +/- 7.67 1 25 270 524 19 +249.76 +/- 20.76
ncm-et-13 06:24:06 1307528 1684 702 173 809 +112.96 +/- 7.52 0 26 277 523 16 +245.9 +/- 20.49
ncm-et-15 06:22:35 1303178 1652 689 165 798 +114.14 +/- 7.75 0 28 265 514 19 +247.07 +/- 20.96
20000 8232 2005 9763 +111.89 +/- 2.2 2 326 3309 6169 194 +242.73 +/- 5.92

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
201257 ncm-dbt-02 1231249 132 55 15 62 +108.69 +/- 33.13 0 6 17 40 3 +220.18 +/- 82.96
201256 ncm-dbt-01 1125671 166 65 8 93 +124.35 +/- 23.7 0 4 18 61 0 +292.46 +/- 82.56
201255 ncm-dbt-03 1252685 164 64 18 82 +100.13 +/- 27.93 0 6 27 46 3 +202.34 +/- 66.72
201254 ncm-dbt-05 1221756 174 74 26 74 +98.39 +/- 23.06 0 4 31 52 0 +215.71 +/- 62.4
201253 ncm-dbt-06 1216240 180 76 15 89 +122.58 +/- 23.32 0 3 25 60 2 +272.73 +/- 70.2
201252 ncm-dbt-04 1216216 184 71 15 98 +109.2 +/- 23.8 0 3 33 53 3 +228.12 +/- 60.3
201251 ncm-dbt-02 1246419 500 200 41 259 +114.45 +/- 14.05 0 10 75 161 4 +251.89 +/- 39.67
201250 ncm-dbt-01 1123657 500 204 50 246 +110.6 +/- 14.06 0 8 86 150 6 +236.51 +/- 36.93
201249 ncm-dbt-05 1218669 500 209 60 231 +106.77 +/- 13.89 0 10 84 153 3 +232.26 +/- 37.43
201248 ncm-dbt-03 1262810 500 208 50 242 +113.68 +/- 13.89 0 7 84 153 6 +245.2 +/- 37.37
201247 ncm-dbt-06 1212428 500 197 47 256 +107.54 +/- 13.4 0 9 83 157 1 +238.66 +/- 37.66
201246 ncm-dbt-04 1217613 500 208 44 248 +118.33 +/- 13.52 0 7 76 163 4 +263.42 +/- 39.41
201245 ncm-dbt-02 1230129 500 211 46 243 +119.11 +/- 14.03 0 7 78 158 7 +258.75 +/- 38.88
201244 ncm-dbt-05 1221891 500 202 55 243 +105.25 +/- 13.39 0 7 92 148 3 +228.08 +/- 35.56
201243 ncm-dbt-01 1118720 500 202 54 244 +106.01 +/- 13.72 0 8 90 148 4 +228.08 +/- 36.04
201242 ncm-dbt-06 1240624 500 209 46 245 +117.55 +/- 13.0 0 5 80 162 3 +263.42 +/- 38.27
201241 ncm-dbt-03 1222680 500 213 42 245 +123.81 +/- 13.46 0 6 72 167 5 +277.93 +/- 40.53
201240 ncm-dbt-04 1243869 500 217 50 233 +120.67 +/- 13.67 0 5 80 158 7 +263.42 +/- 38.27
201239 ncm-dbt-01 1127527 500 211 47 242 +118.33 +/- 13.35 0 5 81 159 5 +261.07 +/- 38.02
201238 ncm-dbt-02 1243160 500 208 48 244 +115.22 +/- 13.72 0 6 84 154 6 +249.64 +/- 37.33
201237 ncm-dbt-04 1256347 500 203 39 258 +118.33 +/- 13.86 0 7 78 159 6 +258.75 +/- 38.88
201236 ncm-dbt-05 1244512 500 217 48 235 +122.24 +/- 13.12 0 5 75 166 4 +275.45 +/- 39.63
201235 ncm-dbt-06 1213074 500 205 56 239 +106.77 +/- 13.23 0 8 86 155 1 +236.51 +/- 36.93
201234 ncm-dbt-03 1261848 500 198 54 248 +102.97 +/- 13.38 0 7 95 145 3 +221.9 +/- 34.93
167372 ncm-et-4 1308630 150 63 13 74 +120.41 +/- 24.37 0 1 25 47 2 +263.42 +/- 69.54
167371 ncm-et-15 1301945 152 66 15 71 +121.26 +/- 25.19 0 3 20 52 1 +274.16 +/- 78.95
167370 ncm-et-3 1289996 154 63 17 74 +107.03 +/- 22.03 0 0 32 44 1 +232.47 +/- 59.23
167369 ncm-et-10 1298474 178 75 21 82 +108.82 +/- 25.77 0 6 25 56 2 +232.41 +/- 69.17
167368 ncm-et-9 1277614 182 76 19 87 +112.6 +/- 26.73 1 3 29 54 4 +237.27 +/- 64.73
167367 ncm-et-13 1320248 184 75 17 92 +113.37 +/- 23.81 0 4 28 58 2 +245.58 +/- 65.95
167366 ncm-et-3 1310479 500 207 49 244 +113.68 +/- 13.89 0 9 78 159 4 +249.64 +/- 38.9
167365 ncm-et-15 1297984 500 216 54 230 +116.77 +/- 14.37 0 9 77 157 7 +251.89 +/- 39.16
167364 ncm-et-4 1300934 500 206 50 244 +112.14 +/- 14.06 0 7 87 149 7 +238.66 +/- 36.67
167363 ncm-et-10 1308964 500 205 43 252 +116.77 +/- 14.04 0 9 75 161 5 +256.44 +/- 39.69
167362 ncm-et-9 1292475 500 196 54 250 +101.46 +/- 13.53 0 7 98 141 4 +215.85 +/- 34.33
167361 ncm-et-13 1300385 500 209 50 241 +114.45 +/- 13.72 0 9 76 162 3 +254.16 +/- 39.42
167360 ncm-et-15 1300634 500 204 47 249 +112.91 +/- 14.69 0 12 75 157 6 +243.0 +/- 39.61
167359 ncm-et-3 1309154 500 199 56 245 +102.22 +/- 14.63 0 14 83 149 4 +217.85 +/- 37.63
167358 ncm-et-4 1304894 500 199 50 251 +106.77 +/- 14.21 0 11 83 152 4 +230.16 +/- 37.67
167357 ncm-et-10 1301405 500 204 51 245 +109.83 +/- 13.73 1 5 88 152 4 +240.82 +/- 36.39
167356 ncm-et-13 1294811 500 211 53 236 +113.68 +/- 13.55 0 6 85 154 5 +247.41 +/- 37.09
167355 ncm-et-9 1294921 500 206 54 240 +109.07 +/- 14.53 0 12 79 154 5 +234.38 +/- 38.61
167354 ncm-et-4 1317577 500 198 63 239 +96.19 +/- 14.11 0 11 97 138 4 +202.15 +/- 34.7
167353 ncm-et-3 1295631 500 196 54 250 +101.46 +/- 15.07 0 13 90 139 8 +207.95 +/- 36.13
167352 ncm-et-15 1312151 500 203 49 248 +110.6 +/- 13.22 0 4 93 148 5 +238.66 +/- 35.13
167351 ncm-et-10 1287366 500 218 52 230 +119.89 +/- 13.85 0 5 82 155 8 +258.75 +/- 37.76
167350 ncm-et-13 1314669 500 207 53 240 +110.6 +/- 13.89 0 7 88 149 6 +236.51 +/- 36.44
167349 ncm-et-9 1293173 500 203 47 250 +112.14 +/- 13.39 0 6 86 154 4 +245.2 +/- 36.85

Commit

Commit ID 876906965b8d552866486c0e6eda1184fdb1d636
Author Linmiao Xu
Date 2023-02-27 21:07:52 UTC
Update default net to nn-52471d67216a.nnue Created by retraining the master net with modifications to the previous best dataset: * Improving T80 oct+nov 2022 endgame lambda accuracy by rescoring with 12-16tb of syzygy 7p tablebases * Filtering T78 jun+jul+aug 2022 with d6pv2 search to remove positions with bestmove captures or one good move * Adding T80 sep 2022 data, rescored with 16tb of 7p tablebases, unfiltered Trained with max-epoch 900, end-lambda 0.7, and early-fen-skipping 28. ``` python3 easy_train.py \ --experiment-name leela96-dfrc99-T80octnovT79aprmayT78junjulaugT60novdec-filt-v2-T78sep12tb7p-T77decT80sep16tb7p-lambda7-sk28 \ --training-dataset /data/leela96-dfrc99-T80octnovT79aprmayT78junjulaugT60novdec-filt-v2-T78sep12tb7p-T77decT80sep16tb7p.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/easy-train-early-fen-skipping \ --early-fen-skipping 28 \ --start-from-engine-test-net True \ --gpus "0," \ --max_epoch 900 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --gamma 0.995 \ --lr 4.375e-4 \ --tui False \ --seed $RANDOM ``` Training data was rescored and d6pv2 filtered in the same way as recent best datasets. For preparing the merged training dataset: ``` python3 interleave_binpacks.py \ leela96-eval-filt-v2.binpack \ dfrc99-eval-filt-v2.binpack \ test80-oct2022-16tb7p-eval-filt-v2-d6.binpack \ test80-nov2022-12tb7p-eval-filt-v2-d6.binpack \ T79-apr2022-12tb7p-eval-filt-v2.binpack \ T79-may2022-12tb7p-eval-filt-v2.binpack \ test78-junjulaug2022-16tb7p-eval-filt-v2-d6.binpack \ T60-nov2021-12tb7p-eval-filt-v2.binpack \ T60-dec2021-12tb7p-eval-filt-v2.binpack \ T78-sep2022-12tb7p.binpack \ test77-dec2021-16gb7p.binpack \ test80-sep2022-16tb7p.binpack \ /data/leela96-dfrc99-T80octnovT79aprmayT78junjulaugT60novdec-filt-v2-T78sep12tb7p-T77decT80sep16tb7p.binpack ``` Links for downloading the training data components can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move: nn-epoch839.nnue : 0.6 +/- 1.4 Passed STC: https://tests.stockfishchess.org/tests/view/63f9ab4be74a12625bcdf02e LLR: 2.95 (-2.94,2.94) <0.00,2.00> Total: 84656 W: 22681 L: 22302 D: 39673 Ptnml(0-2): 271, 9343, 22734, 9696, 284 Passed LTC: https://tests.stockfishchess.org/tests/view/63fa3833e74a12625bce0c0e LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 184664 W: 49933 L: 49344 D: 85387 Ptnml(0-2): 111, 17977, 55561, 18578, 105 closes https://github.com/official-stockfish/Stockfish/pull/4416 bench: 4814343
Copyright 2011–2024 Next Chess Move LLC