Dev Builds » 20230611-1323

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 06:54:18 584048 4014 1317 749 1948 +49.5 ± 5.11 3 219 996 785 4 +100.9 ± 10.76
ncm-dbt-02 06:53:33 586697 4000 1327 756 1917 +49.94 ± 5.06 4 199 1028 760 9 +101.08 ± 10.55
ncm-dbt-03 06:53:07 586770 4000 1335 731 1934 +52.87 ± 5.02 1 190 1024 774 11 +106.39 ± 10.56
ncm-dbt-04 06:53:54 570481 4004 1334 748 1922 +51.22 ± 5.05 2 202 1013 778 7 +103.81 ± 10.64
ncm-dbt-05 06:54:18 584067 3982 1304 743 1935 +49.28 ± 5.01 2 203 1020 764 2 +100.62 ± 10.6
20000 6617 3727 9656 +50.56 ± 2.26 12 1013 5081 3861 33 +102.56 ± 4.75

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
378478 ncm-dbt-04 569310 4 1 0 3 +88.51 ± 137.63 0 0 1 1 0 +190.5 ± 634.21 ↓
378477 ncm-dbt-01 581194 14 5 4 5 +24.82 ± 45.76 0 0 6 1 0 +49.92 ± 94.39 ↓
378476 ncm-dbt-05 583740 482 156 99 227 +41.28 ± 14.3 1 25 131 84 0 +85.29 ± 29.48 ↓
378475 ncm-dbt-02 584453 500 155 106 239 +34.16 ± 14.78 2 31 134 82 1 +70.44 ± 29.29 ↓
378474 ncm-dbt-03 585379 500 171 87 242 +58.93 ± 13.43 0 18 130 102 0 +121.46 ± 29.51 ↓
378473 ncm-dbt-04 569589 500 158 100 242 +40.48 ± 14.12 0 29 135 85 1 +80.63 ± 29.1 ↓
378472 ncm-dbt-01 585253 500 162 101 237 +42.6 ± 14.52 0 31 128 90 1 +85.04 ± 30.07 ↓
378471 ncm-dbt-05 585674 500 163 92 245 +49.67 ± 13.97 0 25 129 96 0 +101.46 ± 29.84 ↓
378470 ncm-dbt-02 588260 500 165 98 237 +46.84 ± 14.49 0 28 129 91 2 +92.46 ± 29.89 ↓
378469 ncm-dbt-03 584622 500 175 87 238 +61.79 ± 13.71 0 18 127 104 1 +126.18 ± 29.94 ↓
378468 ncm-dbt-04 570869 500 165 89 246 +53.22 ± 14.15 0 25 124 101 0 +109.07 ± 30.54 ↓
378467 ncm-dbt-01 583531 500 160 83 257 +53.93 ± 13.19 0 17 140 92 1 +109.07 ± 28.05 ↓
378466 ncm-dbt-05 578877 500 164 88 248 +53.22 ± 14.69 0 29 116 105 0 +109.07 ± 31.72 ↓
378465 ncm-dbt-03 588260 500 176 84 240 +64.66 ± 13.69 0 16 128 104 2 +130.94 ± 29.72 ↓
378464 ncm-dbt-02 584453 500 162 96 242 +46.13 ± 13.05 0 19 147 83 1 +92.46 ± 27.17 ↓
378463 ncm-dbt-04 570789 500 175 90 235 +59.64 ± 14.04 0 21 124 104 1 +121.46 ± 30.47 ↓
378462 ncm-dbt-01 584411 500 163 91 246 +50.38 ± 14.55 1 26 123 100 0 +104.49 ± 30.71 ↓
378461 ncm-dbt-05 585211 500 159 80 261 +55.36 ± 13.11 0 17 137 96 0 +113.68 ± 28.47 ↓
378460 ncm-dbt-02 587622 500 174 92 234 +57.5 ± 14.49 1 22 122 104 1 +118.33 ± 30.8 ↓
378459 ncm-dbt-03 586816 500 150 97 253 +36.97 ± 14.31 0 33 131 86 0 +74.79 ± 29.69 ↓
378458 ncm-dbt-04 571190 500 180 89 231 +63.94 ± 14.23 0 21 118 110 1 +130.94 ± 31.35 ↓
378457 ncm-dbt-01 585000 500 154 95 251 +41.19 ± 14.03 0 29 133 88 0 +83.57 ± 29.37 ↓
378456 ncm-dbt-05 585084 500 156 103 241 +36.97 ± 13.9 0 30 137 83 0 +74.79 ± 28.85 ↓
378455 ncm-dbt-03 587367 500 170 90 240 +56.07 ± 13.73 0 21 128 101 0 +115.23 ± 29.89 ↓
378454 ncm-dbt-02 587155 500 168 107 225 +42.6 ± 13.84 0 27 135 88 0 +86.52 ± 29.06 ↓
378453 ncm-dbt-01 584874 500 172 99 229 +51.09 ± 15.61 1 33 109 106 1 +104.49 ± 32.7 ↓
378452 ncm-dbt-04 567601 500 160 94 246 +46.13 ± 14.18 0 26 134 88 2 +90.97 ± 29.17 ↓
378451 ncm-dbt-05 581902 500 176 99 225 +53.93 ± 14.19 0 23 129 96 2 +107.54 ± 29.8 ↓
378450 ncm-dbt-03 587325 500 161 94 245 +46.84 ± 14.49 0 26 135 85 4 +89.48 ± 29.04 ↓
378449 ncm-dbt-02 587749 500 164 98 238 +46.13 ± 13.91 0 26 132 92 0 +93.95 ± 29.45 ↓
378448 ncm-dbt-01 580530 500 169 101 230 +47.55 ± 14.26 0 28 126 96 0 +96.94 ± 30.31 ↓
378447 ncm-dbt-04 570029 500 165 105 230 +41.9 ± 14.74 1 31 125 93 0 +86.52 ± 30.48 ↓
378446 ncm-dbt-03 586477 500 162 95 243 +46.84 ± 14.22 0 28 127 95 0 +95.44 ± 30.17 ↓
378445 ncm-dbt-02 587028 500 172 74 254 +68.99 ± 14.7 1 19 113 115 2 +142.26 ± 32.1 ↓
378444 ncm-dbt-05 583279 500 163 100 237 +44.01 ± 14.47 1 28 128 93 0 +90.97 ± 30.05 ↓
378443 ncm-dbt-01 585421 500 164 94 242 +48.96 ± 14.74 1 27 124 97 1 +99.95 ± 30.58 ↓
378442 ncm-dbt-04 571994 500 163 94 243 +48.25 ± 14.3 1 24 131 93 1 +98.44 ± 29.57 ↓
378441 ncm-dbt-05 588771 500 167 82 251 +59.64 ± 14.59 0 26 113 111 0 +123.02 ± 32.15 ↓
378440 ncm-dbt-02 586858 500 167 85 248 +57.5 ± 14.89 0 27 116 105 2 +115.23 ± 31.71 ↓
378439 ncm-dbt-04 572961 500 167 87 246 +56.07 ± 14.43 0 25 121 103 1 +113.68 ± 30.97 ↓
378438 ncm-dbt-01 586223 500 168 81 251 +61.07 ± 14.92 0 28 107 115 0 +126.18 ± 33.04 ↓
378437 ncm-dbt-03 587919 500 170 97 233 +51.09 ± 15.61 1 30 118 97 4 +99.95 ± 31.44 ↓

Commit

Commit ID 932f5a2d657c846c282adcf2051faef7ca17ae15
Author Linmiao Xu
Date 2023-06-11 13:23:52 UTC
Update default net to nn-ea57bea57e32.nnue Created by retraining an earlier epoch (ep659) of the experiment that led to the first SFNNv6 net: - First retrained on the nn-0dd1cebea573 dataset - Then retrained with skip 20 on a smaller dataset containing unfiltered Leela data - And then retrained again with skip 27 on the nn-0dd1cebea573 dataset The equivalent 7-step training sequence from scratch that led here was: 1. max-epoch 400, lambda 1.0, constant LR 9.75e-4, T79T77-filter-v6-dd.min.binpack ep379 chosen for retraining in step2 2. max-epoch 800, end-lambda 0.75, T60T70wIsRightFarseerT60T74T75T76.binpack ep679 chosen for retraining in step3 3. max-epoch 800, end-lambda 0.75, skip 28, nn-e1fb1ade4432 dataset ep799 chosen for retraining in step4 4. max-epoch 800, end-lambda 0.7, skip 28, nn-e1fb1ade4432 dataset ep759 became nn-8d69132723e2.nnue (first SFNNv6 net) ep659 chosen for retraining in step5 5. max-epoch 800, end-lambda 0.7, skip 28, nn-0dd1cebea573 dataset ep759 chosen for retraining in step6 6. max-epoch 800, end-lambda 0.7, skip 20, leela-dfrc-v2-T77decT78janfebT79aprT80apr.binpack ep639 chosen for retraining in step7 7. max-epoch 800, end-lambda 0.7, skip 27, nn-0dd1cebea573 dataset ep619 became nn-ea57bea57e32.nnue For the last retraining (step7): python3 easy_train.py --experiment-name L1-1536-Re6-masterShuffled-ep639-sk27-Re5-leela-dfrc-v2-T77toT80small-Re4-masterShuffled-ep659-Re3-sameAs-Re2-leela96-dfrc99-16t-v2-T60novdecT80juntonovjanfebT79aprmayT78jantosepT77dec-v6dd-Re1-LeelaFarseer-new-T77T79 \ --training-dataset /data/leela96-dfrc99-T60novdec-v2-T80juntonovjanfebT79aprmayT78jantosepT77dec-v6dd-T80apr.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes-L1-1536 \ --early-fen-skipping 27 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --max_epoch 800 \ --start-from-engine-test-net False \ --start-from-model /data/L1-1536-Re5-leela-dfrc-v2-T77toT80small-epoch639.nnue \ --lr 4.375e-4 \ --gamma 0.995 \ --tui False \ --seed $RANDOM \ --gpus "0," For preparing the step6 leela-dfrc-v2-T77decT78janfebT79aprT80apr.binpack dataset: python3 interleave_binpacks.py \ leela96-filt-v2.binpack \ dfrc99-16tb7p-eval-filt-v2.binpack \ test77-dec2021-16tb7p.no-db.min-mar2023.binpack \ test78-janfeb2022-16tb7p.no-db.min-mar2023.binpack \ test79-apr2022-16tb7p-filter-v6-dd.binpack \ test80-apr2022-16tb7p.no-db.min-mar2023.binpack \ /data/leela-dfrc-v2-T77decT78janfebT79aprT80apr.binpack The unfiltered Leela data used for the step6 dataset can be found at: https://robotmoon.com/nnue-training-data Local elo at 25k nodes per move: nn-epoch619.nnue : 2.3 +/- 1.9 Passed STC: https://tests.stockfishchess.org/tests/view/6480d43c6e6ce8d9fc6d7cc8 LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 40992 W: 11017 L: 10706 D: 19269 Ptnml(0-2): 113, 4400, 11170, 4689, 124 Passed LTC: https://tests.stockfishchess.org/tests/view/648119ac6e6ce8d9fc6d8208 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 129174 W: 35059 L: 34579 D: 59536 Ptnml(0-2): 66, 12548, 38868, 13050, 55 closes https://github.com/official-stockfish/Stockfish/pull/4611 bench: 2370027
Copyright 2011–2024 Next Chess Move LLC
Feedback & Support Terms of Service Privacy Policy