Dev Builds » 20240307-1853

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 11:34:09 1182461 4000 1776 281 1943 +136.46 +/- 4.61 0 31 495 1422 52 +316.43 +/- 15.32
ncm-dbt-02 11:32:39 1231193 4000 1771 283 1946 +135.76 +/- 4.75 0 32 514 1388 66 +308.94 +/- 15.03
ncm-dbt-03 11:34:57 1240973 4018 1782 305 1931 +133.98 +/- 4.75 0 43 501 1410 55 +306.71 +/- 15.24
ncm-dbt-05 11:30:59 1225730 3996 1762 281 1953 +135.2 +/- 4.94 0 50 488 1389 71 +305.25 +/- 15.43
ncm-dbt-06 11:35:07 1235610 3986 1769 306 1911 +133.76 +/- 4.76 0 41 503 1394 55 +305.78 +/- 15.21
20000 8860 1456 9684 +135.03 +/- 2.13 0 197 2501 7003 299 +308.59 +/- 6.81

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
340143 ncm-dbt-02 1225240 0 0 0 0 -0.0 +/- 1199.83 0 0 0 0 0 -0.0 +/- 1199.83
340142 ncm-dbt-01 1201522 0 0 0 0 -0.0 +/- 1199.83 0 0 0 0 0 -0.0 +/- 1199.83
340141 ncm-dbt-03 1249422 18 8 2 8 +120.35 +/- 61.17 0 0 3 6 0 +279.47 +/- 313.13
340140 ncm-dbt-06 1234259 486 213 31 242 +136.76 +/- 13.42 0 4 60 172 7 +315.47 +/- 44.56
340139 ncm-dbt-05 1227594 496 213 35 248 +130.49 +/- 13.77 0 5 68 167 8 +291.63 +/- 41.74
340138 ncm-dbt-02 1253937 500 219 38 243 +131.74 +/- 14.03 0 4 72 163 11 +288.06 +/- 40.46
340137 ncm-dbt-01 1125456 500 218 40 242 +129.35 +/- 13.36 0 6 65 174 5 +295.94 +/- 42.75
340136 ncm-dbt-03 1233202 500 224 30 246 +142.25 +/- 13.18 0 5 53 185 7 +336.46 +/- 47.52
340135 ncm-dbt-06 1253323 500 219 36 245 +133.34 +/- 13.08 0 3 68 172 7 +304.07 +/- 41.65
340134 ncm-dbt-05 1221837 500 225 26 249 +146.36 +/- 14.18 0 5 54 178 13 +333.32 +/- 47.06
340133 ncm-dbt-02 1243859 500 222 35 243 +136.56 +/- 13.36 0 4 63 175 8 +312.48 +/- 43.44
340132 ncm-dbt-01 1151754 500 226 33 241 +141.44 +/- 12.2 0 1 61 182 6 +336.46 +/- 43.96
340131 ncm-dbt-03 1259323 500 219 42 239 +128.55 +/- 13.91 0 5 72 164 9 +282.94 +/- 40.5
340130 ncm-dbt-06 1206958 500 230 33 237 +144.71 +/- 12.26 0 2 55 187 6 +349.43 +/- 46.59
340129 ncm-dbt-02 1224488 500 223 33 244 +138.99 +/- 12.3 0 3 58 185 4 +333.32 +/- 45.35
340128 ncm-dbt-05 1211593 500 217 27 256 +138.99 +/- 14.03 0 5 61 173 11 +312.48 +/- 44.19
340127 ncm-dbt-01 1179195 500 219 32 249 +136.56 +/- 12.79 0 4 60 181 5 +321.19 +/- 44.57
340126 ncm-dbt-03 1228876 500 226 40 234 +135.76 +/- 13.01 0 4 62 178 6 +315.35 +/- 43.81
340125 ncm-dbt-06 1248680 500 218 41 241 +128.55 +/- 12.82 0 6 63 179 2 +301.33 +/- 43.45
340124 ncm-dbt-05 1228627 500 229 44 227 +134.95 +/- 13.78 0 5 64 172 9 +304.07 +/- 43.1
340123 ncm-dbt-02 1226156 500 228 40 232 +137.37 +/- 13.71 0 6 58 178 8 +315.35 +/- 45.33
340122 ncm-dbt-03 1236738 500 228 39 233 +138.18 +/- 13.5 0 2 68 169 11 +309.64 +/- 41.56
340121 ncm-dbt-01 1192597 500 223 42 235 +131.74 +/- 13.49 0 4 69 169 8 +295.94 +/- 41.39
340120 ncm-dbt-06 1221052 500 226 38 236 +137.37 +/- 13.34 0 5 59 179 7 +318.25 +/- 44.96
340119 ncm-dbt-05 1239207 500 219 28 253 +139.81 +/- 12.47 0 2 61 181 6 +330.23 +/- 44.08
340118 ncm-dbt-02 1219089 500 211 39 250 +124.6 +/- 13.62 0 6 72 166 6 +277.93 +/- 40.53
340117 ncm-dbt-01 1181044 500 223 30 247 +141.44 +/- 13.4 0 4 58 179 9 +327.18 +/- 45.37
340116 ncm-dbt-03 1240706 500 214 39 247 +126.97 +/- 13.76 0 8 64 173 5 +288.06 +/- 43.03
340115 ncm-dbt-06 1242231 500 218 42 240 +127.76 +/- 14.27 0 7 69 165 9 +280.42 +/- 41.44
340114 ncm-dbt-05 1230508 500 214 43 243 +123.81 +/- 14.15 0 11 61 174 4 +280.42 +/- 43.82
340113 ncm-dbt-02 1228154 500 225 41 234 +134.15 +/- 13.97 0 3 72 163 12 +293.29 +/- 40.39
340112 ncm-dbt-01 1216944 500 219 39 242 +130.94 +/- 12.76 0 3 69 173 5 +301.33 +/- 41.33
340111 ncm-dbt-03 1251238 500 222 44 234 +129.35 +/- 13.9 0 8 62 174 6 +293.29 +/- 43.72
340110 ncm-dbt-06 1243289 500 221 46 233 +126.97 +/- 14.11 0 6 72 163 9 +277.93 +/- 40.53
340109 ncm-dbt-05 1223398 500 227 35 238 +140.62 +/- 14.87 0 6 61 168 15 +306.84 +/- 44.17
340108 ncm-dbt-01 1207309 500 223 36 241 +136.56 +/- 12.98 0 6 55 185 4 +324.17 +/- 46.56
340107 ncm-dbt-02 1230017 500 216 26 258 +138.99 +/- 13.48 0 4 61 176 9 +318.25 +/- 44.18
340106 ncm-dbt-03 1238176 500 222 36 242 +135.76 +/- 13.39 0 7 55 183 5 +318.25 +/- 46.48
340105 ncm-dbt-05 1223082 500 218 43 239 +126.97 +/- 14.28 0 11 58 176 5 +288.06 +/- 44.88
340104 ncm-dbt-01 1186331 500 225 29 246 +143.89 +/- 13.31 0 3 58 179 10 +333.32 +/- 45.35
340103 ncm-dbt-06 1235093 500 224 39 237 +134.95 +/- 14.13 0 8 57 177 8 +306.84 +/- 45.58
340102 ncm-dbt-02 1229803 500 227 31 242 +143.89 +/- 12.71 0 2 58 182 8 +339.63 +/- 45.29
340101 ncm-dbt-03 1231081 500 219 33 248 +135.76 +/- 13.01 0 4 62 178 6 +315.35 +/- 43.81

Commit

Commit ID bd579ab5d1a931a09a62f2ed33b5149ada7bc65f
Author Linmiao Xu
Date 2024-03-07 18:53:48 UTC
Update default main net to nn-1ceb1ade0001.nnue Created by retraining the previous main net `nn-b1a57edbea57.nnue` with: - some of the same options as before: - ranger21, more WDL skipping, 15% more loss when Q is too high - removal of the huge 514G pre-interleaved binpack - removal of SF-generated dfrc data (dfrc99-16tb7p-filt-v2.min.binpack) - interleaving many binpacks at training time - training with some bestmove capture positions where SEE < 0 - increased usage of torch.compile to speed up training by up to 40% ```yaml experiment-name: 2560--S10-dfrc0-to-dec2023-skip-more-wdl-15p-more-loss-high-q-see-ge0-sk28 nnue-pytorch-branch: linrock/nnue-pytorch/r21-more-wdl-skip-15p-more-loss-high-q-skip-see-ge0-torch-compile-more start-from-engine-test-net: True early-fen-skipping: 28 training-dataset: # similar, not the exact same as: # https://github.com/official-stockfish/Stockfish/pull/4635 - /data/S5-5af/leela96.v2.min.binpack - /data/S5-5af/test60-2021-11-12-novdec-12tb7p.v6-dd.min.binpack - /data/S5-5af/test77-2021-12-dec-16tb7p.v6-dd.min.binpack - /data/S5-5af/test78-2022-01-to-05-jantomay-16tb7p.v6-dd.min.binpack - /data/S5-5af/test78-2022-06-to-09-juntosep-16tb7p.v6-dd.min.binpack - /data/S5-5af/test79-2022-04-apr-16tb7p.v6-dd.min.binpack - /data/S5-5af/test79-2022-05-may-16tb7p.v6-dd.min.binpack - /data/S5-5af/test80-2022-06-jun-16tb7p.v6-dd.min.unmin.binpack - /data/S5-5af/test80-2022-07-jul-16tb7p.v6-dd.min.binpack - /data/S5-5af/test80-2022-08-aug-16tb7p.v6-dd.min.binpack - /data/S5-5af/test80-2022-09-sep-16tb7p.v6-dd.min.unmin.binpack - /data/S5-5af/test80-2022-10-oct-16tb7p.v6-dd.min.binpack - /data/S5-5af/test80-2022-11-nov-16tb7p.v6-dd.min.binpack - /data/S5-5af/test80-2023-01-jan-16tb7p.v6-sk20.min.binpack - /data/S5-5af/test80-2023-02-feb-16tb7p.v6-dd.min.binpack - /data/S5-5af/test80-2023-03-mar-2tb7p.min.unmin.binpack - /data/S5-5af/test80-2023-04-apr-2tb7p.binpack - /data/S5-5af/test80-2023-05-may-2tb7p.min.dd.binpack # https://github.com/official-stockfish/Stockfish/pull/4782 - /data/S6-1ee1aba5ed/test80-2023-06-jun-2tb7p.binpack - /data/S6-1ee1aba5ed/test80-2023-07-jul-2tb7p.min.binpack # https://github.com/official-stockfish/Stockfish/pull/4972 - /data/S8-baff1edbea57/test80-2023-08-aug-2tb7p.v6.min.binpack - /data/S8-baff1edbea57/test80-2023-09-sep-2tb7p.binpack - /data/S8-baff1edbea57/test80-2023-10-oct-2tb7p.binpack # https://github.com/official-stockfish/Stockfish/pull/5056 - /data/S9-b1a57edbea57/test80-2023-11-nov-2tb7p.binpack - /data/S9-b1a57edbea57/test80-2023-12-dec-2tb7p.binpack num-epochs: 800 lr: 4.375e-4 gamma: 0.995 start-lambda: 1.0 end-lambda: 0.7 ``` This particular net was reached at epoch 759. Use of more torch.compile decorators in nnue-pytorch model.py than in the previous main net training run sped up training by up to 40% on Tesla gpus when using recent pytorch compiled with cuda 12: https://github.com/linrock/nnue-tools/blob/7fb9831/Dockerfile Skipping positions with bestmove captures where static exchange evaluation is >= 0 is based on the implementation from Sopel's NNUE training & experimentation log: https://docs.google.com/document/d/1gTlrr02qSNKiXNZ_SuO4-RjK4MXBiFlLE6jvNqqMkAY Experiment 293 - only skip captures with see>=0 Positions with bestmove captures where score == 0 are always skipped for compatibility with minimized binpacks, since the original minimizer sets scores to 0 for slight improvements in compression. The trainer branch used was: https://github.com/linrock/nnue-pytorch/tree/r21-more-wdl-skip-15p-more-loss-high-q-skip-see-ge0-torch-compile-more Binpacks were renamed to be sorted chronologically by default when sorted by name. The binpack data are otherwise the same as binpacks with similar names in the prior naming convention. Training data can be found at: https://robotmoon.com/nnue-training-data/ Passed STC: https://tests.stockfishchess.org/tests/view/65e3ddd1f2ef6c733362ae5c LLR: 2.92 (-2.94,2.94) <0.00,2.00> Total: 149792 W: 39153 L: 38661 D: 71978 Ptnml(0-2): 675, 17586, 37905, 18032, 698 Passed LTC: https://tests.stockfishchess.org/tests/view/65e4d91c416ecd92c162a69b LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 64416 W: 16517 L: 16135 D: 31764 Ptnml(0-2): 38, 7218, 17313, 7602, 37 closes https://github.com/official-stockfish/Stockfish/pull/5090 Bench: 1373183
Copyright 2011–2024 Next Chess Move LLC