Dev Builds » 20240518-0719

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 11:37:12 1171557 4002 1747 294 1961 +132.17 ± 4.78 0 40 526 1377 58 +299.4 ± 14.86
ncm-dbt-02 11:33:54 1213464 3984 1793 303 1888 +136.56 ± 4.8 1 37 489 1401 64 +312.74 ± 15.42
ncm-dbt-03 11:37:11 1237238 4004 1806 286 1912 +138.84 ± 4.79 1 34 482 1414 71 +318.45 ± 15.53
ncm-dbt-05 11:35:07 1200313 3984 1741 283 1960 +133.33 ± 4.68 0 32 524 1382 54 +304.64 ± 14.88
ncm-dbt-06 11:37:08 1223366 4026 1802 284 1940 +137.8 ± 4.75 1 38 479 1432 63 +317.74 ± 15.58
20000 8889 1450 9661 +135.74 ± 2.13 3 181 2500 7006 310 +310.49 ± 6.81

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
368540 ncm-dbt-03 1260779 4 1 1 2 -0.0 ± 296.81 0 1 0 1 0 -0.0 ± 1199.83
368539 ncm-dbt-06 1213726 26 11 3 12 +110.44 ± 51.3 0 0 5 8 0 +249.23 ± 175.23
368538 ncm-dbt-01 799665 236 101 22 113 +120.96 ± 23.51 0 6 34 71 7 +246.4 ± 59.4
368537 ncm-dbt-05 1199822 484 216 33 235 +138.22 ± 13.61 0 3 62 168 9 +314.63 ± 43.74
368536 ncm-dbt-02 1198835 484 210 35 239 +131.57 ± 14.16 0 5 66 162 9 +291.94 ± 42.39
368535 ncm-dbt-03 1238841 500 224 41 235 +133.34 ± 13.08 1 4 59 183 3 +318.25 ± 44.96
368534 ncm-dbt-01 1224541 266 115 20 131 +129.8 ± 19.75 0 2 41 83 7 +276.47 ± 53.91
368533 ncm-dbt-06 1254495 500 218 32 250 +135.76 ± 12.21 0 2 64 180 4 +321.19 ± 42.95
368532 ncm-dbt-05 1198451 500 227 39 234 +137.37 ± 13.15 0 5 58 181 6 +321.19 ± 45.36
368531 ncm-dbt-02 1197484 500 230 39 231 +139.81 ± 14.19 1 5 55 180 9 +324.17 ± 46.56
368530 ncm-dbt-03 1232283 500 224 29 247 +143.07 ± 12.12 0 1 59 184 6 +342.85 ± 44.76
368529 ncm-dbt-01 1235403 500 218 32 250 +135.76 ± 13.57 0 5 62 175 8 +309.64 ± 43.82
368528 ncm-dbt-06 1243717 500 223 32 245 +139.81 ± 13.26 0 6 53 185 6 +330.23 ± 47.44
368527 ncm-dbt-05 1196091 500 212 34 254 +129.35 ± 13.54 0 5 69 169 7 +290.66 ± 41.43
368526 ncm-dbt-02 1239594 500 225 36 239 +138.18 ± 13.32 0 3 64 174 9 +315.35 ± 43.03
368525 ncm-dbt-03 1238369 500 227 28 245 +146.36 ± 14.54 0 5 56 174 15 +327.18 ± 46.19
368524 ncm-dbt-06 1227643 500 227 30 243 +144.71 ± 13.48 0 4 55 181 10 +336.46 ± 46.65
368523 ncm-dbt-01 1201589 500 217 43 240 +126.17 ± 13.95 0 5 75 161 9 +275.45 ± 39.63
368522 ncm-dbt-05 1200514 500 213 44 243 +122.24 ± 13.83 0 6 76 161 7 +268.17 ± 39.39
368521 ncm-dbt-02 1233945 500 216 37 247 +130.14 ± 13.88 0 6 67 169 8 +290.66 ± 42.08
368520 ncm-dbt-03 1217191 500 224 43 233 +131.74 ± 14.54 0 7 66 166 11 +288.06 ± 42.4
368519 ncm-dbt-06 1199143 500 225 37 238 +137.37 ± 13.53 0 6 57 180 7 +318.25 ± 45.73
368518 ncm-dbt-01 1227824 500 220 39 241 +131.74 ± 13.49 0 4 69 169 8 +295.94 ± 41.39
368517 ncm-dbt-05 1194779 500 234 29 237 +151.34 ± 12.36 0 2 49 191 8 +370.41 ± 49.56
368516 ncm-dbt-02 1204158 500 231 45 224 +135.76 ± 14.98 0 8 61 168 13 +295.94 ± 44.08
368515 ncm-dbt-03 1233653 500 224 41 235 +133.34 ± 13.99 0 7 61 174 8 +301.33 ± 44.13
368514 ncm-dbt-06 1221280 500 214 40 246 +126.17 ± 13.06 0 3 76 165 6 +282.94 ± 39.21
368513 ncm-dbt-01 1214576 500 211 28 261 +133.34 ± 12.29 0 3 64 180 3 +315.35 ± 43.03
368512 ncm-dbt-05 1196248 500 211 38 251 +125.38 ± 12.89 0 4 73 169 4 +285.49 ± 40.16
368511 ncm-dbt-06 1220435 500 227 42 231 +134.95 ± 14.82 1 6 61 171 11 +301.33 ± 44.13
368510 ncm-dbt-03 1219068 500 231 30 239 +148.02 ± 13.15 0 2 56 181 11 +346.12 ± 46.15
368509 ncm-dbt-02 1213061 500 226 33 241 +141.44 ± 12.81 0 3 58 182 7 +333.32 ± 45.35
368508 ncm-dbt-01 1232198 500 227 36 237 +139.81 ± 12.67 0 3 59 182 6 +330.23 ± 44.94
368507 ncm-dbt-03 1258403 500 226 41 233 +134.95 ± 14.31 0 5 67 166 12 +295.94 ± 42.08
368506 ncm-dbt-05 1197743 500 214 33 253 +131.74 ± 12.74 0 3 68 174 5 +304.07 ± 41.65
368505 ncm-dbt-02 1195138 500 227 42 231 +134.95 ± 12.84 0 3 65 176 6 +312.48 ± 42.67
368504 ncm-dbt-06 1201182 500 221 33 246 +137.37 ± 13.34 0 5 59 179 7 +318.25 ± 44.96
368503 ncm-dbt-01 1187598 500 215 46 239 +122.24 ± 13.83 0 8 70 167 5 +273.0 ± 41.13
368502 ncm-dbt-02 1225500 500 228 36 236 +140.62 ± 12.23 0 4 53 190 3 +342.85 ± 47.56
368501 ncm-dbt-06 1228680 500 236 35 229 +148.02 ± 14.12 0 6 49 183 12 +342.85 ± 49.35
368500 ncm-dbt-05 1218861 500 214 33 253 +131.74 ± 13.49 0 4 69 169 8 +295.94 ± 41.39
368499 ncm-dbt-03 1236559 500 225 32 243 +141.44 ± 12.2 0 2 58 185 5 +339.63 ± 45.29
368498 ncm-dbt-01 1220624 500 223 28 249 +143.07 ± 12.54 0 4 52 189 5 +346.12 ± 48.03

Commit

Commit ID 1b7dea3f851cd5c5411ba6f07a2f935bfb7da8a9
Author Linmiao Xu
Date 2024-05-18 07:19:10 UTC
Update default main net to nn-c721dfca8cd3.nnue Created by first retraining the spsa-tuned main net `nn-ae6a388e4a1a.nnue` with: - using v6-dd data without bestmove captures removed - addition of T80 mar2024 data - increasing loss by 20% when Q is too high - torch.compile changes for marginal training speed gains And then SPSA tuning weights of epoch 899 following methods described in: https://github.com/official-stockfish/Stockfish/pull/5149 This net was reached at 92k out of 120k steps in this 70+0.7 th 7 SPSA tuning run: https://tests.stockfishchess.org/tests/view/66413b7df9f4e8fc783c9bbb Thanks to @Viren6 for suggesting usage of: - c value 4 for the weights - c value 128 for the biases Scripts for automating applying fishtest spsa params to exporting tuned .nnue are in: https://github.com/linrock/nnue-tools/tree/master/spsa Before spsa tuning, epoch 899 was nn-f85738aefa84.nnue https://tests.stockfishchess.org/tests/view/663e5c893a2f9702074bc167 After initially training with max-epoch 800, training was resumed with max-epoch 1000. ``` experiment-name: 3072--S11--more-data-v6-dd-t80-mar2024--see-ge0-20p-more-loss-high-q-sk28-l8 nnue-pytorch-branch: linrock/nnue-pytorch/3072-r21-skip-more-wdl-see-ge0-20p-more-loss-high-q-torch-compile-more start-from-engine-test-net: False start-from-model: /data/config/apr2024-3072/nn-ae6a388e4a1a.nnue early-fen-skipping: 28 training-dataset: /data/S11-mar2024/: - leela96.v2.min.binpack - test60-2021-11-12-novdec-12tb7p.v6-dd.min.binpack - test78-2022-01-to-05-jantomay-16tb7p.v6-dd.min.binpack - test80-2022-06-jun-16tb7p.v6-dd.min.binpack - test80-2022-08-aug-16tb7p.v6-dd.min.binpack - test80-2022-09-sep-16tb7p.v6-dd.min.binpack - test80-2023-01-jan-16tb7p.v6-sk20.min.binpack - test80-2023-02-feb-16tb7p.v6-sk20.min.binpack - test80-2023-03-mar-2tb7p.v6-sk16.min.binpack - test80-2023-04-apr-2tb7p.v6-sk16.min.binpack - test80-2023-05-may-2tb7p.v6.min.binpack # https://github.com/official-stockfish/Stockfish/pull/4782 - test80-2023-06-jun-2tb7p.binpack - test80-2023-07-jul-2tb7p.binpack # https://github.com/official-stockfish/Stockfish/pull/4972 - test80-2023-08-aug-2tb7p.v6.min.binpack - test80-2023-09-sep-2tb7p.binpack - test80-2023-10-oct-2tb7p.binpack # S9 new data: https://github.com/official-stockfish/Stockfish/pull/5056 - test80-2023-11-nov-2tb7p.binpack - test80-2023-12-dec-2tb7p.binpack # S10 new data: https://github.com/official-stockfish/Stockfish/pull/5149 - test80-2024-01-jan-2tb7p.binpack - test80-2024-02-feb-2tb7p.binpack # S11 new data - test80-2024-03-mar-2tb7p.binpack /data/filt-v6-dd/: - test77-dec2021-16tb7p-filter-v6-dd.binpack - test78-juntosep2022-16tb7p-filter-v6-dd.binpack - test79-apr2022-16tb7p-filter-v6-dd.binpack - test79-may2022-16tb7p-filter-v6-dd.binpack - test80-jul2022-16tb7p-filter-v6-dd.binpack - test80-oct2022-16tb7p-filter-v6-dd.binpack - test80-nov2022-16tb7p-filter-v6-dd.binpack num-epochs: 1000 lr: 4.375e-4 gamma: 0.995 start-lambda: 0.8 end-lambda: 0.7 ``` Training data can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move: nn-epoch899.nnue : 4.6 +/- 1.4 Passed STC: https://tests.stockfishchess.org/tests/view/6645454893ce6da3e93b31ae LLR: 2.95 (-2.94,2.94) <0.00,2.00> Total: 95232 W: 24598 L: 24194 D: 46440 Ptnml(0-2): 294, 11215, 24180, 11647, 280 Passed LTC: https://tests.stockfishchess.org/tests/view/6645522d93ce6da3e93b31df LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 320544 W: 81432 L: 80524 D: 158588 Ptnml(0-2): 164, 35659, 87696, 36611, 142 closes https://github.com/official-stockfish/Stockfish/pull/5254 bench 1995552
Copyright 2011–2024 Next Chess Move LLC