Dev Builds » 20240108-1734

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 09:49:38 1208057 3340 1490 250 1600 +135.46 +/- 5.1 0 30 413 1184 43 +313.03 +/- 16.78
ncm-dbt-02 09:45:08 1236820 3302 1449 214 1639 +136.57 +/- 5.1 1 26 403 1179 42 +317.67 +/- 16.99
ncm-dbt-03 09:54:52 1235833 3364 1466 220 1678 +135.11 +/- 5.02 0 26 425 1190 41 +312.77 +/- 16.54
ncm-dbt-04 09:49:01 1223218 3340 1453 254 1633 +130.54 +/- 5.23 0 38 438 1151 43 +296.08 +/- 16.3
ncm-dbt-05 09:44:36 1234379 3298 1436 230 1632 +133.21 +/- 5.28 0 37 415 1151 46 +303.7 +/- 16.75
ncm-dbt-06 09:51:45 1225285 3356 1468 231 1657 +134.39 +/- 5.02 0 23 438 1174 43 +309.34 +/- 16.27
20000 8762 1399 9839 +134.21 +/- 2.09 1 180 2532 7029 258 +308.66 +/- 6.77

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
244276 ncm-dbt-05 1249910 298 128 20 150 +131.91 +/- 18.2 0 4 38 102 5 +295.46 +/- 56.4
244275 ncm-dbt-02 1223087 302 131 22 149 +131.31 +/- 16.48 0 1 44 102 4 +298.19 +/- 51.83
244274 ncm-dbt-01 1211799 340 143 23 174 +128.13 +/- 17.03 0 5 45 115 5 +285.79 +/- 51.63
244273 ncm-dbt-04 1215016 340 145 26 169 +126.97 +/- 16.43 0 3 50 112 5 +282.05 +/- 48.78
244272 ncm-dbt-06 1204431 356 155 22 179 +136.4 +/- 15.36 0 2 46 125 5 +314.7 +/- 51.0
244271 ncm-dbt-03 1241727 364 159 37 168 +121.13 +/- 16.1 0 2 63 110 7 +258.67 +/- 42.87
244270 ncm-dbt-05 1224689 500 217 44 239 +125.38 +/- 14.13 0 8 68 167 7 +277.93 +/- 41.74
244269 ncm-dbt-02 1226133 500 216 40 244 +127.76 +/- 13.57 0 7 65 173 5 +290.66 +/- 42.73
244268 ncm-dbt-01 1214739 500 225 34 241 +139.81 +/- 13.26 0 5 56 182 7 +327.18 +/- 46.19
244267 ncm-dbt-04 1236413 500 223 37 240 +135.76 +/- 14.64 0 6 65 166 13 +295.94 +/- 42.75
244266 ncm-dbt-06 1218562 500 223 32 245 +139.81 +/- 13.07 0 4 58 181 7 +327.18 +/- 45.37
244265 ncm-dbt-03 1237829 500 214 36 250 +129.35 +/- 12.8 0 4 68 174 4 +298.62 +/- 41.71
244264 ncm-dbt-05 1232032 500 221 39 240 +132.54 +/- 13.28 0 4 67 172 7 +301.33 +/- 42.04
244263 ncm-dbt-02 1251991 500 221 38 241 +133.34 +/- 13.63 1 3 65 174 7 +306.84 +/- 42.73
244262 ncm-dbt-01 1201584 500 225 44 231 +131.74 +/- 13.12 0 4 67 173 6 +301.33 +/- 42.04
244261 ncm-dbt-04 1225841 500 212 36 252 +127.76 +/- 13.75 0 6 69 168 7 +285.49 +/- 41.44
244260 ncm-dbt-06 1231696 500 219 36 245 +133.34 +/- 13.45 0 4 67 171 8 +301.33 +/- 42.04
244259 ncm-dbt-03 1241641 500 225 27 248 +145.54 +/- 12.22 0 3 51 191 5 +356.21 +/- 48.54
244258 ncm-dbt-05 1251966 500 215 36 249 +130.14 +/- 12.59 0 5 63 180 2 +306.84 +/- 43.45
244257 ncm-dbt-02 1232497 500 228 18 254 +155.54 +/- 10.93 0 1 42 203 4 +406.2 +/- 53.75
244256 ncm-dbt-04 1213210 500 227 38 235 +138.18 +/- 13.5 0 6 56 181 7 +321.19 +/- 46.14
244255 ncm-dbt-01 1203901 500 226 38 236 +137.37 +/- 13.15 0 3 64 175 8 +315.35 +/- 43.03
244254 ncm-dbt-06 1236228 500 214 45 241 +122.24 +/- 12.76 0 3 79 164 4 +275.45 +/- 38.39
244253 ncm-dbt-03 1221257 500 206 32 262 +126.17 +/- 14.29 0 9 65 169 7 +280.42 +/- 42.65
244252 ncm-dbt-05 1242329 500 218 35 247 +133.34 +/- 13.99 0 6 64 171 9 +298.62 +/- 43.1
244251 ncm-dbt-02 1231441 500 204 34 262 +123.02 +/- 13.65 0 7 71 167 5 +275.45 +/- 40.84
244250 ncm-dbt-04 1204452 500 214 36 250 +129.35 +/- 12.8 0 3 71 171 5 +295.94 +/- 40.69
244249 ncm-dbt-01 1222877 500 223 49 228 +126.17 +/- 13.77 0 7 68 169 6 +282.94 +/- 41.76
244248 ncm-dbt-06 1225687 500 217 34 249 +133.34 +/- 11.88 0 1 68 178 3 +315.35 +/- 41.44
244247 ncm-dbt-03 1226920 500 225 28 247 +144.71 +/- 12.47 0 2 56 185 7 +346.12 +/- 46.15
244246 ncm-dbt-05 1225619 500 219 25 256 +142.25 +/- 12.78 0 5 51 189 5 +342.85 +/- 48.46
244245 ncm-dbt-02 1236378 500 226 29 245 +144.71 +/- 12.88 0 4 52 187 7 +346.12 +/- 48.03
244244 ncm-dbt-04 1240683 500 217 38 245 +130.14 +/- 12.97 0 6 62 179 3 +304.07 +/- 43.81
244243 ncm-dbt-01 1225523 500 229 27 244 +148.85 +/- 12.05 0 2 50 192 6 +366.78 +/- 49.02
244242 ncm-dbt-06 1228145 500 226 35 239 +139.81 +/- 14.19 0 5 61 172 12 +312.48 +/- 44.19
244241 ncm-dbt-03 1234972 500 221 32 247 +138.18 +/- 12.33 0 1 65 178 6 +324.17 +/- 42.47
244240 ncm-dbt-05 1214111 500 218 31 251 +136.56 +/- 14.1 0 5 64 170 11 +304.07 +/- 43.1
244239 ncm-dbt-02 1256219 500 223 33 244 +138.99 +/- 13.48 0 3 64 173 10 +315.35 +/- 43.03
244238 ncm-dbt-04 1226911 500 215 43 242 +124.6 +/- 13.45 0 8 65 174 3 +285.49 +/- 42.7
244237 ncm-dbt-01 1175980 500 219 35 246 +134.15 +/- 12.86 0 4 63 178 5 +312.48 +/- 43.44
244236 ncm-dbt-06 1232251 500 214 27 259 +136.56 +/- 12.59 0 4 59 183 4 +324.17 +/- 44.97
244235 ncm-dbt-03 1246488 500 216 28 256 +137.37 +/- 12.96 0 5 57 183 5 +324.17 +/- 45.77

Commit

Commit ID 6deb88728fb141e853243c2873ad0cda4dd19320
Author Linmiao Xu
Date 2024-01-08 17:34:36 UTC
Update default main net to nn-baff1edbea57.nnue Created by retraining the previous main net nn-b1e55edbea57.nnue with: - some of the same options as before: ranger21 optimizer, more WDL skipping - adding T80 aug filter-v6, sep, and oct 2023 data to the previous best dataset - increasing training loss for positions where predicted win rates were higher than estimated match results from training data position scores ```yaml experiment-name: 2560--S8-r21-more-wdl-skip-10p-more-loss-high-q-sk28 training-dataset: # https://github.com/official-stockfish/Stockfish/pull/4782 - /data/S6-1ee1aba5ed.binpack - /data/test80-aug2023-2tb7p.v6.min.binpack - /data/test80-sep2023-2tb7p.binpack - /data/test80-oct2023-2tb7p.binpack early-fen-skipping: 28 start-from-engine-test-net: True nnue-pytorch-branch: linrock/nnue-pytorch/r21-more-wdl-skip-10p-more-loss-high-q num-epochs: 1000 lr: 4.375e-4 gamma: 0.995 start-lambda: 1.0 end-lambda: 0.7 ``` Training data can be found at: https://robotmoon.com/nnue-training-data/ Training loss was increased by 10% for positions where predicted win rates were higher than suggested by the win rate model based on the training data, by multiplying with: ((qf > pt) * 0.1 + 1). This was a variant of experiments from Sopel's NNUE training & experimentation log: https://docs.google.com/document/d/1gTlrr02qSNKiXNZ_SuO4-RjK4MXBiFlLE6jvNqqMkAY Experiment 302 - increase loss when prediction too high, vondele’s idea Experiment 309 - increase loss when prediction too high, normalize in a batch Passed STC: https://tests.stockfishchess.org/tests/view/6597a21c79aa8af82b95fd5c LLR: 2.93 (-2.94,2.94) <0.00,2.00> Total: 148320 W: 37960 L: 37475 D: 72885 Ptnml(0-2): 542, 17565, 37383, 18206, 464 Passed LTC: https://tests.stockfishchess.org/tests/view/659834a679aa8af82b960845 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 55188 W: 13955 L: 13592 D: 27641 Ptnml(0-2): 34, 6162, 14834, 6535, 29 closes https://github.com/official-stockfish/Stockfish/pull/4972 Bench: 1219824
Copyright 2011–2024 Next Chess Move LLC