Dev Builds » 20230114-0712

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 05:01:55 1113872 1670 677 159 834 +111.44 ± 7.82 0 33 270 513 19 +239.53 ± 20.77
ncm-dbt-02 04:59:19 1237029 1656 679 166 811 +111.28 ± 7.76 0 30 273 507 18 +239.65 ± 20.65
ncm-dbt-03 05:02:28 1238920 1668 699 182 787 +111.35 ± 7.75 1 27 279 508 19 +239.91 ± 20.41
ncm-dbt-04 05:02:26 1236505 1672 689 165 818 +112.68 ± 7.55 0 29 267 527 13 +246.99 ± 20.89
ncm-dbt-05 04:59:56 1233125 1668 691 161 816 +114.35 ± 7.8 0 31 262 521 20 +247.14 ± 21.09
ncm-dbt-06 05:01:32 1221198 1666 686 152 828 +115.43 ± 7.34 0 21 270 529 13 +254.99 ± 20.73
ncm-et-3 06:25:18 1300230 1674 702 170 802 +114.38 ± 7.84 1 35 247 539 15 +251.26 ± 21.7
ncm-et-4 06:26:02 1300340 1656 681 180 795 +108.51 ± 7.76 0 32 279 501 16 +233.19 ± 20.42
ncm-et-9 06:26:29 1300285 1662 676 173 813 +108.55 ± 7.82 0 29 292 488 22 +229.55 ± 19.94
ncm-et-10 06:25:51 1290297 1674 711 174 789 +115.53 ± 7.54 0 24 270 525 18 +251.93 ± 20.75
ncm-et-13 06:25:34 1300370 1658 656 161 841 +106.99 ± 7.93 0 36 281 493 19 +227.13 ± 20.36
ncm-et-15 06:25:55 1300092 1676 679 175 822 +107.81 ± 8.0 2 38 268 514 16 +232.65 ± 20.84
20000 8226 2018 9756 +111.52 ± 2.24 4 365 3258 6165 208 +241.04 ± 5.97

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
189841 ncm-dbt-02 1249487 156 72 16 68 +130.53 ± 26.49 0 2 22 50 4 +279.59 ± 75.24
189840 ncm-dbt-06 1245621 166 68 14 84 +117.28 ± 20.99 0 1 27 55 0 +269.73 ± 66.81
189839 ncm-dbt-05 1237687 168 71 16 81 +118.08 ± 23.64 0 2 27 53 2 +258.14 ± 67.17
189838 ncm-dbt-03 1224455 168 68 17 83 +108.9 ± 25.33 0 5 24 54 1 +238.25 ± 71.03
189837 ncm-dbt-04 1239660 172 73 21 78 +108.42 ± 23.33 0 3 29 53 1 +237.06 ± 64.68
189836 ncm-dbt-01 1097782 170 68 15 87 +112.04 ± 23.51 0 3 27 54 1 +247.28 ± 67.26
189835 ncm-dbt-06 1216298 500 210 50 240 +115.22 ± 13.2 0 4 87 154 5 +251.89 ± 36.48
189834 ncm-dbt-02 1231086 500 201 46 253 +111.37 ± 14.06 0 10 79 157 4 +243.0 ± 38.64
189833 ncm-dbt-01 1118449 500 202 46 252 +112.14 ± 14.22 0 11 76 159 4 +245.2 ± 39.38
189832 ncm-dbt-03 1257079 500 222 57 221 +119.11 ± 13.17 0 3 85 156 6 +261.07 ± 36.86
189831 ncm-dbt-05 1232809 500 206 43 251 +117.55 ± 13.36 0 5 82 158 5 +258.75 ± 37.76
189830 ncm-dbt-04 1239009 500 200 38 262 +116.77 ± 14.53 0 12 69 164 5 +256.44 ± 41.25
189829 ncm-dbt-06 1219336 500 204 47 249 +112.91 ± 14.38 0 10 79 155 6 +243.0 ± 38.64
189828 ncm-dbt-02 1242549 500 204 49 247 +111.37 ± 13.56 0 6 88 151 5 +240.82 ± 36.39
189827 ncm-dbt-05 1233740 500 205 60 235 +103.73 ± 15.67 0 17 79 146 8 +213.85 ± 38.42
189826 ncm-dbt-01 1128758 500 202 52 246 +107.54 ± 13.89 0 10 83 154 3 +234.38 ± 37.67
189825 ncm-dbt-03 1245864 500 205 55 240 +107.54 ± 15.13 1 11 82 149 7 +228.08 ± 37.89
189824 ncm-dbt-04 1229526 500 204 57 239 +105.25 ± 13.72 0 8 91 147 4 +226.0 ± 35.82
189823 ncm-dbt-06 1203539 500 204 41 255 +117.55 ± 13.0 0 6 77 165 2 +265.78 ± 39.12
189822 ncm-dbt-02 1224995 500 202 55 243 +105.25 ± 14.51 0 12 84 149 5 +223.94 ± 37.43
189821 ncm-dbt-05 1228264 500 209 42 249 +120.67 ± 13.67 0 7 74 164 5 +268.17 ± 39.97
189820 ncm-dbt-04 1237825 500 212 49 239 +117.55 ± 13.18 0 6 78 163 3 +263.42 ± 38.85
189819 ncm-dbt-01 1110500 500 205 46 249 +114.45 ± 15.0 0 9 84 146 11 +236.51 ± 37.42
189818 ncm-dbt-03 1228282 500 204 53 243 +108.3 ± 13.89 0 8 88 149 5 +232.26 ± 36.48
166454 ncm-et-9 1289383 162 65 14 83 +113.22 ± 23.33 0 2 27 51 1 +250.36 ± 67.08
166453 ncm-et-13 1302979 158 62 19 77 +96.99 ± 28.62 0 7 24 46 2 +199.76 ± 70.2
166452 ncm-et-4 1302269 156 67 20 69 +108.02 ± 27.52 0 4 26 45 3 +221.95 ± 68.45
166451 ncm-et-15 1318909 176 79 16 81 +130.12 ± 22.63 0 2 23 61 2 +296.73 ± 73.59
166450 ncm-et-3 1293177 174 73 17 84 +115.93 ± 25.5 0 3 29 51 4 +239.58 ± 64.7
166449 ncm-et-10 1291198 174 74 18 82 +115.93 ± 26.25 0 6 21 58 2 +252.28 ± 75.01
166448 ncm-et-9 1303895 500 208 59 233 +106.77 ± 14.21 0 9 89 146 6 +226.0 ± 36.29
166447 ncm-et-13 1304746 500 196 38 266 +113.68 ± 14.22 0 7 86 149 8 +240.82 ± 36.9
166446 ncm-et-4 1287126 500 213 59 228 +110.6 ± 13.73 0 8 84 154 4 +240.82 ± 37.4
166445 ncm-et-15 1294241 500 200 53 247 +105.25 ± 14.2 1 9 85 152 3 +230.16 ± 37.2
166444 ncm-et-3 1293952 500 202 51 247 +108.3 ± 14.52 1 11 77 158 3 +238.66 ± 39.1
166443 ncm-et-10 1289330 500 207 47 246 +115.22 ± 13.38 0 5 85 155 5 +251.89 ± 37.03
166442 ncm-et-9 1305022 500 209 55 236 +110.6 ± 14.06 0 7 89 147 7 +234.38 ± 36.22
166441 ncm-et-13 1303923 500 200 56 244 +102.97 ± 14.18 0 10 91 144 5 +217.85 ± 35.89
166440 ncm-et-4 1306874 500 208 45 247 +117.55 ± 13.87 0 7 79 158 6 +256.44 ± 38.62
166439 ncm-et-15 1300687 500 199 64 237 +96.19 ± 15.7 1 18 81 145 5 +202.15 ± 37.89
166438 ncm-et-3 1305893 500 222 59 219 +117.55 ± 14.53 0 11 71 162 6 +256.44 ± 40.73
166437 ncm-et-10 1293640 500 216 53 231 +117.55 ± 13.18 0 6 78 163 3 +263.42 ± 38.85
166436 ncm-et-3 1307901 500 205 43 252 +116.77 ± 13.71 0 10 70 168 2 +263.42 ± 41.06
166435 ncm-et-13 1289835 500 198 48 254 +107.54 ± 14.37 0 12 80 154 4 +232.26 ± 38.36
166434 ncm-et-10 1287023 500 214 56 230 +113.68 ± 14.22 0 7 86 149 8 +240.82 ± 36.9
166433 ncm-et-9 1302840 500 194 45 261 +106.77 ± 14.82 0 11 87 144 8 +221.9 ± 36.76
166432 ncm-et-15 1286533 500 201 42 257 +114.45 ± 14.21 0 9 79 156 6 +247.41 ± 38.64
166431 ncm-et-4 1305092 500 193 56 251 +97.69 ± 14.28 0 13 90 144 3 +207.95 ± 36.13

Commit

Commit ID 3d2381d76d7bf9686ef0e0671f60c3b885a7058a
Author Linmiao Xu
Date 2023-01-14 07:12:11 UTC
Update default net to nn-1e7ca356472e.nnue Created by retraining the master net on a dataset composed of: * The Leela-dfrc_n5000.binpack dataset filtered with depth6 multipv2 search to remove positions with only one good move, in addition to removing positions where either of the two best moves are captures * The same Leela T80 oct+nov 2022 training data used in recent best datasets * Additional Leela training data from T60 nov+dec 2021 and T79 apr+may 2022 Trained with end lambda 0.7 and started with max epoch 800. All positions with ply <= 28 were skipped: ``` python easy_train.py \ --experiment-name leela95-dfrc96-mpv-eval-fonly-T80octnov-T79aprmayT60novdec-12tb7p-sk28-lambda7 \ --training-dataset /data/leela95-dfrc96-mpv-eval-fonly-T80octnov-T79aprmayT60novdec-12tb7p.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes-skip-ply-lteq-28 \ --start-from-engine-test-net True \ --gpus "0," \ --start-lambda 1.0 \ --end-lambda 0.7 \ --gamma 0.995 \ --lr 4.375e-4 \ --tui False \ --seed $RANDOM \ --max_epoch 800 ``` Around epoch 780, training was manually paused and max epoch increased to 920 before resuming. During depth6 multipv2 data filtering, positions were considered to have only one good move if the score of the best move was significantly better than the 2nd best move in a way that changes the outcome of the game: * the best move leads to a significant advantage while the 2nd best move equalizes or loses * the best move is about equal while the 2nd best move loses The modified stockfish branch and exact score thresholds used for filtering are at: https://github.com/linrock/Stockfish/tree/tools-filter-multipv2-eval-diff/src/filter About 95% of the Leela portion and 96% of the DFRC portion of the Leela-dfrc_n5000.binpack dataset was filtered. Unfiltered parts of the dataset were left out. The additional Leela training data from T60 nov+dec 2021 and T79 apr+may 2022 was WDL-rescored with about 12TB of syzygy 7-piece tablebases where the material difference is less than around 6 pawns. Best moves were exported to .plain data files during data conversion with the lc0 rescorer. The exact training data can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move experiment_leela95-dfrc96-mpv-eval-fonly-T80octnov-T79aprmayT60novdec-12tb7p-sk28-lambda7 run_0/nn-epoch899.nnue : 3.8 +/- 1.6 Passed STC https://tests.stockfishchess.org/tests/view/63bed1f540aa064159b9c89b LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 103344 W: 27392 L: 26991 D: 48961 Ptnml(0-2): 333, 11223, 28099, 11744, 273 Passed LTC https://tests.stockfishchess.org/tests/view/63c010415705810de2deb3ec LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 21712 W: 5891 L: 5619 D: 10202 Ptnml(0-2): 12, 2022, 6511, 2304, 7 closes https://github.com/official-stockfish/Stockfish/pull/4338 bench 4106793
Copyright 2011–2024 Next Chess Move LLC