Dev Builds » 20230911-2037

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 09:53:56 1208930 3322 1445 293 1584 +125.69 ± 5.36 1 38 477 1098 47 +279.09 ± 15.61
ncm-dbt-02 09:51:38 1196705 3320 1442 250 1628 +130.56 ± 5.14 0 30 448 1142 40 +297.26 ± 16.1
ncm-dbt-03 09:54:51 1222931 3350 1430 270 1650 +125.49 ± 5.31 0 44 469 1120 42 +280.09 ± 15.75
ncm-dbt-04 09:54:48 1232631 3322 1439 262 1621 +128.68 ± 5.32 0 43 441 1134 43 +289.83 ± 16.24
ncm-dbt-05 09:50:24 1220265 3340 1454 274 1612 +128.27 ± 5.24 0 45 435 1155 35 +291.73 ± 16.35
ncm-dbt-06 09:55:16 1232496 3346 1461 278 1607 +128.38 ± 5.22 0 30 479 1115 49 +286.66 ± 15.55
20000 8671 1627 9702 +127.84 ± 2.15 1 230 2749 6764 256 +287.35 ± 6.49

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
202826 ncm-dbt-01 1190904 322 134 31 157 +115.18 ± 16.86 1 1 56 100 3 +256.06 ± 45.57
202825 ncm-dbt-02 1198188 320 144 23 153 +138.23 ± 14.81 0 1 39 118 2 +333.13 ± 55.52
202824 ncm-dbt-04 1238639 322 140 26 156 +128.57 ± 16.64 0 4 42 112 3 +294.24 ± 53.55
202823 ncm-dbt-05 1222721 340 139 26 175 +120.03 ± 16.24 0 3 55 108 4 +264.11 ± 46.29
202822 ncm-dbt-06 1204396 346 148 26 172 +128.0 ± 17.13 0 4 50 112 7 +278.39 ± 48.86
202821 ncm-dbt-03 1255241 350 144 30 176 +117.44 ± 15.36 0 4 54 116 1 +266.8 ± 46.89
202820 ncm-dbt-01 1231443 500 218 41 241 +128.55 ± 14.59 0 9 64 168 9 +282.94 ± 42.98
202819 ncm-dbt-02 1199587 500 219 43 238 +127.76 ± 14.44 0 6 73 160 11 +275.45 ± 40.24
202818 ncm-dbt-04 1218429 500 222 38 240 +134.15 ± 13.8 0 3 71 165 11 +295.94 ± 40.69
202817 ncm-dbt-05 1229420 500 222 48 230 +126.17 ± 13.77 0 8 65 172 5 +285.49 ± 42.7
202816 ncm-dbt-06 1233493 500 206 46 248 +115.22 ± 12.3 0 2 88 158 2 +258.75 ± 36.04
202815 ncm-dbt-03 1193709 500 216 43 241 +125.38 ± 13.25 0 6 69 171 4 +285.49 ± 41.44
202814 ncm-dbt-01 1183393 500 224 52 224 +124.6 ± 13.27 0 3 79 161 7 +275.45 ± 38.39
202813 ncm-dbt-02 1195696 500 211 35 254 +127.76 ± 12.84 0 5 67 175 3 +295.94 ± 42.08
202812 ncm-dbt-04 1227747 500 205 34 261 +123.81 ± 13.98 0 9 66 170 5 +277.93 ± 42.33
202811 ncm-dbt-05 1213018 500 220 39 241 +131.74 ± 13.67 0 6 64 173 7 +298.62 ± 43.1
202810 ncm-dbt-03 1215383 500 207 44 249 +117.55 ± 14.2 0 9 75 160 6 +256.44 ± 39.69
202809 ncm-dbt-06 1232645 500 227 36 237 +139.81 ± 12.67 0 3 59 182 6 +330.23 ± 44.94
202808 ncm-dbt-01 1202354 500 216 44 240 +124.6 ± 14.14 0 8 69 166 7 +275.45 ± 41.43
202807 ncm-dbt-02 1196387 500 210 24 266 +135.76 ± 13.39 0 4 64 174 8 +309.64 ± 43.08
202806 ncm-dbt-05 1222586 500 223 41 236 +132.54 ± 13.28 0 6 61 178 5 +306.84 ± 44.17
202805 ncm-dbt-04 1237355 500 216 42 242 +126.17 ± 14.46 0 10 63 170 7 +280.42 ± 43.24
202804 ncm-dbt-03 1234443 500 221 33 246 +137.37 ± 13.71 0 4 64 172 10 +309.64 ± 43.08
202803 ncm-dbt-06 1233236 500 220 39 241 +131.74 ± 13.49 0 6 63 175 6 +301.33 ± 43.45
202802 ncm-dbt-01 1209476 500 221 41 238 +130.94 ± 14.39 0 7 66 167 10 +288.06 ± 42.4
202801 ncm-dbt-02 1199152 500 218 39 243 +130.14 ± 13.16 0 5 66 174 5 +298.62 ± 42.41
202800 ncm-dbt-05 1227790 500 210 35 255 +126.97 ± 14.11 0 9 63 172 6 +285.49 ± 43.31
202799 ncm-dbt-04 1239483 500 221 45 234 +127.76 ± 13.21 0 6 66 174 4 +293.29 ± 42.41
202798 ncm-dbt-06 1232804 500 219 40 241 +130.14 ± 13.16 0 3 72 168 7 +293.29 ± 40.39
202797 ncm-dbt-03 1226792 500 217 42 241 +126.97 ± 14.94 0 11 62 168 9 +277.93 ± 43.49
202796 ncm-dbt-01 1244976 500 214 42 244 +124.6 ± 12.53 0 4 72 172 2 +288.06 ± 40.46
202795 ncm-dbt-02 1189774 500 220 44 236 +127.76 ± 14.44 0 8 67 166 9 +280.42 ± 42.05
202794 ncm-dbt-05 1222316 500 220 43 237 +128.55 ± 13.01 0 6 64 177 3 +298.62 ± 43.1
202793 ncm-dbt-04 1239035 500 212 45 243 +120.67 ± 13.5 0 6 76 163 5 +268.17 ± 39.39
202792 ncm-dbt-03 1212329 500 211 44 245 +120.67 ± 13.15 0 5 77 164 4 +270.57 ± 39.07
202791 ncm-dbt-06 1246787 500 214 58 228 +112.14 ± 13.89 0 6 89 148 7 +238.66 ± 36.16
202790 ncm-dbt-01 1199968 500 218 42 240 +127.76 ± 14.1 0 6 71 164 9 +280.42 ± 40.83
202789 ncm-dbt-05 1204007 500 220 42 238 +129.35 ± 13.54 0 7 63 175 5 +295.94 ± 43.42
202788 ncm-dbt-02 1198151 500 220 42 238 +129.35 ± 11.81 0 1 72 175 2 +304.07 ± 40.16
202787 ncm-dbt-04 1227732 500 223 32 245 +139.81 ± 13.45 0 5 57 180 8 +324.17 ± 45.77
202786 ncm-dbt-03 1222621 500 214 34 252 +130.94 ± 13.69 0 5 68 169 8 +293.29 ± 41.75
202785 ncm-dbt-06 1244115 500 227 33 240 +142.25 ± 14.66 0 6 58 172 14 +315.35 ± 45.33

Commit

Commit ID 3d1b067d853d6e8cc22cf18c1abb4cd9833dd38f
Author Linmiao Xu
Date 2023-09-11 20:37:39 UTC
Update default net to nn-1ee1aba5ed4c.nnue Created by retraining the master net on a dataset composed by: - adding Leela data from T60 jul-dec 2020, T77 nov 2021, T80 jun-jul 2023 - deduplicating and unminimizing parts of the dataset before interleaving Trained initially with max epoch 800, then increased near the end of training twice. First to 960, then 1200. After training, post-processing involved: - greedy permuting L1 weights with https://github.com/official-stockfish/Stockfish/pull/4620 - greedy 2- and 3- cycle permuting with https://github.com/official-stockfish/Stockfish/pull/4640 python3 easy_train.py \ --experiment-name 2048-retrain-S6-sk28 \ --training-dataset /data/S6.binpack \ --early-fen-skipping 28 \ --start-from-engine-test-net True \ --max_epoch 1200 \ --lr 4.375e-4 \ --gamma 0.995 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --tui False \ --seed $RANDOM \ --gpus 0 In the list of datasets below, periods in the filename represent the sequence of steps applied to arrive at the particular binpack. For example: test77-dec2021-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack 1. test77 dec2021 data rescored with 16 TB of syzygy tablebases during data conversion 2. filtered with csv_filter_v6_dd.py - v6 filtering and deduplication in one step 3. minimized with the original mar2023 implementation of `minimize_binpack` in the tools branch 4. unminimized by removing all positions with score == 32002 (`VALUE_NONE`) Binpacks were: - filtered with: https://github.com/linrock/nnue-data - unminimized with: https://github.com/linrock/Stockfish/tree/tools-unminify - deduplicated with: https://github.com/linrock/Stockfish/tree/tools-dd DATASETS=( leela96-filt-v2.min.unminimized.binpack dfrc99-16tb7p-eval-filt-v2.min.unminimized.binpack # most of the 0dd1cebea57 v6-dd dataset (without test80-jul2022) # https://github.com/official-stockfish/Stockfish/pull/4606 test60-novdec2021-12tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test77-dec2021-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test78-jantomay2022-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test78-juntosep2022-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test79-apr2022-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test79-may2022-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test80-jun2022-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test80-aug2022-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test80-sep2022-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test80-oct2022-16tb7p.filter-v6-dd.min.binpack test80-nov2022-16tb7p.filter-v6-dd.min.binpack test80-jan2023-3of3-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test80-feb2023-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack # older Leela data, recently converted test60-octnovdec2020-2tb7p.min.unminimized.binpack test60-julaugsep2020-2tb7p.min.binpack test77-nov2021-2tb7p.min.dd.binpack # newer Leela data test80-mar2023-2tb7p.min.unminimized.binpack test80-apr2023-2tb7p.filter-v6-sk16.min.unminimized.binpack test80-may2023-2tb7p.min.dd.binpack test80-jun2023-2tb7p.min.binpack test80-jul2023-2tb7p.binpack ) python3 interleave_binpacks.py ${DATASETS[@]} /data/S6.binpack Training data can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move: nn-epoch1059 : 2.7 +/- 1.6 Passed STC: https://tests.stockfishchess.org/tests/view/64fc8d705dab775b5359db42 LLR: 2.93 (-2.94,2.94) <0.00,2.00> Total: 168352 W: 43216 L: 42704 D: 82432 Ptnml(0-2): 599, 19672, 43134, 20160, 611 Passed LTC: https://tests.stockfishchess.org/tests/view/64fd44a75dab775b5359f065 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 154194 W: 39436 L: 38881 D: 75877 Ptnml(0-2): 78, 16577, 43238, 17120, 84 closes https://github.com/official-stockfish/Stockfish/pull/4782 Bench: 1603079
Copyright 2011–2024 Next Chess Move LLC