Dev Builds » 20230711-2056

You are viewing an old NCM Stockfish dev build test. You may find the most recent dev build tests using Stockfish 15 as the baseline here.

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo
ncm-dbt-01 09:54:19 1196780 3338 1386 305 1647 +116.72 +/- 5.47 0 51 533 1038 47 +251.63 +/- 14.76
ncm-dbt-02 09:50:33 1239438 3316 1389 277 1650 +121.2 +/- 5.25 0 46 482 1102 28 +271.66 +/- 15.53
ncm-dbt-03 09:55:11 1241915 3334 1423 276 1635 +124.61 +/- 5.45 0 47 477 1092 51 +273.9 +/- 15.61
ncm-dbt-04 09:54:45 1235037 3360 1387 325 1648 +113.71 +/- 5.43 1 56 539 1048 36 +247.03 +/- 14.68
ncm-dbt-05 09:51:18 1228388 3318 1397 294 1627 +120.06 +/- 5.46 1 44 513 1053 48 +261.38 +/- 15.04
ncm-dbt-06 09:54:08 1230335 3334 1407 307 1620 +119.08 +/- 5.36 0 51 500 1081 35 +262.75 +/- 15.25
20000 8389 1784 9827 +119.21 +/- 2.21 2 295 3044 6414 245 +261.19 +/- 6.17

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN
196549 ncm-dbt-02 1241470 316 133 33 150 +113.85 +/- 18.68 0 7 49 97 5 +241.51 +/- 49.3
196548 ncm-dbt-05 1217046 318 135 31 152 +117.96 +/- 17.64 0 3 55 95 6 +249.84 +/- 46.18
196547 ncm-dbt-06 1235808 334 144 33 157 +120.02 +/- 17.05 0 4 53 105 5 +260.33 +/- 47.31
196546 ncm-dbt-03 1254606 334 143 30 161 +122.36 +/- 16.06 0 1 57 104 5 +267.39 +/- 44.98
196545 ncm-dbt-01 1198156 338 135 38 165 +102.59 +/- 17.49 0 6 65 93 5 +212.06 +/- 42.44
196544 ncm-dbt-04 1245967 360 145 38 177 +106.48 +/- 15.73 0 4 68 105 3 +229.0 +/- 41.3
196543 ncm-dbt-05 1222462 500 211 51 238 +115.22 +/- 14.69 0 8 84 148 10 +240.82 +/- 37.4
196542 ncm-dbt-02 1239262 500 214 47 239 +120.67 +/- 12.6 0 3 80 164 3 +273.0 +/- 38.12
196541 ncm-dbt-03 1237815 500 222 41 237 +131.74 +/- 13.67 0 5 67 170 8 +295.94 +/- 42.08
196540 ncm-dbt-06 1217508 500 209 43 248 +119.89 +/- 13.16 0 6 75 166 3 +270.57 +/- 39.67
196539 ncm-dbt-01 1213939 500 210 47 243 +117.55 +/- 13.7 0 8 75 163 4 +261.07 +/- 39.69
196538 ncm-dbt-04 1234479 500 197 45 258 +109.07 +/- 14.53 0 12 79 154 5 +234.38 +/- 38.61
196537 ncm-dbt-06 1246988 500 213 48 239 +119.11 +/- 13.52 0 6 78 161 5 +263.42 +/- 38.85
196536 ncm-dbt-05 1210103 500 214 52 234 +116.77 +/- 13.19 0 4 85 156 5 +256.44 +/- 36.95
196535 ncm-dbt-02 1234870 500 201 41 258 +115.22 +/- 13.38 0 7 79 161 3 +256.44 +/- 38.62
196534 ncm-dbt-01 1190028 500 208 58 234 +107.54 +/- 14.21 0 8 91 144 7 +226.0 +/- 35.82
196533 ncm-dbt-03 1244474 500 217 35 248 +132.54 +/- 12.71 0 4 64 178 4 +309.64 +/- 43.08
196532 ncm-dbt-04 1228462 500 201 55 244 +104.49 +/- 13.88 1 6 93 146 4 +226.0 +/- 35.35
196531 ncm-dbt-06 1216995 500 209 46 245 +117.55 +/- 14.04 0 10 71 165 4 +261.07 +/- 40.78
196530 ncm-dbt-02 1236995 500 218 35 247 +133.34 +/- 13.81 0 9 54 182 5 +309.64 +/- 46.67
196529 ncm-dbt-05 1227328 500 212 35 253 +128.55 +/- 13.56 0 6 67 171 6 +290.66 +/- 42.08
196528 ncm-dbt-03 1236551 500 203 33 264 +123.02 +/- 14.49 0 6 79 154 11 +261.07 +/- 38.58
196527 ncm-dbt-01 1186030 500 210 43 247 +120.67 +/- 13.5 0 6 76 163 5 +268.17 +/- 39.39
196526 ncm-dbt-04 1230629 500 213 52 235 +116.0 +/- 13.71 0 6 83 155 6 +251.89 +/- 37.57
196525 ncm-dbt-06 1233495 500 208 44 248 +118.33 +/- 13.86 0 7 78 159 6 +258.75 +/- 38.88
196524 ncm-dbt-05 1248642 500 205 39 256 +119.89 +/- 13.68 0 7 75 163 5 +265.78 +/- 39.69
196523 ncm-dbt-02 1245324 500 207 49 244 +113.68 +/- 14.05 0 10 76 160 4 +249.64 +/- 39.41
196522 ncm-dbt-04 1255030 500 215 40 245 +126.97 +/- 13.58 0 8 63 175 4 +290.66 +/- 43.37
196521 ncm-dbt-01 1190715 500 206 37 257 +122.24 +/- 14.66 0 8 75 157 10 +261.07 +/- 39.69
196520 ncm-dbt-03 1232162 500 206 45 249 +116.0 +/- 14.37 0 10 75 159 6 +251.89 +/- 39.67
196519 ncm-dbt-02 1240361 500 203 32 265 +123.81 +/- 13.28 0 6 71 169 4 +280.42 +/- 40.83
196518 ncm-dbt-06 1222321 500 215 46 239 +122.24 +/- 14.0 0 8 71 165 6 +270.57 +/- 40.83
196517 ncm-dbt-05 1228268 500 214 43 243 +123.81 +/- 15.13 1 7 73 158 11 +265.78 +/- 40.25
196516 ncm-dbt-04 1218738 500 209 49 242 +115.22 +/- 14.37 0 9 79 155 7 +247.41 +/- 38.64
196515 ncm-dbt-01 1212067 500 206 40 254 +119.89 +/- 13.51 0 5 80 159 6 +263.42 +/- 38.27
196514 ncm-dbt-03 1233397 500 221 48 231 +125.38 +/- 15.27 0 9 72 156 13 +263.42 +/- 40.52
196513 ncm-dbt-06 1239232 500 209 47 244 +116.77 +/- 14.37 0 10 74 160 6 +254.16 +/- 39.94
196512 ncm-dbt-02 1237786 500 213 40 247 +125.38 +/- 12.89 0 4 73 169 4 +285.49 +/- 40.16
196511 ncm-dbt-05 1244868 500 206 43 251 +117.55 +/- 14.04 0 9 74 162 5 +258.75 +/- 39.96
196510 ncm-dbt-01 1186528 500 211 42 247 +122.24 +/- 14.98 0 10 71 159 10 +261.07 +/- 40.78
196509 ncm-dbt-03 1254401 500 211 44 245 +120.67 +/- 14.35 0 12 63 171 4 +270.57 +/- 43.07
196508 ncm-dbt-04 1231960 500 207 46 247 +116.0 +/- 14.69 0 11 74 158 7 +249.64 +/- 39.91

Commit

Commit ID af110e02ec96cdb46cf84c68252a1da15a902395
Author Joost VandeVondele
Date 2023-07-11 20:56:49 UTC
Remove classical evaluation since the introduction of NNUE (first released with Stockfish 12), we have maintained the classical evaluation as part of SF in frozen form. The idea that this code could lead to further inputs to the NN or search did not materialize. Now, after five releases, this PR removes the classical evaluation from SF. Even though this evaluation is probably the best of its class, it has become unimportant for the engine's strength, and there is little need to maintain this code (roughly 25% of SF) going forward, or to expend resources on trying to improve its integration in the NNUE eval. Indeed, it had still a very limited use in the current SF, namely for the evaluation of positions that are nearly decided based on material difference, where the speed of the classical evaluation outweights its inaccuracies. This impact on strength is small, roughly 2Elo, and probably decreasing in importance as the TC grows. Potentially, removal of this code could lead to the development of techniques to have faster, but less accurate NN evaluation, for certain positions. STC https://tests.stockfishchess.org/tests/view/64a320173ee09aa549c52157 Elo: -2.35 ± 1.1 (95%) LOS: 0.0% Total: 100000 W: 24916 L: 25592 D: 49492 Ptnml(0-2): 287, 12123, 25841, 11477, 272 nElo: -4.62 ± 2.2 (95%) PairsRatio: 0.95 LTC https://tests.stockfishchess.org/tests/view/64a320293ee09aa549c5215b Elo: -1.74 ± 1.0 (95%) LOS: 0.0% Total: 100000 W: 25010 L: 25512 D: 49478 Ptnml(0-2): 44, 11069, 28270, 10579, 38 nElo: -3.72 ± 2.2 (95%) PairsRatio: 0.96 VLTC SMP https://tests.stockfishchess.org/tests/view/64a3207c3ee09aa549c52168 Elo: -1.70 ± 0.9 (95%) LOS: 0.0% Total: 100000 W: 25673 L: 26162 D: 48165 Ptnml(0-2): 8, 9455, 31569, 8954, 14 nElo: -3.95 ± 2.2 (95%) PairsRatio: 0.95 closes https://github.com/official-stockfish/Stockfish/pull/4674 Bench: 1444646
Copyright 2011–2024 Next Chess Move LLC