Dev Builds » 20180123-1326

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 7. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games Wins Losses Draws Elo
ncm-et-3 17:08:54 2017073 6677 2983 266 3428 +150.06 +/- 5.66
ncm-et-4 17:13:43 2016251 6694 3000 258 3436 +151.19 +/- 5.65
ncm-et-9 04:21:18 1989520 1698 769 58 871 +155.01 +/- 11.19
ncm-et-10 04:21:16 1962229 1667 749 65 853 +151.48 +/- 11.35
ncm-et-13 04:20:50 1942686 1636 733 62 841 +151.41 +/- 11.42
ncm-et-15 04:21:10 1934899 1628 723 64 841 +149.18 +/- 11.42
20000 8957 773 10270 +151.01 +/- 3.27

Test Detail

ID Host Started (UTC) Duration Base NPS Games Wins Losses Draws Elo CLI PGN
54228 ncm-et-15 2018-09-18 12:17 00:21:07 1910700 128 47 3 78 +124.5 +/- 35.8
54227 ncm-et-13 2018-09-18 12:16 00:21:42 1927545 136 61 7 68 +145.98 +/- 40.84
54226 ncm-et-10 2018-09-18 12:12 00:26:17 1980755 167 77 11 79 +145.21 +/- 38.25
54225 ncm-et-3 2018-09-18 12:10 00:28:06 1988619 176 71 10 95 +125.62 +/- 34.17
54224 ncm-et-4 2018-09-18 12:08 00:30:00 1990349 195 90 8 97 +155.76 +/- 33.98
54223 ncm-et-9 2018-09-18 12:07 00:30:28 1991139 198 87 9 102 +144.69 +/- 33.04
54222 ncm-et-15 2018-09-18 10:57 01:18:27 1980443 500 225 19 256 +152.18 +/- 20.73
54221 ncm-et-10 2018-09-18 10:53 01:17:53 1979660 500 230 20 250 +155.54 +/- 21.07
54220 ncm-et-13 2018-09-18 10:51 01:23:36 1864239 500 222 25 253 +144.72 +/- 21.02
54219 ncm-et-3 2018-09-18 10:50 01:18:29 1987666 500 233 18 249 +159.78 +/- 21.08
54218 ncm-et-9 2018-09-18 10:49 01:16:51 1990190 500 230 20 250 +155.54 +/- 21.07
54217 ncm-et-4 2018-09-18 10:49 01:18:11 1989559 500 232 14 254 +162.35 +/- 20.7
54216 ncm-et-13 2018-09-18 09:32 01:17:36 1989717 500 233 18 249 +159.78 +/- 21.08
54215 ncm-et-10 2018-09-18 09:32 01:19:24 1918508 500 224 16 260 +153.86 +/- 20.44
54214 ncm-et-15 2018-09-18 09:32 01:23:59 1876085 500 229 27 244 +148.85 +/- 21.53
54213 ncm-et-3 2018-09-18 09:32 01:17:25 1991772 500 205 24 271 +131.74 +/- 20.07
54212 ncm-et-9 2018-09-18 09:31 01:16:51 1987509 500 230 11 259 +163.21 +/- 20.33
54211 ncm-et-4 2018-09-18 09:31 01:16:37 1990664 500 236 19 245 +161.49 +/- 21.32
54210 ncm-et-4 2018-09-18 08:13 01:16:24 1988454 500 226 25 249 +148.02 +/- 21.23
54209 ncm-et-3 2018-09-18 08:13 01:17:14 1989875 500 237 23 240 +158.93 +/- 21.68
54208 ncm-et-10 2018-09-18 08:13 01:17:42 1969996 500 218 18 264 +147.19 +/- 20.28
54207 ncm-et-15 2018-09-18 08:13 01:17:37 1972370 500 222 15 263 +153.02 +/- 20.25
54206 ncm-et-9 2018-09-18 08:13 01:17:08 1989244 500 222 18 260 +150.51 +/- 20.49
54205 ncm-et-13 2018-09-18 08:13 01:17:56 1989244 500 217 12 271 +151.35 +/- 19.74
356 ncm-et-3 2018-01-24 04:29 00:00:45 2028850 1 0 0 1 0.0 +/- 30.47
355 ncm-et-4 2018-01-24 03:13 01:17:27 2026226 499 201 20 278 +132.03 +/- 19.61
354 ncm-et-3 2018-01-24 03:11 01:16:39 2027046 500 227 17 256 +155.54 +/- 20.68
353 ncm-et-3 2018-01-24 01:53 01:17:00 2028030 500 258 21 221 +179.0 +/- 22.73
352 ncm-et-4 2018-01-24 01:53 01:18:13 2028031 500 237 19 244 +162.35 +/- 21.38
351 ncm-et-4 2018-01-23 23:48 01:16:45 2026882 500 228 20 252 +153.86 +/- 20.97
350 ncm-et-3 2018-01-23 23:43 01:17:14 2029673 500 219 18 263 +148.02 +/- 20.34
349 ncm-et-4 2018-01-23 22:31 01:15:48 2026390 500 231 19 250 +157.24 +/- 21.05
348 ncm-et-3 2018-01-23 22:26 01:16:02 2026063 500 210 19 271 +139.81 +/- 19.95
347 ncm-et-4 2018-01-23 21:12 01:17:31 2027538 500 226 24 250 +148.85 +/- 21.16
346 ncm-et-3 2018-01-23 21:08 01:16:23 2024917 500 230 20 250 +155.54 +/- 21.07
345 ncm-et-4 2018-01-23 19:53 01:17:37 2027374 500 232 14 254 +162.35 +/- 20.7
344 ncm-et-3 2018-01-23 19:50 01:17:36 2029015 500 210 21 269 +138.18 +/- 20.1
343 ncm-et-4 2018-01-23 18:35 01:17:40 2028194 500 231 19 250 +157.24 +/- 21.05
342 ncm-et-3 2018-01-23 18:33 01:15:46 2025081 500 228 16 256 +157.24 +/- 20.65
341 ncm-et-4 2018-01-23 17:16 01:18:08 2026392 500 204 21 275 +133.34 +/- 19.8
340 ncm-et-3 2018-01-23 17:14 01:17:47 2026554 500 212 18 270 +142.26 +/- 19.98
339 ncm-et-3 2018-01-23 15:58 01:15:11 2025408 500 209 25 266 +134.15 +/- 20.35
338 ncm-et-4 2018-01-23 15:57 01:17:15 2025081 500 213 17 270 +143.89 +/- 19.95
337 ncm-et-4 2018-01-23 14:40 01:16:07 2026391 500 213 19 268 +142.26 +/- 20.1
336 ncm-et-3 2018-01-23 14:39 01:17:17 2027538 500 234 16 250 +162.35 +/- 20.97

Commit

Commit ID 254d995e187d8ecd02c3e5613e43aab525e41e22
Author Stéphane Nicolet
Date 2018-01-23 13:26:45 UTC
Contempt 20 Set the default contempt value of Stockfish to 20 centipawns. The contempt feature of Stockfish tries to prevent the engine from simplifying the position too quickly when it feels that it is very slightly behind, instead keeping the tension a little bit longer. Various tests in November 2017 have proved that our current imple- mentation works well against SF7 (which is about 130 Elo weaker than current master) and than the Elo gain is an increasing function of contempt, going (against SF7) from +0 Elo when contempt is set at zero centipawns, to +30 Elo when contempt is 40 centipawns. See pull request 1325 for details: https://github.com/official-stockfish/Stockfish/pull/1325 This november discussion left open the decision of which "default" value for contempt we should use for Stockfish, taking into account the various uses ofStockfish (opening preparation for humans, computer online tournaments,analysis tool for web pages, human/computer play, etc). This pull request proposes to set the default contempt value of SF to twenty centipawns, which turns out to be the highest value which is not a regression against current master, as this seemed to be a good compromise between risk and safety. A couple of SPRT[-3..1] tests were done to bisect this value: Contempt 10: http://tests.stockfishchess.org/tests/view/5a5d42d20ebc5902977e2901 (PASSED) Contempt 15: http://tests.stockfishchess.org/tests/view/5a5d41740ebc5902977e28fa (PASSED) Contempt 20: http://tests.stockfishchess.org/tests/view/5a5d42060ebc5902977e28fc (PASSED) Contempt 25: http://tests.stockfishchess.org/tests/view/5a5d433f0ebc5902977e2904 (FAILED) Surprisingly, a test at "very long time control" hinted that using contempt 20 is not only be non-regressive against contempt 0, but may actually exhibit some small Elo gain, giving a likehood of superio- rity of 88.7% after 8500 games: VLTC: ELO: 2.28 +-3.7 (95%) LOS: 88.7% Total: 8521 W: 1096 L: 1040 D: 6385 http://tests.stockfishchess.org/tests/view/5a60b2820ebc590297b9b7e0 Finally, there was some concerns that a contempt value of 20 would be worse than a value of 7, but a test with 20000 games at STC was neutral: STC: ELO: 0.45 +-3.1 (95%) LOS: 61.2% Total: 20000 W: 4222 L: 4196 D: 11582 http://tests.stockfishchess.org/tests/view/5a64d2fd0ebc590297903868 See the comments in pull request 1361 for the long, nice discussion (180 entries :-)) leading to the decision to propose contempt 20 as the default value: https://github.com/official-stockfish/Stockfish/pull/1361 Whether Stockfish should strictly adhere to the Komodo and Houdini semantics and add the UCI commands to force the contempt to be White in the so-called "analysis mode" is still under discussion, and may be or may not be the object of a future commit. Bench: 5783344
Copyright 2011–2024 Next Chess Move LLC