Dev Builds » 20180314-2101

NCM plays each Stockfish dev build 20,000 times against Stockfish 7. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Results

Host Started (UTC) Duration Base NPS (Avg) Games Wins Losses Draws Elo
ncm-et-3 2018-03-14 23:08 4189:06:54 2001106 2450 1170 63 1217 +169.20 +/- 9.44
ncm-et-4 2018-03-14 23:08 4189:06:36 2000631 2467 1152 75 1240 +162.59 +/- 9.37
ncm-et-5 2018-03-14 23:08 01:57:17 2021180 776 359 23 394 +161.06 +/- 16.61
ncm-et-6 2018-03-14 23:08 01:57:17 2024592 764 340 27 397 +151.22 +/- 16.55
ncm-et-7 2018-03-14 23:08 01:57:06 2007031 768 331 20 417 +149.24 +/- 15.90
ncm-et-8 2018-03-14 23:08 01:57:20 2020999 770 348 29 393 +153.14 +/- 16.70
ncm-et-9 2018-03-14 23:08 4189:06:43 2001873 2432 1068 61 1303 +153.05 +/- 9.01
ncm-et-10 2018-03-14 23:08 4189:07:02 1983071 2398 1106 72 1220 +160.29 +/- 9.43
ncm-et-11 2018-03-14 23:08 01:57:07 2019371 776 340 23 413 +150.72 +/- 16.09
ncm-et-12 2018-03-14 23:08 01:57:17 2013014 761 341 23 397 +154.65 +/- 16.48
ncm-et-13 2018-03-14 23:08 4189:06:52 1984430 2423 1100 92 1231 +153.87 +/- 9.44
ncm-et-14 2018-03-14 23:08 01:57:07 2020348 764 349 21 394 +159.49 +/- 16.54
ncm-et-15 2018-03-14 23:08 4189:06:59 1994002 2451 1123 69 1259 +159.80 +/- 9.25
  20000 9127 598 10275 +158.27 +/- 3.24

Test Detail

ID Host Started (UTC) Duration Base NPS Games Wins Losses Draws Elo CLI PGN
8589 ncm-et-10 2018-03-14 23:08 01:15:09 2021162 500 226 16 258 +155.54 +/- 20.51 Show
8590 ncm-et-8 2018-03-14 23:08 01:15:39 2020836 500 226 22 252 +150.51 +/- 20.98 Show
8591 ncm-et-12 2018-03-14 23:08 01:16:00 2005518 500 221 16 263 +151.35 +/- 20.25 Show
8592 ncm-et-3 2018-03-14 23:08 01:14:05 2024427 500 237 20 243 +161.49 +/- 21.42 Show
8593 ncm-et-15 2018-03-14 23:08 01:14:13 2016452 500 230 12 258 +162.35 +/- 20.39 Show
8594 ncm-et-7 2018-03-14 23:08 01:15:39 2012551 500 214 14 272 +147.19 +/- 19.73 Show
8595 ncm-et-5 2018-03-14 23:08 01:15:06 2021652 500 231 10 259 +164.93 +/- 20.27 Show
8596 ncm-et-6 2018-03-14 23:08 01:16:12 2026226 500 225 15 260 +155.54 +/- 20.38 Show
8597 ncm-et-9 2018-03-14 23:08 01:17:23 2027211 500 214 14 272 +147.19 +/- 19.73 Show
8598 ncm-et-13 2018-03-14 23:08 01:16:05 2025081 500 218 13 269 +151.35 +/- 19.85 Show
8599 ncm-et-11 2018-03-14 23:08 01:13:36 2018720 500 214 17 269 +144.72 +/- 19.97 Show
8600 ncm-et-14 2018-03-14 23:08 01:16:16 2020348 500 238 10 252 +171.02 +/- 20.65 Show
8601 ncm-et-4 2018-03-14 23:08 01:14:16 2024103 500 242 16 242 +169.27 +/- 21.38 Show
8602 ncm-et-11 2018-03-15 00:23 00:42:25 2020022 276 126 6 144 +161.83 +/- 27.23 Show
8603 ncm-et-3 2018-03-15 00:23 00:41:53 2024917 275 131 7 137 +168.80 +/- 28.22 Show
8604 ncm-et-15 2018-03-15 00:23 00:41:54 2019208 274 121 10 143 +149.31 +/- 27.62 Show
8605 ncm-et-4 2018-03-15 00:23 00:41:38 2024920 283 135 8 140 +167.86 +/- 28.00 Show
8606 ncm-et-5 2018-03-15 00:24 00:41:06 2020707 276 128 13 135 +154.14 +/- 28.86 Show
8607 ncm-et-10 2018-03-15 00:24 00:41:00 2015312 265 118 5 142 +158.26 +/- 27.21 Show
8608 ncm-et-8 2018-03-15 00:25 00:40:36 2021162 270 122 7 141 +158.05 +/- 27.61 Show
8609 ncm-et-7 2018-03-15 00:25 00:40:21 2001510 268 117 6 145 +153.10 +/- 26.96 Show
8610 ncm-et-12 2018-03-15 00:25 00:40:12 2020510 261 120 7 134 +161.04 +/- 28.43 Show
8611 ncm-et-13 2018-03-15 00:25 00:40:00 2026063 263 117 7 139 +154.81 +/- 27.76 Show
8612 ncm-et-6 2018-03-15 00:25 00:40:00 2022957 264 115 12 137 +143.14 +/- 28.40 Show
8613 ncm-et-14 2018-03-15 00:25 00:39:46 2020348 264 111 11 142 +138.50 +/- 27.66 Show
8614 ncm-et-9 2018-03-15 00:26 00:38:41 2027046 261 117 5 139 +159.41 +/- 27.55 Show
60381 ncm-et-10 2018-09-05 07:57 01:19:01 1950202 500 240 20 240 +164.07 +/- 21.59 Show
60382 ncm-et-15 2018-09-05 07:57 01:16:35 1983106 500 243 10 247 +175.44 +/- 20.92 Show
60383 ncm-et-4 2018-09-05 07:57 01:15:35 1990032 500 237 19 244 +162.35 +/- 21.35 Show
60384 ncm-et-9 2018-09-05 07:57 01:14:52 1990351 500 216 15 269 +148.02 +/- 19.91 Show
60385 ncm-et-3 2018-09-05 07:57 01:16:08 1988457 500 248 9 243 +180.80 +/- 21.11 Show
60386 ncm-et-13 2018-09-05 07:57 01:16:38 1989561 500 245 25 230 +164.07 +/- 22.24 Show
60387 ncm-et-9 2018-09-05 09:13 01:16:51 1990191 500 220 10 270 +155.54 +/- 19.70 Show
60388 ncm-et-4 2018-09-05 09:14 01:16:29 1987981 500 244 13 243 +173.67 +/- 21.24 Show
60389 ncm-et-3 2018-09-05 09:14 01:15:49 1990034 500 238 11 251 +170.15 +/- 20.73 Show
60390 ncm-et-15 2018-09-05 09:15 01:14:52 1982636 500 220 17 263 +149.68 +/- 20.28 Show
60391 ncm-et-13 2018-09-05 09:15 01:15:59 1984057 500 236 22 242 +158.93 +/- 21.52 Show
60392 ncm-et-10 2018-09-05 09:17 01:16:36 1982008 500 239 16 245 +166.66 +/- 21.21 Show
60393 ncm-et-15 2018-09-05 10:31 01:15:46 1982321 500 237 17 246 +164.07 +/- 21.19 Show
60394 ncm-et-9 2018-09-05 10:31 01:16:40 1986091 500 223 14 263 +154.70 +/- 20.19 Show
60395 ncm-et-3 2018-09-05 10:31 01:16:08 1988771 500 234 12 254 +165.80 +/- 20.61 Show
60396 ncm-et-4 2018-09-05 10:31 01:15:00 1988771 500 213 13 274 +147.19 +/- 19.59 Show
60397 ncm-et-13 2018-09-05 10:32 01:15:30 1975476 500 216 20 264 +143.89 +/- 20.31 Show
60398 ncm-et-10 2018-09-05 10:35 01:18:21 1977007 500 218 11 271 +153.02 +/- 19.68 Show
60399 ncm-et-4 2018-09-05 11:47 00:27:17 1987981 184 81 6 97 +150.35 +/- 33.45 Show
60400 ncm-et-15 2018-09-05 11:48 00:27:15 1980287 177 72 3 102 +143.00 +/- 31.55 Show
60401 ncm-et-3 2018-09-05 11:48 00:26:14 1990032 175 82 4 89 +166.54 +/- 34.89 Show
60402 ncm-et-9 2018-09-05 11:49 00:25:49 1990349 171 78 3 90 +163.47 +/- 34.33 Show
60403 ncm-et-13 2018-09-05 11:49 00:26:00 1906341 160 68 5 87 +144.61 +/- 35.09 Show
60404 ncm-et-10 2018-09-05 11:54 00:20:35 1952736 133 65 4 64 +172.19 +/- 41.80 Show

Commit

Commit ID 8db75dd9ec05410136898aa2f8c6dc720b755eb8
Author Stéphane Nicolet
Date 2018-03-14 21:01:58 UTC
Commit Type
Check the clock every 1024 nodes

This patch checks the clock every 1024, instead of 4096 in current master.

This is a step towards a solution to alleviate the problem reported by
Gian-Carlo Pascutto in pull request #1471, about a situation where SF
would sometimes lose on time in endgames at time control 1min+1sec when
using Syzygy EGTB on a spinning drive.

We made four tests called "Estimate the Elo cost of calling now() every
N nodes", with N in {256,1024} and time control in {STC,LTC}. Each test
was 40000 games with auto-purge off, against master branch.

http://tests.stockfishchess.org/tests/view/5aa4f37c0ebc59029780ff2d
http://tests.stockfishchess.org/tests/view/5aa4f3080ebc59029780ff2b
http://tests.stockfishchess.org/tests/view/5aa5cbb50ebc59029780ffce
http://tests.stockfishchess.org/tests/view/5aa5cb730ebc59029780ffcc

Here is a summary of the results (speed-up and Elo gain against master):

|   Time   |       N=256       |      N=1024      |
|:--------:|:-----------------:|:----------------:|
| Speed-up |    0.1% slower    |   0.2% faster    |
|   STC    |  -0.33 +-2.2 Elo  |  1.33 +-2.2 Elo  |
|   LTC    |   1.08 +-2.3 Elo  |  2.34 +-2.2 Elo  |

Based on these results, we tested N=1024 as a non-regression against
master at LTC:

LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 80746 W: 12430 L: 12399 D: 55917
http://tests.stockfishchess.org/tests/view/5aa7ba660ebc590297810116

Closes https://github.com/official-stockfish/Stockfish/pull/1482

No functional change.