Dev Builds » 20180314-2101

NCM plays each Stockfish dev build 20,000 times against Stockfish 7. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Notice: NCM upgraded all servers to to Ubuntu 18.04 (from 16.04) on October 23rd. Binaries compiled on the upgraded disto are skewing measurements upwards. We are working on a fix.

Results

Host Started (UTC) Duration Base NPS (Avg) Games Wins Losses Draws Elo
ncm-et-3 2018-03-14 23:08 4189:06:54 2001106 2450 1170 63 1217 +169.20 +/- 9.44
ncm-et-4 2018-03-14 23:08 4189:06:36 2000631 2467 1152 75 1240 +162.59 +/- 9.37
ncm-et-5 2018-03-14 23:08 01:57:17 2021180 776 359 23 394 +161.06 +/- 16.61
ncm-et-6 2018-03-14 23:08 01:57:17 2024592 764 340 27 397 +151.22 +/- 16.55
ncm-et-7 2018-03-14 23:08 01:57:06 2007031 768 331 20 417 +149.24 +/- 15.90
ncm-et-8 2018-03-14 23:08 01:57:20 2020999 770 348 29 393 +153.14 +/- 16.70
ncm-et-9 2018-03-14 23:08 4189:06:43 2001873 2432 1068 61 1303 +153.05 +/- 9.01
ncm-et-10 2018-03-14 23:08 4189:07:02 1983071 2398 1106 72 1220 +160.29 +/- 9.43
ncm-et-11 2018-03-14 23:08 01:57:07 2019371 776 340 23 413 +150.72 +/- 16.09
ncm-et-12 2018-03-14 23:08 01:57:17 2013014 761 341 23 397 +154.65 +/- 16.48
ncm-et-13 2018-03-14 23:08 4189:06:52 1984430 2423 1100 92 1231 +153.87 +/- 9.44
ncm-et-14 2018-03-14 23:08 01:57:07 2020348 764 349 21 394 +159.49 +/- 16.54
ncm-et-15 2018-03-14 23:08 4189:06:59 1994002 2451 1123 69 1259 +159.80 +/- 9.25
  20000 9127 598 10275 +158.27 +/- 3.24

Test Detail

ID Host Started (UTC) Duration Base NPS Games Wins Losses Draws Elo CLI PGN
8589 ncm-et-10 2018-03-14 23:08 01:15:09 2021162 500 226 16 258 +155.54 +/- 20.51 Show
8590 ncm-et-8 2018-03-14 23:08 01:15:39 2020836 500 226 22 252 +150.51 +/- 20.98 Show
8591 ncm-et-12 2018-03-14 23:08 01:16:00 2005518 500 221 16 263 +151.35 +/- 20.25 Show
8592 ncm-et-3 2018-03-14 23:08 01:14:05 2024427 500 237 20 243 +161.49 +/- 21.42 Show
8593 ncm-et-15 2018-03-14 23:08 01:14:13 2016452 500 230 12 258 +162.35 +/- 20.39 Show
8594 ncm-et-7 2018-03-14 23:08 01:15:39 2012551 500 214 14 272 +147.19 +/- 19.73 Show
8595 ncm-et-5 2018-03-14 23:08 01:15:06 2021652 500 231 10 259 +164.93 +/- 20.27 Show
8596 ncm-et-6 2018-03-14 23:08 01:16:12 2026226 500 225 15 260 +155.54 +/- 20.38 Show
8597 ncm-et-9 2018-03-14 23:08 01:17:23 2027211 500 214 14 272 +147.19 +/- 19.73 Show
8598 ncm-et-13 2018-03-14 23:08 01:16:05 2025081 500 218 13 269 +151.35 +/- 19.85 Show
8599 ncm-et-11 2018-03-14 23:08 01:13:36 2018720 500 214 17 269 +144.72 +/- 19.97 Show
8600 ncm-et-14 2018-03-14 23:08 01:16:16 2020348 500 238 10 252 +171.02 +/- 20.65 Show
8601 ncm-et-4 2018-03-14 23:08 01:14:16 2024103 500 242 16 242 +169.27 +/- 21.38 Show
8602 ncm-et-11 2018-03-15 00:23 00:42:25 2020022 276 126 6 144 +161.83 +/- 27.23 Show
8603 ncm-et-3 2018-03-15 00:23 00:41:53 2024917 275 131 7 137 +168.80 +/- 28.22 Show
8604 ncm-et-15 2018-03-15 00:23 00:41:54 2019208 274 121 10 143 +149.31 +/- 27.62 Show
8605 ncm-et-4 2018-03-15 00:23 00:41:38 2024920 283 135 8 140 +167.86 +/- 28.00 Show
8606 ncm-et-5 2018-03-15 00:24 00:41:06 2020707 276 128 13 135 +154.14 +/- 28.86 Show
8607 ncm-et-10 2018-03-15 00:24 00:41:00 2015312 265 118 5 142 +158.26 +/- 27.21 Show
8608 ncm-et-8 2018-03-15 00:25 00:40:36 2021162 270 122 7 141 +158.05 +/- 27.61 Show
8609 ncm-et-7 2018-03-15 00:25 00:40:21 2001510 268 117 6 145 +153.10 +/- 26.96 Show
8610 ncm-et-12 2018-03-15 00:25 00:40:12 2020510 261 120 7 134 +161.04 +/- 28.43 Show
8611 ncm-et-13 2018-03-15 00:25 00:40:00 2026063 263 117 7 139 +154.81 +/- 27.76 Show
8612 ncm-et-6 2018-03-15 00:25 00:40:00 2022957 264 115 12 137 +143.14 +/- 28.40 Show
8613 ncm-et-14 2018-03-15 00:25 00:39:46 2020348 264 111 11 142 +138.50 +/- 27.66 Show
8614 ncm-et-9 2018-03-15 00:26 00:38:41 2027046 261 117 5 139 +159.41 +/- 27.55 Show
60381 ncm-et-10 2018-09-05 07:57 01:19:01 1950202 500 240 20 240 +164.07 +/- 21.59 Show
60382 ncm-et-15 2018-09-05 07:57 01:16:35 1983106 500 243 10 247 +175.44 +/- 20.92 Show
60383 ncm-et-4 2018-09-05 07:57 01:15:35 1990032 500 237 19 244 +162.35 +/- 21.35 Show
60384 ncm-et-9 2018-09-05 07:57 01:14:52 1990351 500 216 15 269 +148.02 +/- 19.91 Show
60385 ncm-et-3 2018-09-05 07:57 01:16:08 1988457 500 248 9 243 +180.80 +/- 21.11 Show
60386 ncm-et-13 2018-09-05 07:57 01:16:38 1989561 500 245 25 230 +164.07 +/- 22.24 Show
60387 ncm-et-9 2018-09-05 09:13 01:16:51 1990191 500 220 10 270 +155.54 +/- 19.70 Show
60388 ncm-et-4 2018-09-05 09:14 01:16:29 1987981 500 244 13 243 +173.67 +/- 21.24 Show
60389 ncm-et-3 2018-09-05 09:14 01:15:49 1990034 500 238 11 251 +170.15 +/- 20.73 Show
60390 ncm-et-15 2018-09-05 09:15 01:14:52 1982636 500 220 17 263 +149.68 +/- 20.28 Show
60391 ncm-et-13 2018-09-05 09:15 01:15:59 1984057 500 236 22 242 +158.93 +/- 21.52 Show
60392 ncm-et-10 2018-09-05 09:17 01:16:36 1982008 500 239 16 245 +166.66 +/- 21.21 Show
60393 ncm-et-15 2018-09-05 10:31 01:15:46 1982321 500 237 17 246 +164.07 +/- 21.19 Show
60394 ncm-et-9 2018-09-05 10:31 01:16:40 1986091 500 223 14 263 +154.70 +/- 20.19 Show
60395 ncm-et-3 2018-09-05 10:31 01:16:08 1988771 500 234 12 254 +165.80 +/- 20.61 Show
60396 ncm-et-4 2018-09-05 10:31 01:15:00 1988771 500 213 13 274 +147.19 +/- 19.59 Show
60397 ncm-et-13 2018-09-05 10:32 01:15:30 1975476 500 216 20 264 +143.89 +/- 20.31 Show
60398 ncm-et-10 2018-09-05 10:35 01:18:21 1977007 500 218 11 271 +153.02 +/- 19.68 Show
60399 ncm-et-4 2018-09-05 11:47 00:27:17 1987981 184 81 6 97 +150.35 +/- 33.45 Show
60400 ncm-et-15 2018-09-05 11:48 00:27:15 1980287 177 72 3 102 +143.00 +/- 31.55 Show
60401 ncm-et-3 2018-09-05 11:48 00:26:14 1990032 175 82 4 89 +166.54 +/- 34.89 Show
60402 ncm-et-9 2018-09-05 11:49 00:25:49 1990349 171 78 3 90 +163.47 +/- 34.33 Show
60403 ncm-et-13 2018-09-05 11:49 00:26:00 1906341 160 68 5 87 +144.61 +/- 35.09 Show
60404 ncm-et-10 2018-09-05 11:54 00:20:35 1952736 133 65 4 64 +172.19 +/- 41.80 Show

Commit

Commit ID 8db75dd9ec05410136898aa2f8c6dc720b755eb8
Author Stéphane Nicolet
Date 2018-03-14 21:01:58 UTC
Commit Type
Check the clock every 1024 nodes

This patch checks the clock every 1024, instead of 4096 in current master.

This is a step towards a solution to alleviate the problem reported by
Gian-Carlo Pascutto in pull request #1471, about a situation where SF
would sometimes lose on time in endgames at time control 1min+1sec when
using Syzygy EGTB on a spinning drive.

We made four tests called "Estimate the Elo cost of calling now() every
N nodes", with N in {256,1024} and time control in {STC,LTC}. Each test
was 40000 games with auto-purge off, against master branch.

http://tests.stockfishchess.org/tests/view/5aa4f37c0ebc59029780ff2d
http://tests.stockfishchess.org/tests/view/5aa4f3080ebc59029780ff2b
http://tests.stockfishchess.org/tests/view/5aa5cbb50ebc59029780ffce
http://tests.stockfishchess.org/tests/view/5aa5cb730ebc59029780ffcc

Here is a summary of the results (speed-up and Elo gain against master):

|   Time   |       N=256       |      N=1024      |
|:--------:|:-----------------:|:----------------:|
| Speed-up |    0.1% slower    |   0.2% faster    |
|   STC    |  -0.33 +-2.2 Elo  |  1.33 +-2.2 Elo  |
|   LTC    |   1.08 +-2.3 Elo  |  2.34 +-2.2 Elo  |

Based on these results, we tested N=1024 as a non-regression against
master at LTC:

LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 80746 W: 12430 L: 12399 D: 55917
http://tests.stockfishchess.org/tests/view/5aa7ba660ebc590297810116

Closes https://github.com/official-stockfish/Stockfish/pull/1482

No functional change.