Dev Builds » 20180327-2231

NCM plays each Stockfish dev build 20,000 times against Stockfish 7. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Notice: NCM upgraded all servers to to Ubuntu 18.04 (from 16.04) on October 23rd. Binaries compiled on the upgraded disto are skewing measurements upwards. We are working on a fix.

Results

Host Started (UTC) Duration Base NPS (Avg) Games Wins Losses Draws Elo
ncm-et-3 2018-03-27 23:43 3767:50:17 2001636 2388 1073 59 1256 +157.50 +/- 9.20
ncm-et-4 2018-03-27 23:41 3767:52:11 1997168 2392 1126 94 1172 +160.40 +/- 9.73
ncm-et-5 2018-03-27 23:40 01:50:56 2023284 736 335 26 375 +155.48 +/- 17.08
ncm-et-6 2018-03-27 23:41 01:48:54 2024755 711 332 21 358 +162.96 +/- 17.44
ncm-et-7 2018-03-27 23:46 01:44:37 1992538 678 289 20 369 +145.85 +/- 16.95
ncm-et-8 2018-03-27 23:42 01:48:26 2001619 713 324 13 376 +162.43 +/- 16.73
ncm-et-9 2018-03-27 23:39 3767:53:50 2000973 2417 1097 70 1250 +157.62 +/- 9.28
ncm-et-10 2018-03-27 23:41 3767:51:58 1980625 2367 1105 71 1191 +162.71 +/- 9.56
ncm-et-11 2018-03-27 23:42 01:48:36 2009977 702 329 15 358 +167.22 +/- 17.30
ncm-et-12 2018-03-27 23:41 01:49:11 2021081 726 359 19 348 +176.47 +/- 17.77
ncm-et-13 2018-03-27 23:40 3767:53:09 1985123 2383 1132 75 1176 +165.61 +/- 9.66
ncm-et-14 2018-03-27 23:41 01:49:55 2020351 725 326 26 373 +152.93 +/- 17.10
ncm-et-15 2018-03-27 23:42 3767:50:57 1971551 2360 1079 70 1211 +158.74 +/- 9.45
ncm-et-16 2018-03-27 23:40 01:50:05 1960272 702 294 17 391 +144.96 +/- 16.31
  20000 9200 596 10204 +159.87 +/- 3.26

Test Detail

ID Host Started (UTC) Duration Base NPS Games Wins Losses Draws Elo CLI PGN
12886 ncm-et-9 2018-03-27 23:39 01:15:34 2027702 500 223 16 261 +153.02 +/- 20.36 Show
12887 ncm-et-5 2018-03-27 23:40 01:14:14 2024264 500 236 19 245 +161.49 +/- 21.29 Show
12888 ncm-et-13 2018-03-27 23:40 01:13:30 2026227 500 229 19 252 +155.54 +/- 20.91 Show
12889 ncm-et-16 2018-03-27 23:40 01:18:15 1897914 500 211 13 276 +145.54 +/- 19.49 Show
12890 ncm-et-14 2018-03-27 23:41 01:14:49 2018395 500 224 18 258 +152.18 +/- 20.57 Show
12891 ncm-et-12 2018-03-27 23:41 01:14:21 2021814 500 246 13 241 +175.44 +/- 21.35 Show
12892 ncm-et-4 2018-03-27 23:41 01:15:07 2024755 500 231 18 251 +158.08 +/- 20.94 Show
12893 ncm-et-10 2018-03-27 23:41 01:14:35 2018721 500 233 15 252 +162.35 +/- 20.81 Show
12894 ncm-et-6 2018-03-27 23:41 01:14:52 2023610 500 230 19 251 +156.39 +/- 20.97 Show
12895 ncm-et-11 2018-03-27 23:42 01:16:46 2002853 500 231 12 257 +163.21 +/- 20.45 Show
12896 ncm-et-8 2018-03-27 23:42 01:15:00 2020510 500 230 11 259 +163.21 +/- 20.31 Show
12897 ncm-et-15 2018-03-27 23:42 01:14:12 2018558 500 225 19 256 +152.18 +/- 20.70 Show
12898 ncm-et-3 2018-03-27 23:43 01:16:24 2023447 500 233 10 257 +166.66 +/- 20.38 Show
12899 ncm-et-7 2018-03-27 23:46 01:15:50 1997087 500 219 18 263 +148.02 +/- 20.31 Show
12900 ncm-et-13 2018-03-28 00:55 00:35:38 2024753 232 104 7 121 +154.74 +/- 29.92 Show
12901 ncm-et-5 2018-03-28 00:55 00:35:38 2022303 236 99 7 130 +143.00 +/- 28.55 Show
12902 ncm-et-9 2018-03-28 00:56 00:34:32 2025736 230 99 9 122 +143.61 +/- 29.89 Show
12903 ncm-et-12 2018-03-28 00:56 00:33:45 2020347 226 113 6 107 +178.76 +/- 32.21 Show
12904 ncm-et-14 2018-03-28 00:56 00:34:02 2022306 225 102 8 115 +154.61 +/- 30.91 Show
12905 ncm-et-10 2018-03-28 00:57 00:33:24 2019371 223 97 4 122 +154.30 +/- 29.22 Show
12906 ncm-et-4 2018-03-28 00:57 00:33:03 1998433 216 104 10 102 +162.00 +/- 33.36 Show
12907 ncm-et-6 2018-03-28 00:57 00:32:57 2025900 211 102 2 107 +178.97 +/- 31.41 Show
12908 ncm-et-15 2018-03-28 00:58 00:32:42 2018722 212 101 7 104 +165.54 +/- 32.67 Show
12909 ncm-et-8 2018-03-28 00:58 00:32:20 1982727 213 94 2 117 +160.61 +/- 29.55 Show
12910 ncm-et-16 2018-03-28 01:00 00:30:46 2022630 202 83 4 115 +143.52 +/- 29.87 Show
12911 ncm-et-11 2018-03-28 01:00 00:30:45 2017100 202 98 3 101 +177.35 +/- 32.60 Show
12912 ncm-et-3 2018-03-28 01:00 00:30:01 2026554 194 84 3 107 +154.50 +/- 31.08 Show
12913 ncm-et-7 2018-03-28 01:03 00:27:43 1987988 178 70 2 106 +139.82 +/- 30.46 Show
59805 ncm-et-3 2018-08-31 19:14 01:16:32 1988454 500 243 11 246 +174.55 +/- 21.01 Show
59806 ncm-et-15 2018-08-31 19:14 01:17:43 1946330 500 214 13 273 +148.02 +/- 19.64 Show
59807 ncm-et-4 2018-08-31 19:14 01:15:44 1989876 500 232 22 246 +155.54 +/- 21.30 Show
59808 ncm-et-10 2018-08-31 19:14 01:16:40 1980598 500 234 17 249 +161.49 +/- 21.02 Show
59809 ncm-et-13 2018-08-31 19:14 01:16:34 1964006 500 226 19 255 +153.02 +/- 20.75 Show
59810 ncm-et-9 2018-08-31 19:15 01:15:16 1988454 500 230 12 258 +162.35 +/- 20.39 Show
59811 ncm-et-9 2018-08-31 20:31 01:16:00 1989085 500 228 14 258 +158.93 +/- 20.46 Show
59812 ncm-et-4 2018-08-31 20:31 01:15:45 1989719 500 250 14 236 +178.11 +/- 21.66 Show
59813 ncm-et-3 2018-08-31 20:32 01:14:45 1989875 500 206 14 280 +140.62 +/- 19.32 Show
59814 ncm-et-13 2018-08-31 20:32 01:17:43 1956658 500 263 9 228 +194.57 +/- 21.97 Show
59815 ncm-et-10 2018-08-31 20:32 01:18:12 1943564 500 233 13 254 +164.07 +/- 20.64 Show
59816 ncm-et-15 2018-08-31 20:33 01:17:11 1942321 500 220 11 269 +154.70 +/- 19.78 Show
59817 ncm-et-3 2018-08-31 21:48 01:14:28 1989874 500 222 14 264 +153.86 +/- 20.14 Show
59818 ncm-et-9 2018-08-31 21:48 01:15:33 1987037 500 232 17 251 +159.78 +/- 20.92 Show
59819 ncm-et-4 2018-08-31 21:48 01:16:13 1990981 500 229 26 245 +149.68 +/- 21.43 Show
59820 ncm-et-13 2018-08-31 21:51 01:17:36 1950326 500 248 15 237 +175.44 +/- 21.63 Show
59821 ncm-et-15 2018-08-31 21:51 01:17:42 1952960 500 249 13 238 +178.11 +/- 21.52 Show
59822 ncm-et-10 2018-08-31 21:51 01:18:28 1941518 500 252 15 233 +179.00 +/- 21.86 Show
59823 ncm-et-3 2018-08-31 23:03 00:29:59 1991614 194 85 7 102 +148.04 +/- 32.70 Show
59824 ncm-et-9 2018-08-31 23:05 00:28:29 1987823 187 85 2 100 +165.73 +/- 32.22 Show
59825 ncm-et-4 2018-08-31 23:05 00:28:04 1989243 176 80 4 92 +160.56 +/- 34.15 Show
59826 ncm-et-13 2018-08-31 23:10 00:23:46 1988770 151 62 6 83 +135.30 +/- 36.07 Show
59827 ncm-et-15 2018-08-31 23:10 00:23:11 1950417 148 70 7 71 +157.95 +/- 40.02 Show
59828 ncm-et-10 2018-08-31 23:11 00:22:14 1979977 144 56 7 81 +123.13 +/- 36.53 Show

Commit

Commit ID 9953bfff621b287f5e08139f51d6663c2620db1e
Author Jerry Donald Watson
Date 2018-03-27 22:31:47 UTC
Commit Type Elo
Change reduction/improving interaction

Adjust criterion for applying extra reduction if not improving.
We now add an extra ply of reduction if r > 1.0, instead of the
previous condition Reductions[NonPV][imp][d][mc] >= 2.

Why does this work? Previously, reductions when not improving had
a discontinuity as the depth and/or move count increases due to the
Reductions[NonPV][imp][d][mc] >= 2 condition. Hence, values of r
such that 0.5 < r < 1.5 would be mapped to a reduction of 1, while
1.5 < r < 2.5 would be mapped to a reduction of 3. This patch allows
values of r satisfying 1.0 < r < 1.5 to be mapped to a reduction of 2,
making the reduction formula more continuous.

STC:
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 35908 W: 7382 L: 7087 D: 21439
http://tests.stockfishchess.org/tests/view/5aba723a0ebc5902a4743e8f

LTC:
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 23087 W: 3584 L: 3378 D: 16125
http://tests.stockfishchess.org/tests/view/5aba89070ebc5902a4743ea9

Ideas for future work:
- We could look at retuning the LMR formula.
- We could look at adjusting the reductions in PV nodes if not improving.

Bench: 5326261