Dev Builds » 20170408-0002

NCM plays each Stockfish dev build 20,000 times against Stockfish 7. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Notice: NCM upgraded all servers to to Ubuntu 18.04 (from 16.04) on October 23rd. Binaries compiled on the upgraded distro are skewing measurements upwards. We are working on a fix.

Results

Host Started (UTC) Duration Base NPS (Avg) Games Wins Losses Draws Elo
ncm-et-3 2018-01-23 01:05 7515:31:47 2009912 6693 2107 322 4264 +94.96 +/- 4.83
ncm-et-4 2018-01-23 01:05 7515:31:52 2009658 6649 2034 283 4332 +93.70 +/- 4.73
ncm-et-9 2018-12-02 00:24 04:12:16 1972770 1661 514 68 1079 +95.64 +/- 9.49
ncm-et-10 2018-12-02 00:24 04:12:35 1965122 1663 551 64 1048 +104.81 +/- 9.72
ncm-et-13 2018-12-02 00:24 04:12:40 1972848 1672 519 76 1077 +94.30 +/- 9.56
ncm-et-15 2018-12-02 00:24 04:12:43 1964459 1662 519 86 1057 +92.65 +/- 9.74
  20000 6244 899 12857 +95.16 +/- 2.77

Test Detail

ID Host Started (UTC) Duration Base NPS Games Wins Losses Draws Elo CLI PGN
318 ncm-et-3 2018-01-23 01:05 01:13:41 2028194 500 144 32 324 +79.17 +/- 17.61 Show
319 ncm-et-4 2018-01-23 01:05 01:12:46 2028358 500 156 25 319 +93.20 +/- 17.70 Show
320 ncm-et-4 2018-01-23 02:19 01:11:49 2028030 500 133 14 353 +84.31 +/- 15.72 Show
321 ncm-et-3 2018-01-23 02:19 01:12:25 2027866 500 164 21 315 +102.22 +/- 17.80 Show
322 ncm-et-4 2018-01-23 03:32 01:13:42 2028522 500 155 24 321 +93.20 +/- 17.58 Show
323 ncm-et-3 2018-01-23 03:33 01:12:27 2027210 500 157 25 318 +93.95 +/- 17.75 Show
324 ncm-et-4 2018-01-23 04:46 01:14:03 2027704 500 158 20 322 +98.44 +/- 17.43 Show
325 ncm-et-3 2018-01-23 04:46 01:12:59 2028030 500 151 27 322 +88.00 +/- 17.60 Show
326 ncm-et-3 2018-01-23 06:01 01:10:37 2027374 500 146 26 328 +85.04 +/- 17.29 Show
327 ncm-et-4 2018-01-23 06:02 01:12:54 2027374 500 134 27 339 +75.52 +/- 16.76 Show
328 ncm-et-3 2018-01-23 07:12 01:13:58 2026718 500 148 29 323 +84.31 +/- 17.60 Show
329 ncm-et-4 2018-01-23 07:16 01:13:47 2026230 500 174 24 302 +107.54 +/- 18.51 Show
330 ncm-et-3 2018-01-23 08:27 01:12:58 2026719 500 166 24 310 +101.46 +/- 18.12 Show
331 ncm-et-4 2018-01-23 08:30 01:12:29 2027538 500 140 26 334 +80.63 +/- 16.99 Show
332 ncm-et-3 2018-01-23 09:41 01:12:52 2024427 500 159 23 318 +96.94 +/- 17.70 Show
333 ncm-et-4 2018-01-23 09:44 01:13:55 2025899 500 149 22 329 +90.22 +/- 17.14 Show
334 ncm-et-3 2018-01-23 10:55 01:13:49 2025735 500 150 23 327 +90.22 +/- 17.26 Show
335 ncm-et-4 2018-01-23 10:59 01:12:42 2026227 500 138 22 340 +82.10 +/- 16.60 Show
336 ncm-et-3 2018-01-23 12:10 01:13:07 2024099 500 174 21 305 +109.83 +/- 18.29 Show
337 ncm-et-4 2018-01-23 12:13 01:12:27 2029343 496 163 15 318 +106.92 +/- 17.43 Show
338 ncm-et-3 2018-01-23 13:25 00:00:52 2027866 4 1 0 3 +88.74 +/- 172.51 Show
114352 ncm-et-15 2018-12-02 00:24 01:15:32 1976259 500 142 17 341 +88.74 +/- 16.41 Show
114353 ncm-et-13 2018-12-02 00:24 01:15:33 1965534 500 164 26 310 +98.44 +/- 18.16 Show
114354 ncm-et-3 2018-12-02 00:24 01:13:40 1960615 500 172 29 299 +102.22 +/- 18.76 Show
114355 ncm-et-10 2018-12-02 00:24 01:16:14 1957098 500 156 27 317 +91.71 +/- 17.85 Show
114356 ncm-et-4 2018-12-02 00:24 01:16:47 1958475 500 168 16 316 +109.07 +/- 17.61 Show
114357 ncm-et-9 2018-12-02 00:24 01:15:03 1967398 500 138 20 342 +83.57 +/- 16.44 Show
114358 ncm-et-3 2018-12-02 01:39 01:14:24 1965841 500 174 18 308 +112.14 +/- 18.06 Show
114359 ncm-et-9 2018-12-02 01:41 01:15:04 1984875 500 153 22 325 +93.20 +/- 17.34 Show
114360 ncm-et-15 2018-12-02 01:41 01:15:50 1958951 500 168 30 302 +98.44 +/- 18.64 Show
114361 ncm-et-13 2018-12-02 01:41 01:13:44 1969430 500 154 19 327 +96.19 +/- 17.16 Show
114362 ncm-et-10 2018-12-02 01:42 01:15:42 1968312 500 179 12 309 +120.67 +/- 17.82 Show
114363 ncm-et-4 2018-12-02 01:42 01:13:04 1962153 500 155 20 325 +96.19 +/- 17.28 Show
114364 ncm-et-3 2018-12-02 02:54 01:13:51 1964149 500 140 16 344 +88.00 +/- 16.23 Show
114365 ncm-et-13 2018-12-02 02:56 01:14:33 1972038 500 153 23 324 +92.46 +/- 17.41 Show
114366 ncm-et-4 2018-12-02 02:56 01:15:32 1966311 500 158 22 320 +96.94 +/- 17.58 Show
114367 ncm-et-9 2018-12-02 02:57 01:14:38 1966466 500 171 18 311 +109.83 +/- 17.91 Show
114368 ncm-et-15 2018-12-02 02:58 01:13:39 1961228 500 157 33 310 +88.00 +/- 18.31 Show
114369 ncm-et-10 2018-12-02 02:58 01:12:50 1967076 500 151 20 329 +93.20 +/- 17.09 Show
114370 ncm-et-3 2018-12-02 04:09 00:27:08 1963839 189 61 8 120 +100.11 +/- 28.81 Show
114371 ncm-et-13 2018-12-02 04:11 00:25:39 1984390 172 48 8 116 +82.30 +/- 28.64 Show
114372 ncm-et-10 2018-12-02 04:12 00:24:37 1968003 163 65 5 93 +134.19 +/- 33.55 Show
114373 ncm-et-15 2018-12-02 04:12 00:24:29 1961396 162 52 6 104 +101.44 +/- 30.70 Show
114374 ncm-et-9 2018-12-02 04:12 00:24:19 1972339 161 52 8 101 +97.43 +/- 31.72 Show
114375 ncm-et-4 2018-12-02 04:13 00:23:44 1973052 153 53 6 94 +110.29 +/- 32.89 Show

Commit

Commit ID 35b77b120ef5ca5d9db848e02e42642b1a93e982
Author VoyagerOne
Date 2017-04-08 00:02:31 UTC
Commit Type Simplification
Standardize stat penalty

STC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 90631 W: 16325 L: 16323 D: 57983

LTC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 97679 W: 12779 L: 12759 D: 72141

Bench: 6340591

Closes #1053

Tests From Startpos

Host Wins Losses Draws Elo
ncm-et-3 5003 1582 134 3287 +103.51 +/- 5.34
ncm-et-4 4997 1665 141 3191 +109.44 +/- 5.50
  10000 3247 275 6478 +106.47 +/- 3.83