Dev Builds » 20170101-0956

NCM plays each Stockfish dev build 20,000 times against Stockfish 7. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Notice: NCM upgraded all servers to to Ubuntu 18.04 (from 16.04) on October 23rd. Binaries compiled on the upgraded disto are skewing measurements upwards. We are working on a fix.

Results

Host Started (UTC) Duration Base NPS (Avg) Games Wins Losses Draws Elo
ncm-et-3 2018-03-06 20:24 01:46:29 2025327 713 230 23 460 +103.85 +/- 14.48
ncm-et-4 2018-03-06 20:24 01:46:30 2024263 721 226 37 458 +93.25 +/- 14.81
ncm-et-5 2018-03-06 20:24 01:46:40 2024264 716 208 27 481 +89.78 +/- 13.97
ncm-et-6 2018-03-06 20:24 01:46:30 2026803 724 195 37 492 +77.06 +/- 13.87
ncm-et-7 2018-03-06 20:24 01:46:28 1977447 721 229 27 465 +100.01 +/- 14.46
ncm-et-8 2018-03-06 20:24 01:46:38 1979507 703 213 36 454 +89.40 +/- 14.78
ncm-et-9 2018-03-06 20:24 01:46:30 2024018 717 208 34 475 +86.03 +/- 14.25
ncm-et-10 2018-03-06 20:24 01:46:30 2020673 707 184 30 493 +76.91 +/- 13.57
ncm-et-11 2018-03-06 20:24 01:46:28 2018965 718 204 28 486 +86.94 +/- 13.86
ncm-et-12 2018-03-06 20:24 01:46:01 1989095 699 195 42 462 +77.30 +/- 14.59
ncm-et-13 2018-03-06 20:24 01:46:30 2024754 730 222 29 479 +94.09 +/- 14.17
ncm-et-14 2018-03-06 20:24 01:46:36 2018316 710 210 22 478 +94.24 +/- 13.91
ncm-et-15 2018-03-06 20:24 01:46:26 2017259 713 193 38 482 +76.75 +/- 14.07
ncm-et-16 2018-03-06 20:24 01:45:57 2000549 708 213 30 465 +91.89 +/- 14.41
  10000 2930 440 6630 +88.37 +/- 3.80

Test Detail

ID Host Started (UTC) Duration Base NPS Games Wins Losses Draws Elo CLI PGN
5742 ncm-et-8 2018-03-06 20:24 01:14:05 2015714 500 161 31 308 +92.46 +/- 18.37 Show
5743 ncm-et-5 2018-03-06 20:24 01:13:04 2023772 500 140 22 338 +83.57 +/- 16.70 Show
5744 ncm-et-14 2018-03-06 20:24 01:14:02 2017749 500 152 14 334 +98.44 +/- 16.67 Show
5745 ncm-et-15 2018-03-06 20:24 01:13:39 2016447 500 127 22 351 +74.06 +/- 16.04 Show
5746 ncm-et-13 2018-03-06 20:24 01:12:02 2025245 500 149 21 330 +90.97 +/- 17.07 Show
5747 ncm-et-6 2018-03-06 20:24 01:13:03 2026227 500 140 23 337 +82.83 +/- 16.77 Show
5748 ncm-et-3 2018-03-06 20:24 01:13:21 2024427 500 158 18 324 +99.95 +/- 17.28 Show
5749 ncm-et-10 2018-03-06 20:24 01:13:56 2021325 500 133 22 345 +78.44 +/- 16.35 Show
5750 ncm-et-7 2018-03-06 20:24 01:12:48 2001412 500 150 20 330 +92.46 +/- 17.04 Show
5751 ncm-et-9 2018-03-06 20:24 01:13:36 2023773 500 150 24 326 +89.48 +/- 17.34 Show
5752 ncm-et-12 2018-03-06 20:24 01:14:08 1989406 500 141 23 336 +83.57 +/- 16.82 Show
5753 ncm-et-11 2018-03-06 20:24 01:12:45 2018721 500 138 21 341 +82.83 +/- 16.52 Show
5754 ncm-et-4 2018-03-06 20:24 01:12:56 2024590 500 135 26 339 +76.98 +/- 16.74 Show
5755 ncm-et-16 2018-03-06 20:24 01:13:18 2021652 500 147 20 333 +90.22 +/- 16.89 Show
5756 ncm-et-13 2018-03-06 21:37 00:33:04 2024263 230 73 8 149 +100.94 +/- 25.48 Show
5757 ncm-et-7 2018-03-06 21:38 00:32:16 1953482 221 79 7 135 +117.47 +/- 27.30 Show
5758 ncm-et-11 2018-03-06 21:38 00:32:19 2019209 218 66 7 145 +96.43 +/- 25.47 Show
5759 ncm-et-5 2018-03-06 21:38 00:32:13 2024755 216 68 5 143 +104.37 +/- 25.49 Show
5760 ncm-et-6 2018-03-06 21:38 00:32:04 2027379 224 55 14 155 +64.32 +/- 24.68 Show
5761 ncm-et-4 2018-03-06 21:38 00:32:10 2023936 221 91 11 119 +131.74 +/- 30.37 Show
5762 ncm-et-3 2018-03-06 21:39 00:31:44 2026226 213 72 5 136 +113.12 +/- 26.56 Show
5763 ncm-et-16 2018-03-06 21:39 00:31:16 1979446 208 66 10 132 +95.90 +/- 27.58 Show
5764 ncm-et-15 2018-03-06 21:39 00:31:22 2018070 213 66 16 131 +83.11 +/- 28.39 Show
5765 ncm-et-9 2018-03-06 21:39 00:31:30 2024263 217 58 10 149 +78.14 +/- 25.01 Show
5766 ncm-et-10 2018-03-06 21:39 00:31:10 2020021 207 51 8 148 +73.24 +/- 24.34 Show
5767 ncm-et-14 2018-03-06 21:39 00:31:10 2018883 210 58 8 144 +84.34 +/- 25.30 Show
5768 ncm-et-8 2018-03-06 21:39 00:31:08 1943300 203 52 5 146 +81.93 +/- 24.08 Show
5769 ncm-et-12 2018-03-06 21:40 00:30:28 1988783 199 54 19 126 +61.75 +/- 28.90 Show

Commit

Commit ID 881a9dfb0a8fec3b1472791e2d98415e4a9a182a
Author Sergei Antonov
Date 2017-01-01 09:56:46 UTC
Commit Type Simplification
Threefold repetition detection

Implement a threefold repetition detection. Below are the examples of
problems fixed by this change.

    Loosing move in a drawn position.
    position fen 8/k7/3p4/p2P1p2/P2P1P2/8/8/K7 w - - 0 1 moves a1a2 a7a8 a2a1
    The old code suggested a loosing move "bestmove a8a7", the new code suggests "bestmove a8b7" leading to a draw.

    Incorrect evaluation (happened in a real game in TCEC Season 9).
    position fen 4rbkr/1q3pp1/b3pn2/7p/1pN5/1P1BBP1P/P1R2QP1/3R2K1 w - - 5 31 moves e3d4 h8h6 d4e3
    The old code evaluated it as "cp 0", the new code evaluation is around "cp -50" which is adequate.

Brings 0.5-1 ELO gain. Passes [-3.00,1.00].

STC: http://tests.stockfishchess.org/tests/view/584ece040ebc5903140c5aea
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 47744 W: 8537 L: 8461 D: 30746

LTC: http://tests.stockfishchess.org/tests/view/584f134d0ebc5903140c5b37
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 36775 W: 4739 L: 4639 D: 27397

Patch has been rewritten into current form for simplification and
logic slightly changed so that return a draw score if the position
repeats once earlier but after or at the root, or repeats twice
strictly before the root. In its original form, repetition at root
was not returned as an immediate draw.

After retestimng testing both version with SPRT[-3, 1], both passed
succesfully, but this version was chosen becuase more natural. There is
an argument about MultiPV in which an extended draw at root may be sensible.
See discussion here:

   https://github.com/official-stockfish/Stockfish/pull/925

For documentation, current version passed both at STC and LTC:

STC
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 51562 W: 9314 L: 9245 D: 33003

LTC
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 115663 W: 14904 L: 14906 D: 85853

bench: 5468995

Tests From Startpos

Host Wins Losses Draws Elo
ncm-et-3 4992 1403 156 3433 +88.67 +/- 5.13
ncm-et-4 5008 1429 147 3432 +90.96 +/- 5.13
  10000 2832 303 6865 +89.82 +/- 3.63