NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.
| Host | Duration | Avg Base NPS | Games | WLD | Standard Elo | Ptnml(0-2) | Gamepair Elo |
|---|---|---|---|---|---|---|---|
| ncm-dbt-01 | 02:19:38 | 580974 | 1450 | 27 940 483 | -257.39 ± 11.26 | 223 467 35 0 0 | -642.67 ± 59.62 |
| ncm-dbt-02 | 02:19:45 | 587693 | 1426 | 25 938 463 | -263.56 ± 11.85 | 235 443 35 0 0 | -639.69 ± 59.62 |
| ncm-dbt-03 | 02:19:36 | 585322 | 1444 | 24 917 503 | -251.0 ± 10.71 | 203 488 30 1 0 | -657.87 ± 64.78 |
| ncm-dbt-04 | 02:19:06 | 566193 | 1432 | 22 935 475 | -261.99 ± 11.31 | 224 466 25 1 0 | -686.53 ± 71.59 |
| ncm-dbt-05 | 02:18:31 | 583897 | 1412 | 14 921 477 | -264.8 ± 11.26 | 223 461 22 0 0 | -720.22 ± 77.08 |
| 7164 | 112 4651 2401 | -259.67 ± 5.04 | 1108 2325 147 2 0 | -666.77 ± 28.31 | |||
| ID | Host | Base NPS | Games | WLD | Standard Elo | Ptnml(0-2) | Gamepair Elo | CLI | PGN | ||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 450676 | ncm-dbt-05 | 581777 | 412 | 3 267 142 | -263.87 ± 20.21 | 63 138 5 0 0 | -764.18 ± 230.48 | ||||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 2037113567 \
-pgnout ncm-dbt-20200610-1107-015.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20200610-1107 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:3af083a7cd9be1659f1d8a39a65e33b87608f762 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 450675 | ncm-dbt-02 | 586901 | 426 | 6 274 146 | -257.07 ± 21.02 | 66 136 11 0 0 | -630.63 ± 116.56 | ||||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 3149707192 \
-pgnout ncm-dbt-20200610-1107-014.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20200610-1107 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:3af083a7cd9be1659f1d8a39a65e33b87608f762 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 450674 | ncm-dbt-04 | 567522 | 432 | 4 291 137 | -278.14 ± 21.49 | 76 135 5 0 0 | -772.51 ± 230.75 | ||||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 2087836398 \
-pgnout ncm-dbt-20200610-1107-013.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20200610-1107 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:3af083a7cd9be1659f1d8a39a65e33b87608f762 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 450673 | ncm-dbt-01 | 580862 | 450 | 11 293 146 | -255.67 ± 20.99 | 71 140 14 0 0 | -597.32 ± 99.95 | ||||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 2165204425 \
-pgnout ncm-dbt-20200610-1107-012.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20200610-1107 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:3af083a7cd9be1659f1d8a39a65e33b87608f762 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 450672 | ncm-dbt-03 | 581902 | 444 | 8 276 160 | -242.78 ± 18.17 | 56 156 10 0 0 | -654.96 ± 124.39 | ||||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 3065649673 \
-pgnout ncm-dbt-20200610-1107-011.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20200610-1107 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:3af083a7cd9be1659f1d8a39a65e33b87608f762 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 450671 | ncm-dbt-05 | 585042 | 500 | 3 321 176 | -261.07 ± 17.97 | 74 170 6 0 0 | -766.17 ± 188.02 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 1995244198 \
-pgnout ncm-dbt-20200610-1107-010.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20200610-1107 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:3af083a7cd9be1659f1d8a39a65e33b87608f762 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 450670 | ncm-dbt-02 | 587537 | 500 | 6 337 157 | -276.68 ± 20.45 | 89 153 8 0 0 | -715.51 ± 146.53 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 1584454443 \
-pgnout ncm-dbt-20200610-1107-009.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20200610-1107 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:3af083a7cd9be1659f1d8a39a65e33b87608f762 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 450669 | ncm-dbt-04 | 564683 | 500 | 11 322 167 | -253.02 ± 19.17 | 75 161 14 0 0 | -616.18 ± 100.06 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 151257120 \
-pgnout ncm-dbt-20200610-1107-008.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20200610-1107 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:3af083a7cd9be1659f1d8a39a65e33b87608f762 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 450668 | ncm-dbt-03 | 588558 | 500 | 3 326 171 | -266.97 ± 19.27 | 80 164 5 1 0 | -739.1 ± 208.89 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 269009437 \
-pgnout ncm-dbt-20200610-1107-007.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20200610-1107 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:3af083a7cd9be1659f1d8a39a65e33b87608f762 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 450667 | ncm-dbt-01 | 579909 | 500 | 7 326 167 | -262.24 ± 18.7 | 77 165 8 0 0 | -715.51 ± 146.53 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 3933279675 \
-pgnout ncm-dbt-20200610-1107-006.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20200610-1107 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:3af083a7cd9be1659f1d8a39a65e33b87608f762 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 450666 | ncm-dbt-05 | 584874 | 500 | 8 333 159 | -269.36 ± 20.41 | 86 153 11 0 0 | -659.13 ± 116.77 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 2491735561 \
-pgnout ncm-dbt-20200610-1107-005.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20200610-1107 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:3af083a7cd9be1659f1d8a39a65e33b87608f762 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 450665 | ncm-dbt-04 | 566375 | 500 | 7 322 171 | -257.59 ± 18.4 | 73 170 6 1 0 | -715.55 ± 177.2 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 2772872691 \
-pgnout ncm-dbt-20200610-1107-004.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20200610-1107 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:3af083a7cd9be1659f1d8a39a65e33b87608f762 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 450664 | ncm-dbt-02 | 588643 | 500 | 13 327 160 | -256.44 ± 20.12 | 80 154 16 0 0 | -592.27 ± 92.24 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 1054221461 \
-pgnout ncm-dbt-20200610-1107-003.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20200610-1107 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:3af083a7cd9be1659f1d8a39a65e33b87608f762 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 450663 | ncm-dbt-01 | 582152 | 500 | 9 321 170 | -254.15 ± 19.05 | 75 162 13 0 0 | -629.41 ± 104.8 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 45881744 \
-pgnout ncm-dbt-20200610-1107-002.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20200610-1107 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:3af083a7cd9be1659f1d8a39a65e33b87608f762 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| 450662 | ncm-dbt-03 | 585506 | 500 | 13 315 172 | -243.0 ± 18.13 | 67 168 15 0 0 | -603.84 ± 95.91 | ↓ | |||
cutechess-cli \
-rounds 266 \
-games 2 \
-concurrency 16 \
-srand 1962859106 \
-pgnout ncm-dbt-20200610-1107-001.pgn \
-openings \
file=UHO_4060_v2.epd \
format=epd \
order=random \
-repeat \
-resign \
movecount=3 \
score=600 \
-draw \
movenumber=34 \
movecount=8 \
score=5 \
-each \
tc=30+0.3 \
timemargin=10000 \
proto=uci \
option.Hash=128 \
option.Threads=8 \
-engine \
name=20200610-1107 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=dev_build:3af083a7cd9be1659f1d8a39a65e33b87608f762 \
-engine \
name=sf15 \
cmd=docker \
arg=run \
arg=-i \
arg=--rm \
arg=--entrypoint=/engine \
arg=stockfish:15
|
|||||||||||
| Commit ID | 3af083a7cd9be1659f1d8a39a65e33b87608f762 |
|---|---|
| Author | Stéphane Nicolet |
| Date | 2020-06-10 11:07:12 UTC |
|
Improve the anti-shuffling policy
We replace the current decrease of the complexity term in initiative
when shuffling by a direct damping of the evaluation. This scheme may
have two benefits over the initiative approach:
a) the damping effect is more brutal for fortresses with heavy pieces
on the board, because the initiative term is almost an endgame term;
b) the initiative implementation had a funny side effect, almost a bug,
in the rare positions where mg > 0, eg < 0 and the tampered eval
returned a positive value (ie with heavy pieces still on the board):
sending eg to zero via shuffling would **increase** the tampered
eval instead of decreasing it, which is somewhat illogical. This
patch avoids this phenomenon.
STC:
LLR: 2.94 (-2.94,2.94) {-0.50,1.50}
Total: 43072 W: 8373 L: 8121 D: 26578
Ptnml(0-2): 729, 4954, 9940, 5162, 751
https://tests.stockfishchess.org/tests/view/5ee008ebf29b40b0fc95ade2
LTC:
LLR: 2.94 (-2.94,2.94) {0.25,1.75}
Total: 37376 W: 4816 L: 4543 D: 28017
Ptnml(0-2): 259, 3329, 11286, 3508, 306
https://tests.stockfishchess.org/tests/view/5ee03b06f29b40b0fc95ae0c
Closes https://github.com/official-stockfish/Stockfish/pull/2727
Bench: 4757174
|
|