NCM plays each Stockfish dev build 20,000 times against Stockfish 14. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.
Host | Duration | Avg Base NPS | Games | WLD | Standard Elo | Ptnml(0-2) | Gamepair Elo |
---|---|---|---|---|---|---|---|
ncm-dbt-01 | 00:28:56 | 1496876 | 170 | 9 100 61 | -207.6 +/- 38.19 | 23 48 11 3 0 | -381.7 +/- 107.72 |
ncm-dbt-02 | 00:29:05 | 1487664 | 162 | 9 95 58 | -205.45 +/- 33.66 | 18 51 11 1 0 | -423.7 +/- 113.61 |
ncm-dbt-03 | 00:29:21 | 1504394 | 168 | 6 92 70 | -196.39 +/- 29.17 | 15 56 13 0 0 | -430.53 +/- 102.66 |
ncm-dbt-04 | 00:28:52 | 1500294 | 164 | 8 96 60 | -208.22 +/- 30.77 | 17 54 11 0 0 | -457.29 +/- 114.27 |
ncm-dbt-05 | 00:28:57 | 1501402 | 164 | 13 98 53 | -199.41 +/- 31.72 | 17 51 14 0 0 | -411.96 +/- 97.92 |
ncm-dbt-06 | 00:29:44 | 1499517 | 172 | 12 109 51 | -221.87 +/- 36.8 | 25 48 12 1 0 | -421.01 +/- 107.77 |
1000 | 57 590 353 | -206.49 +/- 13.63 | 115 308 72 5 0 | -419.61 +/- 40.65 |
ID | Host | Base NPS | Games | WLD | Standard Elo | Ptnml(0-2) | Gamepair Elo | CLI | PGN | ||
---|---|---|---|---|---|---|---|---|---|---|---|
178192 | ncm-dbt-04 | 1500294 | 164 | 8 96 60 | -208.22 +/- 30.77 | 17 54 11 0 0 | -457.29 +/- 114.27 | ↓ | |||
cutechess-cli \ -rounds 250 \ -games 2 \ -concurrency 9 \ -srand 3615506552 \ -pgnout ncm-dbt-20200820-1913-006.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20200820-1913 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a1ad8604a11459a94189f857e368d0fbb72da25d \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14 |
|||||||||||
178191 | ncm-dbt-06 | 1499517 | 172 | 12 109 51 | -221.87 +/- 36.8 | 25 48 12 1 0 | -421.01 +/- 107.77 | ↓ | |||
cutechess-cli \ -rounds 250 \ -games 2 \ -concurrency 9 \ -srand 2695651325 \ -pgnout ncm-dbt-20200820-1913-005.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20200820-1913 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a1ad8604a11459a94189f857e368d0fbb72da25d \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14 |
|||||||||||
178190 | ncm-dbt-05 | 1501402 | 164 | 13 98 53 | -199.41 +/- 31.72 | 17 51 14 0 0 | -411.96 +/- 97.92 | ↓ | |||
cutechess-cli \ -rounds 250 \ -games 2 \ -concurrency 9 \ -srand 4122595324 \ -pgnout ncm-dbt-20200820-1913-004.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20200820-1913 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a1ad8604a11459a94189f857e368d0fbb72da25d \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14 |
|||||||||||
178189 | ncm-dbt-01 | 1496876 | 170 | 9 100 61 | -207.6 +/- 38.19 | 23 48 11 3 0 | -381.7 +/- 107.72 | ↓ | |||
cutechess-cli \ -rounds 250 \ -games 2 \ -concurrency 9 \ -srand 2176372205 \ -pgnout ncm-dbt-20200820-1913-003.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20200820-1913 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a1ad8604a11459a94189f857e368d0fbb72da25d \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14 |
|||||||||||
178188 | ncm-dbt-02 | 1487664 | 162 | 9 95 58 | -205.45 +/- 33.66 | 18 51 11 1 0 | -423.7 +/- 113.61 | ↓ | |||
cutechess-cli \ -rounds 250 \ -games 2 \ -concurrency 9 \ -srand 1580233822 \ -pgnout ncm-dbt-20200820-1913-002.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20200820-1913 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a1ad8604a11459a94189f857e368d0fbb72da25d \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14 |
|||||||||||
178187 | ncm-dbt-03 | 1504394 | 168 | 6 92 70 | -196.39 +/- 29.17 | 15 56 13 0 0 | -430.53 +/- 102.66 | ↓ | |||
cutechess-cli \ -rounds 250 \ -games 2 \ -concurrency 9 \ -srand 2533096101 \ -pgnout ncm-dbt-20200820-1913-001.pgn \ -openings \ file=UHO_4060_v2.epd \ format=epd \ order=random \ -repeat \ -resign \ movecount=3 \ score=600 \ -draw \ movenumber=34 \ movecount=8 \ score=5 \ -each \ tc=30+0.3 \ timemargin=10000 \ proto=uci \ option.Hash=64 \ option.Threads=2 \ -engine \ name=20200820-1913 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=dev_build:a1ad8604a11459a94189f857e368d0fbb72da25d \ -engine \ name=sf14 \ cmd=docker \ arg=run \ arg=-i \ arg=--rm \ arg=--entrypoint=/engine \ arg=stockfish:14 |
Commit ID | a1ad8604a11459a94189f857e368d0fbb72da25d |
---|---|
Author | Joost VandeVondele |
Date | 2020-08-20 19:13:07 UTC |
Send error message as an UCI info string
some GUIs do not show the error message when the engine terminates in the no-net case, as it is send to cerr.
Instead send it as an info string, which the GUI will more likely display.
closes https://github.com/official-stockfish/Stockfish/pull/3031
No functional change.
|