Dev Builds » 20210614-0724

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 15. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo

Test Detail

ID Host Base NPS Games WLD Standard Elo Ptnml(0-2) Gamepair Elo CLI PGN

Commit

Commit ID f8c779dbe538315aa6f65556d0acf11640558504
Author JWmer
Date 2021-06-14 07:24:07 UTC
Update default net to nn-8e47cf062333.nnue This net is the result of training on data used by the Leela project. More precisely, we shuffled T60 and T74 data kindly provided by borg (for different Tnn, the data is a result of Leela selfplay with differently sized Leela nets). The data is available at vondele's google drive: https://drive.google.com/drive/folders/1mftuzYdl9o6tBaceR3d_VBQIrgKJsFpl. The Leela data comes in small chunks of .binpack files. To shuffle them, we simply used a small python script to randomly rename the files, and then concatenated them using `cat`. As validation data we picked a file of T60 data. We will further investigate T74 data. The training for the NNUE architecture used 200 epochs with the Python trainer from the Stockfish project. Unlike the previous run we tried with this data, this run does not have adjusted scaling — not because we didn't want to, but because we forgot. However, this training randomly skips 40% more positions than previous run. The loss was very spiky and decreased slower than it does usually. Training loss: https://github.com/official-stockfish/images/blob/main/training-loss-8e47cf062333.png Validation loss: https://github.com/official-stockfish/images/blob/main/validation-loss-8e47cf062333.png This is the exact training command: python train.py --smart-fen-skipping --random-fen-skipping 14 --batch-size 16384 --threads 4 --num-workers 4 --gpus 1 trainingdata\training_data.binpack validationdata\val.binpack --- 10k STC result: ELO: 3.61 +-3.3 (95%) LOS: 98.4% Total: 10000 W: 1241 L: 1137 D: 7622 Ptnml(0-2): 68, 841, 3086, 929, 76 https://tests.stockfishchess.org/tests/view/60c67e50457376eb8bcaae70 10k LTC result: ELO: 2.71 +-2.4 (95%) LOS: 98.8% Total: 10000 W: 659 L: 581 D: 8760 Ptnml(0-2): 22, 485, 3900, 579, 14 https://tests.stockfishchess.org/tests/view/60c69deb457376eb8bcaae98 Passed LTC: LLR: 2.93 (-2.94,2.94) <0.50,3.50> Total: 9648 W: 685 L: 545 D: 8418 Ptnml(0-2): 22, 448, 3740, 596, 18 https://tests.stockfishchess.org/tests/view/60c6d41c457376eb8bcaaecf --- closes https://github.com/official-stockfish/Stockfish/pull/3550 Bench: 4877339
Copyright 2011–2024 Next Chess Move LLC