Dev Builds » 20160916-0615

Use this dev build

NCM plays each Stockfish dev build 20,000 times against Stockfish 7. This yields an approximate Elo difference and establishes confidence in the strength of the dev builds.

Summary

Host Duration Avg Base NPS Games Wins Losses Draws Elo
ncm-et-3 05:58:56 1987205 2451 624 140 1687 +69.52 +/- 7.48
ncm-et-4 05:58:24 1961808 2385 637 131 1617 +74.85 +/- 7.69
ncm-et-5 01:52:06 2019004 761 213 45 503 +77.98 +/- 13.99
ncm-et-6 01:52:02 2026390 791 199 29 563 +75.85 +/- 12.5
ncm-et-7 01:52:02 2001007 759 204 43 512 +74.83 +/- 13.71
ncm-et-8 01:52:02 1986794 754 190 29 535 +75.35 +/- 12.86
ncm-et-9 05:59:09 1990293 2452 635 152 1665 +69.34 +/- 7.6
ncm-et-10 05:58:28 1986913 2467 619 130 1718 +69.79 +/- 7.34
ncm-et-11 01:51:59 2004478 758 212 41 505 +79.75 +/- 13.87
ncm-et-12 01:51:56 1981511 769 206 43 520 +74.78 +/- 13.59
ncm-et-13 05:59:11 1990818 2456 643 135 1678 +72.92 +/- 7.52
ncm-et-14 01:52:07 2019859 768 224 51 493 +79.63 +/- 14.36
ncm-et-15 05:58:28 1986336 2429 636 144 1649 +71.36 +/- 7.63
20000 5242 1113 13645 +72.77 +/- 2.64

Test Detail

ID Host Started (UTC) Duration Base NPS Games Wins Losses Draws Elo CLI PGN
67262 ncm-et-4 2019-01-24 04:07 00:18:45 1972808 122 25 9 88 +45.83 +/- 32.17
67261 ncm-et-9 2019-01-24 04:01 00:24:41 1973895 170 46 7 117 +81.15 +/- 28.13
67260 ncm-et-13 2019-01-24 04:01 00:25:05 1977941 168 44 10 114 +71.3 +/- 29.09
67259 ncm-et-15 2019-01-24 04:00 00:25:09 1972807 170 53 8 109 +94.21 +/- 30.27
67258 ncm-et-3 2019-01-24 03:58 00:27:32 1964458 185 53 11 121 +80.28 +/- 28.72
67257 ncm-et-10 2019-01-24 03:57 00:28:11 1959081 185 48 11 126 +70.44 +/- 27.61
67256 ncm-et-4 2019-01-24 02:51 01:15:15 1960933 500 140 30 330 +77.71 +/- 17.3
67255 ncm-et-15 2019-01-24 02:47 01:11:49 1969555 500 126 36 338 +63.23 +/- 17.01
67254 ncm-et-9 2019-01-24 02:46 01:13:32 1977486 500 115 30 355 +59.64 +/- 16.04
67253 ncm-et-13 2019-01-24 02:45 01:14:58 1974840 500 124 27 349 +68.27 +/- 16.28
67252 ncm-et-10 2019-01-24 02:44 01:11:47 1984919 500 114 27 359 +61.08 +/- 15.77
67251 ncm-et-3 2019-01-24 02:44 01:12:56 1967874 500 115 30 355 +59.64 +/- 16.04
67250 ncm-et-4 2019-01-24 01:35 01:15:00 1939878 500 120 25 355 +66.82 +/- 15.93
67249 ncm-et-15 2019-01-24 01:31 01:14:45 1972198 500 132 27 341 +74.06 +/- 16.69
67248 ncm-et-9 2019-01-24 01:31 01:14:46 1985212 500 128 38 334 +63.23 +/- 17.24
67247 ncm-et-13 2019-01-24 01:30 01:13:17 1972518 500 144 35 321 +76.98 +/- 17.83
67246 ncm-et-10 2019-01-24 01:29 01:13:51 1967861 500 138 22 340 +82.1 +/- 16.62
67245 ncm-et-3 2019-01-24 01:29 01:14:14 1974860 500 149 28 323 +85.78 +/- 17.6
67244 ncm-et-4 2019-01-24 00:16 01:17:47 1893810 500 136 32 332 +73.34 +/- 17.24
67243 ncm-et-15 2019-01-24 00:16 01:14:47 1965534 500 118 31 351 +61.08 +/- 16.26
67242 ncm-et-10 2019-01-24 00:16 01:12:53 1966314 500 125 26 349 +69.71 +/- 16.26
67241 ncm-et-3 2019-01-24 00:15 01:12:17 1965382 500 129 21 350 +76.25 +/- 16.09
67240 ncm-et-13 2019-01-24 00:15 01:13:47 1968469 500 114 27 359 +61.08 +/- 15.77
67239 ncm-et-9 2019-01-24 00:15 01:14:11 1984722 500 143 34 323 +76.98 +/- 17.72
11168 ncm-et-11 2018-03-23 10:18 00:37:18 2018557 258 77 12 169 +89.46 +/- 24.06
11167 ncm-et-7 2018-03-23 10:17 00:38:00 1996484 259 71 16 172 +74.92 +/- 23.94
11166 ncm-et-5 2018-03-23 10:17 00:38:04 2023611 261 71 16 174 +74.33 +/- 23.75
11165 ncm-et-15 2018-03-23 10:16 00:38:36 2018882 259 62 14 183 +65.14 +/- 22.34
11164 ncm-et-8 2018-03-23 10:16 00:39:17 1950795 254 66 7 181 +82.2 +/- 21.86
11163 ncm-et-4 2018-03-23 10:15 00:39:24 1979486 263 77 15 171 +83.47 +/- 24.16
11162 ncm-et-14 2018-03-23 10:15 00:39:42 2019534 268 80 20 168 +79.12 +/- 24.94
11161 ncm-et-12 2018-03-23 10:15 00:40:23 1944140 269 72 15 182 +74.75 +/- 22.98
11160 ncm-et-9 2018-03-23 10:14 00:40:22 1996019 282 69 14 199 +68.64 +/- 21.37
11159 ncm-et-10 2018-03-23 10:14 00:40:24 2021488 282 65 9 208 +69.92 +/- 19.96
11158 ncm-et-3 2018-03-23 10:14 00:40:43 2024102 266 70 17 179 +70.16 +/- 23.35
11157 ncm-et-6 2018-03-23 10:14 00:41:05 2026882 291 70 14 207 +67.7 +/- 20.82
11156 ncm-et-13 2018-03-23 10:14 00:41:06 2025901 288 72 19 197 +64.67 +/- 22.09
11155 ncm-et-4 2018-03-23 09:01 01:12:13 2023936 500 139 20 341 +84.31 +/- 16.52
11154 ncm-et-12 2018-03-23 09:01 01:11:33 2018883 500 134 28 338 +74.79 +/- 16.86
11153 ncm-et-15 2018-03-23 09:01 01:13:22 2019045 500 145 28 327 +82.83 +/- 17.4
11152 ncm-et-3 2018-03-23 09:01 01:11:14 2026554 500 108 33 359 +52.51 +/- 15.88
11151 ncm-et-7 2018-03-23 09:01 01:14:02 2005531 500 133 27 340 +74.79 +/- 16.74
11150 ncm-et-11 2018-03-23 09:01 01:14:41 1990400 500 135 29 336 +74.79 +/- 16.98
11149 ncm-et-6 2018-03-23 09:01 01:10:57 2025899 500 129 15 356 +80.63 +/- 15.62
11148 ncm-et-10 2018-03-23 09:01 01:11:22 2021818 500 129 35 336 +66.1 +/- 17.09
11147 ncm-et-9 2018-03-23 09:01 01:11:37 2024426 500 134 29 337 +74.06 +/- 16.93
11146 ncm-et-13 2018-03-23 09:01 01:10:58 2025244 500 145 17 338 +90.97 +/- 16.58
11145 ncm-et-14 2018-03-23 09:01 01:12:25 2020184 500 144 31 325 +79.9 +/- 17.56
11144 ncm-et-5 2018-03-23 09:01 01:14:02 2014398 500 142 29 329 +79.9 +/- 17.33
11143 ncm-et-8 2018-03-23 09:01 01:12:45 2022793 500 124 22 354 +71.89 +/- 15.91

Commit

Commit ID 5c58d1f5cb4871595c07e6c2f6931780b5ac05b5
Author Marco Costalba
Date 2016-09-16 06:15:07 UTC
Use per-thread counterMoveHistory Drops a scalability bottleneck due to memory contention of a single shared table across threads. The effect starts to be sensible with a high number of threads. Specifically we have a small regression with 7 threads both at 60 and 180 seconds TC: 10000 @ 60+0.6 th 7 ELO: -2.46 +-3.2 (95%) LOS: 6.5% Total: 9896 W: 1037 L: 1107 D: 7752 5000 @ 180+0.6 th 7 ELO: -1.95 +-4.1 (95%) LOS: 17.7% Total: 5000 W: 444 L: 472 D: 4084 We have a regression because counterMoveHistory table is quite big and it takes time for a single thread to fill it. Sharing the table yields to a higher fill rate and better quality of moves and up to 7 threads the benefits of sharing more then compensate the loss in speed due to contention. Interestingly even with a 3X longer TC, so with more time for the single thread to catch up, the improvment is quite limited and below noise level. It seems we really need much longer TC to saturate the table. When we move to high threads number it's another story: 5000 @ 60+0.6 th 22 ELO: 3.49 +-4.3 (95%) LOS: 94.6% Total: 4880 W: 490 L: 441 D: 3949 2000 @ 60+0.6 th 32 ELO: 8.34 +-6.9 (95%) LOS: 99.1% Total: 2000 W: 229 L: 181 D: 1590 As expected the speed-up more than compensates the filling rate, and we expect that with tournament TC, where single thread is able to saturate the table, the difference will be even stronger. For instance for TCEC 9 super-final time control will be 180 minutes + 15 seconds and this scalability improvement seems definitely the way to go. So, summarizing: GOOD: Measured big improvement in high core scenario Suitable for TCEC 9 superfinal (big hardware, very long TC) Consistent and natural patch that extends to counterMoveHistory what we already do for remaining history tables, that are all per-thread Non functional change for the common case of a single core Very simple (just 6 lines modified, no added ones) BAD: Small regression (within 2-3 ELO) with few threads and short TC bench: 5341477
Copyright 2011–2024 Next Chess Move LLC