Russell Sherwood

Goldilocks and the Hash Table

Russell Sherwood  Tuesday, October 20, 2020

Following on from selecting the "best" version of an engine to use we now move onto setting the Hash table.

The Hash table is a storage location the engine uses to store position evaluations. If it sees the same position come up again it looks it up in the Hash table.

In theory, as long as looking up the position is faster than evaluating it again, the engine will search more efficiently. This means setting the Hash table to the "right" size can have a significant impact on both the efficiency and effectiveness of the engine search. 

If the table is too small then, it will fill up quickly and all positions will have to be calculated every time met, slowing down the search.

If the table is too large then it will be slower to search for the position that it is to recalculate it from scratch.

So how to we get the Goldilocks zone? It is a combination of:

  1. The Processor speed
  2. The available RAM
  3. The number of Cores being used to analyse
  4. The time being spent analysing each move

Processor Speed and Available RAM can be determined from the Windows system settings or via utilities such as CPU-Z. In general the amount of RAM being allocated to Hash should be no more than 50% the Total RAM of the system. Going above this level tends to slow down the PC in general

The number of Cores (not threads) you set for the engine - if you are using the PC mainly for analysis this will normally be the maximum, less one being used for the Windows system.

The time being spent on each move is a function of your analysis methods.  Rather than go through the formula, a small spreadsheet is attached which will estimate the setting to use.  

 

 

Download

Engine SettingsHashStockfish

Stockfish: Make the best choice!

Russell Sherwood  Saturday, October 17, 2020

Since the inception of the NNUE networks with the Stockfish framework, a very regular question has been: Which version should i use? BMI or Popcnt? AVX2 or SSSE?

A simple method to determine the best option is as follows: 

  1. Place all the candidate exe files into a folder.
  2. Navigate to the folder via File Explorer
  3. Double-click on the first engine. If it is not compatible with your hardware either nothing will happen or a black screen will open up briefly and then close again. These versions of the engine can be ignored.
  4. If when you double-click the screen opens up type "Bench". This will run for a while and at the end you will see a "Nodes per second" result. Not this number and close the engine window.
  5. Repeat this process for all the exe files.
  6. The file with the highest 'Nodes per second" will be the version of the engine best suited for your hardware.

This process will work for all Stockfish variant engines and many more. In a test I saw a 44% difference in speed between the least and most suitable versions. This in practice means that an engine will get to your normal depth faster and may achieve the odd or two greater depth in the same time, which in the long run converts to elo gain.

EngineStockfish

British Championship - Entries to Date - Closing Date 31st August

Russell Sherwood  Saturday, August 15, 2020

If you believe you have entered and are not on the list , please get in touch ASAP

Asquith JENG
Bailey MENG
Beckett PJENG
Brotherton TENG
Bruce RENG
Clark SENG
Cowan AENG
Elwood DENG
Franks KENG
Gardner AENG
Graham OENG
Grayland SJENG
Griling CENG
Grummitt WENG
Herman TENG
Hollands GENG
Illingworth JBENG
Kitson KENG
Law KENG
Lockett JENG
Lumley WENG
Maguire GENG
Marchant AENG
O'Mahoney MENG
Owen CENG
Perrin RENG
Pope PENG
Raby EBDENG
Rawlings AJCENG
Richardson DENG
Roberts AENG
Roberts SENG
Ruffle AENG
Squires MENG
Sutton AENG
Tibbert PHENG
Walsh HJENG
Ward RENG
Webster RENG
Wharam GENG
Wood BAENG
Beecham CRSCO
Buchan ASCO
Burridge RSCO
Clark SLSCO
Cumming DSCO
Hardwick MESCO
Lloyd GSCO
Whittaker IPSCO
Bailey S WLS
Bishop WWLS
Bullen AWLS
Jones GWLS
Jones IanWLS
Jones RWLS
Scott PWLS
Sherwood AWLS
Sherwood HWLS
Sherwood RWLS
Wakeham MRWLS
BCCC Entries

Setting up Stockfish NNEU in Fritz/Chessbase

Russell Sherwood  Tuesday, July 21, 2020

  1. Go to   https://github.com/nodchip/Stockfish/releases and download the latest copy – about 11mb in size and with a lot of different versions included
  2. Confirm what your CPU can handle – use CPU Z if unsure about which one you have.
  3.  Go to https://discord.com/channels/435943710472011776/733547343319924808 and download the network of choice
  4. Unzip the download from (1) in a folder of your choice
  5. Add a subfolder to that directory called eval
  6. Unzip or place the file nn.bin into the eval directory
  7. From Step 2 determine which version of the engine you need. For modern Ryzen machines avx2
filenameCPU instructionsSupported CPUs
sse2Intel Streaming SIMD Extensions 2>= AMD Athlon 64
sse3Intel Streaming SIMD Extensions 3>= Intel Pentium 4 (Prescott model F)
sse3-popcntIntel Streaming SIMD Extensions 3 + POPCNT>= AMD Phenom
ssse3Intel Supplemental Streaming SIMD Extensions 3>= Intel Core 2 Duo (Merom)
sse41Intel Streaming SIMD Extensions 4.1>= Intel Core 2 Duo (Penryn)
sse42Intel Streaming SIMD Extensions 4.2>= Intel Core i7 (Bloomfield), AMD FX (Bulldozer)
avx2Intel Advanced Vector Extensions 2>= AMD APU (Carrizo)
bmi2Intel Advanced Vector Extensions 2 + BMI2>= Intel Core i7 (Haswell)
avx512Intel Advanced Vector Extensions 512>= Intel Core i9/i7 (Skylake-X)
  1. Go into Fritz or Chessbase and “Create UCI Engine”
  2. Click on the three dots and navigate to the folder you created in (4)
  1. Select the file – you want the section after stockfish to be the correct one you identified in (7). So in my case a avx2. You then want the file with .profile in the name. This should have reduced the list to two choices – either a 256 or 384 file. You need the correct one to match the network. At this point select the 256

 

  1. The engine will now populate the details
  1. Click OK
  2. The engine should now be installed
  3. Click Add kibitzer to check the engine – it should be running at a decent rate 
  4. If get zero evals you don’t have a network!
  5. If it wont run at all its probably the wrong version for your machine
  6. Parameter Setting: Click on the Engine/ Advanced and Parameter
  7. Hash – half of your total ram should be max, generally a little less
  8. Contempt – set to zero
  9. Analysis Contempt – Off
  10. 50 Move rule off – ICCF don’t use it as of a couple of years ago
  11. Threads – set to your total minus a couple if you want to concentrate on chess analysis
  12. Latest versions have Eval File, older ones Eval Folder – if you want to switch nets this is the place to do it.

 

Notes of Caution

  1. Generally, NNUE nets are built on quite shallow evaluations and testing have been in very rapid games – its benefits are unknown in deep search. Cross check with vanilla SF any interesting moves it suggests
  2. The net needs to be kept up to date.
  3. If you go wrong installing to remove the engine you need to go to your Users/Logonname/app folder/Roaming/Chessbase/Engines.UCI and delete the offending File before trying again

Download

Stockfish NNUE

Return to the Engine Room

Russell Sherwood  Wednesday, July 8, 2020

It's been sometime since I was last in the Engine room and quite a bit has happened in the world of Chess Engine development in that time!

ASMfish is no more. This port of Stockfish into Assembler had proven very popular with enthusiasts due to its additional speed boost but it is now so far behind Stockfish in terms of development that any speed advantage pales into insignificance compared with the deficits in search and evaluation.

It appeared CFish had gone the same way but in recent weeks an effort has been made to bring the code up to date. CFish offers some speed benefits, so this is worth watching to see if the efforts are successful.

Houdini has had no new releases in several years but in recent months, what is allegedly the source code has appeared in public forums. Whilst the veracity of this is not of interest, it would be surprising to see an updated version of this engine appear in some form.

In terms of other engines outside of the main players the main newcomer has been Ethereal. Whilst not quite at the same level as the top engines it does have an interesting playing style.

Neural net development has continued to advance           

Significant work has been undertaken to make LC0 far easier to install and many specialised nets have been developed, Elo improvement has slowed, and development of Stockfish has been spurred on to compete with LC0.  A neural net is an essential part of an aspiring CC players arsenal, but it is now much easier to achieve for two reasons:

Fat Fritz has been updated and runs well on CPU based hardware (as opposed to LC0 which really requires a high-quality graphics card). Fat Fritz is based on a different network development method but does add to players options.

Stockfish development has thrown up two fascinating projects. Stockfish NNEU is a method to create a neural net which operates with Stockfish. It is early days yet, but early testing shows performance approaching Stockfish itself. In my own research it does show some interesting moves and runs much faster than a LC0 network but as I said early days!  The other project is Stockfish WDL, which is an attempt to generate a Win-Draw-Loss score for Stockfish. This is in exceedingly early days now but does appear to show some promise.

With the release of Fritz 17, we saw the appearance of the Fritz 17 engine (which I believe many users think is Fat Fritz!). Whilst a pleasing upgrade from Fritz 16 (which was simply Rybka modernised) this engine is a long way of the pace in CC terms and should not be utilised as a main engine

Honey is a specialised development of Stockfish which adds several features especially useful for CC players, for example Defensive mode and Deep Profound Analysis. Each release does provide several versions which include the best tweaks of several other engines. Personally, a favourite!

Eman is something of a controversial engine, for non-chess reasons I will not go into and ridiculously hard to get hold of but does provide interesting features including an ability to sniff out interesting moves. Well worth the effort to get hold of!

Bluefish is something of a marmite engine. Its “Bluefish” mode is somewhat misleading as it is simple reports the depth and node count incorrectly. It does, however, tend to utilise a narrower search giving a greater depth. Dependent on position this can either be a game winner or a game loser. Worth a look but personally I would never use it as a main engine

ShashChess has just updated as is an interesting engine, with former CC World Champions and Grandmasters involved in its development. It has three different modes, which either the engine self-selects, or the user can select, which approach the position differently. Well worth a look.

Raubfisch has been around for a long time and includes a lot of CC specific features, including ICCF Mode. Worth close examination.

Komodo has two modes – traditional and Monte Carlo. Both have a following with CC. If you want to use a Monte Carlo engine with a CPU then you probably need to use Komodo. Do not be fooled into thinking Stockfish is far superior simply based on ratings. Komodo plays a much more solid game and tends to be popular with those who prefer solid positions. It of course one of the few commercial engines in this review.

Rating Differences

Russell Sherwood  Tuesday, June 16, 2020

In the postbag (well the electronic one anyway!) I have been asked several questions about the ICCF rating system.  One wide misunderstanding  is about rating points being gained or lost playing against lower rated opposition. 

Looking into it two factors have the most underlying effect on the result – your k value and the difference in rating.

The k value is defined as part of the rating equation:

 

I put a K calculator into the spreadsheet to make life easier, but we can see from the above the k value gets smaller as your rating and number of games completed increase.

From looking at the formula we can see that the k value starts out around 25 and then trends down to around 10 for the majority of players – or 20 if your rating is below 2000. Where do you find your k value – either on the spreadsheet or in your rating forecast on ICCF.com

Once we know this we can determine that the number of rating points lost or gained in a game is calculated from your actual score in the game – your expect score multiplied by your k value.

Your actual score can only take one of three values 0,0.5 or 1. 

Your expected score comes from the difference between you and your opponents rating, by the formula:

Hmmmm…..

In practice this means

Rating difference

Diff%
2552%
5054%
10059%
15063%
20067%
25071%
30075%

 

By a very long and painful journey we come to a number of generalizations, which have been the questions I have been asked:

  1. You have to be rated more than 100 elo more than your opponent to lose more than 1 point from a draw (for most people with a K factor around the 10 mark)
  2. In a two game match (Friendlies in particular), this drops to 50 elo for a “double draw” costing more than 1  point
  3. If the difference in elo is less than 300, than 1.5/2 will be a net rating gain for the higher rated player 
  4. Once the gap is more than 300 elo then 2/2 is required to not lose rating points.

 

Download

Rating system

Neural Nets on the cheap

Russell Sherwood  Tuesday, June 9, 2020

As I finalize an article on setting up Neural Net based engines, it is worth sharing that you do not need to upgrade to an expensive new machine or graphics card to make use of neutral nets.

http://www.ezchess.org/

It is possible to put together a stand-alone unit for a little for a fraction of the cost of a high end PC.  The unit is not super-powered but will out perform most CPU powered versions.  If you are interested in this approach, get in touch 

 

 Thanks to SIM John Claridge for the tip on this!

 

p.s. An even cheaper alternative is https://leelanalysis.com/  which is currently free but in Beta Stage

NN

Counties and Districts Update

Russell Sherwood  Tuesday, June 9, 2020

The 2019-2020 C&D CCC is moving towards an interesting finish, with 20 games remaining on the top tier

https://www.iccf.com/event?id=83944

Hertfordshire top the table, having completed their games but with expected results Essex A look to be heading towards the title chased home by West Wales and Lancashire. The shock of the event is that former powerhouse Yorkshire look to be have been sucked into the relegation battle.

Probably the most interesting game of the event is https://www.iccf.com/game?id=1143988

 

https://www.iccf.com/event?id=83949  

In the 2nd Tier West Wales B look set for promotion 

Euro Team Cup Update

Russell Sherwood  Sunday, June 7, 2020

Following a number of new results our position continues to strengthen in this event.

Currently, we stand at 16-11, with 13 games still in play - so we only need to score 4.5 from 13 to advance, which is looking more than likely

Congratulations to Tony Balshaw, Paul Keevil, Phil Morgan, Fred Clough and Sean Denton in securing wins!

In the other fixtures.

Team AA ScoreB ScoreTeam BRemaing GamesLikely to Advance
Gemany136Noway21Gemany
Gemany1210Slovenia18
Slovenia18.58.5Noway13
 
Italy17.515.5Ukraine7Italy
Netherlands15.513.5Romania11To close to call
Russia1716France7Russia
Spain12.511.5Poland16Spain
Sweden1116Wales13Wales
Euro Team Cup

Interesting Annotated Games

Russell Sherwood  Friday, June 5, 2020

Tim Spanton publishes some interesting annotated games from the BWSTT on his blog

https://beauchess.blogspot.com/2020/06/more-lockdown-chess.html

6BWSTT
Welsh Correspondence Chess FederationBritish Correspondence Chess AssociationSchemingMind Internet Correspondence Chess ClubSocial Correspondence Chess AssociationNational Correspondence Chess ClubWelsh Chess UnionInternational Correspondence Chess Association