I was recently triggered to investigate (finally!) a new Leela feature called WDL Contempt. This feature was beautifully described in this blog article (The Lc0 v0.30.0 WDL rescale/contempt implementation – Leela Chess Zero (lczero.org)) by the Leela team, but the implementation is slightly fiddly so you need an hour of quiet time to set things up right.
The idea of WDL Contempt is that you tell Leela “Look I’m playing White against an opponent rated 400 ELO lower than me. Can you suggest some ideas in the opening to maximise my winning chances against such an opponent?” During analysis, Leela will look to sharpen the play to a degree appropriate to the ELO difference.
There are a couple of things to mention about this. Firstly, the assumption is that the weaker your opponent, the more wildly and tactically you want to play. That’s not always the case of course – it depends somewhat on your own strengths and those of your opponent. Secondly, this feature is not changing Leela’s evaluation: it’s a feature that develops as Leela searches deeper. That means that Leela should be searching about 10,000 nodes per move at least.
There is more however, and this is where it gets really interesting. You can then get Leela to play its recommendation against you, with its playing style altered by the WDL Contempt settings used to generate the opening idea. This is important because if you throw a Leela idea generated by WDL Contempt to Stockfish and Komodo, they will run away screaming and desperately try and claw their way back to a worse but holdable endgame! You need an engine that “believes” in its idea and plays it with the goal of achieving an advantage.
In a subsequent article I will demonstrate how this works in practice. My experience was pretty cool! Essentially a couple of minutes of clicking generated an outrageous and almost-never-seen idea in a well-known line which was then supplemented by the comprehensive demolition of an eminent English grandmaster 😉
This article focuses on how to set things up. The basic things that you need are
1. The Nibbler GUI and
2. A Leela installation.
I won’t go into how to do those 2 basics because instructions for precisely that are available from the free supplementary material I made for my book “The Silicon Road to Chess Improvement”. This supplementary material can be downloaded here: https://www.newinchess.nl/Na/Downloads/SiliconRoad-Supplementary.zip
Having got those 2 essential requirements up and running, let’s tweak our Leela installation with WDL Contempt for the scenario described above.
We will tweak our Leela installation with some options that Leela will activate on startup. This is achieved by manually editing the engines.json file that regulates how engines behave inside Nibbler. The easiest way to find this file is to open the Dev menu from within Nibbler and click on show engines.json.
Clicking on that will open Windows Explorer in the correct directory.
As you can see, I often make a backup copy of this file before editing it – old IT guy’s habit 😉
Before you make any changes to the engines.json file, CLOSE Nibbler! Otherwise none of your changes will be saved!
Most likely you don’t have any file association for .json files but you can right-click the file and choose Open with and then select the trusty Notepad!
Basically the engines.json file is a definition of all of the engines you can select within Nibbler. What you see here…
…is displayed in the Nibbler GUI under the menu Engine – Choose Known Engine
The most important point to understand is that the specific startup options you need to define for WDL Contempt are put into the options branch of the specific engine definition in the engines.json file. I struggled for ages with that in the past before a very helpful and friendly Leela developer helped me out!
So let’s have a look at what you need to put into your Nibbler file to analyse from the White point of view with a 400 ELO difference with the Black player
The first entry defines where my Leela installation is stored.
You don’t need to type this in this odd notation yourself. Before you start to work on this, open Nibbler, go to the Engine menu and select Choose Engine and then navigate to where your lc0.exe executable is stored and click OK. This will create a plain vanilla entry in the engines.json file with which you can work later.
Edit: Everything in italics from now on are corrections / clarifications made after a discussion with the Leela developer responsible for the feature! The figures I picked are pretty OK to start with, but it’s definitely useful for future experiments to understand them better!
“WDLDrawRateReference”: 0.58 is the estimated draw rate in a balanced starting position. I’d read somewhere that 0.58 (which equates to 58%) was a good value to use but for the net I’m using (t2-768x15x24h-swa-5230000.pb, recommended download at the time from https://lczero.org/play/networks/bestnets/) then 0.63 is the optimal value. It apparently shouldn’t make a massive amount of difference though!
“WDLCalibrationElo”: 2700 is the White ELO. I’d written that the actual number doesn’t matter at all but it does make a difference (so be accurate!) Leela tries to estimate the overall accuracy (and thus expected draw rate) at the respective level, and is supposed to reflect fast rapid ratings. The main effect is that the same amount of contempt has a stronger effect at lower Elo.
“Contempt”: 400 means that White is 400 ELO stronger than the opponent. i.e., we have a 2700 player with White taking on a 2300 player as Black!
“ContemptMode”: “white_side_analysis” tells Leela that all its cunning and effort should be for White!
“WDLContemptAttenuation”: 0.5 This actually scales down contempt by half! So with “Contempt”: 400 and “WDLContemptAttenuation”: 0.5 used together, I was actually configuring 200 ELO Contempt! The Leela developer said that the line “WDLContemptAttenuation”: 0.5 could be removed which sets it at its default value of 1.0 (which means no scaling of WDL Contempt. In other words, “Contempt”: 400 really does mean 400 ELO.)
“ScoreType”: “WDL_mu” is now the default anyway so you can omit this line: it doesn’t have any effect.
Now there is an extra little caveat in there (it wouldn’t be IT if there wasn’t!). There is currently a small bug in the way that WDL Contempt is applied to ELO ratings below 2800 which basically doubles the Contempt value! So in my example, I specified a rating of 2700 with “Contempt”: 400 and “WDLContemptAttenuation”: 0.5; with this bug I ended up all the same with Contempt of 400! (which is what I intended, yeah!) A new release of Leela will fix that. Update 28-02-2024: There is a development version of Leela that is available here: https://ci.appveyor.com/api/buildjobs/qg3f608s3phgwdjk/artifacts/build%2Flc0.exe Just replace your lc0.exe with this one and Contempt should behave exactly as you expect!
Now let’s have a look at the settings we need when you need Leela to play White assuming the Black player is rated 400 ELO difference less.
The difference here is “ContemptMode”: “play” which tells Leela to play rather than analyse!
Do be careful with the syntax! You need commas in exactly the right place, the accolades need to be balanced etc. Just take an existing example and compare carefully to get it right!
After saving the file, you can then open Nibbler, and select the engine you want from the Engine – Known Engine menu
I want to play Leela this time, so I select the bottom engine!
Nibbler loads your Leela! To verify whether the right options have indeed been loaded, you can go once again to the Dev menu and select List sent options
Yep! That looks about perfect to me!
A final comment: the Leela developers felt that 400 was really pushing it for a WDL Contempt value! I’ll definitely take that on board and experiment with smaller my values in the future but from a chess point of view, I was pretty comfortable with the types of ideas that Leela is coming up with at these settings!
OK, we are ready to go! In the next article, we are going to look at a practical example of what Leela can help you discover with WDL Contempt enabled!
Thanks for the post. Could you please also share what would be the configuration for Black? and also if the opponent is higher rated.
For Black you just replace “ContemptMode”: “white_side_analysis” with “ContemptMode”: “black_side_analysis”. If your opponent is higher rated then you put e.g. “Contempt”: -400 to show that your opponent is 400 points higher than you.
Can you help with the syntax? I just can’t get it right. I get this error message: “Unexpected token ” in JSON at position 523″
This is my JSON:
“C:\\Users\\bealo\\Downloads\\lc0-v0.30.0-windows-cpu-dnnl\\lc0.exe”: {
“args”: [],
“options”: {
“WDLDrawRateReference”: 0.58,
“WDLCalibrationElo”: 2400,
“Contempt”: 400,
“ContemptMode”: “white_side_analysis”,
“WDLEvalObjectivity”: 1.0,
“WDLContemptAttenuation”: 0.5,
“ScoreType”: “WDL_mu”
},
“search_nodes”: null,
“search_nodes_special”: 1,
“limit_by_time”: false
}
}
Hi, at first sight that looks fine. What I’ve found is that it’s important to do the following. Starting from a blank engines.json, you select your engine (from the Engine menu and then Choose engine) and point that to the installation you want to use. That creates a default vanilla entry – just make sure that works. Then close Nibbler, go the engines.json file and add the parameters. And then restart Nibbler and see if that works. I also recommend using Notepad to edit the engines.json file. Any other editors sometimes add random stuff which you can’t see but that interferes. Hope that helps! Best Wishes, Matthew
Hello, the ” signs changed when I copy pasted from this page to notepad and I had the same kind of errors. Changing back the ” signs to the same that were in the original engines.json file did the trick. Hope it helps, I was able to recreate the famous g4, so now I can move on to exploring my own lines haha.
Thanks! That is really useful – I think it has to be something like that. What also helped for me once after doing lots of fiddling around was to select Analysis ~ Forget all analysis from the Nibbler menu. I kept on getting sensible values for everything until that point but afterwards all was good… Best Wishes, Matthew
Hello,
Sorry if 3 or 4 of my replies appears but the website keep telling me that I have already sent my replies but I don’t see it after refreshing the page… I got a few issues similar as yours after copy pasting the changes to make from this page to notepad. I found that the ” signs changed to a slightly different format. Changing the ” signs back to the same that were in the original engines.json file did the trick. Hope it helps, I was able to recreate the famous g4 Semi-Slav grob and now am having a blast in the open Sicilian. Below 1800, Leela wants to play the following line (1. e4 c5 2. Nf3 Nc6 3. d4 cxd4 4. Nxd4 e5 5. Nb5 d6 6. Bg5 !), which would never work at master level I am sure but will be a lot of fun OTB at my local club if I try that haha ! I just tried it against the Maia9 bot on Lichess and got a quick checkmate !
Hi, no worries! We are currently on moderated posts due to an impressive amount of SPAM recently! Haha, that Bg5 idea is funny! 🙂
Unless it was just the way you cut-and-pasted the JSON, you are missing a “{” character at the start (i.e. you have too many closing “}”s)
Hello, have you copy/paste the json configuration values from the blog? If so, be very careful with the italics, specially with quotation marks. Try rewriting the quotes by hand.
Yes that’s a very good tip. The Leela guys mention this too in their latest blog: https://lczero.org/blog/2024/03/gm-matthew-sadler-on-wdl-contempt/
Hi Matthew,
Thank you very much again for your great blog and videos.
It doesn’t look to work on my Nibble. I set the position 1.d4 d5 2.c4 c6 3.Nc3 Nf6 4.Nf3 e6 and press Analysis ==> Go
But after 25′, it still show 5.Bg5, 5.e3 or 5.cxd5 with a weight of 52%, while 5.g4 stays far away at 44.2%
My config seems to be the same than yours :
{
“C:\\Users\\Fabrice\\Documents\\ChessBase\\Engines.uci\\Lc0-2\\lc0.exe”: {
“args”: [],
“options”: {
“WDLDrawRateReference”: 0.58,
“WDLCalibrationElo”: 2700,
“Contempt”: 400,
“ContemptMode”: “white_side_analysis”,
“WDLEvalObjectivity”: 1,
“WDLContemptAttenuation”: 0.5,
“ScoreType”: “WDL_mu”
},
“search_nodes”: null,
“search_nodes_special”: 10000000,
“limit_by_time”: false
}
}
I checked the parameters with Dev > List sent options and all seems ok
I have this Network file: 50dacb37fa30c5d27c32f76b6a0756a2ec776e76b9a82c59f0ef32b9e2685c2c
I don’t know if it is a good one, but I downloaded it this week.
Hmmm… there’s only one other thing I can think of – you do have the latest 0.30 Leela executable? https://lczero.org/play/download/ This WDL Contempt was only introduced in this later version.
Yes, the LC0.exe (CUDA) is from 21/07/23
Analysis shows WDL values, but without priority
Hi Matthew,
Always a pleasure to read your posts and see your engine videos !
Have you seen that Komodo Dragon seems to have a contempt feature implemented too (see https://komodochess.com/store/pages.php?cmsid=14) ?
Ah thanks! Lovely to hear! Ah yes – someone mentioned to me recently that Dragon personalities were really worth playing with but never looked at them yet! Best Wishes, Matthew
I was very much gassed up over this post, but nothing worked for me. Perhaps my laptop is not strong enough to make work even the new release. It would be quite helpful if you (or the developers) can explain it better, or perhaps send it as a text to just copy&paste into it…
It shouldn’t have anything to do with your laptop: if you can run Leela, then you can do this. A lot of people havce got this working using these instructions (and the ones on https://lczero.org/blog/2023/07/the-lc0-v0.30.0-wdl-rescale/contempt-implementation/) so I think they are OK. Things to check are:
1. Using the 0.30 Leela version
2. List Sent Options in Nibbler to be sure that the options in the enginesjson file are being applied
3. Check a few of the other comments to this post. People have been having problems with the “” when copying from other text formats so best just to copy those from existing entries in the file.
Hope that helps, Best Wishes, Matthew
It works now with a correct Network file 🙂 ==> t2-768x15x24h-swa-5230000.pb.gz
Thank you very much!
Ah super! 🙂 🙂 Hope you have lots of fun with it! Best Wishes, Matthew
Hi,
Very interesting stuff. Any idea on how I would set this up using the Leela engine in the Chessbase UCI interface?
Thanks!
Hey, I haven’t tried it yet myself but I have heard that it is possible. When you use the right Leela version (0.30+) then you see all the Contempt values exposed in the parameters of the engine. So you can change them there instead of fiddling around with a config file in Nibbler 🙂
Thanks! I’ll give it a try.