Answers.Body: <p>Lc0 is fully compatible with UCI. It's strength depends a ton on what hardware it's being run on. On any hardware (tested down to Raspberry Pi), it will be superhuman (over 2900 elo). With a RTX 2060 ($200ish) it should be roughly even with SF on 8 cores (although you may need to mess with settings a bit to get it there). A 2080ti puts decently (20 or so elo) ahead of 24 cores of SF. The current MultiPV mode is only a display tweak. It isn't weaker, but it doesn't give as good analysis of the non-primary lines. If you have more questions, the best place to ask is probably the Lc0 discord.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-06T09:42:41.807Z
Answers.Text: Thank you for benchmark information. By GTX 2060 you mean RTX 2060 using FP16 backend, right?
Answers.UserDisplayName: hoacin
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-06T15:16:49.067Z
Answers.Text: Correct. Updated to be more clear
Answers.UserDisplayName: Oscar Smith
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-06T07:26:53.437Z
Answers.LastActivityDate: 2019-06-06T15:17:11.993Z
Answers.LastEditDate: 2019-06-06T15:17:11.993Z
Answers.UserDisplayName: Oscar Smith
Answers.Body: <p>Can't comment on hardware but will say something about Leela's strengths and weaknesses -</p>
  
  <p>If you look through the games from the TCEC superfinal, sometimes Leela is brilliant. For example in this position from game 61:</p>
  
  <pre><code>[fen "r1bbnr2/pp1n1q1k/3p4/2pPp1pP/2P1PpP1/2NQ1N2/PP2BB2/2KR2R1 w - - 10 26"]
  </code></pre>
  
  <p>Stockfish (White) evaluated this as +0.63, while Leela was at -2.01. From the human point of view, I suspect most will prefer Black. That's because White has nothing on the kingside while Black is more capable of playing on the queenside. The game proved Leela right - it took a lot more moves for Stockfish to realize it was inferior, and even more moves after that for Stockfish to resign (it is a fantastic defender), but the fact that Leela saw it was superior so much earlier than Stockfish is very impressive. There are many similar games from the superfinal - Leela seems very capable of outplaying Stockfish positionally, often converting a Stockfish advantage into one of its own, even if the game eventually ends in a draw.</p>
  
  <p>Another advantage of Leela is multi-PV. MCTS engines get this for free, while Stockfish (which runs on AB) needs to invest computational power. If this matters for you then that's an additional advantage for Leela.</p>
  
  <p>However the infamous "Leela blunder" is still a thing. Take this position from game 81:</p>
  
  <pre><code>[fen "4rr2/2pb2bk/1p1p4/2nP3P/p1P1p3/P1N1Bq2/1P3N2/1K2Q1RR w - - 2 26"]
  </code></pre>
  
  <p>Leela just played 25...Rae8 and showed a +0.67 eval. It thinks its position is inferior, but not by that much. Its principal variation went 25. .. Rae8 26. h6 Be5 27. Rg7+ Kh8 28. Ka2 Rf7 29. Nfd1 Nd3 30. Qh4. Stockfish had the same moves, but saw that 28. Qg1! wins, and its eval shot up to +7.68 one move later. Leela remained blissfully unaware until Stockfish played Qg1, and resigned shortly afterwards. Stockfish virtually never makes this kind of one-move blunder.</p>
  
  <p>Another problem with Leela is that it doesn't seem to defend very well. When Stockfish loses, it defends grimly and it takes a long time for Leela to grind a win. On the other hand when Leela loses, it often implodes. In the ongoing TCEC Champions bonus, <a href="https://cd.tcecbeta.club/archive.html?season=15&amp;div=champions&amp;game=29#/" rel="nofollow noreferrer">its loss to Houdini 3 (!!) is illustrative</a>. Houdini 3 is an engine many years old, rated hundreds of elo below Stockfish (and ostensibly Leela). But Leela hated the starting position so much it managed to lose. Comparatively you would need to play hundreds if not thousands of games before Stockfish loses to Houdini 3.</p>
  
  <p>Finally a noticeable weakness of Leela's is that it isn't very good at beating inferior opponents. For example in the ongoing TCEC Champions bonus, which includes engines from many years ago, Leela is performing worse than 1-year old versions of Stockfish. In this, Leela is kind of similar to Petrosian - it doesn't win many games, but it's good in match play.</p>
  
  <p>In the end, I think Leela is inconsistent. In some positions it's much stronger than Stockfish, and in others it's just worse. As a practical player, I'd say use both. Their relative strengths and weaknesses should be pretty apparent. As a heuristic, trust Stockfish if the position is highly tactical, or if it's an endgame. Otherwise trust Leela (but run Leela's principal variation through Stockfish to eliminate the "Leela blunder"). </p>
  
  <p><strong>EDIT:</strong> Will probably expand on this later, but GM Matthew Sadler wrote <a href="https://tcec.chessdom.com/articles/Sufi_15_-_Sadler.pdf" rel="nofollow noreferrer">a piece</a> on the relative differences between Stockfish and Leela based on the 15th TCEC superfinal which you might find interesting (especially <a href="https://en.wikipedia.org/wiki/TCEC_Season_15#Games_43/44" rel="nofollow noreferrer">games 43/44</a>) In a nutshell, Stockfish is comfortable going to a slightly inferior endgame and defending endlessly, while Leela hates that playstyle and prefers to seek active play.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-06T15:19:20.400Z
Answers.Text: Your comments about Multi-PV are somewhat misleading. The reason multi-pv doesn't weaken leela is that it's version of it isn't very good at what multi-pv is supposed to do (ie provide better analysis of weaker lines).
Answers.UserDisplayName: Oscar Smith
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-06T21:08:08.297Z
Answers.Text: @OscarSmith you sure? If the node that Leela visits most is the line it plays, then the node that Leela visits second-most is the second-best line, and so on.
Answers.UserDisplayName: Allure
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-06T21:17:33.003Z
Answers.Text: Yes, the problem is that since the 2nd best line for leela often has considerably fewer visits (10% of main line or less), everything past the 2nd best line is fairly under-analyzed.
Answers.UserDisplayName: Oscar Smith
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-06T21:18:14.733Z
Answers.Text: Ideally there would be 3 modes for multi-PV, off, cosmetic, and analysis. Leela supports 1 and 2, most other engines support 1 and 3
Answers.UserDisplayName: Oscar Smith
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-05T01:45:25.567Z
Answers.Text: I decided not to expand too much on the edit, since GM Sadler's article & the Wikipedia section are, I think, pretty good.
Answers.UserDisplayName: Allure
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-06T13:18:14.460Z
Answers.LastActivityDate: 2019-07-05T01:46:14.067Z
Answers.LastEditDate: 2019-07-05T01:46:14.067Z
Answers.UserDisplayName: Allure
Body: <p>What is current use status of Leela for practical player? Is it already working well as UCI? Could anybody give some insight on strength with various thinking times, especially in analysis mode? As leela uses different thinking process, I must ask also for multiPV search. Is the speed down of many PVs still as devastating as with other engines?</p>
  
  <p>I have desktop with 6x3.2Ghz CPU. As graphics card like RTX2080ti is 'just' around 1300€ with good use outside of chess too, I wonder, what performance change would it mean.</p>
  
  <p>With mentioned hardware, how long would approximately leela have to think to compare to Stockfish?</p>
  
  <p>Stockfish 0.1s
  Stockfish 1s
  Stockfish 10s
  Stockfish 1min
  Stockfish 10min
  Stockfish 1hour</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-18T05:34:59.533Z
Comments.Text: Leela is currently leading Stockfish in TCEC super final by 5 points, so it would seem that it is competitive with traditional engines (to say the least).
Comments.UserDisplayName: Qudit
Tags: <engines><leela>
Title: Leela Chess Zero - UCI engine
ThreadText: # Leela Chess Zero - UCI engine
  
  (Asked by: username_0 on 2019-05-18)
  
  What is current use status of Leela for practical player? Is it already working well as UCI? Could anybody give some insight on strength with various thinking times, especially in analysis mode? As leela uses different thinking process, I must ask also for multiPV search. Is the speed down of many PVs still as devastating as with other engines?
  
  I have desktop with 6x3.2Ghz CPU. As graphics card like RTX2080ti is 'just' around 1300€ with good use outside of chess too, I wonder, what performance change would it mean.
  
  With mentioned hardware, how long would approximately leela have to think to compare to Stockfish?
  
  Stockfish 0.1s
  Stockfish 1s
  Stockfish 10s
  Stockfish 1min
  Stockfish 10min
  Stockfish 1hour
  
  **Tags:** engines, leela
  
  > **Comments:**
  > **username_1**: Leela is currently leading Stockfish in TCEC super final by 5 points, so it would seem that it is competitive with traditional engines (to say the least).  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2019-06-06. Score: 6 | Accepted answer
  
  Lc0 is fully compatible with UCI. It's strength depends a ton on what hardware it's being run on. On any hardware (tested down to Raspberry Pi), it will be superhuman (over 2900 elo). With a RTX 2060 ($200ish) it should be roughly even with SF on 8 cores (although you may need to mess with settings a bit to get it there). A 2080ti puts decently (20 or so elo) ahead of 24 cores of SF. The current MultiPV mode is only a display tweak. It isn't weaker, but it doesn't give as good analysis of the non-primary lines. If you have more questions, the best place to ask is probably the Lc0 discord.
  
  > **Comments:**
  > **username_0**: Thank you for benchmark information. By GTX 2060 you mean RTX 2060 using FP16 backend, right?  
  > **username_2**: Correct. Updated to be more clear  
  
  ---
  
  ### Answer by username_3 on 2019-06-06. Score: 6
  
  Can't comment on hardware but will say something about Leela's strengths and weaknesses -
  
  If you look through the games from the TCEC superfinal, sometimes Leela is brilliant. For example in this position from game 61:
  
  ```
  [fen "r1bbnr2/pp1n1q1k/3p4/2pPp1pP/2P1PpP1/2NQ1N2/PP2BB2/2KR2R1 w - - 10 26"]
  
  ```
  
  Stockfish (White) evaluated this as +0.63, while Leela was at -2.01. From the human point of view, I suspect most will prefer Black. That's because White has nothing on the kingside while Black is more capable of playing on the queenside. The game proved Leela right - it took a lot more moves for Stockfish to realize it was inferior, and even more moves after that for Stockfish to resign (it is a fantastic defender), but the fact that Leela saw it was superior so much earlier than Stockfish is very impressive. There are many similar games from the superfinal - Leela seems very capable of outplaying Stockfish positionally, often converting a Stockfish advantage into one of its own, even if the game eventually ends in a draw.
  
  Another advantage of Leela is multi-PV. MCTS engines get this for free, while Stockfish (which runs on AB) needs to invest computational power. If this matters for you then that's an additional advantage for Leela.
  
  However the infamous "Leela blunder" is still a thing. Take this position from game 81:
  
  ```
  [fen "4rr2/2pb2bk/1p1p4/2nP3P/p1P1p3/P1N1Bq2/1P3N2/1K2Q1RR w - - 2 26"]
  
  ```
  
  Leela just played 25...Rae8 and showed a +0.67 eval. It thinks its position is inferior, but not by that much. Its principal variation went 25. .. Rae8 26. h6 Be5 27. Rg7+ Kh8 28. Ka2 Rf7 29. Nfd1 Nd3 30. Qh4. Stockfish had the same moves, but saw that 28. Qg1! wins, and its eval shot up to +7.68 one move later. Leela remained blissfully unaware until Stockfish played Qg1, and resigned shortly afterwards. Stockfish virtually never makes this kind of one-move blunder.
  
  Another problem with Leela is that it doesn't seem to defend very well. When Stockfish loses, it defends grimly and it takes a long time for Leela to grind a win. On the other hand when Leela loses, it often implodes. In the ongoing TCEC Champions bonus, [its loss to Houdini 3 (!!) is illustrative](https://cd.tcecbeta.club/archive.html?season=15&div=champions&game=29#/). Houdini 3 is an engine many years old, rated hundreds of elo below Stockfish (and ostensibly Leela). But Leela hated the starting position so much it managed to lose. Comparatively you would need to play hundreds if not thousands of games before Stockfish loses to Houdini 3.
  
  Finally a noticeable weakness of Leela's is that it isn't very good at beating inferior opponents. For example in the ongoing TCEC Champions bonus, which includes engines from many years ago, Leela is performing worse than 1-year old versions of Stockfish. In this, Leela is kind of similar to Petrosian - it doesn't win many games, but it's good in match play.
  
  In the end, I think Leela is inconsistent. In some positions it's much stronger than Stockfish, and in others it's just worse. As a practical player, I'd say use both. Their relative strengths and weaknesses should be pretty apparent. As a heuristic, trust Stockfish if the position is highly tactical, or if it's an endgame. Otherwise trust Leela (but run Leela's principal variation through Stockfish to eliminate the "Leela blunder").
  
  **EDIT:** Will probably expand on this later, but GM Matthew Sadler wrote [a piece](https://tcec.chessdom.com/articles/Sufi_15_-_Sadler.pdf) on the relative differences between Stockfish and Leela based on the 15th TCEC superfinal which you might find interesting (especially [games 43/44](https://en.wikipedia.org/wiki/TCEC_Season_15#Games_43/44)) In a nutshell, Stockfish is comfortable going to a slightly inferior endgame and defending endlessly, while Leela hates that playstyle and prefers to seek active play.
  
  > **Comments:**
  > **username_2**: Your comments about Multi-PV are somewhat misleading. The reason multi-pv doesn't weaken leela is that it's version of it isn't very good at what multi-pv is supposed to do (ie provide better analysis of weaker lines).  
  > **username_3**: @username_2 you sure? If the node that Leela visits most is the line it plays, then the node that Leela visits second-most is the second-best line, and so on.  
  > **username_2**: Yes, the problem is that since the 2nd best line for leela often has considerably fewer visits (10% of main line or less), everything past the 2nd best line is fairly under-analyzed.  
  > **username_2**: Ideally there would be 3 modes for multi-PV, off, cosmetic, and analysis. Leela supports 1 and 2, most other engines support 1 and 3  
  > **username_3**: I decided not to expand too much on the edit, since GM Sadler's article & the Wikipedia section are, I think, pretty good.  
  
  ---
  
  