Answers.Body: <p>In short, because <strong>DeepMind has TPUs</strong>. Lots and lots of TPUs.</p>
  
  <p>By far, the most time-consuming part of the training process is <strong>generating self-play games</strong>. DeepMind used 5000 TPUs to generate self-play games, which is a lot of processing power. Meanwhile, Lc0 crowdsources processing power from volunteer clients, which is much slower.</p>
  
  <p>From the <a href="https://arxiv.org/pdf/1712.01815.pdf" rel="noreferrer">DeepMind paper</a> published December 2017 (emphasis mine):</p>
  
  <blockquote>
    <p>We trained a separate instance of AlphaZero for each game. Training proceeded for 700,000 steps (mini-batches of size 4,096) starting from randomly initialised parameters, using <strong>5,000 first-generation TPUs</strong> to generate self-play games and <strong>64 second-generation TPUs</strong> to train the neural networks.</p>
  </blockquote>
  
  <p>To achieve the level of performance that AlphaZero got against the (sub-optimally tuned) Stockfish version 8, the training run took about 9 hours and 44 million self-play games for chess:</p>
  
  <p><img src="https://i.sstatic.net/ispsC.png" alt="Table S3: Selected statistics of AlphaZero training in Chess, Shogi, and Go: Mini-batches, training time, training games, and thinking time"></p>
  
  <p>Meanwhile, the Lc0 project got started sometime in <a href="https://github.com/gcp/leela-zero/issues/369#issuecomment-354701069" rel="noreferrer">January 2018</a>, and by <a href="https://en.chessbase.com/post/leela-chess-zero-alphazero-for-the-pc" rel="noreferrer">April 2018</a>, they'd only generated <strong>8.2 million out of the needed 44 million games</strong>. I believe they were generating about 200k games per day at the time.</p>
  
  <p>They also came across a lot of bugs in the process and were figuring out a lot of things as they went, so when the first training run plateaued, the best net was not as good as AlphaZero. Since then, the Lc0 project has actually <a href="https://github.com/LeelaChessZero/lc0/wiki/Training-runs" rel="noreferrer">restarted the training run</a> multiple times:</p>
  
  <ul>
  <li><strong>old main:</strong> The original 192x15 "main" run, using the original <code>lczero.exe</code> engine. It ended in August 2018, with the last net being ID595. (These nets don't seem to be listed anymore though? The early IDs at the bottom of the <a href="http://lczero.org/networks/" rel="noreferrer">net list</a> seem to be these <a href="https://github.com/LeelaChessZero/lc0/wiki/lc0-transition#runs-on-the-test-pipeline" rel="noreferrer">test nets</a>.)</li>
  <li><strong>test10</strong>: The original 256x20 test run. This was a separate pipeline using a completely rewritten <code>lc0.exe</code> engine that was much faster at generating self-play games. Net numbering for this run <a href="http://archive.is/oiuQu" rel="noreferrer">started</a> with ID10000 on June 27, 2018, and ended with ID11262 on August 30, 2018.</li>
  <li><strong>test20</strong>: A run with <a href="http://blog.lczero.org/2018/08/training-run-reset.html" rel="noreferrer">various changes</a>, starting with ID20001 on August 30, 2018 and ending with ID22201 on November 16, 2018.</li>
  <li><strong>test30</strong>: A run testing <a href="http://blog.lczero.org/2018/09/tb-rescoring.html" rel="noreferrer">various new ideas</a>, starting with ID30001 on September 9, 2018. As of January 14, 2019, the best net was ID32598.</li>
  <li><strong>test40</strong>: The December 6, 2018 AlphaZero paper revealed a lot of <a href="http://blog.lczero.org/2018/12/alphazero-paper-and-lc0-v0191.html?m=1" rel="noreferrer">differences between what DeepMind did and what Lc0 did</a>, and it looks like a <a href="http://blog.lczero.org/2019/01/leela-in-december.html" rel="noreferrer">new test run</a> started with ID40001 on January 8, 2019. As of January 14, 2019, the best net was ID40202.</li>
  </ul>
  
  <p>To get a sense of how well Lc0 is doing compared to AlphaZero, it's more useful to look at the Elo with respect to total number of games generated, not time elapsed. Here's what the graph looks like for the entire Lc0 project as of January 14, 2019 (note that this is self-play Elo, which is inflated compared to actual Elo):</p>
  
  <p><a href="http://web.archive.org/web/20190114061353/http://lczero.org/?full_elo=1" rel="noreferrer"><img src="https://i.sstatic.net/f8iTb.png" alt="graph of self-play Elo vs. number of accumulated games"></a></p>
  
  <p>As you can see, they've reset the training run many times. There were also a number of <a href="http://archive.is/tnj6k" rel="noreferrer">bootstrap mini-resets</a> during the original main run to fix some critical bugs, and there was also a temporary <a href="http://blog.lczero.org/2019/01/leela-in-december.html" rel="noreferrer">test35</a> to test the newly revealed configuration settings from the December 2018 AlphaGo paper. Yep, this isn't confusing at all! (/s)</p>
  
  <p>There are lots of calculations and graphs in <a href="https://docs.google.com/spreadsheets/d/1XSJiCcQpCLv0fNwrUn7jXjdkZFU63YFEWpdXv6dSSg0/edit?usp=sharing" rel="noreferrer">this spreadsheet here</a> that compares lc0's performance with AlphaZero. For example, here's the graph from the first old main run (under the "Graphs" tab):</p>
  
  <p><a href="https://docs.google.com/spreadsheets/d/1XSJiCcQpCLv0fNwrUn7jXjdkZFU63YFEWpdXv6dSSg0/edit#gid=2014732845" rel="noreferrer"><img src="https://i.sstatic.net/adHkF.png" alt="AlphaZero vs. Leela progression comparison"></a>
  From that link, an explanation of what "Leela Max" is:</p>
  
  <blockquote>
    <p>"Leela Max" is Leela's theoretical maximum performance when given hardware and/or time to the point of absolute diminished returns. It may be more comparable to AlphaZero, as AlphaZero operated on 4.8 million nodes per move, whereas a conventional setup only gets a small fraction of that.</p>
  </blockquote>
  
  <hr>
  
  <p>These days, the self-play generation speed is much faster than it was at the beginning. As of January 14, 2019, it looks like the project is generating about <a href="http://web.archive.org/web/20190114060043/http://lczero.org/" rel="noreferrer">1.5 million games per day</a>. But at this speed, it'll still take a month to achieve what DeepMind achieved in 9 hours. More like double the time, really, if the training games are being diverted towards two separate runs.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-16T20:25:27.797Z
Answers.Text: In addition to using many TPUs for training, DeepMind also used TPUs during the match itself.  Thus, AlphaZero also ran on significantly more powerful hardware during the match than LC0.
Answers.UserDisplayName: Qudit
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-14T06:44:52.050Z
Answers.LastActivityDate: 2019-01-14T07:06:19.910Z
Answers.LastEditDate: 2019-01-14T07:06:19.910Z
Answers.UserDisplayName: ahiijny
Answers.Body: <p>This is happening because AlphaZero and Leela aren't playing against the same Stockfish. If you read the paper, AlphaZero beat Stockfish 8 (it also played a series against Stockfish 9 - same logic applies however). Both these versions of Stockfish are old, and <a href="https://nextchessmove.com/dev-builds" rel="noreferrer">significantly inferior to the latest version of Stockfish</a>.</p>
  
  <p>AlphaZero beat Stockfish 8 by +155 -6 =839. That's a ~50 elo difference. On the other hand Stockfish 10 is better than Stockfish 8 by ~100 elo. Leela, playing against Stockfish Dev (which is even better than Stockfish 10 - the engine is continuously updated) and not Stockfish 8, could be as good as AlphaZero yet fail to crush Stockfish. It would crush Stockfish 8 alright, but not Stockfish Dev.</p>
  
  <p>On a side note: it's not clear that AlphaZero would crush Stockfish 10 either, or vice versa, because elo isn't transitive: if A beats B by 50 elo and B beats C by 50 elo, it doesn't mean that A beats C by 100 elo.</p>
  
  <p><strong>Edit:</strong> To illustrate, in the <a href="http://legacy-tcec.chessdom.com/archive.php" rel="noreferrer">TCEC cup semifinals</a>, Leela vs. Stockfish, the version of Stockfish that played was Stockfish 270918. The last six numbers is the date - Stockfish as of the 27th of the 9th month of 2018. You can see from the link above that this version of Stockfish is roughly 100 elo stronger than Stockfish 8, so the fact that Leela performs worse against it is not a surprise.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-12-09T11:20:11.020Z
Answers.LastActivityDate: 2018-12-09T22:37:43.843Z
Answers.LastEditDate: 2018-12-09T22:37:43.843Z
Answers.UserDisplayName: Allure
Body: <p>Alpha Zero defeated Stockfish after <a href="https://en.wikipedia.org/wiki/AlphaZero" rel="noreferrer">only 9 hours</a> of self-play. On the other hand, Lc0, has been training for <a href="http://blog.lczero.org/2018/09/guide-setting-up-leela-on-chess-gui.html" rel="noreferrer">several months</a> and while attaining impressive strength (to say the least...) it's still not crushing Stockfish. What can be the reason for such a dramatic difference in learning efficiency?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-12-08T14:59:46.857Z
Comments.Text: Note that Stockfish wasn't running under optimal conditions in the Alpha0 match. It was a bit weaker than a full-power version of Stockfish.
Comments.UserDisplayName: Scounged
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-12-08T15:24:01.813Z
Comments.Text: Well, during the 2nd match it seems the conditions were fair...
Comments.UserDisplayName: acye
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-12-08T15:25:38.713Z
Comments.Text: As far as I can see, Lc0 is running on much more normal hardware than AlphaZero.
Comments.UserDisplayName: user1583209
Tags: <engines><alphazero><leela>
Title: Alpha Zero vs Lc0 - time for self-play
ThreadText: # Alpha Zero vs Lc0 - time for self-play
  
  (Asked by: username_0 on 2018-12-08)
  
  Alpha Zero defeated Stockfish after [only 9 hours](https://en.wikipedia.org/wiki/AlphaZero) of self-play. On the other hand, Lc0, has been training for [several months](http://blog.lczero.org/2018/09/guide-setting-up-leela-on-chess-gui.html) and while attaining impressive strength (to say the least...) it's still not crushing Stockfish. What can be the reason for such a dramatic difference in learning efficiency?
  
  **Tags:** engines, alphazero, leela
  
  > **Comments:**
  > **username_1**: Note that Stockfish wasn't running under optimal conditions in the Alpha0 match. It was a bit weaker than a full-power version of Stockfish.  
  > **username_0**: Well, during the 2nd match it seems the conditions were fair...  
  > **username_2**: As far as I can see, Lc0 is running on much more normal hardware than AlphaZero.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2019-01-14. Score: 11
  
  In short, because **DeepMind has TPUs**. Lots and lots of TPUs.
  
  By far, the most time-consuming part of the training process is **generating self-play games**. DeepMind used 5000 TPUs to generate self-play games, which is a lot of processing power. Meanwhile, Lc0 crowdsources processing power from volunteer clients, which is much slower.
  
  From the [DeepMind paper](https://arxiv.org/pdf/1712.01815.pdf) published December 2017 (emphasis mine):
  
  > We trained a separate instance of AlphaZero for each game. Training proceeded for 700,000 steps (mini-batches of size 4,096) starting from randomly initialised parameters, using **5,000 first-generation TPUs** to generate self-play games and **64 second-generation TPUs** to train the neural networks.
  
  To achieve the level of performance that AlphaZero got against the (sub-optimally tuned) Stockfish version 8, the training run took about 9 hours and 44 million self-play games for chess:
  
  ![Table S3: Selected statistics of AlphaZero training in Chess, Shogi, and Go: Mini-batches, training time, training games, and thinking time](https://i.sstatic.net/ispsC.png)
  
  Meanwhile, the Lc0 project got started sometime in [January 2018](https://github.com/gcp/leela-zero/issues/369#issuecomment-354701069), and by [April 2018](https://en.chessbase.com/post/leela-chess-zero-alphazero-for-the-pc), they'd only generated **8.2 million out of the needed 44 million games**. I believe they were generating about 200k games per day at the time.
  
  They also came across a lot of bugs in the process and were figuring out a lot of things as they went, so when the first training run plateaued, the best net was not as good as AlphaZero. Since then, the Lc0 project has actually [restarted the training run](https://github.com/LeelaChessZero/lc0/wiki/Training-runs) multiple times:
  
  * **old main:** The original 192x15 "main" run, using the original `lczero.exe` engine. It ended in August 2018, with the last net being ID595. (These nets don't seem to be listed anymore though? The early IDs at the bottom of the [net list](http://lczero.org/networks/) seem to be these [test nets](https://github.com/LeelaChessZero/lc0/wiki/lc0-transition#runs-on-the-test-pipeline).)
  * **test10**: The original 256x20 test run. This was a separate pipeline using a completely rewritten `lc0.exe` engine that was much faster at generating self-play games. Net numbering for this run [started](http://archive.is/oiuQu) with ID10000 on June 27, 2018, and ended with ID11262 on August 30, 2018.
  * **test20**: A run with [various changes](http://blog.lczero.org/2018/08/training-run-reset.html), starting with ID20001 on August 30, 2018 and ending with ID22201 on November 16, 2018.
  * **test30**: A run testing [various new ideas](http://blog.lczero.org/2018/09/tb-rescoring.html), starting with ID30001 on September 9, 2018. As of January 14, 2019, the best net was ID32598.
  * **test40**: The December 6, 2018 AlphaZero paper revealed a lot of [differences between what DeepMind did and what Lc0 did](http://blog.lczero.org/2018/12/alphazero-paper-and-lc0-v0191.html?m=1), and it looks like a [new test run](http://blog.lczero.org/2019/01/leela-in-december.html) started with ID40001 on January 8, 2019. As of January 14, 2019, the best net was ID40202.
  
  To get a sense of how well Lc0 is doing compared to AlphaZero, it's more useful to look at the Elo with respect to total number of games generated, not time elapsed. Here's what the graph looks like for the entire Lc0 project as of January 14, 2019 (note that this is self-play Elo, which is inflated compared to actual Elo):
  
  [![graph of self-play Elo vs. number of accumulated games](https://i.sstatic.net/f8iTb.png)](http://web.archive.org/web/20190114061353/http://lczero.org/?full_elo=1)
  
  As you can see, they've reset the training run many times. There were also a number of [bootstrap mini-resets](http://archive.is/tnj6k) during the original main run to fix some critical bugs, and there was also a temporary [test35](http://blog.lczero.org/2019/01/leela-in-december.html) to test the newly revealed configuration settings from the December 2018 AlphaGo paper. Yep, this isn't confusing at all! (/s)
  
  There are lots of calculations and graphs in [this spreadsheet here](https://docs.google.com/spreadsheets/d/1XSJiCcQpCLv0fNwrUn7jXjdkZFU63YFEWpdXv6dSSg0/edit?usp=sharing) that compares lc0's performance with AlphaZero. For example, here's the graph from the first old main run (under the "Graphs" tab):
  
  [![AlphaZero vs. Leela progression comparison](https://i.sstatic.net/adHkF.png)](https://docs.google.com/spreadsheets/d/1XSJiCcQpCLv0fNwrUn7jXjdkZFU63YFEWpdXv6dSSg0/edit#gid=2014732845)
  From that link, an explanation of what "Leela Max" is:
  
  > "Leela Max" is Leela's theoretical maximum performance when given hardware and/or time to the point of absolute diminished returns. It may be more comparable to AlphaZero, as AlphaZero operated on 4.8 million nodes per move, whereas a conventional setup only gets a small fraction of that.
  
  ---
  
  These days, the self-play generation speed is much faster than it was at the beginning. As of January 14, 2019, it looks like the project is generating about [1.5 million games per day](http://web.archive.org/web/20190114060043/http://lczero.org/). But at this speed, it'll still take a month to achieve what DeepMind achieved in 9 hours. More like double the time, really, if the training games are being diverted towards two separate runs.
  
  > **Comments:**
  > **username_4**: In addition to using many TPUs for training, DeepMind also used TPUs during the match itself.  Thus, AlphaZero also ran on significantly more powerful hardware during the match than LC0.  
  
  ---
  
  ### Answer by username_5 on 2018-12-09. Score: 6
  
  This is happening because AlphaZero and Leela aren't playing against the same Stockfish. If you read the paper, AlphaZero beat Stockfish 8 (it also played a series against Stockfish 9 - same logic applies however). Both these versions of Stockfish are old, and [significantly inferior to the latest version of Stockfish](https://nextchessmove.com/dev-builds).
  
  AlphaZero beat Stockfish 8 by +155 -6 =839. That's a ~50 elo difference. On the other hand Stockfish 10 is better than Stockfish 8 by ~100 elo. Leela, playing against Stockfish Dev (which is even better than Stockfish 10 - the engine is continuously updated) and not Stockfish 8, could be as good as AlphaZero yet fail to crush Stockfish. It would crush Stockfish 8 alright, but not Stockfish Dev.
  
  On a side note: it's not clear that AlphaZero would crush Stockfish 10 either, or vice versa, because elo isn't transitive: if A beats B by 50 elo and B beats C by 50 elo, it doesn't mean that A beats C by 100 elo.
  
  **Edit:** To illustrate, in the [TCEC cup semifinals](http://legacy-tcec.chessdom.com/archive.php), Leela vs. Stockfish, the version of Stockfish that played was Stockfish 270918. The last six numbers is the date - Stockfish as of the 27th of the 9th month of 2018. You can see from the link above that this version of Stockfish is roughly 100 elo stronger than Stockfish 8, so the fact that Leela performs worse against it is not a surprise.
  
  
  ---
  
  