Body: <p>During my research for Google DeepMind's Go-playing program <a href="https://www.nature.com/articles/nature16961" rel="nofollow noreferrer">Alpha Go</a> and its successor <a href="https://www.nature.com/articles/nature24270" rel="nofollow noreferrer">Alpha Go Zero</a>, I discovered that the system uses a clever pipeline and an interplay of blocks of both policy and value networks to play the game of Go in such a way, that it is able to outperform even the best players in the world. This is in particular remarkable, because the game of Go was considered to be unsolvable a few years ago. This success gained international attention and it was labeled as a breakthrough in the community of AI. It is also not a secret that the research team behind AlphaGo and AlphaGo Zero used lots of computation power to create such a sophisticated system.</p>
  <p>But, since each board configuration is considered as a distinct state, where algorithms can be applied really well, and just consider AlphaGo Zero, which uses no prior knowledge and can figure out how the play the game of go from scratch, my question is the following:</p>
  <p>Is there any way to state (theoretically) how the performance of AlphaGo would be in continuous action spaces (e.g. self-driving cars)?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-11T00:18:41.573Z
Comments.Text: The paper [A0C: Alpha Zero in Continuous Action Space](https://arxiv.org/pdf/1805.09613.pdf) (although a pre-print) could be useful.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-11T07:42:33.433Z
Comments.Text: I'm not familiar with reinforcement learning terminology, but are real-time video games (like tetris, breakout, pong, etc.) considered continuous action spaces? Their action space is technically discrete, but each pixel is very small compared to the total field. If yes, you might be interested in reading MuZero, which is AlphaZero's successor (AlphaZero is AlphaGo Zero's successor). MuZero generalized AlphaZero to play Atari games, which are real time video games. It achieved state of the art. https://arxiv.org/pdf/1911.08265.pdf
Comments.UserDisplayName: user3667125
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-11T11:47:07.210Z
Comments.Text: Thanks @nbro & @user3667125!
  I'll check that out.
Comments.UserDisplayName: maven
Tags: <monte-carlo-tree-search><alphazero><alphago><deepmind><continuous-action-spaces>
Title: What would be the AlphaGo's performance in continuous action space?
UserDisplayName: maven
ThreadText: # What would be the AlphaGo's performance in continuous action space?
  
  (Asked by: username_0 on 2020-12-09)
  
  During my research for Google DeepMind's Go-playing program [Alpha Go](https://www.nature.com/articles/nature16961) and its successor [Alpha Go Zero](https://www.nature.com/articles/nature24270), I discovered that the system uses a clever pipeline and an interplay of blocks of both policy and value networks to play the game of Go in such a way, that it is able to outperform even the best players in the world. This is in particular remarkable, because the game of Go was considered to be unsolvable a few years ago. This success gained international attention and it was labeled as a breakthrough in the community of AI. It is also not a secret that the research team behind AlphaGo and AlphaGo Zero used lots of computation power to create such a sophisticated system.
  
  But, since each board configuration is considered as a distinct state, where algorithms can be applied really well, and just consider AlphaGo Zero, which uses no prior knowledge and can figure out how the play the game of go from scratch, my question is the following:
  
  Is there any way to state (theoretically) how the performance of AlphaGo would be in continuous action spaces (e.g. self-driving cars)?
  
  **Tags:** monte-carlo-tree-search, alphazero, alphago, deepmind, continuous-action-spaces
  > **Comments:**
  > **username_1**: The paper [A0C: Alpha Zero in Continuous Action Space](https://arxiv.org/pdf/1805.09613.pdf) (although a pre-print) could be useful.  
  > **username_2**: I'm not familiar with reinforcement learning terminology, but are real-time video games (like tetris, breakout, pong, etc.) considered continuous action spaces? Their action space is technically discrete, but each pixel is very small compared to the total field. If yes, you might be interested in reading MuZero, which is AlphaZero's successor (AlphaZero is AlphaGo Zero's successor). MuZero generalized AlphaZero to play Atari games, which are real time video games. It achieved state of the art. https://arxiv.org/pdf/1911.08265.pdf  
  > **username_0**: Thanks @nbro & @user3667125! I'll check that out.  
  
  ---
  
  ## 0 answers
  
  