Answers.Body: <p>The reason why Cepheus can't generalize has to do with the number of decision points.</p>
  
  <p>The same authors recently let loose Deep Stack (DeepStack: Expert-Level Artificial Intelligence in Heads-Up No-Limit (HUNL) Poker) which is freaking many professional poker players out.</p>
  
  <p>In the <a href="https://arxiv.org/pdf/1701.01724.pdf" rel="nofollow noreferrer">DeepStack arxiv paper</a>, they say</p>
  
  <blockquote>
    <p>AI techniques (Cepheus) have previously shown
    success in the simpler game of heads-up limit Texas hold’em, where all bets are of a fixed size resulting in just under 10^14 decision points.</p>
  </blockquote>
  
  <p>...</p>
  
  <blockquote>
    <p>The imperfect information game HUNL is comparable in size to go, with
    the number of decision points exceeding 10^160</p>
  </blockquote>
  
  <p>...</p>
  
  <blockquote>
    <p>Imperfect information games require more complex reasoning than similarly sized perfect
    information games. The correct decision at a particular moment depends upon the probability
    distribution over private information that the opponent holds, which is revealed through their
    past actions.</p>
  </blockquote>
  
  <p>Using the same strategy for HUNL as Cepheus did is out of the question. Rather, taking an educated guess or using intuition based on the previous play (referred to as Continual Re-solving in the paper) is a method which can better handle this gargantuan game. For more information check out the <a href="https://www.deepstack.ai/" rel="nofollow noreferrer">DeepStack website</a>.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-25T17:31:48.203Z
Answers.LastActivityDate: 2017-07-27T22:17:19.757Z
Answers.LastEditDate: 2017-07-27T22:17:19.757Z
Answers.UserDisplayName: Jaden Travnik
Body: <p><a href="http://www.vocativ.com/culture/science/most-powerful-poker-computer-cepheus/" rel="nofollow noreferrer">Cepheus</a> is an artificial intelligence designed to play Texas Hold'em. By playing against itself and learning where it could have done better, it became very good at the game. <a href="http://slatestarcodex.com/2015/01/17/links-12014-link-for-you-know-not-whence-you-came-nor-why/" rel="nofollow noreferrer">Slate Star Codex</a> comments:</p>
  
  <blockquote>
    <p>I was originally confused why they published this result instead of heading to online casinos and becoming rich enough to buy small countries, but it seems that it’s a very simplified version of the game with only two players. More interesting, the strategy was reinforcement learning – the computer started with minimal domain knowledge, then played poker against itself a zillion times until it learned everything it needed to know. </p>
  </blockquote>
  
  <p>Apparently Cepheus currently just plays against one person. Seeing as it managed to develop amazing strategy for this "very simplified" environment, what's stopping it from working on real/full poker games?</p>
  
Tags: <reinforcement-learning><gaming>
Title: What's stopping Cepheus from generalizing to full poker games?
UserDisplayName: Ben N
ThreadText: # What's stopping Cepheus from generalizing to full poker games?
  
  (Asked by: username_0 on 2017-07-24)
  
  [Cepheus](http://www.vocativ.com/culture/science/most-powerful-poker-computer-cepheus/) is an artificial intelligence designed to play Texas Hold'em. By playing against itself and learning where it could have done better, it became very good at the game. [Slate Star Codex](http://slatestarcodex.com/2015/01/17/links-12014-link-for-you-know-not-whence-you-came-nor-why/) comments:
  
  > I was originally confused why they published this result instead of heading to online casinos and becoming rich enough to buy small countries, but it seems that it’s a very simplified version of the game with only two players. More interesting, the strategy was reinforcement learning – the computer started with minimal domain knowledge, then played poker against itself a zillion times until it learned everything it needed to know.
  
  Apparently Cepheus currently just plays against one person. Seeing as it managed to develop amazing strategy for this "very simplified" environment, what's stopping it from working on real/full poker games?
  
  **Tags:** reinforcement-learning, gaming
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2017-07-25. Score: 3 | Accepted answer
  
  The reason why Cepheus can't generalize has to do with the number of decision points.
  
  The same authors recently let loose Deep Stack (DeepStack: Expert-Level Artificial Intelligence in Heads-Up No-Limit (HUNL) Poker) which is freaking many professional poker players out.
  
  In the [DeepStack arxiv paper](https://arxiv.org/pdf/1701.01724.pdf), they say
  
  > AI techniques (Cepheus) have previously shown
  > success in the simpler game of heads-up limit Texas hold’em, where all bets are of a fixed size resulting in just under 10^14 decision points.
  
  ...
  
  > The imperfect information game HUNL is comparable in size to go, with
  > the number of decision points exceeding 10^160
  
  ...
  
  > Imperfect information games require more complex reasoning than similarly sized perfect
  > information games. The correct decision at a particular moment depends upon the probability
  > distribution over private information that the opponent holds, which is revealed through their
  > past actions.
  
  Using the same strategy for HUNL as Cepheus did is out of the question. Rather, taking an educated guess or using intuition based on the previous play (referred to as Continual Re-solving in the paper) is a method which can better handle this gargantuan game. For more information check out the [DeepStack website](https://www.deepstack.ai/).
  
  
  ---
  
  