Answers.Body: <p>120 inputs can be handled by a complex enough network. Dealing with high complexity is one of NN's strengths.</p>
  
  <p>Using a (120,) vector or a (3,40) matrix is the same, they're still 120 inputs. Your binary encoding should work. Another option is a single (40,) vector, with 0 being "still in deck", 1 being "in hand", 2 being "on table", 3 being "already played". </p>
  
  <p>If the types of cards are irrelevant, you could actually have a (3,10) matrix with counters of cards (1 in hand, 1 in table, 2 already played). You can try different approaches and see what works best.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-08T11:25:39.550Z
Answers.Text: What I do not get here is what are your actions? Play card x? Does this simply come out of:  state0 = torch.tensor(state, dtype=torch.float)
     prediction = self.model(state0)
     action = torch.argmax(prediction).item()
Answers.UserDisplayName: sqp_125
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-08T12:09:23.820Z
Answers.Text: @sqp_125 4 outpouts representing the 4 cards in hand, ordered, could work
Answers.UserDisplayName: BlueMoon93
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-06T18:12:59.540Z
Answers.Text: Never knew that we can encode features as non binary... Can you tell me the name of this "technique"? Thank you for the informations.
Answers.UserDisplayName: Haytam
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-06T18:30:01.623Z
Answers.Text: For the (40,) vector, wouldn't encoding like that mean that there is some kind of order?
Answers.UserDisplayName: Haytam
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-06T18:33:44.847Z
Answers.Text: Imagine you had one ace in hand, one on table, 2 played already. How could you code that with a single (10,) vector? You need 3 rows, one for each place. Its simply called non binary input, its very common. Image processing NNs use inputs with pixel values [0,255]
Answers.UserDisplayName: BlueMoon93
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-06T18:36:07.273Z
Answers.Text: I edited my second comment in case, please take a look and thank you!
Answers.UserDisplayName: Haytam
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-06T18:39:02.040Z
Answers.Text: When using tokens to code information (1 means something, 2 something else, etc) you need a more complex network that learns to distinguish the meaning of these codes, and learns that they do not represent order or quantity, yes.
Answers.UserDisplayName: BlueMoon93
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-06T20:27:53.850Z
Answers.Text: One last question please: Does it matter if the score increases when we have the same number twice in our hand? Like if we have 2, 2, 5, 7 then the pair of 2 gives us extra score.
Answers.UserDisplayName: Haytam
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-06T21:29:42.423Z
Answers.Text: This type of thing (having a pair of the same number) needs to be called by the player, should I add an action to the NN?
Answers.UserDisplayName: Haytam
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-07T11:11:24.337Z
Answers.Text: I didnt et the question, but it might be worth openig up a new question in the stack
Answers.UserDisplayName: BlueMoon93
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-06T14:38:47.160Z
Answers.LastActivityDate: 2018-07-06T14:38:47.160Z
Answers.UserDisplayName: BlueMoon93
Body: <p>In order to learn about DP and RL, I chose to start a side project where I would train an AI to play a "simple" card game. I will be doing this using the DQN with replay memory.<br>
  The problem is, I can't get the intuition behind how to represent the input to the neural network..</p>
  
  <p><strong>About the game</strong></p>
  
  <p>It's a fairly simple 2-players game. There is a deck of 40 unique cards (4 types of cards, 10 numbered cards in each type).<br>
  Each player gets 4 cards and each turn a player must put a card on the table.<br>
  If a player puts a card and there is already a card with the same number on the table, the player wins both cards.<br>
  If for example a player plays Card 2 and on the table there is Cards 2, 3, 4, 5 then the player wins all those cards (sequence).<br>
  Cards won don't go back to the hand nor to the deck, they are just kept as like a score.<br>
  When the players have 0 cards in hand, another 4 cards are dealt to each one untile the deck has 0 cards left where then we decide who won based on the number of cards eaten/won.</p>
  
  <p><strong>Question</strong></p>
  
  <p>As the input, I will be using the following:</p>
  
  <ul>
  <li>Current cards in the AI hand (40 one-hot-encoded features?)</li>
  <li>Current cards on the table (40 one-hot-encoded features?)</li>
  <li>History of played cards (40 one-hot-encoded features?)</li>
  </ul>
  
  <p>This would give 120 columns/features in each state.<br>
  I am wondering wheter this is too much for the NN or wheter my input representation would be bad for the NN?<br>
  Should the features be represented as a (120,) vector or as a 3x40 matrix?</p>
  
  <p>I am also wondering if it's a good idea to represent the current cards on the table as just a 10 one-hot-encoded features since the type of the cards don't matter and the same number can't exist 2 times in the table?</p>
  
  <p>Thank you in advance.</p>
  
Tags: <reinforcement-learning><dqn>
Title: DQN input representation for a card game
UserDisplayName: Haytam
ThreadText: # DQN input representation for a card game
  
  (Asked by: username_0 on 2018-07-06)
  
  In order to learn about DP and RL, I chose to start a side project where I would train an AI to play a "simple" card game. I will be doing this using the DQN with replay memory.  
  The problem is, I can't get the intuition behind how to represent the input to the neural network..
  
  **About the game**
  
  It's a fairly simple 2-players game. There is a deck of 40 unique cards (4 types of cards, 10 numbered cards in each type).  
  Each player gets 4 cards and each turn a player must put a card on the table.  
  If a player puts a card and there is already a card with the same number on the table, the player wins both cards.  
  If for example a player plays Card 2 and on the table there is Cards 2, 3, 4, 5 then the player wins all those cards (sequence).  
  Cards won don't go back to the hand nor to the deck, they are just kept as like a score.  
  When the players have 0 cards in hand, another 4 cards are dealt to each one untile the deck has 0 cards left where then we decide who won based on the number of cards eaten/won.
  
  **Question**
  
  As the input, I will be using the following:
  
  * Current cards in the AI hand (40 one-hot-encoded features?)
  * Current cards on the table (40 one-hot-encoded features?)
  * History of played cards (40 one-hot-encoded features?)
  
  This would give 120 columns/features in each state.  
  I am wondering wheter this is too much for the NN or wheter my input representation would be bad for the NN?  
  Should the features be represented as a (120,) vector or as a 3x40 matrix?
  
  I am also wondering if it's a good idea to represent the current cards on the table as just a 10 one-hot-encoded features since the type of the cards don't matter and the same number can't exist 2 times in the table?
  
  Thank you in advance.
  
  **Tags:** reinforcement-learning, dqn
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-07-06. Score: 3 | Accepted answer
  
  120 inputs can be handled by a complex enough network. Dealing with high complexity is one of NN's strengths.
  
  Using a (120,) vector or a (3,40) matrix is the same, they're still 120 inputs. Your binary encoding should work. Another option is a single (40,) vector, with 0 being "still in deck", 1 being "in hand", 2 being "on table", 3 being "already played".
  
  If the types of cards are irrelevant, you could actually have a (3,10) matrix with counters of cards (1 in hand, 1 in table, 2 already played). You can try different approaches and see what works best.
  
  > **Comments:**
  > **username_0**: Never knew that we can encode features as non binary... Can you tell me the name of this "technique"? Thank you for the informations.  
  > **username_0**: For the (40,) vector, wouldn't encoding like that mean that there is some kind of order?  
  > **username_1**: Imagine you had one ace in hand, one on table, 2 played already. How could you code that with a single (10,) vector? You need 3 rows, one for each place. Its simply called non binary input, its very common. Image processing NNs use inputs with pixel values [0,255]  
  > **username_0**: I edited my second comment in case, please take a look and thank you!  
  > **username_1**: When using tokens to code information (1 means something, 2 something else, etc) you need a more complex network that learns to distinguish the meaning of these codes, and learns that they do not represent order or quantity, yes.  
  > **username_0**: One last question please: Does it matter if the score increases when we have the same number twice in our hand? Like if we have 2, 2, 5, 7 then the pair of 2 gives us extra score.  
  > **username_0**: This type of thing (having a pair of the same number) needs to be called by the player, should I add an action to the NN?  
  > **username_1**: I didnt et the question, but it might be worth openig up a new question in the stack  
  > **username_2**: What I do not get here is what are your actions? Play card x? Does this simply come out of:  state0 = torch.tensor(state, dtype=torch.float)    prediction = self.model(state0)    action = torch.argmax(prediction).item()  
  > **username_1**: @username_2 4 outpouts representing the 4 cards in hand, ordered, could work  
  
  ---
  
  