Answers.Body: <p>Let's define your problem from another point of view. Let's say that in this RL problem you have two agents (agent1 and agent2) that compete with each other in order to accomplish their own goal, i.e., wining connect4 game.</p>
  
  <p>Therefore, we could say that from agent1's point of view, he is player1 and the player2 is agent2. The same way, from agent2's point of view, he is player1 and the player2 is agent1.  </p>
  
  <p>Here, we can use the same definition as you proposed:</p>
  
  <p>"let's say player1's pieces on the board are represented as -1, empty board as 0 and player2's pieces as 1"</p>
  
  <p>When it is the turn for agent1, he will receive an observation (representing the board configuration) from the environment. In this case, agent1's pieces are represented as -1, empty board as 0 and for player2's (agent2's) pieces as 1. Then, you could use the neural network to compute, e.g., the next action for agent1.</p>
  
  <p>When it is the turn for agent2, he will receive another observation from the environment, but this time you represent its own pieces as -1, while the agent1's (player2's from agent2's point of view) as 1.</p>
  
  <p>With this approach, the neural network will always receives as an input a representation in which the pieces 'it' controls are -1, while its rival's pieces are 1 and empty space 0. From the neural network's point of view, he does not care about which agent is playing as the actual observation is indistinguishable to calculate the next action.</p>
  
  <p>Therefore, you will have to generate two different observations, in which you invert the sign of the ones on it, depending on which turn it is (agent1's or agent2's turn).</p>
  
  <p>For example:</p>
  
  <p><a href="https://i.sstatic.net/KlgSW.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/KlgSW.png" alt="agent1&#39;s board"></a></p>
  
  <p><a href="https://i.sstatic.net/ER9xb.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ER9xb.png" alt="agent2&#39;s board"></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-24T20:40:03.493Z
Answers.Text: oh my, that makes a lot of sense. Is that what a canonical form is?  
  So to be sure I got this right.  
  My first layer will be exactly 7x6 = 42 Neurons, one for each position.  
  Every time a player took a turn, the board for the next player is simply multiplied by -1, to flip the board.  
  And at the end of the game, the result z € [-1,1] will be backpropgated as the same all the way, without flipping it, right?
Answers.UserDisplayName: Leviathan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-24T20:22:53.700Z
Answers.LastActivityDate: 2019-07-24T20:22:53.700Z
Answers.UserDisplayName: rjgp
Body: <p>First of all, it is great to have found this community!</p>
  
  <p>I am currently implementing my own Alpha Zero clone on Connect4. However, I have a mental barrier I cannot overcome.</p>
  
  <p>How can I use one neural network for both players? I do not understand what the input should be. </p>
  
  <p>Do I just put in the board position (<span class="math-container">$6 \times 7$</span>) and let's say Player1's pieces on the board are represented as <span class="math-container">$-1$</span>, empty board as <span class="math-container">$0$</span> and Player2's pieces as <span class="math-container">$1$</span>? </p>
  
  <p>To me, that seems the most efficient. But then, in the backpropagation, I feel like this cannot be working. If I update the same network for both players (which Alpha Zero does), don't I try to optimize Player1 and Player 2 at the same time?  </p>
  
  <p>I just can't get my head around it. 2 Neural networks, each for one player is understandable for me. But one network? I don't understand how to backpropagate? Should I just flip my "z" (the result of the game) every time I go one layer backward? Is that all there is to using one network?</p>
  
  <p>I hope I made this clear enough. I am quite confused, I tried my best. </p>
  
  <p>Thank you for reading this!</p>
  
Tags: <neural-networks><deep-learning><backpropagation><deep-neural-networks><alphazero>
Title: How can I use one neural network for both players in Alpha Zero (Connect 4)?
UserDisplayName: Leviathan
ThreadText: # How can I use one neural network for both players in Alpha Zero (Connect 4)?
  
  (Asked by: username_0 on 2019-07-24)
  
  First of all, it is great to have found this community!
  
  I am currently implementing my own Alpha Zero clone on Connect4. However, I have a mental barrier I cannot overcome.
  
  How can I use one neural network for both players? I do not understand what the input should be.
  
  Do I just put in the board position ($6 \times 7$) and let's say Player1's pieces on the board are represented as $-1$, empty board as $0$ and Player2's pieces as $1$?
  
  To me, that seems the most efficient. But then, in the backpropagation, I feel like this cannot be working. If I update the same network for both players (which Alpha Zero does), don't I try to optimize Player1 and Player 2 at the same time?
  
  I just can't get my head around it. 2 Neural networks, each for one player is understandable for me. But one network? I don't understand how to backpropagate? Should I just flip my "z" (the result of the game) every time I go one layer backward? Is that all there is to using one network?
  
  I hope I made this clear enough. I am quite confused, I tried my best.
  
  Thank you for reading this!
  
  **Tags:** neural-networks, deep-learning, backpropagation, deep-neural-networks, alphazero
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-07-24. Score: 3 | Accepted answer
  
  Let's define your problem from another point of view. Let's say that in this RL problem you have two agents (agent1 and agent2) that compete with each other in order to accomplish their own goal, i.e., wining connect4 game.
  
  Therefore, we could say that from agent1's point of view, he is player1 and the player2 is agent2. The same way, from agent2's point of view, he is player1 and the player2 is agent1.
  
  Here, we can use the same definition as you proposed:
  
  "let's say player1's pieces on the board are represented as -1, empty board as 0 and player2's pieces as 1"
  
  When it is the turn for agent1, he will receive an observation (representing the board configuration) from the environment. In this case, agent1's pieces are represented as -1, empty board as 0 and for player2's (agent2's) pieces as 1. Then, you could use the neural network to compute, e.g., the next action for agent1.
  
  When it is the turn for agent2, he will receive another observation from the environment, but this time you represent its own pieces as -1, while the agent1's (player2's from agent2's point of view) as 1.
  
  With this approach, the neural network will always receives as an input a representation in which the pieces 'it' controls are -1, while its rival's pieces are 1 and empty space 0. From the neural network's point of view, he does not care about which agent is playing as the actual observation is indistinguishable to calculate the next action.
  
  Therefore, you will have to generate two different observations, in which you invert the sign of the ones on it, depending on which turn it is (agent1's or agent2's turn).
  
  For example:
  
  [![agent1's board](https://i.sstatic.net/KlgSW.png)](https://i.sstatic.net/KlgSW.png)
  
  [![agent2's board](https://i.sstatic.net/ER9xb.png)](https://i.sstatic.net/ER9xb.png)
  
  > **Comments:**
  > **username_0**: oh my, that makes a lot of sense. Is that what a canonical form is?   So to be sure I got this right.   My first layer will be exactly 7x6 = 42 Neurons, one for each position.   Every time a player took a turn, the board for the next player is simply multiplied by -1, to flip the board.   And at the end of the game, the result z € [-1,1] will be backpropgated as the same all the way, without flipping it, right?  
  
  ---
  
  