Answers.Body: <p>From <a href="https://en.wikipedia.org/wiki/Connect_Four" rel="nofollow noreferrer">wikipedia article on  Connect Four</a>:</p>
  
  <blockquote>
    <p>Connect Four is a solved game. The first player can always win by
    playing the right moves.</p>
  </blockquote>
  
  <p>It's pretty reasonable that even without training 1st player win more often by small margin.
  With some training  1st player should mostly win.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-26T07:00:56.820Z
Answers.LastActivityDate: 2019-06-26T07:00:56.820Z
Answers.UserDisplayName: mirror2image
Body: <p>I'm testing various learning rates and neural network configurations. I'm testing over 10000 games, with the first 2000 having random starting moves and general randomness throughout of about 20%, i.e. 20% of moves are random.</p>
  
  <p>In all configurations, I initiate the weights to random values.</p>
  
  <p>What I've found is that in all configurations, Player 1 will win the majority of games, or Player 2 will. There's no 50/50 split.</p>
  
  <p>Is this expected or normal? I am training on the Connect Four game.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-12-14T03:11:10.240Z
Comments.Text: This is weird, how big are the majorities? I would also decrease your epsilon (probability of random move) as training progresses. Do you test with an epsilon?
Comments.UserDisplayName: Jaden Travnik
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-12-14T04:13:10.297Z
Comments.Text: @jaden What are "the majorities", I'm not familiar with this term. I do reduce the randomness over time, it starts at something like 50% and drops to 10% after the first 10% of moves. 20% was way off.
Comments.UserDisplayName: NeomerArcana
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-12-14T12:55:00.200Z
Comments.Text: By majorities i mean by how much does player 1 beat player 2 (eg 90%) and vice versa.
Comments.UserDisplayName: Jaden Travnik
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-12-14T19:07:49.193Z
Comments.Text: It's hard to tell, it's a win or lose, there's no score or anything.
Comments.UserDisplayName: NeomerArcana
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-12-14T23:53:44.850Z
Comments.Text: Sorry, I meant in regards to what you said about there is no 50/50 split. What is the split? Like 90/10?
Comments.UserDisplayName: Jaden Travnik
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-12-15T05:54:50.177Z
Comments.Text: @Jaden, yeah it's like 90/10 but starts to go up the more i increase the amount of random moves, but only because those moves ending randomly are getting wins for the other player. It's  also sometimes player 1 or sometimes player 2, I believe related to what the random weights maybe lean to at the start.
Comments.UserDisplayName: NeomerArcana
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-08-17T15:33:19.503Z
Comments.Text: What Reinforcement Learning algorithm are you using? Something like Q-learning, or Policy Gradients? Can you also elaborate on how the "self-play" is set up? Does each player have a Network of their own (and if so, are they using same or different configs), or are they both using and learning parameters for the same network at the same time? I realise this request for more info is a bit late, hopefully you're still around :)
Comments.UserDisplayName: Dennis Soemers
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-08-17T23:00:42.580Z
Comments.Text: @dennis_soemers I am always around. It was using backward propagation. They were using the same network. The game was Connect4 so I figured they didn't need their own networks as the state given is always the same, just swapped out the "player" vs "opponent" values in the input.
Comments.UserDisplayName: NeomerArcana
Tags: <neural-networks><reinforcement-learning>
Title: Is it expected that during self-play reinforcement learning that player 1 or player 2 wins the majority of games?
UserDisplayName: NeomerArcana
ThreadText: # Is it expected that during self-play reinforcement learning that player 1 or player 2 wins the majority of games?
  
  (Asked by: username_0 on 2017-12-10)
  
  I'm testing various learning rates and neural network configurations. I'm testing over 10000 games, with the first 2000 having random starting moves and general randomness throughout of about 20%, i.e. 20% of moves are random.
  
  In all configurations, I initiate the weights to random values.
  
  What I've found is that in all configurations, Player 1 will win the majority of games, or Player 2 will. There's no 50/50 split.
  
  Is this expected or normal? I am training on the Connect Four game.
  
  **Tags:** neural-networks, reinforcement-learning
  > **Comments:**
  > **username_1**: This is weird, how big are the majorities? I would also decrease your epsilon (probability of random move) as training progresses. Do you test with an epsilon?  
  > **username_0**: @jaden What are "the majorities", I'm not familiar with this term. I do reduce the randomness over time, it starts at something like 50% and drops to 10% after the first 10% of moves. 20% was way off.  
  > **username_1**: By majorities i mean by how much does player 1 beat player 2 (eg 90%) and vice versa.  
  > **username_0**: It's hard to tell, it's a win or lose, there's no score or anything.  
  > **username_1**: Sorry, I meant in regards to what you said about there is no 50/50 split. What is the split? Like 90/10?  
  > **username_0**: @Jaden, yeah it's like 90/10 but starts to go up the more i increase the amount of random moves, but only because those moves ending randomly are getting wins for the other player. It's  also sometimes player 1 or sometimes player 2, I believe related to what the random weights maybe lean to at the start.  
  > **username_2**: What Reinforcement Learning algorithm are you using? Something like Q-learning, or Policy Gradients? Can you also elaborate on how the "self-play" is set up? Does each player have a Network of their own (and if so, are they using same or different configs), or are they both using and learning parameters for the same network at the same time? I realise this request for more info is a bit late, hopefully you're still around :)  
  > **username_0**: @dennis_soemers I am always around. It was using backward propagation. They were using the same network. The game was Connect4 so I figured they didn't need their own networks as the state given is always the same, just swapped out the "player" vs "opponent" values in the input.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2019-06-26. Score: -2
  
  From [wikipedia article on Connect Four](https://en.wikipedia.org/wiki/Connect_Four):
  
  > Connect Four is a solved game. The first player can always win by
  > playing the right moves.
  
  It's pretty reasonable that even without training 1st player win more often by small margin.
  With some training 1st player should mostly win.
  
  
  ---
  
  