<p>From <a href="https://en.wikipedia.org/wiki/Connect_Four" rel="nofollow noreferrer">wikipedia article on  Connect Four</a>:</p>

<blockquote>
  <p>Connect Four is a solved game. The first player can always win by
  playing the right moves.</p>
</blockquote>

<p>It's pretty reasonable that even without training 1st player win more often by small margin.
With some training  1st player should mostly win.</p>

CC BY-SA 4.0
2019-06-26T07:00:56.820Z
2019-06-26T07:00:56.820Z
mirror2image
<p>I'm testing various learning rates and neural network configurations. I'm testing over 10000 games, with the first 2000 having random starting moves and general randomness throughout of about 20%, i.e. 20% of moves are random.</p>

<p>In all configurations, I initiate the weights to random values.</p>

<p>What I've found is that in all configurations, Player 1 will win the majority of games, or Player 2 will. There's no 50/50 split.</p>

<p>Is this expected or normal? I am training on the Connect Four game.</p>

CC BY-SA 3.0
2017-12-14T03:11:10.240Z
This is weird, how big are the majorities? I would also decrease your epsilon (probability of random move) as training progresses. Do you test with an epsilon?
Jaden Travnik
CC BY-SA 3.0
2017-12-14T04:13:10.297Z
@jaden What are "the majorities", I'm not familiar with this term. I do reduce the randomness over time, it starts at something like 50% and drops to 10% after the first 10% of moves. 20% was way off.
NeomerArcana
CC BY-SA 3.0
2017-12-14T12:55:00.200Z
By majorities i mean by how much does player 1 beat player 2 (eg 90%) and vice versa.
Jaden Travnik
CC BY-SA 3.0
2017-12-14T19:07:49.193Z
It's hard to tell, it's a win or lose, there's no score or anything.
NeomerArcana
CC BY-SA 3.0
2017-12-14T23:53:44.850Z
Sorry, I meant in regards to what you said about there is no 50/50 split. What is the split? Like 90/10?
Jaden Travnik
CC BY-SA 3.0
2017-12-15T05:54:50.177Z
@Jaden, yeah it's like 90/10 but starts to go up the more i increase the amount of random moves, but only because those moves ending randomly are getting wins for the other player. It's  also sometimes player 1 or sometimes player 2, I believe related to what the random weights maybe lean to at the start.
NeomerArcana
CC BY-SA 4.0
2018-08-17T15:33:19.503Z
What Reinforcement Learning algorithm are you using? Something like Q-learning, or Policy Gradients? Can you also elaborate on how the "self-play" is set up? Does each player have a Network of their own (and if so, are they using same or different configs), or are they both using and learning parameters for the same network at the same time? I realise this request for more info is a bit late, hopefully you're still around :)
Dennis Soemers
CC BY-SA 4.0
2018-08-17T23:00:42.580Z
@dennis_soemers I am always around. It was using backward propagation. They were using the same network. The game was Connect4 so I figured they didn't need their own networks as the state given is always the same, just swapped out the "player" vs "opponent" values in the input.
NeomerArcana
<neural-networks><reinforcement-learning>
Is it expected that during self-play reinforcement learning that player 1 or player 2 wins the majority of games?
NeomerArcana
# Is it expected that during self-play reinforcement learning that player 1 or player 2 wins the majority of games?

(Asked by: username_0 on 2017-12-10)

I'm testing various learning rates and neural network configurations. I'm testing over 10000 games, with the first 2000 having random starting moves and general randomness throughout of about 20%, i.e. 20% of moves are random.

In all configurations, I initiate the weights to random values.

What I've found is that in all configurations, Player 1 will win the majority of games, or Player 2 will. There's no 50/50 split.

Is this expected or normal? I am training on the Connect Four game.

**Tags:** neural-networks, reinforcement-learning

> **Comments:**
> **username_1**: This is weird, how big are the majorities? I would also decrease your epsilon (probability of random move) as training progresses. Do you test with an epsilon?  
> **username_0**: @jaden What are "the majorities", I'm not familiar with this term. I do reduce the randomness over time, it starts at something like 50% and drops to 10% after the first 10% of moves. 20% was way off.  
> **username_1**: By majorities i mean by how much does player 1 beat player 2 (eg 90%) and vice versa.  
> **username_0**: It's hard to tell, it's a win or lose, there's no score or anything.  
> **username_1**: Sorry, I meant in regards to what you said about there is no 50/50 split. What is the split? Like 90/10?  
> **username_0**: @Jaden, yeah it's like 90/10 but starts to go up the more i increase the amount of random moves, but only because those moves ending randomly are getting wins for the other player. It's  also sometimes player 1 or sometimes player 2, I believe related to what the random weights maybe lean to at the start.  
> **username_2**: What Reinforcement Learning algorithm are you using? Something like Q-learning, or Policy Gradients? Can you also elaborate on how the "self-play" is set up? Does each player have a Network of their own (and if so, are they using same or different configs), or are they both using and learning parameters for the same network at the same time? I realise this request for more info is a bit late, hopefully you're still around :)  
> **username_0**: @dennis_soemers I am always around. It was using backward propagation. They were using the same network. The game was Connect4 so I figured they didn't need their own networks as the state given is always the same, just swapped out the "player" vs "opponent" values in the input.  

---

## 1 answers

### Answer by username_3 on 2019-06-26. Score: -2

From [wikipedia article on Connect Four](https://en.wikipedia.org/wiki/Connect_Four):

> Connect Four is a solved game. The first player can always win by
> playing the right moves.

It's pretty reasonable that even without training 1st player win more often by small margin.
With some training 1st player should mostly win.


---

