Answers.Body: <p>1) your input should be so that you describe your entire environment. This could be done by 8 (length)* 8 (height)* 3 (either empty space, opponent chip or your chip) = 192 input neurons. you can just import a image of the current boardstate (which is width pixels * height pixels input neurons), but this means you task the neural network with learning to read the image as wel as playing the game.</p>
  
  <p>2) rewards for a game like this are (most of the time) 1 for a winning move and 0 for a losing move. if you do it your way it might have side effects like prioritizing lining 2 chips up instead of going for a winning move.</p>
  
  <p>3) Tensorflow is most commenly used. (this is python and not javascript though)</p>
  
  <p>4) I liked the book Reinforcement learning, an introduction and would recommand this to anyone.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-04T14:10:17.663Z
Answers.LastActivityDate: 2019-12-04T14:10:17.663Z
Answers.UserDisplayName: Lustwelpintje
Body: <p>I've made a connect 4 game in javascript, and I want to design an AI for it. I made a post the other day about what output would be needed, and I think I could use images of the board and a CNN. I did some research into Reinforcement learning, and I think that's what I need to do. I don't have much experience with ML in general, much less RL with Q-learning, but that is what I'd like to do. </p>
  
  <p>Now, I don't really know how to start out with such a big project. I have a few questions first:</p>
  
  <ol>
  <li><p>What do I do with my input? I'm thinking I give the AI 7 options for moves to make, one for each row of the board. How do I implement a way that it can "look" at the board? Can I just import an image of the current board state?</p></li>
  <li><p>How do I make a reward table? How should I do the points system for the Q-learning? I'm thinking something like: If it drops a chip it gets a point, if it lines up 2 chips in a row it gets 5 points, if it gets 3 in a row it gets 30, and if it gets 4 in a row it gets 100. Would that be an effective way to do this? How do I implement this?</p></li>
  <li><p>Is there a library I can use to do any of the work where I make an algorithm and board states and reward tables? Or do I have to hard code any of it?</p></li>
  <li><p>I've done some research, are there any links or tutorials you think I should read or follow along with? Any other general advice or help for me?</p></li>
  </ol>
  
  <p>I greatly appreciate anyone who answers one or all of these questions! Thank you so much!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-04T10:22:28.020Z
Comments.Text: You don't need reinforcement learning for this, or any kind of ML, but it would work and is a fun thing to study and apply in this kind of game environment. Could you clarify whether your question is about AI routines for connect 4 (and you don't really care whether it is ML or RL), or is it about using connect 4 as a case study for Reinforcement Learning? Or do you just want a connect 4 bot (I expect they already exist)?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-04T12:18:43.223Z
Comments.Text: @NeilSlater I want to use ML or RL as a case study. I'm really interested in making my own AIs or bots, and learning how to write neural networks and things like that. I don't want just the solution, I want to design it and make it the hard way. I just don't know how to start.
Comments.UserDisplayName: joelanbanks3
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-04T14:39:41.070Z
Comments.Text: You're asking too many questions here. Please, ask only one question per post.
Comments.UserDisplayName: nbro
Tags: <deep-learning><reinforcement-learning><ai-design><game-ai>
Title: Designing a reinforcement learning AI for a game of connect 4
UserDisplayName: joelanbanks3
ThreadText: # Designing a reinforcement learning AI for a game of connect 4
  
  (Asked by: username_0 on 2019-12-03)
  
  I've made a connect 4 game in javascript, and I want to design an AI for it. I made a post the other day about what output would be needed, and I think I could use images of the board and a CNN. I did some research into Reinforcement learning, and I think that's what I need to do. I don't have much experience with ML in general, much less RL with Q-learning, but that is what I'd like to do.
  
  Now, I don't really know how to start out with such a big project. I have a few questions first:
  
  1. What do I do with my input? I'm thinking I give the AI 7 options for moves to make, one for each row of the board. How do I implement a way that it can "look" at the board? Can I just import an image of the current board state?
  2. How do I make a reward table? How should I do the points system for the Q-learning? I'm thinking something like: If it drops a chip it gets a point, if it lines up 2 chips in a row it gets 5 points, if it gets 3 in a row it gets 30, and if it gets 4 in a row it gets 100. Would that be an effective way to do this? How do I implement this?
  3. Is there a library I can use to do any of the work where I make an algorithm and board states and reward tables? Or do I have to hard code any of it?
  4. I've done some research, are there any links or tutorials you think I should read or follow along with? Any other general advice or help for me?
  
  I greatly appreciate anyone who answers one or all of these questions! Thank you so much!
  
  **Tags:** deep-learning, reinforcement-learning, ai-design, game-ai
  
  > **Comments:**
  > **username_1**: You don't need reinforcement learning for this, or any kind of ML, but it would work and is a fun thing to study and apply in this kind of game environment. Could you clarify whether your question is about AI routines for connect 4 (and you don't really care whether it is ML or RL), or is it about using connect 4 as a case study for Reinforcement Learning? Or do you just want a connect 4 bot (I expect they already exist)?  
  > **username_0**: @username_1 I want to use ML or RL as a case study. I'm really interested in making my own AIs or bots, and learning how to write neural networks and things like that. I don't want just the solution, I want to design it and make it the hard way. I just don't know how to start.  
  > **username_2**: You're asking too many questions here. Please, ask only one question per post.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2019-12-04. Score: 1
  
  1) your input should be so that you describe your entire environment. This could be done by 8 (length)\* 8 (height)\* 3 (either empty space, opponent chip or your chip) = 192 input neurons. you can just import a image of the current boardstate (which is width pixels \* height pixels input neurons), but this means you task the neural network with learning to read the image as wel as playing the game.
  
  2) rewards for a game like this are (most of the time) 1 for a winning move and 0 for a losing move. if you do it your way it might have side effects like prioritizing lining 2 chips up instead of going for a winning move.
  
  3) Tensorflow is most commenly used. (this is python and not javascript though)
  
  4) I liked the book Reinforcement learning, an introduction and would recommand this to anyone.
  
  
  ---
  
  