Answers.Body: <p>If it is a game you can try a simple weightage calculation where if the bot perform an action that yields a positive result - killed an enemy, gained an advantageous position etc. Add a 'weight' to that action that in similar circumstances the chances of performing that action that will lead to a positive result is higher.</p>
  
  <p>Yet due to the chance of not performing an action that was remembered to yield positive results, there is a little bit of 'randomness' and also a chance to discover new possibilities. Just remember not to let a single occurrence shift the weightage too much or allow a single action's weightage to become so high that the AI stops trying different actions on similar situations.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-12-16T01:43:35.673Z
Answers.LastActivityDate: 2016-12-16T01:43:35.673Z
Answers.UserDisplayName: Ben Ong
Answers.Body: <h2>Reinforcement learning</h2>
  
  <p>The problem that you describe, namely, choosing a good sequence of actions based on a reward/score received based on the whole sequence (and possibly significantly delayed), is pretty much the textbook definition of <a href="https://en.wikipedia.org/wiki/Reinforcement_learning" rel="nofollow noreferrer">Reinforcement learning</a>.</p>
  
  <p>As with quite a few other topics, deep neural networks currently seem to be a promising way for solving this type of problems. <a href="http://karpathy.github.io/2016/05/31/rl/" rel="nofollow noreferrer">This</a> may be a beginner-friendly description of this approach.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-12-19T01:55:56.100Z
Answers.LastActivityDate: 2016-12-19T01:55:56.100Z
Answers.UserDisplayName: Peteris
Body: <p>How to train a bot, given a series of games in which he did (initially random) actions, to improve its behavior based on previous experiences?</p>
  
  <p>The bot has some actions: e.g. shoot, wait, move, etc. It's a turn based "game" in which, for know, I'm running the bots with some objectives (e.g. kill some other bot) and random actions. So every bot will have a score function that at the end of the game will say, from X to Y (0 to 100?) if they did well or not.</p>
  
  <p>So how to make the bots to learn of their previous experiences? Because this is not a fixed input as the neural networks take, this is kind of a list of games, each one in which the bot took several actions (one by every "turn"). The IA functions that I know are used to <em>predict</em> future values.. I'm not sure is the same.</p>
  
  <p>Maybe I should have a function that gets the "more similar previous games" that the bot played and checked what were the actions he took, if the results were bad he should take another action, if the results were good then he should take the same action. But this seems kind of hardcoded.</p>
  
  <p>Another option would be to train a neural network (somehow fixing the problem of the fixed input) based on previous game actions and to predict the future action's results in score (something that I guess it's similar to how chess and Go games work) and choose the one that seems to have better outcome.</p>
  
  <p>I hope this is not too abstract. I don't want to hardcode much stuff in the bots, I'd like them to learn by their own starting from a blank page.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-12-16T08:54:47.740Z
Comments.Text: I'm not sure how to answer your question, but I would point you to OpenAI Universe framework which seems to be quite fresh implementation of reinforcement learning envinroment: https://github.com/openai/universe-starter-agent. It is Open Source so you might find some inspiration there.
Comments.UserDisplayName: Damian Melniczuk
Tags: <ai-design>
Title: How to make a bot to learn from previous games
UserDisplayName: hhaamm
ThreadText: # How to make a bot to learn from previous games
  
  (Asked by: username_0 on 2016-12-16)
  
  How to train a bot, given a series of games in which he did (initially random) actions, to improve its behavior based on previous experiences?
  
  The bot has some actions: e.g. shoot, wait, move, etc. It's a turn based "game" in which, for know, I'm running the bots with some objectives (e.g. kill some other bot) and random actions. So every bot will have a score function that at the end of the game will say, from X to Y (0 to 100?) if they did well or not.
  
  So how to make the bots to learn of their previous experiences? Because this is not a fixed input as the neural networks take, this is kind of a list of games, each one in which the bot took several actions (one by every "turn"). The IA functions that I know are used to *predict* future values.. I'm not sure is the same.
  
  Maybe I should have a function that gets the "more similar previous games" that the bot played and checked what were the actions he took, if the results were bad he should take another action, if the results were good then he should take the same action. But this seems kind of hardcoded.
  
  Another option would be to train a neural network (somehow fixing the problem of the fixed input) based on previous game actions and to predict the future action's results in score (something that I guess it's similar to how chess and Go games work) and choose the one that seems to have better outcome.
  
  I hope this is not too abstract. I don't want to hardcode much stuff in the bots, I'd like them to learn by their own starting from a blank page.
  
  **Tags:** ai-design
  > **Comments:**
  > **username_1**: I'm not sure how to answer your question, but I would point you to OpenAI Universe framework which seems to be quite fresh implementation of reinforcement learning envinroment: https://github.com/openai/universe-starter-agent. It is Open Source so you might find some inspiration there.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2016-12-19. Score: 2 | Accepted answer
  
  ## Reinforcement learning
  
  The problem that you describe, namely, choosing a good sequence of actions based on a reward/score received based on the whole sequence (and possibly significantly delayed), is pretty much the textbook definition of [Reinforcement learning](https://en.wikipedia.org/wiki/Reinforcement_learning).
  
  As with quite a few other topics, deep neural networks currently seem to be a promising way for solving this type of problems. [This](http://karpathy.github.io/2016/05/31/rl/) may be a beginner-friendly description of this approach.
  
  
  ---
  
  ### Answer by username_2 on 2016-12-16. Score: 0
  
  If it is a game you can try a simple weightage calculation where if the bot perform an action that yields a positive result - killed an enemy, gained an advantageous position etc. Add a 'weight' to that action that in similar circumstances the chances of performing that action that will lead to a positive result is higher.
  
  Yet due to the chance of not performing an action that was remembered to yield positive results, there is a little bit of 'randomness' and also a chance to discover new possibilities. Just remember not to let a single occurrence shift the weightage too much or allow a single action's weightage to become so high that the AI stops trying different actions on similar situations.
  
  
  ---
  
  