Answers.Body: <p>Yes, but we're talking about <em>a lot</em> of ratings, like, millions of ratings. You have to automate rating generation, human feedback would just get way too expensive.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-09T06:11:30.457Z
Answers.Text: One alternative would be to have the AI play versus another AI in a reinforcement learning scenario, which is a fairly common strategy. (You can do this in parallel for many hours.) Of course, there are some drawbacks. It gets more complicated to code and they don't always behave like you want them to. If your simulated AIs start taking advantage of edge glitches or decide it's far better to not compete at all, it wouldn't be very useful for normal gameplay.
Answers.UserDisplayName: NotANumber
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-09T17:08:34.083Z
Answers.Text: @NotANumber Sure. You've suggested a very reasonable means of automating it. :)
Answers.UserDisplayName: Emilio M Bumachar
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-08T13:11:21.327Z
Answers.LastActivityDate: 2024-04-08T13:11:21.327Z
Answers.UserDisplayName: Emilio M Bumachar
Answers.Body: <p>Yes, this is a stereotypical reinforcement learning problem.  Instead of trying to calculate the dynamics of the environment the agent is given a reward or punishment for its behavior in the environment (shooting an enemy, etc).  The training process tries to find a policy to maximize the reward signal.  It’s a rather deep field and a bit much for a single post.  Maybe try reading one or two papers on Deep Q Network (DQN) or Proximal policy optimization (PPO) and see if either can be formulated in a way to work for your problem.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-09T01:41:53.247Z
Answers.Text: If the neural network outputs a sequence of actions, and we only find out at the end whether that entire sequence was successful, it is a RL problem.  The problem as stated does not appear to have that form.  Instead, the neural network appears to make a single choice (the angle to shoot at) and then we learn whether that was a good choice.  That is a simple supervised learning problem -- no need for the machinery of reinforcement learning.
Answers.UserDisplayName: D.W.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-09T10:50:42.350Z
Answers.Text: It would probably be helpful to spell out the full names of DQN and PPO (Deep Q-Network and Proximal Policy Optimization?).
Answers.UserDisplayName: jcaron
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-09T12:19:25.097Z
Answers.Text: @D.W. I can see your argument there.  I took there being a bot and a human to mean  that it might be competitive and round based.  A supervised learner trained to make a locally optimal choice may miss a broader strategy.
Answers.UserDisplayName: foreverska
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-08T01:24:38.697Z
Answers.LastActivityDate: 2024-04-09T12:02:01.750Z
Answers.LastEditDate: 2024-04-09T12:02:01.750Z
Answers.UserDisplayName: foreverska
Body: <p>I am currently trying to train a bot for a game I am creating. It is a 2d game with a complex map made of various shapes. The bot and character shoot bullets that are capable of ricocheting. The neural network outputs a vector in which the bot will turn and then fire. I myself cannot calculate the correct trajectory and find the Loss of the network. But, I can give a rating on how well the neural network performs when it fires. Can I train it by simply giving it a rating, and if, how so?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-04-08T02:26:58.700Z
Comments.Text: @foreverska, can I just rate it on the damage the bot deals to the character? Higher damage = lower loss and vice-versa, then just back-propagate?
Comments.UserDisplayName: Beluker
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-04-08T06:00:44.253Z
Comments.Text: No, it's more complex than that, although there are simplified reinforcement learning approaches. If your action choices are discrete, you might like to look into the Cross Entropy Method (CEM), which is where you run e.g. 100 episodes, take the top 10 performing ones and treat every action taken in them as being correct to train a multi-class classifier. Clearly that's not true, but the small degree to which it is true (some of the actions will be better choices) can be enough to train the neural network.
Comments.UserDisplayName: Neil Slater
Tags: <neural-networks><training>
Title: Can you train a neural network by simply giving it ratings each time it runs?
UserDisplayName: Beluker
ThreadText: # Can you train a neural network by simply giving it ratings each time it runs?
  
  (Asked by: username_0 on 2024-04-08)
  
  I am currently trying to train a bot for a game I am creating. It is a 2d game with a complex map made of various shapes. The bot and character shoot bullets that are capable of ricocheting. The neural network outputs a vector in which the bot will turn and then fire. I myself cannot calculate the correct trajectory and find the Loss of the network. But, I can give a rating on how well the neural network performs when it fires. Can I train it by simply giving it a rating, and if, how so?
  
  **Tags:** neural-networks, training
  
  > **Comments:**
  > **username_0**: @username_4, can I just rate it on the damage the bot deals to the character? Higher damage = lower loss and vice-versa, then just back-propagate?  
  > **username_1**: No, it's more complex than that, although there are simplified reinforcement learning approaches. If your action choices are discrete, you might like to look into the Cross Entropy Method (CEM), which is where you run e.g. 100 episodes, take the top 10 performing ones and treat every action taken in them as being correct to train a multi-class classifier. Clearly that's not true, but the small degree to which it is true (some of the actions will be better choices) can be enough to train the neural network.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_4 on 2024-04-08. Score: 19 | Accepted answer
  
  Yes, this is a stereotypical reinforcement learning problem. Instead of trying to calculate the dynamics of the environment the agent is given a reward or punishment for its behavior in the environment (shooting an enemy, etc). The training process tries to find a policy to maximize the reward signal. It’s a rather deep field and a bit much for a single post. Maybe try reading one or two papers on Deep Q Network (DQN) or Proximal policy optimization (PPO) and see if either can be formulated in a way to work for your problem.
  
  > **Comments:**
  > **username_5**: If the neural network outputs a sequence of actions, and we only find out at the end whether that entire sequence was successful, it is a RL problem.  The problem as stated does not appear to have that form.  Instead, the neural network appears to make a single choice (the angle to shoot at) and then we learn whether that was a good choice.  That is a simple supervised learning problem -- no need for the machinery of reinforcement learning.  
  > **username_6**: It would probably be helpful to spell out the full names of DQN and PPO (Deep Q-Network and Proximal Policy Optimization?).  
  > **username_4**: @D.W. I can see your argument there.  I took there being a bot and a human to mean  that it might be competitive and round based.  A supervised learner trained to make a locally optimal choice may miss a broader strategy.  
  
  ---
  
  ### Answer by username_2 on 2024-04-08. Score: 11
  
  Yes, but we're talking about *a lot* of ratings, like, millions of ratings. You have to automate rating generation, human feedback would just get way too expensive.
  
  > **Comments:**
  > **username_3**: One alternative would be to have the AI play versus another AI in a reinforcement learning scenario, which is a fairly common strategy. (You can do this in parallel for many hours.) Of course, there are some drawbacks. It gets more complicated to code and they don't always behave like you want them to. If your simulated AIs start taking advantage of edge glitches or decide it's far better to not compete at all, it wouldn't be very useful for normal gameplay.  
  > **username_2**: @username_3 Sure. You've suggested a very reasonable means of automating it. :)  
  
  ---
  
  