Answers.Body: <p>This looks like a job for a multiclass classifier. You can build a classifier model to predict one of your three classes and train it on test data with known correct outputs very easily.</p>
  <p>Although you can fit reinforcement learning (RL) to the problem, there is no real benefit to doing so, and learning will be less efficient, since RL works on trial-and error. There is no need for trial-and-error here, or conversion into a reward score. Instead you can directly use an error gradient from the prediction.</p>
  <p>If your purpose is to find a simple game where RL could be applied, so you can practise RL, then you need to make a more complex game, which has time steps and actions that have consequences involving changes to state. A common toy game for teaching yourself RL would be Blackjack (aka 21), where the actions are &quot;twist&quot; or &quot;stick&quot; - see <a href="http://incompleteideas.net/book/the-book.html" rel="nofollow noreferrer">Sutton &amp; Barto: Reinforcement Learning, an Introduction</a> chapter 5, section 5.3 which presents a simplified version of the game and what a solution looks like.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-09T18:32:56.000Z
Answers.Text: I'd suggest multi armed bandit as an even easier entry point
Answers.UserDisplayName: chessprogrammer
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-12T14:59:00.897Z
Answers.LastActivityDate: 2022-07-12T15:04:54.097Z
Answers.LastEditDate: 2022-07-12T15:04:54.097Z
Answers.UserDisplayName: Neil Slater
Body: <p>Let's imagine this simple environment :</p>
  <p>Each episode has a length of 1 step. Each action leads to a reward for this action.</p>
  <p>The action space is of 3 : 'UP', 'DOWN', 'UNKNOWN'</p>
  <p>Most of the time, the observation is a random vector. But sometimes, it is a sinusoid. The player has to predict if the next value is greater or smaller than the last one. If the vector is a sinusoid, it is very easy to predict what the next value would be. If it is random, there is no way to know.</p>
  <p>If the answer is correct, the player gets a reward of 1. If it is incorrect, -1. If the action taken is 'UNKNOWN', 0.</p>
  <p>A good model would predict 'UP' or 'DOWN' when the observation is a sinusoid, and 'UNKNOW' when it is random.</p>
  <p>I thought it would be possible to solve this game with a Reinforcment Learning algorithm, what are your thougths ?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-07-12T14:54:37.313Z
Comments.Text: Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.
Comments.UserDisplayName: Community
Tags: <reinforcement-learning>
Title: Is this simple game solvable with reinforcment learning?
UserDisplayName: Lucien Leseigle
ThreadText: # Is this simple game solvable with reinforcment learning?
  
  (Asked by: username_0 on 2022-07-12)
  
  Let's imagine this simple environment :
  
  Each episode has a length of 1 step. Each action leads to a reward for this action.
  
  The action space is of 3 : 'UP', 'DOWN', 'UNKNOWN'
  
  Most of the time, the observation is a random vector. But sometimes, it is a sinusoid. The player has to predict if the next value is greater or smaller than the last one. If the vector is a sinusoid, it is very easy to predict what the next value would be. If it is random, there is no way to know.
  
  If the answer is correct, the player gets a reward of 1. If it is incorrect, -1. If the action taken is 'UNKNOWN', 0.
  
  A good model would predict 'UP' or 'DOWN' when the observation is a sinusoid, and 'UNKNOW' when it is random.
  
  I thought it would be possible to solve this game with a Reinforcment Learning algorithm, what are your thougths ?
  
  **Tags:** reinforcement-learning
  
  > **Comments:**
  > **username_1**: Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-07-12. Score: 1
  
  This looks like a job for a multiclass classifier. You can build a classifier model to predict one of your three classes and train it on test data with known correct outputs very easily.
  
  Although you can fit reinforcement learning (RL) to the problem, there is no real benefit to doing so, and learning will be less efficient, since RL works on trial-and error. There is no need for trial-and-error here, or conversion into a reward score. Instead you can directly use an error gradient from the prediction.
  
  If your purpose is to find a simple game where RL could be applied, so you can practise RL, then you need to make a more complex game, which has time steps and actions that have consequences involving changes to state. A common toy game for teaching yourself RL would be Blackjack (aka 21), where the actions are "twist" or "stick" - see [Sutton & Barto: Reinforcement Learning, an Introduction](http://incompleteideas.net/book/the-book.html) chapter 5, section 5.3 which presents a simplified version of the game and what a solution looks like.
  
  > **Comments:**
  > **username_3**: I'd suggest multi armed bandit as an even easier entry point  
  
  ---
  
  