Body: <p>I'm working currently on a problem and I'm using RL (bandit problem). </p>
  
  <p>In my system, I have an agent that chooses an action among <span class="math-container">$k$</span> possible actions, and a user that decides whether the agent chooses the right action or not. If the user is satisfied with the decision made by the agent, he rewards with <span class="math-container">$+1$</span>, otherwise <span class="math-container">$-1$</span>.</p>
  
  <p>Is this is a good reward function, knowing that in my problem the values are in the range <span class="math-container">$[0, 1]$</span>? </p>
  
  <p>Are there any guidelines to follow for defining the reward function? Are there any references (books or articles) that tackle this problem and present a solution?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-18T15:55:08.077Z
Comments.Text: Why would you let a user decide if the action is good or not? if you have a classical bandit problem where each arm returns a value between 2 values. And you are saying these 2 values are 0 and 1. Why not return this value the arm returns as reward?
Comments.UserDisplayName: Lustwelpintje
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-18T16:32:25.397Z
Comments.Text: I need the user to be part of the system, it's the same thing to say that the arm return a value or the user send the value.
Comments.UserDisplayName: Walid
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-18T21:27:13.673Z
Comments.Text: No, because now you need the user to decide on every action. You should automate the reward function so you can run through thousands of training steps in seconds.
Comments.UserDisplayName: Lustwelpintje
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-19T07:47:38.270Z
Comments.Text: For the training, I automated it but when the system will be deployed the user will be involved.
Comments.UserDisplayName: Walid
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-19T09:43:36.220Z
Comments.Text: And how did you automate it for training?
Comments.UserDisplayName: Lustwelpintje
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-20T21:02:36.103Z
Comments.Text: each arm returns 1 or -1
Comments.UserDisplayName: Walid
Tags: <reinforcement-learning><ai-design><rewards>
Title: What are the guidelines for defining a reward function in reinforcement learning (bandit problem)?
UserDisplayName: Walid
ThreadText: # What are the guidelines for defining a reward function in reinforcement learning (bandit problem)?
  
  (Asked by: username_0 on 2020-03-18)
  
  I'm working currently on a problem and I'm using RL (bandit problem).
  
  In my system, I have an agent that chooses an action among $k$ possible actions, and a user that decides whether the agent chooses the right action or not. If the user is satisfied with the decision made by the agent, he rewards with $+1$, otherwise $-1$.
  
  Is this is a good reward function, knowing that in my problem the values are in the range $[0, 1]$?
  
  Are there any guidelines to follow for defining the reward function? Are there any references (books or articles) that tackle this problem and present a solution?
  
  **Tags:** reinforcement-learning, ai-design, rewards
  
  > **Comments:**
  > **username_1**: Why would you let a user decide if the action is good or not? if you have a classical bandit problem where each arm returns a value between 2 values. And you are saying these 2 values are 0 and 1. Why not return this value the arm returns as reward?  
  > **username_0**: I need the user to be part of the system, it's the same thing to say that the arm return a value or the user send the value.  
  > **username_1**: No, because now you need the user to decide on every action. You should automate the reward function so you can run through thousands of training steps in seconds.  
  > **username_0**: For the training, I automated it but when the system will be deployed the user will be involved.  
  > **username_1**: And how did you automate it for training?  
  > **username_0**: each arm returns 1 or -1  
  
  ---
  
  ## 0 answers
  
  