<p>I'm working currently on a problem and I'm using RL (bandit problem). </p>

<p>In my system, I have an agent that chooses an action among <span class="math-container">$k$</span> possible actions, and a user that decides whether the agent chooses the right action or not. If the user is satisfied with the decision made by the agent, he rewards with <span class="math-container">$+1$</span>, otherwise <span class="math-container">$-1$</span>.</p>

<p>Is this is a good reward function, knowing that in my problem the values are in the range <span class="math-container">$[0, 1]$</span>? </p>

<p>Are there any guidelines to follow for defining the reward function? Are there any references (books or articles) that tackle this problem and present a solution?</p>

CC BY-SA 4.0
2020-03-18T15:55:08.077Z
Why would you let a user decide if the action is good or not? if you have a classical bandit problem where each arm returns a value between 2 values. And you are saying these 2 values are 0 and 1. Why not return this value the arm returns as reward?
Lustwelpintje
CC BY-SA 4.0
2020-03-18T16:32:25.397Z
I need the user to be part of the system, it's the same thing to say that the arm return a value or the user send the value.
Walid
CC BY-SA 4.0
2020-03-18T21:27:13.673Z
No, because now you need the user to decide on every action. You should automate the reward function so you can run through thousands of training steps in seconds.
Lustwelpintje
CC BY-SA 4.0
2020-03-19T07:47:38.270Z
For the training, I automated it but when the system will be deployed the user will be involved.
Walid
CC BY-SA 4.0
2020-03-19T09:43:36.220Z
And how did you automate it for training?
Lustwelpintje
CC BY-SA 4.0
2020-03-20T21:02:36.103Z
each arm returns 1 or -1
Walid
<reinforcement-learning><ai-design><rewards>
What are the guidelines for defining a reward function in reinforcement learning (bandit problem)?
Walid
# What are the guidelines for defining a reward function in reinforcement learning (bandit problem)?

(Asked by: username_0 on 2020-03-18)

I'm working currently on a problem and I'm using RL (bandit problem).

In my system, I have an agent that chooses an action among $k$ possible actions, and a user that decides whether the agent chooses the right action or not. If the user is satisfied with the decision made by the agent, he rewards with $+1$, otherwise $-1$.

Is this is a good reward function, knowing that in my problem the values are in the range $[0, 1]$?

Are there any guidelines to follow for defining the reward function? Are there any references (books or articles) that tackle this problem and present a solution?

**Tags:** reinforcement-learning, ai-design, rewards

> **Comments:**
> **username_1**: Why would you let a user decide if the action is good or not? if you have a classical bandit problem where each arm returns a value between 2 values. And you are saying these 2 values are 0 and 1. Why not return this value the arm returns as reward?  
> **username_0**: I need the user to be part of the system, it's the same thing to say that the arm return a value or the user send the value.  
> **username_1**: No, because now you need the user to decide on every action. You should automate the reward function so you can run through thousands of training steps in seconds.  
> **username_0**: For the training, I automated it but when the system will be deployed the user will be involved.  
> **username_1**: And how did you automate it for training?  
> **username_0**: each arm returns 1 or -1  

---

## 0 answers

