Answers.Body: <p>In this case, the word "system" refers to a <a href="https://en.wikipedia.org/wiki/Markov_decision_process" rel="nofollow noreferrer">Markov decision process</a> (MDP), which is the mathematical model used to represent the reinforcement learning (RL) problem or, in general, a decision making problem. Recall that, in RL, the problem consists in finding an (optimal) policy, which is a policy that allows the agent to collect the highest amount of reward (in the long run). Hence, in RL, the MDP is the problem and the optimal policy (for that MDP) is the solution.</p>
  
  <p>The MDP is composed of the set of states of the environment <span class="math-container">$S$</span>, the set of possible actions that the RL agent can take <span class="math-container">$A$</span>, a transition function, <span class="math-container">$P(s_{t+1}=s'\mid s_{t}=s, a_t = a)$</span>, which is a probability distribution and describes the dynamics of the environment, and a reward function <span class="math-container">$R_a(s', s)$</span>, which is a function that describes the "reward system" of the environment. <span class="math-container">$R_a(s', s)$</span> can be thought of as the reward (signal) that the agent receives after having taken action <span class="math-container">$a$</span> in state <span class="math-container">$s$</span> and having landed in state <span class="math-container">$s'$</span>.</p>
  
  <p>In other words, the reward system is just this function <span class="math-container">$R_a(s', s)$</span>, which is often the hardest function to define when modelling an RL problem. However, in certain cases, this function can be easily defined. For example, in the case of chess, you could define <span class="math-container">$R_a(s', s) = 1$</span>, if <span class="math-container">$s'$</span> is the terminal state (checkmate), else <span class="math-container">$R_a(s', s) = 0$</span>. Nonetheless, we could define the reward function for a chess environment differently. For example, we could give a reward of <span class="math-container">$0.5$</span> for certain "clever" moves. So, the reward function needs to be defined (by the programmer) in order to solve an RL problem and it highly affects the way the agent will learn.</p>
  
  <p>The reward function can also be denoted by <span class="math-container">$R_a(s)$</span>, which can be thought of as the reward that the agent receives after having taken action <span class="math-container">$a$</span> in state <span class="math-container">$s$</span> (no matter which state it lands in), or <span class="math-container">$R(s)$</span>, which can be thought of as the reward the agent receives either when it enters or exits state <span class="math-container">$s$</span>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-11T08:49:28.940Z
Answers.Text: Worth adding that in general the output of all the reward functions you show is a *distribution* of scalar reward values. Although very common to just define it as a fixed value, it may actually be random in some environments, e.g. amount of money made on a bet.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-11T00:40:09.840Z
Answers.LastActivityDate: 2019-05-11T00:45:40.447Z
Answers.LastEditDate: 2019-05-11T00:45:40.447Z
Answers.UserDisplayName: nbro
Body: <p>Can you describe this reward system in more detail?</p>
  <p>I understand that the environment sends a signal indicating whether or not the action taken by the agent was 'good' or not, but it seems too simple.</p>
  <p>Basically, can you detail the nitty-gritty workings of this system? I dunno, I may just be overthinking things.</p>
  
Tags: <reinforcement-learning><markov-decision-process><rewards><reward-functions>
Title: What is the reward system of reinforcement learning?
UserDisplayName: naisuu42
ThreadText: # What is the reward system of reinforcement learning?
  
  (Asked by: username_0 on 2019-05-10)
  
  Can you describe this reward system in more detail?
  
  I understand that the environment sends a signal indicating whether or not the action taken by the agent was 'good' or not, but it seems too simple.
  
  Basically, can you detail the nitty-gritty workings of this system? I dunno, I may just be overthinking things.
  
  **Tags:** reinforcement-learning, markov-decision-process, rewards, reward-functions
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-05-11. Score: 2 | Accepted answer
  
  In this case, the word "system" refers to a [Markov decision process](https://en.wikipedia.org/wiki/Markov_decision_process) (MDP), which is the mathematical model used to represent the reinforcement learning (RL) problem or, in general, a decision making problem. Recall that, in RL, the problem consists in finding an (optimal) policy, which is a policy that allows the agent to collect the highest amount of reward (in the long run). Hence, in RL, the MDP is the problem and the optimal policy (for that MDP) is the solution.
  
  The MDP is composed of the set of states of the environment $S$, the set of possible actions that the RL agent can take $A$, a transition function, $P(s\_{t+1}=s'\mid s\_{t}=s, a\_t = a)$, which is a probability distribution and describes the dynamics of the environment, and a reward function $R\_a(s', s)$, which is a function that describes the "reward system" of the environment. $R\_a(s', s)$ can be thought of as the reward (signal) that the agent receives after having taken action $a$ in state $s$ and having landed in state $s'$.
  
  In other words, the reward system is just this function $R\_a(s', s)$, which is often the hardest function to define when modelling an RL problem. However, in certain cases, this function can be easily defined. For example, in the case of chess, you could define $R\_a(s', s) = 1$, if $s'$ is the terminal state (checkmate), else $R\_a(s', s) = 0$. Nonetheless, we could define the reward function for a chess environment differently. For example, we could give a reward of $0.5$ for certain "clever" moves. So, the reward function needs to be defined (by the programmer) in order to solve an RL problem and it highly affects the way the agent will learn.
  
  The reward function can also be denoted by $R\_a(s)$, which can be thought of as the reward that the agent receives after having taken action $a$ in state $s$ (no matter which state it lands in), or $R(s)$, which can be thought of as the reward the agent receives either when it enters or exits state $s$.
  
  > **Comments:**
  > **username_2**: Worth adding that in general the output of all the reward functions you show is a *distribution* of scalar reward values. Although very common to just define it as a fixed value, it may actually be random in some environments, e.g. amount of money made on a bet.  
  
  ---
  
  