Answers.Body: <ol>
  <li><p>you have in front of you 10 slot machines</p>
  </li>
  <li><p>you can play with any of them, and each of them have a specific winrate (reward function)</p>
  </li>
  <li><p>the only state of this MDP is the initial state, the one where you are in front of these slot machines</p>
  </li>
  <li><p>how do you model the reward?</p>
  <p>a. <span class="math-container">$R(s)$</span>: well, the state is only one, so the distribution of the rewards does not depend on the action you take/ slot machine you play</p>
  <p>b. <span class="math-container">$R(s,a)$</span>: well, now you have actions, so you can learn statistics about <span class="math-container">$R(s,a)$</span> (aka statistics about each slot machine), and thus take actions according to these statistics (policy)</p>
  </li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-25T13:10:27.177Z
Answers.LastActivityDate: 2023-09-25T13:10:27.177Z
Answers.UserDisplayName: Alberto
Body: <p>I am quite new to RL. I would like to know why a state-dependent reward function R(s) is more restrictive than a state-action-dependent reward function R(s, a)? And why is it that a policy can be optimal for the latter R(s, a) but not for the former(R(s).</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-03T13:09:59.287Z
Comments.Text: Does this answer your question? [How are the reward functions $R(s)$, $R(s, a)$ and $R(s, a, s')$ equivalent?](https://ai.stackexchange.com/questions/10442/how-are-the-reward-functions-rs-rs-a-and-rs-a-s-equivalent)
Comments.UserDisplayName: nbro
Tags: <markov-decision-process><reward-functions><optimality>
Title: Why is R(s) more restrictive than R(s, a) in an MDP?
UserDisplayName: TicTacToemat
ThreadText: # Why is R(s) more restrictive than R(s, a) in an MDP?
  
  (Asked by: username_0 on 2023-09-25)
  
  I am quite new to RL. I would like to know why a state-dependent reward function R(s) is more restrictive than a state-action-dependent reward function R(s, a)? And why is it that a policy can be optimal for the latter R(s, a) but not for the former(R(s).
  
  **Tags:** markov-decision-process, reward-functions, optimality
  > **Comments:**
  > **username_1**: Does this answer your question? [How are the reward functions $R(s)$, $R(s, a)$ and $R(s, a, s')$ equivalent?](https://ai.stackexchange.com/questions/10442/how-are-the-reward-functions-rs-rs-a-and-rs-a-s-equivalent)  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-09-25. Score: 1
  
  1. you have in front of you 10 slot machines
  2. you can play with any of them, and each of them have a specific winrate (reward function)
  3. the only state of this MDP is the initial state, the one where you are in front of these slot machines
  4. how do you model the reward?
  
     a. $R(s)$: well, the state is only one, so the distribution of the rewards does not depend on the action you take/ slot machine you play
  
     b. $R(s,a)$: well, now you have actions, so you can learn statistics about $R(s,a)$ (aka statistics about each slot machine), and thus take actions according to these statistics (policy)
  
  
  ---
  
  