Body: <p>I am learning Reinforcement learning for games following Gridworld examples. Apologies in advance if this is a basic question, very new to reinforcement learning.</p>
  <p>I am slightly confused in scenarios where probability of moving up, down, left and right are not provided or stated. In this scenario, I assume we assume the optimal policy and therefore, you would apply the Bellman equation as:</p>
  <p><span class="math-container">$V(s) = max_a(R(s,a)+\gamma V(s'))$</span></p>
  <p>Cost for any movement is 0 and an agent can choose to terminate at a numbered grid to collect a reward amount of the grid number. This is why my square closest to the reward takes in the value 8 since it will terminate with the action to the next state to collect the reward.</p>
  <p>Would this be the correct way to determine the value for the surrounding grid squares?
  <a href="https://i.sstatic.net/NXq2S.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/NXq2S.png" alt="enter image description here" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-04-14T21:51:35.193Z
Comments.Text: Could you please put your **specific question** in the title? "Determine Gridworld values" is not a question and it's also not specific. Thank you.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-04-20T12:49:32.207Z
Comments.Text: Why is "In this scenario, I assume we assume the optimal policy" true? Why would you assume the optimal policy you don't know the transition dynamics? That doesn't make sense to me. Anyway, are you asking if that equation is the right one to update the values?
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning><deep-learning><deep-rl><bellman-equations><optimal-policy>
Title: Determine Gridworld values
UserDisplayName: Krellex
ThreadText: # Determine Gridworld values
  
  (Asked by: username_0 on 2022-04-14)
  
  I am learning Reinforcement learning for games following Gridworld examples. Apologies in advance if this is a basic question, very new to reinforcement learning.
  
  I am slightly confused in scenarios where probability of moving up, down, left and right are not provided or stated. In this scenario, I assume we assume the optimal policy and therefore, you would apply the Bellman equation as:
  
  $V(s) = max\_a(R(s,a)+\gamma V(s'))$
  
  Cost for any movement is 0 and an agent can choose to terminate at a numbered grid to collect a reward amount of the grid number. This is why my square closest to the reward takes in the value 8 since it will terminate with the action to the next state to collect the reward.
  
  Would this be the correct way to determine the value for the surrounding grid squares?
  [![enter image description here](https://i.sstatic.net/NXq2S.png)](https://i.sstatic.net/NXq2S.png)
  
  **Tags:** reinforcement-learning, deep-learning, deep-rl, bellman-equations, optimal-policy
  
  > **Comments:**
  > **username_1**: Could you please put your **specific question** in the title? "Determine Gridworld values" is not a question and it's also not specific. Thank you.  
  > **username_1**: Why is "In this scenario, I assume we assume the optimal policy" true? Why would you assume the optimal policy you don't know the transition dynamics? That doesn't make sense to me. Anyway, are you asking if that equation is the right one to update the values?  
  
  ---
  
  ## 0 answers
  
  