<p>If the state not changing is a valid transition for the environment (e.g. due to an action result such as running into a wall in a maze, or just random chance), then yes it should be accounted for as normal in updates. After all, time is still ticking, and it will impact mean reward or discounted return calculations.</p>

CC BY-SA 4.0
2023-05-11T16:06:33.293Z
2023-05-11T16:06:33.293Z
Neil Slater
<p>Assuming the agent in my environment does an action, however the agent's state does not change. Does that mean the Q-Table gets updated regardless of respective states (current and next) being the same, or do we skip calculating a new Q-Value in this instance?</p>

<reinforcement-learning><q-learning>
Should the Q-Value of a state-action tuple be updated, if $s_{t} == s_{t+1}$
Ralph
# Should the Q-Value of a state-action tuple be updated, if $s_{t} == s_{t+1}$

(Asked by: username_0 on 2023-05-11)

Assuming the agent in my environment does an action, however the agent's state does not change. Does that mean the Q-Table gets updated regardless of respective states (current and next) being the same, or do we skip calculating a new Q-Value in this instance?

**Tags:** reinforcement-learning, q-learning


---

## 1 answers

### Answer by username_1 on 2023-05-11. Score: 3 | Accepted answer

If the state not changing is a valid transition for the environment (e.g. due to an action result such as running into a wall in a maze, or just random chance), then yes it should be accounted for as normal in updates. After all, time is still ticking, and it will impact mean reward or discounted return calculations.


---

