<p>That notation should mean to go from time step <span class="math-container">$T$</span> to time step <span class="math-container">$1$</span> by a negative step <span class="math-container">$-1$</span>, i.e. backward, so <span class="math-container">$T$</span>, then <span class="math-container">$T-1$</span>, then <span class="math-container">$T-2$</span>, and so on until <span class="math-container">$1$</span>. If you know Python, this should be familiar. However, note that this is just a guess, because I am not familiar with this algorithm.</p>

CC BY-SA 4.0
2021-12-20T18:46:41.917Z
2021-12-20T18:46:41.917Z
nbro
<p>In looking at an algorithm in the paper <a href="https://arxiv.org/abs/1605.06676" rel="nofollow noreferrer">Learning to Communicate with Deep Multi-Agent Reinforcement Learning</a>.</p>
<p>Here is the full algorithm:</p>
<p><a href="https://i.sstatic.net/g1vq2.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/g1vq2.png" alt="enter image description here" /></a></p>
<p>What does the notation <code>for t=T to 1,−1 do:</code> refer to in terms of <strong>time steps</strong>?</p>
<p>The network structure is a deep recurrent q network.</p>
<p>Secondly, why do the gradients need to be reset to zero?</p>

CC BY-SA 4.0
2020-07-03T13:23:38.600Z
That notation should mean to go from time step $T$ to time step $1$ by a negative step $-1$, i.e. backward, so $T$, then $T-1$, then $T-2$, and so on until $1$. If you know Python, this should be familiar. However, note that this is just a guess because I am not familiar with this algorithm.
nbro
CC BY-SA 4.0
2020-07-03T17:41:10.037Z
Yes, this makes sense, and fits the problem. Thank you!
kikram
<reinforcement-learning><recurrent-neural-networks><dqn><notation>
What does the notation "for t=T to 1,−1 do" in terms of time steps, in deep recurrent q network?
kikram
# What does the notation "for t=T to 1,−1 do" in terms of time steps, in deep recurrent q network?

(Asked by: username_0 on 2020-07-03)

In looking at an algorithm in the paper [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://arxiv.org/abs/1605.06676).

Here is the full algorithm:

[![enter image description here](https://i.sstatic.net/g1vq2.png)](https://i.sstatic.net/g1vq2.png)

What does the notation `for t=T to 1,−1 do:` refer to in terms of **time steps**?

The network structure is a deep recurrent q network.

Secondly, why do the gradients need to be reset to zero?

**Tags:** reinforcement-learning, recurrent-neural-networks, dqn, notation

> **Comments:**
> **username_1**: That notation should mean to go from time step $T$ to time step $1$ by a negative step $-1$, i.e. backward, so $T$, then $T-1$, then $T-2$, and so on until $1$. If you know Python, this should be familiar. However, note that this is just a guess because I am not familiar with this algorithm.  
> **username_0**: Yes, this makes sense, and fits the problem. Thank you!  

---

## 1 answers

### Answer by username_1 on 2021-12-20. Score: 1

That notation should mean to go from time step $T$ to time step $1$ by a negative step $-1$, i.e. backward, so $T$, then $T-1$, then $T-2$, and so on until $1$. If you know Python, this should be familiar. However, note that this is just a guess, because I am not familiar with this algorithm.


---

