Answers.Body: <p>That notation should mean to go from time step <span class="math-container">$T$</span> to time step <span class="math-container">$1$</span> by a negative step <span class="math-container">$-1$</span>, i.e. backward, so <span class="math-container">$T$</span>, then <span class="math-container">$T-1$</span>, then <span class="math-container">$T-2$</span>, and so on until <span class="math-container">$1$</span>. If you know Python, this should be familiar. However, note that this is just a guess, because I am not familiar with this algorithm.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-20T18:46:41.917Z
Answers.LastActivityDate: 2021-12-20T18:46:41.917Z
Answers.UserDisplayName: nbro
Body: <p>In looking at an algorithm in the paper <a href="https://arxiv.org/abs/1605.06676" rel="nofollow noreferrer">Learning to Communicate with Deep Multi-Agent Reinforcement Learning</a>.</p>
  <p>Here is the full algorithm:</p>
  <p><a href="https://i.sstatic.net/g1vq2.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/g1vq2.png" alt="enter image description here" /></a></p>
  <p>What does the notation <code>for t=T to 1,−1 do:</code> refer to in terms of <strong>time steps</strong>?</p>
  <p>The network structure is a deep recurrent q network.</p>
  <p>Secondly, why do the gradients need to be reset to zero?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-07-03T13:23:38.600Z
Comments.Text: That notation should mean to go from time step $T$ to time step $1$ by a negative step $-1$, i.e. backward, so $T$, then $T-1$, then $T-2$, and so on until $1$. If you know Python, this should be familiar. However, note that this is just a guess because I am not familiar with this algorithm.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-07-03T17:41:10.037Z
Comments.Text: Yes, this makes sense, and fits the problem. Thank you!
Comments.UserDisplayName: kikram
Tags: <reinforcement-learning><recurrent-neural-networks><dqn><notation>
Title: What does the notation "for t=T to 1,−1 do" in terms of time steps, in deep recurrent q network?
UserDisplayName: kikram
ThreadText: # What does the notation "for t=T to 1,−1 do" in terms of time steps, in deep recurrent q network?
  
  (Asked by: username_0 on 2020-07-03)
  
  In looking at an algorithm in the paper [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://arxiv.org/abs/1605.06676).
  
  Here is the full algorithm:
  
  [![enter image description here](https://i.sstatic.net/g1vq2.png)](https://i.sstatic.net/g1vq2.png)
  
  What does the notation `for t=T to 1,−1 do:` refer to in terms of **time steps**?
  
  The network structure is a deep recurrent q network.
  
  Secondly, why do the gradients need to be reset to zero?
  
  **Tags:** reinforcement-learning, recurrent-neural-networks, dqn, notation
  
  > **Comments:**
  > **username_1**: That notation should mean to go from time step $T$ to time step $1$ by a negative step $-1$, i.e. backward, so $T$, then $T-1$, then $T-2$, and so on until $1$. If you know Python, this should be familiar. However, note that this is just a guess because I am not familiar with this algorithm.  
  > **username_0**: Yes, this makes sense, and fits the problem. Thank you!  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-12-20. Score: 1
  
  That notation should mean to go from time step $T$ to time step $1$ by a negative step $-1$, i.e. backward, so $T$, then $T-1$, then $T-2$, and so on until $1$. If you know Python, this should be familiar. However, note that this is just a guess, because I am not familiar with this algorithm.
  
  
  ---
  
  