Answers.Body: <p>One word answer for your question "<strong>Do you need to store previous values of weights and layers on recurrent layer while BPTT?</strong>" is <strong>YES</strong></p>
  
  <p>Let us go through the details.</p>
  
  <p>For training an RNN using BPTT, we need gradients of error w.r.t all three parameters <strong>U</strong>, <strong>V</strong>, <strong>W</strong></p>
  
  <p>Notation of my explanation is different from notation in the figure of question.
  My notation is as below:</p>
  
  <ol>
  <li><strong>V</strong> - Hidden Layer - Output Layer (gradients of V are independent of
  previous time steps) </li>
  <li><strong>U</strong> - Input Layer - Hidden Layer (gradients of U are dependent on previous time steps) </li>
  <li><strong>W</strong> - Hidden Layer - Hidden Layer (gradients of W are also dependent on previous time steps)</li>
  </ol>
  
  <p>And for calculating these gradients, we use chain rule of differentiation, the same rule that we used to calculate gradients in a fully connected neural network. </p>
  
  <p>The gradient w.r.t V only depends on current time step (doesn't need any values from previous time step). </p>
  
  <p>The gradients w.r.t U, W depends on current time step and also all previous time steps (so needs values from all time steps)</p>
  
  <p>Basically, we need to back propagate gradients from current time step all the way to t=0.</p>
  
  <p>How this back propagation is different from the back propagation we use in fully connected neural network is that, in fully connected neural network we don't have the concept of t and also we don't share any weights across layers. But, here we share weights across layers and time instants. So, gradients depend on all time instants.</p>
  
  <p><strong>Note: Be careful with notation difference between several articles. I followed slightly different notation than in the diagram in question.</strong></p>
  
  <p>Some links that will help you explore.</p>
  
  <p><a href="https://www.youtube.com/watch?v=RrB605Mbpic" rel="nofollow noreferrer">https://www.youtube.com/watch?v=RrB605Mbpic</a> (clearly explains about gradients of all three U, V, W; but notation is different from diagram in question)</p>
  
  <p><a href="http://www.wildml.com/2015/10/recurrent-neural-networks-tutorial-part-3-backpropagation-through-time-and-vanishing-gradients/" rel="nofollow noreferrer">http://www.wildml.com/2015/10/recurrent-neural-networks-tutorial-part-3-backpropagation-through-time-and-vanishing-gradients/</a><a href="http://www.wildml.com/2015/10/recurrent-neural-networks-tutorial-part-3-backpropagation-through-time-and-vanishing-gradients/" rel="nofollow noreferrer">enter link description here</a></p>
  
  <p><a href="http://ir.hit.edu.cn/~jguo/docs/notes/bptt.pdf" rel="nofollow noreferrer">http://ir.hit.edu.cn/~jguo/docs/notes/bptt.pdf</a></p>
  
  <p><a href="https://www.d2l.ai/chapter_recurrent-neural-networks/bptt.html" rel="nofollow noreferrer">https://www.d2l.ai/chapter_recurrent-neural-networks/bptt.html</a></p>
  
  <p>Remember, you should understand chain rule of partial derivative very clearly to do the derivation yourself and understand it.</p>
  
  <p>Also, dont think BPTT is separate than BP. It is one and the same. Since neural network architecture in RNN includes time instants and sharing of weights across time instants, just using chain rule on this network makes back propagation also dependent on time and so is the name. </p>
  
  <p>Hope it helps. Feedback is welcome.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-12T11:22:43.963Z
Answers.Text: Wait a second, its confusin again. That means, everything above ht must also be recalculated, so i store also 'U'-s and also make sum it each time? W does not have chain from V, so does not need to be stored, eight?
Answers.UserDisplayName: user8426627
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-12T12:09:12.583Z
Answers.Text: I didn't understand what you mean by everything above ht must also be recalculated.
  Calculation of gradients of both Us and Ws will have time (summation over time).
Answers.UserDisplayName: Kartik Podugu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-12T12:16:00.140Z
Answers.Text: why W is also summed over time?  It  is not recurrent and gaings no information from U. if you look seperatels just h->o ther ewill be just 'normal' BP.
Answers.UserDisplayName: user8426627
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-13T11:45:27.067Z
Answers.Text: I am sorry. Since you also used U, V, W; i thought notation is same. That is where the confusion came. 
  
  U --> Input Layer - Hidden Layer
  V --> Hidden Layer - Output Layer
  W --> Hidden Layer - Hidden Layer
  According to my convention V is independent of previous time steps.
  U, W depend on previous time steps.
  
  But, as per your convention in the above figure in the question, W is independent of previous time steps. U and V depend on previous time steps.
Answers.UserDisplayName: Kartik Podugu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-13T12:20:29.467Z
Answers.Text: reread my answer, i have updated notation details and another youtube link which has clear derivation details. I felt video is better than deriving here.
Answers.UserDisplayName: Kartik Podugu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-14T18:08:32.130Z
Answers.Text: ok, the best answer i have find is https://codeburst.io/recurrent-neural-network-4ca9fd4f242 . Please, confirm if it is valie implementation, because i m not sure it does not match. There is like all steps ,except forward and backprop of hidden layer is done like in normal minibatch training , so if you have minibatch implementation, no extra info must be stored
Answers.UserDisplayName: user8426627
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-15T02:16:05.607Z
Answers.Text: The implementation you mentioned is different from what i tried to explain in my answer. In a vanilla BPTT, for gradients of U and W, you have to have recursion from particular time instant all the way to 0. But, i don't see any kind of recursion in the implementation of codeburst. In codeburst implementation, W only depends on current time instant and previous time instant and U depends only on current time instant. It doesn't go all the way till 0. Equations for this is mentioned, but derivation is not there.
Answers.UserDisplayName: Kartik Podugu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-15T02:17:15.517Z
Answers.Text: So, i am not sure of its correctness.
  Vanilla RNN is seldom used because of vanishing gradient and exploding gradient problem because of BPTT. There is more practical method called as Truncated BPTT where instead of going till time instant 0, we only go back till fixed time instants. (may be 20 time instants). May be codeburst implementation is some kind of Truncated BPTT with only one time instant. I am not sure of what that implementation is, but it is quite different from my answer especially w.r.t gradients.
Answers.UserDisplayName: Kartik Podugu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-12T11:09:04.690Z
Answers.LastActivityDate: 2019-06-13T12:19:39.980Z
Answers.LastEditDate: 2019-06-13T12:19:39.980Z
Answers.UserDisplayName: Kartik Podugu
Body: <p>The Back propagation through time on recurrent layer is defined similar to  normal one, means somethin like </p>
  
  <p><code>self.deltas[x] =  self.deltas[x+1].dot(self.weights[x].T) * self.layers[x] * (1- self.layers[x])</code> where </p>
  
  <p><code>self.deltas[x+1]</code> is error from prevous layer, <code>self.weights[x]</code> is weights map and <code>self.layers[x](1- self.layers[x])</code> is bakwards activation of sigmoid function where <code>self.layers[x]</code> is vector of sigmoid. But while normal backpropagation the values are there, while BPTT i can not take the current <code>self.layers[x]</code> : i need the previous ones, right ? </p>
  
  <p>So unlike normal BP, do i need extra store old weights and layers, for example in circular queue, and then apply the formula where <code>self.deltas[x+1]</code> is layer from next time ?</p>
  
  <p>Not realy implementation, just basic understanding in order to can implement it.</p>
  
  <p>Lets see the picture:</p>
  
  <p><a href="https://i.sstatic.net/NwXQ9.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/NwXQ9.png" alt="enter image description here"></a></p>
  
  <p>Here are : self.layers[0] = <span class="math-container">$x_{t+1}$</span>, self.layers[<em>1</em>] = <span class="math-container">$h_{t+1}$</span> , self.layers[2] = <span class="math-container">$o_{t+1}$</span>, in order to perform backprop <span class="math-container">$h_{t+1}$</span> -> <span class="math-container">$h_{t}$</span> -> <span class="math-container">$h_{t-1}$</span>...  <strong>I DO NEED to have layers <span class="math-container">$h_t$</span> ,<span class="math-container">$h_{t-1}$</span>... and weights <span class="math-container">$v_{t+1}$</span>, <span class="math-container">$v_t$</span>...  EXTRA stored in additional to the network <span class="math-container">$x_{t+1}$</span> -> <span class="math-container">$h_{t+1}$</span> -> <span class="math-container">$o_{t+1}$</span>, right?</strong> 
  Thats all the question.</p>
  
  <p>And i do not need to store previous outputs <span class="math-container">$o[t, o_{t-1}, etc..]$</span>, because backprop from them ot->ht, etc was <em>already</em> calculated. </p>
  
Tags: <recurrent-neural-networks><backpropagation>
Title: Do you need to store prevous values of weights and layers on recurrent layer while BPTT?
UserDisplayName: user8426627
ThreadText: # Do you need to store prevous values of weights and layers on recurrent layer while BPTT?
  
  (Asked by: username_0 on 2019-06-03)
  
  The Back propagation through time on recurrent layer is defined similar to normal one, means somethin like
  
  `self.deltas[x] = self.deltas[x+1].dot(self.weights[x].T) * self.layers[x] * (1- self.layers[x])` where
  
  `self.deltas[x+1]` is error from prevous layer, `self.weights[x]` is weights map and `self.layers[x](1- self.layers[x])` is bakwards activation of sigmoid function where `self.layers[x]` is vector of sigmoid. But while normal backpropagation the values are there, while BPTT i can not take the current `self.layers[x]` : i need the previous ones, right ?
  
  So unlike normal BP, do i need extra store old weights and layers, for example in circular queue, and then apply the formula where `self.deltas[x+1]` is layer from next time ?
  
  Not realy implementation, just basic understanding in order to can implement it.
  
  Lets see the picture:
  
  [![enter image description here](https://i.sstatic.net/NwXQ9.png)](https://i.sstatic.net/NwXQ9.png)
  
  Here are : self.layers[0] = $x\_{t+1}$, self.layers[*1*] = $h\_{t+1}$ , self.layers[2] = $o\_{t+1}$, in order to perform backprop $h\_{t+1}$ -> $h\_{t}$ -> $h\_{t-1}$... **I DO NEED to have layers $h\_t$ ,$h\_{t-1}$... and weights $v\_{t+1}$, $v\_t$... EXTRA stored in additional to the network $x\_{t+1}$ -> $h\_{t+1}$ -> $o\_{t+1}$, right?**
  Thats all the question.
  
  And i do not need to store previous outputs $o[t, o\_{t-1}, etc..]$, because backprop from them ot->ht, etc was *already* calculated.
  
  **Tags:** recurrent-neural-networks, backpropagation
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-06-12. Score: 1 | Accepted answer
  
  One word answer for your question "**Do you need to store previous values of weights and layers on recurrent layer while BPTT?**" is **YES**
  
  Let us go through the details.
  
  For training an RNN using BPTT, we need gradients of error w.r.t all three parameters **U**, **V**, **W**
  
  Notation of my explanation is different from notation in the figure of question.
  My notation is as below:
  
  1. **V** - Hidden Layer - Output Layer (gradients of V are independent of
     previous time steps)
  2. **U** - Input Layer - Hidden Layer (gradients of U are dependent on previous time steps)
  3. **W** - Hidden Layer - Hidden Layer (gradients of W are also dependent on previous time steps)
  
  And for calculating these gradients, we use chain rule of differentiation, the same rule that we used to calculate gradients in a fully connected neural network.
  
  The gradient w.r.t V only depends on current time step (doesn't need any values from previous time step).
  
  The gradients w.r.t U, W depends on current time step and also all previous time steps (so needs values from all time steps)
  
  Basically, we need to back propagate gradients from current time step all the way to t=0.
  
  How this back propagation is different from the back propagation we use in fully connected neural network is that, in fully connected neural network we don't have the concept of t and also we don't share any weights across layers. But, here we share weights across layers and time instants. So, gradients depend on all time instants.
  
  **Note: Be careful with notation difference between several articles. I followed slightly different notation than in the diagram in question.**
  
  Some links that will help you explore.
  
  <https://www.youtube.com/watch?v=RrB605Mbpic> (clearly explains about gradients of all three U, V, W; but notation is different from diagram in question)
  
  <http://www.wildml.com/2015/10/recurrent-neural-networks-tutorial-part-3-backpropagation-through-time-and-vanishing-gradients/>[enter link description here](http://www.wildml.com/2015/10/recurrent-neural-networks-tutorial-part-3-backpropagation-through-time-and-vanishing-gradients/)
  
  <http://ir.hit.edu.cn/~jguo/docs/notes/bptt.pdf>
  
  <https://www.d2l.ai/chapter_recurrent-neural-networks/bptt.html>
  
  Remember, you should understand chain rule of partial derivative very clearly to do the derivation yourself and understand it.
  
  Also, dont think BPTT is separate than BP. It is one and the same. Since neural network architecture in RNN includes time instants and sharing of weights across time instants, just using chain rule on this network makes back propagation also dependent on time and so is the name.
  
  Hope it helps. Feedback is welcome.
  
  > **Comments:**
  > **username_0**: Wait a second, its confusin again. That means, everything above ht must also be recalculated, so i store also 'U'-s and also make sum it each time? W does not have chain from V, so does not need to be stored, eight?  
  > **username_1**: I didn't understand what you mean by everything above ht must also be recalculated. Calculation of gradients of both Us and Ws will have time (summation over time).  
  > **username_0**: why W is also summed over time?  It  is not recurrent and gaings no information from U. if you look seperatels just h->o ther ewill be just 'normal' BP.  
  > **username_1**: I am sorry. Since you also used U, V, W; i thought notation is same. That is where the confusion came.   U --> Input Layer - Hidden Layer V --> Hidden Layer - Output Layer W --> Hidden Layer - Hidden Layer According to my convention V is independent of previous time steps. U, W depend on previous time steps.  But, as per your convention in the above figure in the question, W is independent of previous time steps. U and V depend on previous time steps.  
  > **username_1**: reread my answer, i have updated notation details and another youtube link which has clear derivation details. I felt video is better than deriving here.  
  > **username_0**: ok, the best answer i have find is https://codeburst.io/recurrent-neural-network-4ca9fd4f242 . Please, confirm if it is valie implementation, because i m not sure it does not match. There is like all steps ,except forward and backprop of hidden layer is done like in normal minibatch training , so if you have minibatch implementation, no extra info must be stored  
  > **username_1**: The implementation you mentioned is different from what i tried to explain in my answer. In a vanilla BPTT, for gradients of U and W, you have to have recursion from particular time instant all the way to 0. But, i don't see any kind of recursion in the implementation of codeburst. In codeburst implementation, W only depends on current time instant and previous time instant and U depends only on current time instant. It doesn't go all the way till 0. Equations for this is mentioned, but derivation is not there.  
  > **username_1**: So, i am not sure of its correctness. Vanilla RNN is seldom used because of vanishing gradient and exploding gradient problem because of BPTT. There is more practical method called as Truncated BPTT where instead of going till time instant 0, we only go back till fixed time instants. (may be 20 time instants). May be codeburst implementation is some kind of Truncated BPTT with only one time instant. I am not sure of what that implementation is, but it is quite different from my answer especially w.r.t gradients.  
  
  ---
  
  