Answers.Body: <p><a href="https://www.bioinf.jku.at/publications/older/2604.pdf" rel="nofollow noreferrer">LSTMs</a> or <a href="https://arxiv.org/abs/1406.1078" rel="nofollow noreferrer">GRUs</a> are <strong>computationally</strong> more effective than the standard RNNs because they explicitly attempt to address the <a href="http://ai.dinfo.unifi.it/paolo/ps/tnn-94-gradient.pdf" rel="nofollow noreferrer">vanishing and exploding gradient problems</a>, which are <a href="https://en.wikipedia.org/wiki/Numerical_analysis" rel="nofollow noreferrer">numerical problems</a> related to the vanishing or explosion of the values of the gradient vector (the vector that contains the partial derivatives of the loss function with respect to the parameters of the model) that arise when training recurrent neural networks with gradient descent and <a href="http://axon.cs.byu.edu/~martinez/classes/678/Papers/Werbos_BPTT.pdf" rel="nofollow noreferrer">back-propagation through time</a>. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-02T16:29:05.860Z
Answers.LastActivityDate: 2019-11-02T23:34:58.110Z
Answers.LastEditDate: 2019-11-02T23:34:58.110Z
Answers.UserDisplayName: nbro
Answers.Body: <p>These newer RNNs (LSTMs and GRUs) have greater memory control, allowing previous values to persist or to be reset as necessary for many sequences of steps, avoiding "gradient decay" or eventual degradation of the values passed from step to step. LSTM and GRU networks make this memory control possible with memory blocks and structures called "gates" that pass or reset values as appropriate.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-14T09:15:34.727Z
Answers.LastActivityDate: 2020-06-15T21:00:21.550Z
Answers.LastEditDate: 2020-06-15T21:00:21.550Z
Answers.UserDisplayName: Simbarashe Timothy Motsi
Body: <p>It seems that older RNNs have a limitation for their use cases and have been outperformed by other recurrent architectures, such as the LSTM and GRU.</p>
  
Tags: <comparison><recurrent-neural-networks><long-short-term-memory><gated-recurrent-unit>
Title: Why are GRU and LSTM better than standard RNNs?
UserDisplayName: Deep Analytics
ThreadText: # Why are GRU and LSTM better than standard RNNs?
  
  (Asked by: username_0 on 2018-06-14)
  
  It seems that older RNNs have a limitation for their use cases and have been outperformed by other recurrent architectures, such as the LSTM and GRU.
  
  **Tags:** comparison, recurrent-neural-networks, long-short-term-memory, gated-recurrent-unit
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2018-06-14. Score: 4 | Accepted answer
  
  These newer RNNs (LSTMs and GRUs) have greater memory control, allowing previous values to persist or to be reset as necessary for many sequences of steps, avoiding "gradient decay" or eventual degradation of the values passed from step to step. LSTM and GRU networks make this memory control possible with memory blocks and structures called "gates" that pass or reset values as appropriate.
  
  
  ---
  
  ### Answer by username_1 on 2019-11-02. Score: 3
  
  [LSTMs](https://www.bioinf.jku.at/publications/older/2604.pdf) or [GRUs](https://arxiv.org/abs/1406.1078) are **computationally** more effective than the standard RNNs because they explicitly attempt to address the [vanishing and exploding gradient problems](http://ai.dinfo.unifi.it/paolo/ps/tnn-94-gradient.pdf), which are [numerical problems](https://en.wikipedia.org/wiki/Numerical_analysis) related to the vanishing or explosion of the values of the gradient vector (the vector that contains the partial derivatives of the loss function with respect to the parameters of the model) that arise when training recurrent neural networks with gradient descent and [back-propagation through time](http://axon.cs.byu.edu/~martinez/classes/678/Papers/Werbos_BPTT.pdf).
  
  
  ---
  
  