Answers.Body: <p>On the same problems, sometimes GRU is better, sometimes LSTM. </p>
  
  <p>In short, having more parameters (more "knobs") is not always a good thing. The training process needs to learn those parameters. There is a higher chance of over-fitting, amongst other problems. </p>
  
  <p>The parameters are assigned specific roles inside either GRU or LSTM, so if that role is less important for a specific learning challenge, then it can be wasteful or even counter-productive to have the system attempt to learn values for them.</p>
  
  <p>The only way to find out if LSTM is better than GRU on a problem is a hyperparameter search. Unfortunately, you cannot simply swap one for the other, and test that, because the number of cells that optimises a LSTM solution will be different to the number that optimises a GRU.</p>
  
  <blockquote>
    <p>When would one use Long Short-Term Memory (LSTM) over Gated Recurrent Units (GRU)?</p>
  </blockquote>
  
  <p>When it proves better, experimentally. In some problem domains, this may be established and you can check. However, in other problem domains if either GRU or LSTM works well enough to solve a problem (and the superiority of either LSTM or GRU is not the main point of the work), then it may not so clear.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-23T00:07:24.333Z
Answers.Text: Overall, this answer seems to be correct, but I think this answer would improve considerably if you provided some concrete examples of applications/problems where GRU was used over LSTM (or vice-versa) and why it was used. This would at least provide some more "proof" that the preference for one or the other may be context-dependent.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-21T08:17:16.437Z
Answers.LastActivityDate: 2020-05-21T08:17:16.437Z
Answers.UserDisplayName: Neil Slater
Body: <p>I have been reading about LSTMs and GRUs, which are recurrent neural networks (RNNs). The difference between the two is the number and specific type of gates that they have. The GRU has an <em>update</em> gate, which has a similar role to the role of the <em>input</em> and <em>forget</em> gates in the LSTM.</p>
  <p>Here's a diagram that illustrates both units (or RNNs).</p>
  <p><a href="https://miro.medium.com/max/1280/1*yBXV9o5q7L_CvY7quJt3WQ.png" rel="nofollow noreferrer"><img src="https://miro.medium.com/max/1280/1*yBXV9o5q7L_CvY7quJt3WQ.png" alt="enter image description here" /></a></p>
  <p>With respect to the vanilla RNN, the LSTM has more &quot;knobs&quot; or parameters. So, why do we make use of the GRU, when we clearly have more control over the neural network through the LSTM model?</p>
  <p>Here are two more specific questions.</p>
  <ol>
  <li><p>When would one use Long Short-Term Memory (LSTM) over Gated Recurrent Units (GRU)?</p>
  </li>
  <li><p>What are the advantages/disadvantages of using LSTM over GRU?</p>
  </li>
  </ol>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-21T02:24:19.147Z
Comments.Text: Did you see this post? [When to use GRU over LSTM?](https://datascience.stackexchange.com/q/14581/27665) at Data Science SE.
Comments.UserDisplayName: OmG
Tags: <comparison><recurrent-neural-networks><long-short-term-memory><gated-recurrent-unit>
Title: What's the difference between LSTM and GRU?
UserDisplayName: Pluviophile
ThreadText: # What's the difference between LSTM and GRU?
  
  (Asked by: username_0 on 2020-05-21)
  
  I have been reading about LSTMs and GRUs, which are recurrent neural networks (RNNs). The difference between the two is the number and specific type of gates that they have. The GRU has an *update* gate, which has a similar role to the role of the *input* and *forget* gates in the LSTM.
  
  Here's a diagram that illustrates both units (or RNNs).
  
  [![enter image description here](https://miro.medium.com/max/1280/1*yBXV9o5q7L_CvY7quJt3WQ.png)](https://miro.medium.com/max/1280/1*yBXV9o5q7L_CvY7quJt3WQ.png)
  
  With respect to the vanilla RNN, the LSTM has more "knobs" or parameters. So, why do we make use of the GRU, when we clearly have more control over the neural network through the LSTM model?
  
  Here are two more specific questions.
  
  1. When would one use Long Short-Term Memory (LSTM) over Gated Recurrent Units (GRU)?
  2. What are the advantages/disadvantages of using LSTM over GRU?
  
  **Tags:** comparison, recurrent-neural-networks, long-short-term-memory, gated-recurrent-unit
  
  > **Comments:**
  > **username_1**: Did you see this post? [When to use GRU over LSTM?](https://datascience.stackexchange.com/q/14581/27665) at Data Science SE.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-05-21. Score: 1 | Accepted answer
  
  On the same problems, sometimes GRU is better, sometimes LSTM.
  
  In short, having more parameters (more "knobs") is not always a good thing. The training process needs to learn those parameters. There is a higher chance of over-fitting, amongst other problems.
  
  The parameters are assigned specific roles inside either GRU or LSTM, so if that role is less important for a specific learning challenge, then it can be wasteful or even counter-productive to have the system attempt to learn values for them.
  
  The only way to find out if LSTM is better than GRU on a problem is a hyperparameter search. Unfortunately, you cannot simply swap one for the other, and test that, because the number of cells that optimises a LSTM solution will be different to the number that optimises a GRU.
  
  > When would one use Long Short-Term Memory (LSTM) over Gated Recurrent Units (GRU)?
  
  When it proves better, experimentally. In some problem domains, this may be established and you can check. However, in other problem domains if either GRU or LSTM works well enough to solve a problem (and the superiority of either LSTM or GRU is not the main point of the work), then it may not so clear.
  
  > **Comments:**
  > **username_3**: Overall, this answer seems to be correct, but I think this answer would improve considerably if you provided some concrete examples of applications/problems where GRU was used over LSTM (or vice-versa) and why it was used. This would at least provide some more "proof" that the preference for one or the other may be context-dependent.  
  
  ---
  
  