Answers.Body: <p>Let's write down Fibonacci!</p>
  
  <blockquote>
    <p><strong>K = 0 1 1 2 3 5 ...</strong></p>
  </blockquote>
  
  <p>And another series that is derived from Fibo; </p>
  
  <blockquote>
    <p><strong>X = 1 4 7 12 30 <span class="math-container">$X_{5}$</span></strong></p>
  </blockquote>
  
  <p>Guessing <span class="math-container">$X_{5}$</span> is our task and both series are available to you (Fibonacci as an additional feature). </p>
  
  <p>One unit that you feed with <strong>X</strong> will try to capture the relation of <span class="math-container">$X_{t}$</span> and <span class="math-container">$X_{t-1}$</span> <em>only</em>;</p>
  
  <p><span class="math-container">$X_{t}$</span> = <span class="math-container">$X_{t-1}$</span> + <span class="math-container">$X_{t-2}$</span></p>
  
  <p>However, an additional unit that you insert&amp;feed with <strong>K</strong> will not only try to capture <span class="math-container">$K_{t}$</span> and <span class="math-container">$K_{t-1}$</span> relation but also the relation between <span class="math-container">$K_{t}$</span> and <span class="math-container">$X_{t}$</span>.</p>
  
  <p><span class="math-container">$K_{t}$</span> = <span class="math-container">$K_{t-1}$</span> + <span class="math-container">$2*$</span><span class="math-container">$X_{t}$</span> + <span class="math-container">$1$</span> </p>
  
  <p>In the example above, there is a clear correlation between <span class="math-container">$K_{t}$</span> and <span class="math-container">$X_{t}$</span> (which is not always the case) and it will support the network to capture the sequential relation. Even the CORRELATION is not crystal clear, almost every additional feature data correlates with the other features and will support the network to grab the relation.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-15T02:43:12.250Z
Answers.LastActivityDate: 2019-10-15T02:43:12.250Z
Answers.UserDisplayName: laserwoman
Body: <p>What is the point of having multiple LSTM units in a single layer?</p>
  
  <p>Surely if we have a single unit it should be able to capture (remember) all the data anyway and using more units in the same layer would just make the other units learn exactly the same historical features?</p>
  
  <p>I've even shown myself empirically that using multiple LSTMs in a single layer improves performance, but in my head it still doesn't make sense, because I don't see what is it that other units are learning that others aren't? Is this sort of similar to how we use multiple filters in a single CNN layer?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-15T22:15:53.330Z
Comments.Text: Your assumption "Surely if we have a single unit it should be able to capture (remember) all the data anyway" is likely incorrect. Why do you think a single unit would be sufficient? This is just an interpretation, but, if you have to keep track of multiple concepts at the same time step, then a single RNN unit might not be sufficient.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-06-15T01:05:36.413Z
Comments.Text: that was my question i got here : https://ai.stackexchange.com/questions/12657/do-you-need-to-store-prevous-values-of-weights-and-layers-on-recurrent-layer-whi : for RNN, multiple cells/units, whatever store the data from previous time stamps and use it for backpropagate. Just RNN cell store only weights abd biases, and  LSTM stores more stuff what is in there
Comments.UserDisplayName: user8426627
Tags: <machine-learning><deep-learning><long-short-term-memory>
Title: Why do we need multiple LSTM units in a layer?
UserDisplayName: Dema Ushchapovskyy
ThreadText: # Why do we need multiple LSTM units in a layer?
  
  (Asked by: username_0 on 2019-05-15)
  
  What is the point of having multiple LSTM units in a single layer?
  
  Surely if we have a single unit it should be able to capture (remember) all the data anyway and using more units in the same layer would just make the other units learn exactly the same historical features?
  
  I've even shown myself empirically that using multiple LSTMs in a single layer improves performance, but in my head it still doesn't make sense, because I don't see what is it that other units are learning that others aren't? Is this sort of similar to how we use multiple filters in a single CNN layer?
  
  **Tags:** machine-learning, deep-learning, long-short-term-memory
  > **Comments:**
  > **username_1**: Your assumption "Surely if we have a single unit it should be able to capture (remember) all the data anyway" is likely incorrect. Why do you think a single unit would be sufficient? This is just an interpretation, but, if you have to keep track of multiple concepts at the same time step, then a single RNN unit might not be sufficient.  
  > **username_2**: that was my question i got here : https://ai.stackexchange.com/questions/12657/do-you-need-to-store-prevous-values-of-weights-and-layers-on-recurrent-layer-whi : for RNN, multiple cells/units, whatever store the data from previous time stamps and use it for backpropagate. Just RNN cell store only weights abd biases, and  LSTM stores more stuff what is in there  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2019-10-15. Score: 2 | Accepted answer
  
  Let's write down Fibonacci!
  
  > **K = 0 1 1 2 3 5 ...**
  
  And another series that is derived from Fibo;
  
  > **X = 1 4 7 12 30 $X\_{5}$**
  
  Guessing $X\_{5}$ is our task and both series are available to you (Fibonacci as an additional feature).
  
  One unit that you feed with **X** will try to capture the relation of $X\_{t}$ and $X\_{t-1}$ *only*;
  
  $X\_{t}$ = $X\_{t-1}$ + $X\_{t-2}$
  
  However, an additional unit that you insert&feed with **K** will not only try to capture $K\_{t}$ and $K\_{t-1}$ relation but also the relation between $K\_{t}$ and $X\_{t}$.
  
  $K\_{t}$ = $K\_{t-1}$ + $2\*$$X\_{t}$ + $1$
  
  In the example above, there is a clear correlation between $K\_{t}$ and $X\_{t}$ (which is not always the case) and it will support the network to capture the sequential relation. Even the CORRELATION is not crystal clear, almost every additional feature data correlates with the other features and will support the network to grab the relation.
  
  
  ---
  
  