Answers.Body: <p>You are correct: you get one hidden state per time-step. The hidden size, however, refers to the number of features that are used to compute each hidden state, and are therefore independent of your sequence length.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-22T15:37:58.657Z
Answers.LastActivityDate: 2023-03-22T15:37:58.657Z
Answers.UserDisplayName: postnubilaphoebus
Body: <p>It does not come clear to me how the <code>seq_length</code> is not the <strong>exact same</strong> as the <code>hidden_size</code> in LSTMs.</p>
  <p>For example, in the next image, the <code>seq_length</code> is T since there are <span class="math-container">$x_{1}, x_{2}, ..., x_{T}$</span> time steps and there are is one LSTM cell (green LSTM blocks) per input <span class="math-container">$x_{i}$</span>. So in this image <code>seq_length</code> == <code>hidden_size</code></p>
  <img src="https://miro.medium.com/v2/resize:fit:1400/format:webp/1*xRe8z9Znm2CqGeeYRwfmBw.png" width="450">
  <p>In addition, every LSTM cell requires 3 inputs:</p>
  <ol>
  <li><span class="math-container">$x_{t}$</span> the input value at time t.</li>
  <li><span class="math-container">$h_{t-1}$</span> the hidden state value from the previous time step.</li>
  <li><span class="math-container">$c_{t-1}$</span> the cell state value from the previous time step.</li>
  </ol>
  <p>This can be seen in the following image:</p>
  <img src="https://upload.wikimedia.org/wikipedia/commons/thumb/9/93/LSTM_Cell.svg/1280px-LSTM_Cell.svg.png" width="350">
  <p>How can the <code>hidden_size</code> be lower or higher than the <code>seq_length</code>? In any of both cases the LSTM cells would not have the <span class="math-container">$x_{i}$</span> input which is required.</p>
  
Tags: <deep-learning><recurrent-neural-networks><long-short-term-memory><time-series>
Title: Difference between sequence length and hidden size in LSTM
UserDisplayName: moth123
ThreadText: # Difference between sequence length and hidden size in LSTM
  
  (Asked by: username_0 on 2023-03-22)
  
  It does not come clear to me how the `seq_length` is not the **exact same** as the `hidden_size` in LSTMs.
  
  For example, in the next image, the `seq_length` is T since there are $x\_{1}, x\_{2}, ..., x\_{T}$ time steps and there are is one LSTM cell (green LSTM blocks) per input $x\_{i}$. So in this image `seq_length` == `hidden_size`
  
  ![](https://miro.medium.com/v2/resize:fit:1400/format:webp/1*xRe8z9Znm2CqGeeYRwfmBw.png)
  
  In addition, every LSTM cell requires 3 inputs:
  
  1. $x\_{t}$ the input value at time t.
  2. $h\_{t-1}$ the hidden state value from the previous time step.
  3. $c\_{t-1}$ the cell state value from the previous time step.
  
  This can be seen in the following image:
  
  ![](https://upload.wikimedia.org/wikipedia/commons/thumb/9/93/LSTM_Cell.svg/1280px-LSTM_Cell.svg.png)
  
  How can the `hidden_size` be lower or higher than the `seq_length`? In any of both cases the LSTM cells would not have the $x\_{i}$ input which is required.
  
  **Tags:** deep-learning, recurrent-neural-networks, long-short-term-memory, time-series
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-03-22. Score: 1
  
  You are correct: you get one hidden state per time-step. The hidden size, however, refers to the number of features that are used to compute each hidden state, and are therefore independent of your sequence length.
  
  
  ---
  
  