Answers.Body: <p><a href="https://i.sstatic.net/vd6d3.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/vd6d3.png" alt="Image from wikipedia" /></a></p>
  <p>This image shows how an lstm cell operates (taken from Wikipedia). The carry state as implemented in tensorflow is the <code>c_t</code> which is different from the final memory state h_t (which is also the state passed as sequence output). So <code>seq_output - h_t = 0</code> and <code>seq_output - c_t != 0</code>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-22T10:22:45.170Z
Answers.LastActivityDate: 2023-02-22T10:22:45.170Z
Answers.UserDisplayName: Robin van Hoorn
Body: <p>when <code>return_sequences=True</code> and <code>return_state=True</code>, a TensorFlow LSTM outputs the hidden states of the LSTM cell along with the memory state and hidden state of the cell as described in Tensorflow docs</p>
  <pre><code>lstm = tf.keras.layers.LSTM(4, return_sequences=True, return_state=True)
  whole_seq_output, final_memory_state, final_carry_state = lstm(inputs)
  </code></pre>
  <p>So in my understanding, the last timestep of each item of <code>whole_seq_output</code> should be the same as <code>final_carry_state</code>, but it turned out it's not. Could someone please explain to me why?</p>
  <pre><code>whole_seq_output[:, -1, :] - final_carry_state # this should be equals to 0, but its not
  whole_seq_output[:, -1, :] - final_memory_state # this equals to 0
  </code></pre>
  
Tags: <tensorflow><long-short-term-memory>
Title: Why final memory state equals to the last hidden state of entire hidden state sequence?
UserDisplayName: noone
ThreadText: # Why final memory state equals to the last hidden state of entire hidden state sequence?
  
  (Asked by: username_0 on 2023-02-22)
  
  when `return_sequences=True` and `return_state=True`, a TensorFlow LSTM outputs the hidden states of the LSTM cell along with the memory state and hidden state of the cell as described in Tensorflow docs
  
  ```
  lstm = tf.keras.layers.LSTM(4, return_sequences=True, return_state=True)
  whole_seq_output, final_memory_state, final_carry_state = lstm(inputs)
  
  ```
  
  So in my understanding, the last timestep of each item of `whole_seq_output` should be the same as `final_carry_state`, but it turned out it's not. Could someone please explain to me why?
  
  ```
  whole_seq_output[:, -1, :] - final_carry_state # this should be equals to 0, but its not
  whole_seq_output[:, -1, :] - final_memory_state # this equals to 0
  
  ```
  
  **Tags:** tensorflow, long-short-term-memory
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-02-22. Score: 2 | Accepted answer
  
  [![Image from wikipedia](https://i.sstatic.net/vd6d3.png)](https://i.sstatic.net/vd6d3.png)
  
  This image shows how an lstm cell operates (taken from Wikipedia). The carry state as implemented in tensorflow is the `c_t` which is different from the final memory state h\_t (which is also the state passed as sequence output). So `seq_output - h_t = 0` and `seq_output - c_t != 0`.
  
  
  ---
  
  