Body: <p>If my understanding of an LSTM is correct then the output from each LSTM unit is the hidden state from that layer. For the final layer if I wanted to predict e.g. a scalar real number, would I want to add a dense layer with 1 neuron or is it recommended to have a final LSTM layer where the output has just one hidden unit (i.e. the output dimension of the final hidden state is 1)? If we didn't add the dense layer, then the output from the hidden layer I believe would be between (-1,1), if you use the traditional activations in the LSTM unit. </p>
  
  <p>Apologies if I've used wrong terminology, there seems to be some inconsistency with LSTM's when going between literature and definition in TensorFlow etc. </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-10T22:31:26.107Z
Comments.Text: Have you tried to have a look at some reference implementations? [Here](https://machinelearningmastery.com/how-to-develop-lstm-models-for-time-series-forecasting/) the author seems to use a dense layer with 1 unit after the last LSTM layer.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-10T22:34:39.820Z
Comments.Text: Yes, I have used LSTM's in the past and added a dense layer with 1 unit on. This was from a walkthrough I found -- I was more questioning whether you _have_ to do this, although intuitively I think that you do, especially if it is was in a scenario like I had described.
  e: I just clicked the link and that was the walkthrough I had followed haha.
Comments.UserDisplayName: David
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-10T22:44:26.583Z
Comments.Text: Your question is actually interesting. It's been a while I've used LSTMs, so I cannot answer it now. Of course, if the output of the last LSTM layer is a number in the range $[-1, 1]$ and you need a number in $\mathbb{R}$, then you probably need to fix that (maybe by changing the activation function), if you want to try not to use the dense layers.
Comments.UserDisplayName: nbro
Tags: <deep-learning><recurrent-neural-networks><long-short-term-memory>
Title: Do you have to add a dense layer onto the final layer of an LSTM?
UserDisplayName: David
ThreadText: # Do you have to add a dense layer onto the final layer of an LSTM?
  
  (Asked by: username_0 on 2020-06-10)
  
  If my understanding of an LSTM is correct then the output from each LSTM unit is the hidden state from that layer. For the final layer if I wanted to predict e.g. a scalar real number, would I want to add a dense layer with 1 neuron or is it recommended to have a final LSTM layer where the output has just one hidden unit (i.e. the output dimension of the final hidden state is 1)? If we didn't add the dense layer, then the output from the hidden layer I believe would be between (-1,1), if you use the traditional activations in the LSTM unit.
  
  Apologies if I've used wrong terminology, there seems to be some inconsistency with LSTM's when going between literature and definition in TensorFlow etc.
  
  **Tags:** deep-learning, recurrent-neural-networks, long-short-term-memory
  > **Comments:**
  > **username_1**: Have you tried to have a look at some reference implementations? [Here](https://machinelearningmastery.com/how-to-develop-lstm-models-for-time-series-forecasting/) the author seems to use a dense layer with 1 unit after the last LSTM layer.  
  > **username_0**: Yes, I have used LSTM's in the past and added a dense layer with 1 unit on. This was from a walkthrough I found -- I was more questioning whether you _have_ to do this, although intuitively I think that you do, especially if it is was in a scenario like I had described. e: I just clicked the link and that was the walkthrough I had followed haha.  
  > **username_1**: Your question is actually interesting. It's been a while I've used LSTMs, so I cannot answer it now. Of course, if the output of the last LSTM layer is a number in the range $[-1, 1]$ and you need a number in $\mathbb{R}$, then you probably need to fix that (maybe by changing the activation function), if you want to try not to use the dense layers.  
  
  ---
  
  ## 0 answers
  
  