Answers.Body: <p>I'm not sure about what you are referring to when you say &quot;number of hidden units&quot;, but I will assume that it's the dimension of the hidden vector <span class="math-container">$h_t \in \mathbb{R}^N$</span> in <a href="https://en.wikipedia.org/wiki/Long_short-term_memory#Variants" rel="nofollow noreferrer">this definition of an LSTM</a>.</p>
  <p>In general, the larger your model, in your case <span class="math-container">$N$</span>, the more capacity your model has and, therefore, the more complex a function it can represent.</p>
  <p>If by &quot;better result&quot; you mean smaller loss on the training dataset, it's very likely that you will indeed overfit more the more you increase <span class="math-container">$N$</span>.</p>
  <p>However, there are many techniques to increase your model expressiveness without overfitting, such as <a href="https://en.wikipedia.org/wiki/Dilution_(neural_networks)#Dropout" rel="nofollow noreferrer">dropout</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-05T18:59:44.327Z
Answers.LastActivityDate: 2022-05-05T18:59:44.327Z
Answers.UserDisplayName: Raphael Lopez Kaufman
Body: <p>Is there any rule of thumb for choosing the number of hidden units in an LSTM? Is it similar to hidden neurons in a regular feedforward neural network? I'm getting better results with my LSTM when I have a much bigger amount of hidden units (like 300 Hidden units for a problem with 14 inputs and 5 outputs),  is it normal that hidden units in an LSTM are usually much more than hidden neurons in a feedforward ANN? or am I just greatly overfitting my problem?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-05-05T09:58:16.130Z
Comments.Text: When you say "I'm getting better results with my LSTM", you need to be more precise for us to understand whether you're over-fitting or not. Can you provide more info about 1. how you're measuring the performance of your model, 2. your training dataset (i.e. how many training data points do you have?), 3. which task you're trying to solve, 4. which loss you're using, 4. how you're splitting the dataset into training, validation and test datasets, if at all, etc. Having said that, your question is partially a duplicate of https://ai.stackexchange.com/q/3156/2444.
Comments.UserDisplayName: nbro
Tags: <neural-networks><long-short-term-memory><hidden-layers>
Title: How Many Hidden Units in an LSTM?
UserDisplayName: Chris T
ThreadText: # How Many Hidden Units in an LSTM?
  
  (Asked by: username_0 on 2022-05-05)
  
  Is there any rule of thumb for choosing the number of hidden units in an LSTM? Is it similar to hidden neurons in a regular feedforward neural network? I'm getting better results with my LSTM when I have a much bigger amount of hidden units (like 300 Hidden units for a problem with 14 inputs and 5 outputs), is it normal that hidden units in an LSTM are usually much more than hidden neurons in a feedforward ANN? or am I just greatly overfitting my problem?
  
  **Tags:** neural-networks, long-short-term-memory, hidden-layers
  
  > **Comments:**
  > **username_1**: When you say "I'm getting better results with my LSTM", you need to be more precise for us to understand whether you're over-fitting or not. Can you provide more info about 1. how you're measuring the performance of your model, 2. your training dataset (i.e. how many training data points do you have?), 3. which task you're trying to solve, 4. which loss you're using, 4. how you're splitting the dataset into training, validation and test datasets, if at all, etc. Having said that, your question is partially a duplicate of https://ai.stackexchange.com/q/3156/2444.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-05-05. Score: 0
  
  I'm not sure about what you are referring to when you say "number of hidden units", but I will assume that it's the dimension of the hidden vector $h\_t \in \mathbb{R}^N$ in [this definition of an LSTM](https://en.wikipedia.org/wiki/Long_short-term_memory#Variants).
  
  In general, the larger your model, in your case $N$, the more capacity your model has and, therefore, the more complex a function it can represent.
  
  If by "better result" you mean smaller loss on the training dataset, it's very likely that you will indeed overfit more the more you increase $N$.
  
  However, there are many techniques to increase your model expressiveness without overfitting, such as [dropout](https://en.wikipedia.org/wiki/Dilution_(neural_networks)#Dropout).
  
  
  ---
  
  