Answers.Body: <p>Yes, your understanding of the hidden state is correct. But the size of the hidden state is a hyperparameter that needs to found by trial-and-error. There is no closed-form formula or solution which links the size of the hidden state and the problem at hand. But, there are some rules of thumb like to start out with the size of the hidden state to be a power of 2, etc. Keep tuning the hyperparameter until you get very good predictions.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-23T05:42:36.170Z
Answers.LastActivityDate: 2020-04-12T18:30:55.687Z
Answers.LastEditDate: 2020-04-12T18:30:55.687Z
Answers.UserDisplayName: varsh
Body: <p>I'm trying to understand how the size of the hidden state affects the GRU. </p>
  
  <p>For example, suppose I want to make a GRU count. I'm gonna feed it with three numbers, and I expect it to predict the fourth. </p>
  
  <p>How should I choose the size of the hidden state of a GRU?</p>
  
Tags: <machine-learning><recurrent-neural-networks><hyperparameter-optimization><hidden-layers><gated-recurrent-unit>
Title: How do I choose the size of the hidden state of a GRU?
UserDisplayName: razvanc92
ThreadText: # How do I choose the size of the hidden state of a GRU?
  
  (Asked by: username_0 on 2019-01-22)
  
  I'm trying to understand how the size of the hidden state affects the GRU.
  
  For example, suppose I want to make a GRU count. I'm gonna feed it with three numbers, and I expect it to predict the fourth.
  
  How should I choose the size of the hidden state of a GRU?
  
  **Tags:** machine-learning, recurrent-neural-networks, hyperparameter-optimization, hidden-layers, gated-recurrent-unit
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-01-23. Score: 3 | Accepted answer
  
  Yes, your understanding of the hidden state is correct. But the size of the hidden state is a hyperparameter that needs to found by trial-and-error. There is no closed-form formula or solution which links the size of the hidden state and the problem at hand. But, there are some rules of thumb like to start out with the size of the hidden state to be a power of 2, etc. Keep tuning the hyperparameter until you get very good predictions.
  
  
  ---
  
  