Answers.Body: <p>Definitely you can have multiple hidden layers in RNN. One the most common approaches to determine the hidden units is to start with a very small network (one hidden unit) and apply the K-fold cross validation ( k over 30 will give very good accuracy) and estimate the average prediction risk. Then you will have to repeat the procedure for increasing growing networks, for example for 1 to 10 hidden units or more if needed.</p>
  
  <p>However, in my experience, if you are interested to get best possible accuracy, you should start with small number of hidden layers and more simple structure, and if you are not satisfied with the corresponding accuracy, then we should go on increasing the learning rate by fixed but small small steps and each time start training fresh. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-28T02:44:12.723Z
Answers.LastActivityDate: 2019-10-28T02:44:12.723Z
Answers.UserDisplayName: aminrd
Body: <p>Did I get it right, that RNNs most often have just one hidden neuron layer? Is there a reason for that? Will RNNs with several hidden layers in each cell work worse?
  Thank you!! </p>
  
Tags: <neural-networks><recurrent-neural-networks>
Title: Why RNNs often use just one hidden layer?
UserDisplayName: StanislavOssovsky
ThreadText: # Why RNNs often use just one hidden layer?
  
  (Asked by: username_0 on 2019-10-28)
  
  Did I get it right, that RNNs most often have just one hidden neuron layer? Is there a reason for that? Will RNNs with several hidden layers in each cell work worse?
  Thank you!!
  
  **Tags:** neural-networks, recurrent-neural-networks
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-10-28. Score: 3 | Accepted answer
  
  Definitely you can have multiple hidden layers in RNN. One the most common approaches to determine the hidden units is to start with a very small network (one hidden unit) and apply the K-fold cross validation ( k over 30 will give very good accuracy) and estimate the average prediction risk. Then you will have to repeat the procedure for increasing growing networks, for example for 1 to 10 hidden units or more if needed.
  
  However, in my experience, if you are interested to get best possible accuracy, you should start with small number of hidden layers and more simple structure, and if you are not satisfied with the corresponding accuracy, then we should go on increasing the learning rate by fixed but small small steps and each time start training fresh.
  
  
  ---
  
  