Answers.Body: <p>In deep learning era, there are two possible choices. Caveat approach and Panda approach. </p>
  
  <p><strong>Caviar Approach</strong></p>
  
  <p>In this approach, it is supposed that you have a very powerful cluster system that enables you to run different models simultaneously on different nodes. In this manner, you construct a d-dimensional space which each corresponds to a special hyperparameter. Then you can have a grid approach to partition the space and for each intersection of the grids, you can have a possible initialisation although there are some better ways in order to find better initialisations.</p>
  
  <p><strong>Panda Approach</strong></p>
  
  <p>This approach is widely used among students due to using desktop computers. In this approach, you usually try to find an initialisation of hyperparameters based on your experience or the other architectures' which are available and you try to refine them step by step by using cross-validation.</p>
  
  <hr />
  
  <p>To answer your question, the efficiency depends on your computing power. For more details, you can take a look at the <a href="https://www.coursera.org/learn/deep-neural-network" rel="nofollow noreferrer">contents of the third week</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-02-27T16:37:55.557Z
Answers.Text: I have never heard of these terms/expressions. Where did you hear them? In general, you don't need a cluster. It depends on your NN model. A GPU (or even a CPU) will suffice if you have a simple NN model. It also depends on your time requirements. Furthermore, note that there are other hyper-parameter techniques other than grid search.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-02-27T16:44:40.277Z
Answers.Text: I added the link. You can't train a YOLO model on a single GPU.
Answers.UserDisplayName: Green Falcon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-02-27T16:46:49.020Z
Answers.Text: The question is not clear. Maybe the OP is asking for techniques to look for combinations of hyper-parameters. In that case, your answer is not valid. I would first ask for clarifications.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-02-27T16:49:53.527Z
Answers.Text: Combination of hyperparameters? Isn't it exactly what I've tried to convey?
Answers.UserDisplayName: Green Falcon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-02-27T16:58:13.160Z
Answers.Text: No, I'm talking about something different...
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-02-27T16:34:53.750Z
Answers.LastActivityDate: 2019-02-27T16:43:42.053Z
Answers.LastEditDate: 2019-02-27T16:43:42.053Z
Answers.UserDisplayName: Green Falcon
Body: <p>How do you efficiently choose the hyper-parameters of a neural network (e.g. the learning rate, number of layer, weights, etc.)?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-02-27T16:43:07.303Z
Comments.Text: Are you asking which techniques exist to search for the best combination of hyper-parameters?
Comments.UserDisplayName: nbro
Tags: <neural-networks><machine-learning><deep-learning><hyperparameter-optimization><hyper-parameters>
Title: How do you efficiently choose the hyper-parameters of a neural network?
UserDisplayName: bobbyoiji
ThreadText: # How do you efficiently choose the hyper-parameters of a neural network?
  
  (Asked by: username_0 on 2019-02-27)
  
  How do you efficiently choose the hyper-parameters of a neural network (e.g. the learning rate, number of layer, weights, etc.)?
  
  **Tags:** neural-networks, machine-learning, deep-learning, hyperparameter-optimization, hyper-parameters
  
  > **Comments:**
  > **username_1**: Are you asking which techniques exist to search for the best combination of hyper-parameters?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-02-27. Score: 1
  
  In deep learning era, there are two possible choices. Caveat approach and Panda approach.
  
  **Caviar Approach**
  
  In this approach, it is supposed that you have a very powerful cluster system that enables you to run different models simultaneously on different nodes. In this manner, you construct a d-dimensional space which each corresponds to a special hyperparameter. Then you can have a grid approach to partition the space and for each intersection of the grids, you can have a possible initialisation although there are some better ways in order to find better initialisations.
  
  **Panda Approach**
  
  This approach is widely used among students due to using desktop computers. In this approach, you usually try to find an initialisation of hyperparameters based on your experience or the other architectures' which are available and you try to refine them step by step by using cross-validation.
  
  ---
  
  To answer your question, the efficiency depends on your computing power. For more details, you can take a look at the [contents of the third week](https://www.coursera.org/learn/deep-neural-network).
  
  > **Comments:**
  > **username_1**: I have never heard of these terms/expressions. Where did you hear them? In general, you don't need a cluster. It depends on your NN model. A GPU (or even a CPU) will suffice if you have a simple NN model. It also depends on your time requirements. Furthermore, note that there are other hyper-parameter techniques other than grid search.  
  > **username_2**: I added the link. You can't train a YOLO model on a single GPU.  
  > **username_1**: The question is not clear. Maybe the OP is asking for techniques to look for combinations of hyper-parameters. In that case, your answer is not valid. I would first ask for clarifications.  
  > **username_2**: Combination of hyperparameters? Isn't it exactly what I've tried to convey?  
  > **username_1**: No, I'm talking about something different...  
  
  ---
  
  