Answers.Body: <p>It's not clearly stated (it's not stated <strong>at all</strong> on Wikipedia), but, after a bit of searching, I found an answer <a href="https://towardsdatascience.com/neural-architecture-search-limitations-and-extensions-8141bec7681f" rel="nofollow noreferrer">here</a> about a third of the way down the page:</p>
  <blockquote>
  <p>The best performing architecture observed during the training of the controller is taken, and <strong>a grid search is performed over some basic hyperparameters such as learning rate and weight decay</strong> in order to achieve near STOTA (state of the art) performance.</p>
  </blockquote>
  <p>So, as a direct answer: The norm; <a href="https://medium.com/datadriveninvestor/an-introduction-to-grid-search-ff57adcc0998" rel="nofollow noreferrer">A grid search</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-30T05:33:43.280Z
Answers.LastActivityDate: 2021-10-29T15:01:32.670Z
Answers.LastEditDate: 2021-10-29T15:01:32.670Z
Answers.UserDisplayName: Recessive
Body: <p>I have read a lot about NAS, but I still do not understand one concept: When setting up a neural network, hyperparameters (such as the learning rate, dropout rate, batch size, filter size, etc.) need to be set up.</p>
  <p>In NAS, only the best architecture is decided, e.g. how many layers and neurons. But what about the hyperparameters? Are they randomly chosen?</p>
  
Tags: <neural-networks><hyper-parameters><neuroevolution><neural-architecture-search>
Title: When using Neural Architecture Search, how are the hyper-parameters chosen?
UserDisplayName: cocojambo
ThreadText: # When using Neural Architecture Search, how are the hyper-parameters chosen?
  
  (Asked by: username_0 on 2019-10-29)
  
  I have read a lot about NAS, but I still do not understand one concept: When setting up a neural network, hyperparameters (such as the learning rate, dropout rate, batch size, filter size, etc.) need to be set up.
  
  In NAS, only the best architecture is decided, e.g. how many layers and neurons. But what about the hyperparameters? Are they randomly chosen?
  
  **Tags:** neural-networks, hyper-parameters, neuroevolution, neural-architecture-search
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-10-30. Score: 2 | Accepted answer
  
  It's not clearly stated (it's not stated **at all** on Wikipedia), but, after a bit of searching, I found an answer [here](https://towardsdatascience.com/neural-architecture-search-limitations-and-extensions-8141bec7681f) about a third of the way down the page:
  
  > The best performing architecture observed during the training of the controller is taken, and **a grid search is performed over some basic hyperparameters such as learning rate and weight decay** in order to achieve near STOTA (state of the art) performance.
  
  So, as a direct answer: The norm; [A grid search](https://medium.com/datadriveninvestor/an-introduction-to-grid-search-ff57adcc0998).
  
  
  ---
  
  