Answers.Body: <p>This might work for your case but isn't necessarily true and depends on how much data the network goes through in an iteration. You should be able to test this by making a small change and training until 100 iterations and seeing if the performance significantly changes and if it can be predicted from the 20th iteration.</p>
  
  <p>Another way which may work for you is preloading lower layers of your network (if you have more than one layer).
  For instance, if you have 5 layers and are making changes to the last 2, you could preload the bottom 3 layers with previously trained weights. This should decrease the amount of training that needs to take place as your network can already discern some primary features of your problem.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-02T09:48:55.003Z
Answers.LastActivityDate: 2017-08-02T09:48:55.003Z
Answers.UserDisplayName: Jaden Travnik
Answers.Body: <p>Your scenario is common.</p>
  
  <p>The most straightforward approach is to subsample your data randomly. Unless your data or your model has strong bias, your performance to the smaller data set should be comparable. The accuracy might be lower, but the purpose is to do quick sanity check.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-02T15:48:56.697Z
Answers.Text: ‎I tried this and got good results.
Answers.UserDisplayName: Erel Segal-Halevi
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-02T12:03:15.540Z
Answers.LastActivityDate: 2017-08-02T12:03:15.540Z
Answers.UserDisplayName: SmallChess
Body: <p>I am developing an LSTM for sequence tagging. During the development, I do various changes in the system, for example, add new features, change the number of nodes in the hidden layers, etc.  After each change, I check the accuracy using cross-validation on a development dataset.</p>
  <p>Currently, in each check, I use 100 iterations to train the system, which takes a lot of time. So I thought that, maybe, during development, I can use only e.g. 20 iterations. Then, each check will be faster. After I find the best configuration, I can switch back to 100 iterations to get better accuracy.</p>
  <p>My question is: is this consideration correct? More specifically, if model A is better than model B with 20 training iterations, is it likely that A will be better than B also with 100 training iterations?</p>
  <p>Alternatively, is there a better way to speed up the development process?</p>
  
Tags: <neural-networks><training><long-short-term-memory><hyperparameter-optimization><epochs>
Title: How to shorten the development time of a neural network?
UserDisplayName: Erel Segal-Halevi
ThreadText: # How to shorten the development time of a neural network?
  
  (Asked by: username_0 on 2017-08-02)
  
  I am developing an LSTM for sequence tagging. During the development, I do various changes in the system, for example, add new features, change the number of nodes in the hidden layers, etc. After each change, I check the accuracy using cross-validation on a development dataset.
  
  Currently, in each check, I use 100 iterations to train the system, which takes a lot of time. So I thought that, maybe, during development, I can use only e.g. 20 iterations. Then, each check will be faster. After I find the best configuration, I can switch back to 100 iterations to get better accuracy.
  
  My question is: is this consideration correct? More specifically, if model A is better than model B with 20 training iterations, is it likely that A will be better than B also with 100 training iterations?
  
  Alternatively, is there a better way to speed up the development process?
  
  **Tags:** neural-networks, training, long-short-term-memory, hyperparameter-optimization, epochs
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2017-08-02. Score: 5 | Accepted answer
  
  Your scenario is common.
  
  The most straightforward approach is to subsample your data randomly. Unless your data or your model has strong bias, your performance to the smaller data set should be comparable. The accuracy might be lower, but the purpose is to do quick sanity check.
  
  > **Comments:**
  > **username_0**: ‎I tried this and got good results.  
  
  ---
  
  ### Answer by username_1 on 2017-08-02. Score: 2
  
  This might work for your case but isn't necessarily true and depends on how much data the network goes through in an iteration. You should be able to test this by making a small change and training until 100 iterations and seeing if the performance significantly changes and if it can be predicted from the 20th iteration.
  
  Another way which may work for you is preloading lower layers of your network (if you have more than one layer).
  For instance, if you have 5 layers and are making changes to the last 2, you could preload the bottom 3 layers with previously trained weights. This should decrease the amount of training that needs to take place as your network can already discern some primary features of your problem.
  
  
  ---
  
  