Answers.Body: <p>First, I assume you've tuned your hyperparameters. Because, instead of re-train the network (use the weights that resulted from the previously training process) that needs more times, I'll invest more on hyperparameters tuning of the available network.</p>
  
  <p>Then, there are several methods and considerations:</p>
  
  <ul>
  <li><p>You can use the weight resulted from your first network as the initial of your next training process. But as the network will face the same data/problem if you use "initial" value for your hyperparameters (e.g. high learning rate) then I'm afraid it will lead to overfitting. Because it's simply "nothing change" in your architecture.</p></li>
  <li><p>If you previously didn't use adaptive learning for your training process (e.g. Adadelta, Adam), you can re-train your network with a smaller learning rate. So, your model can find a better result.</p></li>
  <li><p>Or, you can use the concept of the <em>selffer</em> network, you use the weight for some layers from previous training process (you can freeze it or not) and randomly initialize other layers. And then, train the network using "initial" value of hyperparameters</p></li>
  </ul>
  
  <p>You can read more about <a href="https://arxiv.org/pdf/1411.1792.pdf" rel="nofollow noreferrer">the transfer learning concept (or <em>selffer</em> network)</a> to get the most appropriate method for your case. There is also a paper about <a href="https://arxiv.org/pdf/1803.10232.pdf" rel="nofollow noreferrer">"incremental training on CNN"</a> that I think is similar with the <em>selffer</em> network but with some modifications.</p>
  
  <p>Hope it helps</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-19T12:02:25.180Z
Answers.Text: "I assume you've tuned your hyperparameters", if i understood you correctly, yes, i reached this accuracy after trying different learning rates, and adding dropout layers.  (i have not change other parameters of vgg). 
  i tried using the previous weights into training the network again by only changing learning rate (learning rate of the new training is smaller than the previous training). but this method did not change the accuracy.
Answers.UserDisplayName: norahik
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-18T05:38:38.783Z
Answers.LastActivityDate: 2019-04-18T05:47:54.210Z
Answers.LastEditDate: 2019-04-18T05:47:54.210Z
Answers.UserDisplayName: malioboro
Body: <p>I have trained  a modified VGG classification CNN, with random initialized weights; therefor the validation accuracy was not high enough for me to accept (around 66%). 
  now using the weights resulted from training the network, how can i use those weights in training the network again to improve accuracy? (e.g. using previous training weights with different learning rate, or increase epochs, ..)</p>
  
Tags: <convolutional-neural-networks><training><keras>
Title: how to benefit from previous training weights in training again to increase accuracy?
UserDisplayName: norahik
ThreadText: # how to benefit from previous training weights in training again to increase accuracy?
  
  (Asked by: username_0 on 2019-04-17)
  
  I have trained a modified VGG classification CNN, with random initialized weights; therefor the validation accuracy was not high enough for me to accept (around 66%).
  now using the weights resulted from training the network, how can i use those weights in training the network again to improve accuracy? (e.g. using previous training weights with different learning rate, or increase epochs, ..)
  
  **Tags:** convolutional-neural-networks, training, keras
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-04-18. Score: 2
  
  First, I assume you've tuned your hyperparameters. Because, instead of re-train the network (use the weights that resulted from the previously training process) that needs more times, I'll invest more on hyperparameters tuning of the available network.
  
  Then, there are several methods and considerations:
  
  * You can use the weight resulted from your first network as the initial of your next training process. But as the network will face the same data/problem if you use "initial" value for your hyperparameters (e.g. high learning rate) then I'm afraid it will lead to overfitting. Because it's simply "nothing change" in your architecture.
  * If you previously didn't use adaptive learning for your training process (e.g. Adadelta, Adam), you can re-train your network with a smaller learning rate. So, your model can find a better result.
  * Or, you can use the concept of the *selffer* network, you use the weight for some layers from previous training process (you can freeze it or not) and randomly initialize other layers. And then, train the network using "initial" value of hyperparameters
  
  You can read more about [the transfer learning concept (or *selffer* network)](https://arxiv.org/pdf/1411.1792.pdf) to get the most appropriate method for your case. There is also a paper about ["incremental training on CNN"](https://arxiv.org/pdf/1803.10232.pdf) that I think is similar with the *selffer* network but with some modifications.
  
  Hope it helps
  
  > **Comments:**
  > **username_0**: "I assume you've tuned your hyperparameters", if i understood you correctly, yes, i reached this accuracy after trying different learning rates, and adding dropout layers.  (i have not change other parameters of vgg).  i tried using the previous weights into training the network again by only changing learning rate (learning rate of the new training is smaller than the previous training). but this method did not change the accuracy.  
  
  ---
  
  