Answers.Body: <p>Yes this is possible. Many hyperparameters of training can be changed at any time when training a neural network model, and that can include the objective function. You can also change the dataset, freeze/unfreeze layers (i.e. decide this time around that only some layers will be trained), and in some situations you can even usefully add/remove layers or otherwise adjust the NN architecture.</p>
  <p>If you intend that the extended training run is still for the same end result, then you should keep the validation and test <em>metrics</em> (e.g. accuracy, <a href="https://glassboxmedicine.com/2019/02/23/measuring-performance-auc-auroc" rel="nofollow noreferrer">AUROC</a>, <a href="https://en.wikipedia.org/wiki/Mean_absolute_percentage_error" rel="nofollow noreferrer">MAPE</a>) the same as before, so you can make a comparison with other approaches, including the state of the model before you continued the training with new hyperparameters.</p>
  <p>Whether or not your idea works for your particular situation is impossible to say without doing the experiment. You will also need to try multiple times, with different learning rates or other hyperparameters, before finding the best possible values to use when switching the loss.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-26T10:34:31.557Z
Answers.LastActivityDate: 2023-11-26T12:07:44.790Z
Answers.LastEditDate: 2023-11-26T12:07:44.790Z
Answers.UserDisplayName: Neil Slater
Body: <p>I have a saved tensorflow neural network model. I was wondering if it's possible to incrementally train the model but with different nt loss function.</p>
  
Tags: <neural-networks><tensorflow><long-short-term-memory><models><incremental-learning>
Title: Can I do incremental learning with different loss function in neural networks?
UserDisplayName: SUNITA GUPTA
ThreadText: # Can I do incremental learning with different loss function in neural networks?
  
  (Asked by: username_0 on 2023-11-26)
  
  I have a saved tensorflow neural network model. I was wondering if it's possible to incrementally train the model but with different nt loss function.
  
  **Tags:** neural-networks, tensorflow, long-short-term-memory, models, incremental-learning
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-11-26. Score: 1
  
  Yes this is possible. Many hyperparameters of training can be changed at any time when training a neural network model, and that can include the objective function. You can also change the dataset, freeze/unfreeze layers (i.e. decide this time around that only some layers will be trained), and in some situations you can even usefully add/remove layers or otherwise adjust the NN architecture.
  
  If you intend that the extended training run is still for the same end result, then you should keep the validation and test *metrics* (e.g. accuracy, [AUROC](https://glassboxmedicine.com/2019/02/23/measuring-performance-auc-auroc), [MAPE](https://en.wikipedia.org/wiki/Mean_absolute_percentage_error)) the same as before, so you can make a comparison with other approaches, including the state of the model before you continued the training with new hyperparameters.
  
  Whether or not your idea works for your particular situation is impossible to say without doing the experiment. You will also need to try multiple times, with different learning rates or other hyperparameters, before finding the best possible values to use when switching the loss.
  
  
  ---
  
  