title: Fine-tuning PyTorch: No deepcopy
body: <p>Regarding fine-tuning CNNs in PyTorch, as per <a href="https://pytorch.org/tutorials/beginner/saving_loading_models.html" rel="nofollow noreferrer">SAVING AND LOADING MODELS</a>:</p>
  <blockquote>
  <p>If you only plan to keep the best performing model (according to the acquired validation loss), … You must serialize best_model_state or use best_model_state = deepcopy(model.state_dict()) otherwise your best best_model_state will keep getting updated by the subsequent training iterations. As a result, the final model state will be the state of the overfitted model.</p>
  </blockquote>
  <p>However, I have done something like this:</p>
  <pre><code>def train_model(model, ...):
         ...
         if validation_loss improves:
              delete previous best model
              torch.save(model.state_dict(), best_model_path)
         else:
               ....
          ...
          return model
  
   def test_model(model, best_model_path, ...):
       model.load_state_dict(torch.load(best_model_path))
       model.eval()
       ...
  
  ...
  
  my_model = train_model(my_model, ...)
  
  test_model(my_model, my_path, ...)
  </code></pre>
  <p>In other words, the model returned by the training phase is the final one which is likely to present overfitting (I did not use deepcopy). But since I saved the best model during training, I have no problem during the test/inference phase because I load the best model, overloading the final model obtained during testing.</p>
  <p>Is something wrong with this solution?</p>
  <p>Thank you.</p>