Hello,
Regarding this point, as per  SAVING AND LOADING MODELS:

If you only plan to keep the best performing model (according to the acquired validation loss), … You must serialize best_model_state or use best_model_state = deepcopy(model.state_dict()) otherwise your best best_model_state will keep getting updated by the subsequent training iterations. As a result, the final model state will be the state of the overfitted model.

However, I have done something like this:
def train_model(model, ...):
       ...
       if validation_loss improves:
            delete previous best model
            torch.save(model.state_dict(), best_model_path)
       else:
             ....
        ...
        return model

 def test_model(model, best_model_path, ...):
     model.load_state_dict(torch.load(best_model_path))
     model.eval()
     ...

...

my_model = train_model(my_model, ...)

test_model(my_model, my_path, ...)

In other words, the model returned by the training phase is the final one which is likely to present overfitting. But since I saved the best model during training, I have no problem during the test/inference phase because I load it during testing.
Is something wrong with this solution?
Thanks.