Hi @moreshud
You could do something like:
if accuracy_val > max_accuracy_val:

            checkpoint = {
                'epoch': epoch,
                'model_state': model.state_dict(),
                'optimizer_state': optimizer.state_dict(),
            }
            torch.save(checkpoint, 'path/to/folder/filename.pth')

            max_accuracy_val = accuracy_val

That way you save the state of the model whenever you have reached a new maximum accuracy on the validation set.
When you want to continue training you can do:
loaded_checkpoint = torch.load('path/to/folder/filename.pth')
loaded_epoch = loaded_checkpoint['epoch']
loaded_model = model() # instantiate your model
loaded_model.load_state_dict(loaded_checkpoint['model_state'])
loaded_optimizer = torch.optim.SGD(loaded_model.parameters(), lr=0, momentum=0) # or whatever optimizer you use
loaded_optimizer.load_state_dict(loaded_checkpoint['optimizer_state'])

You can then continue training with the loaded model and loaded optimizer.
DISCLAIMER: I am relatively new to PyTorch myself. This approach works for me but I cannot guarantee that there are no better options 
Anyways, I hope it helps!
All the best
snowe