You could divide your entire dataset into training (80) , validation (10), and testing (10) , and do a traning scheme like https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html 14 . Also don’t forget to set the model in training mode with model.train() and when evaluate, either in validation or training, set to evaluation mode with model.eval()