Whenever you want to test your model you want to set it to model.eval() before which will disable dropout (and do the appropriate scaling of the weights), also it will make batchnorm work on the averages computed during training. Your code where you’ve commented model.eval() looks like like the right spot to set it to evaluation mode. Then after you simply do model.train() and you’ve enabled dropout, batchnorm to work as previously.