Before calling the evaluation code you should set your model into evaluation mode by calling model.eval() on it.
This will make sure to set all layers to eval, such as Dropout and BachNorm layers.
You don’t need to remove these layers or do something else with it.
Also during evaluation you should use a context manager to disable the gradient calculation:
model.eval()
with torch.no_grad():
    # your evaluation code

Remember to set the model to train before training with model.train().
Regarding the overfitting:
Are you observing the training and validation accuracy? Does the validation accuracy drop after a while?
Have you considered early stopping?
It’s good to see that your resubstitution error is low, i.e. that you can overfit on your training data.
Now you could add some more regularization techniques like weight decay, more dropout etc.