Are you calling model.eval() for the validation and test case?
Also, make sure to use the same preprocessing for both use cases.
If that doesn’t help, could you post your training and validation/testing loop, so that we could have a look?