Did you make sure to call model.eval() before the evaluation step?
This will make sure to set all modules to eval, e.g. nn.Dropout won’t drop units and nn.BatchNorm will use the running estimates instead of the batch statistics.