I think it is the latter. model.eval() has effect on dropout, batchnorm etc. You can use model.eval() in combination with with torch.no_grad() during inference phase.
from the docs:
Disabling gradient calculation is useful for inference, when you are sure that you will not call Tensor.backward(). It will reduce memory consumption for computations that would otherwise have requires_grad=True. In this mode, the result of every computation will have requires_grad=False, even when the inputs have requires_grad=True.