Without torch.no_grad() any operation that involves the tensors with requires_grad=True is recorded by autograd and this process consumes memory. Plus, it can make your code less error-prone (it can save you from occasions when you call backward in the eval phase by accident).
Calling net.eval() is mandatory on the model if the model contains layers/modules with different behaviour between the train and eval phase (dropout, batchnorm, etc.).