Switching correctly train()/eval() during training/validation is tricky in the presence of exceptions or other early exits. Once in the wrong state your training might be totally off. Context managers help in such situations. I’m using the following context manager:
# Following snippet is licensed under MIT license

from contextlib import contextmanager

@contextmanager
def evaluating(net):
    '''Temporarily switch to evaluation mode.'''
    istrain = net.training
    try:
        net.eval()
        yield net
    finally:
        if istrain:
            net.train()

Intended usage: with evaluating(net): .... Also in case you are concerned having too many nested blocks, you can collapse multiple context managers into a single line with evaluating(net), torch.no_grad(): in py3.x and later. In 2.7 you need to use nested from contextlib.