Did the model fit into your GPU in training?
If so, it’s strange that it doesn’t fit anymore in the evaulaution.
To save some memory you could set your Variables to volatile=True.
By this, the intermediate results of the forward pass won’t be saved anymore, which are needed for your backward pass (weight updates).
Anyway, if this does not solve your problem, you could shard your model on different GPUs like this:
class MyModel(nn.Module):
def __init__(self, split_gpus):
    self.large_submodule1 = ...
    self.large_submodule2 = ...

    self.split_gpus = split_gpus
    if split_gpus:
        self.large_submodule1.cuda(0)
        self.large_submodule1.cuda(1)

def forward(self, x):
    x = self.large_submodule1(x)
    if split_gpus:
        x = x.cuda(1) # P2P GPU transfer
    return self.large_submodule2(x)