I have a training scheme with multiple losses which looks something like this:
x1 = generator(img1)
x2 = generator(img2)

L1 = nn.NLLLoss2d()(nn.LogSoftmax()(x1),target1)
if some_cond:
   L2 = nn.NLLLoss2d()(nn.LogSoftmax()(x2),target2)
   L_total = L1 + L2
else:
    L_total = L1

optimizer.zero_grad()
L_total.backward()
optimizer.step()

If the some_cond is False, then my GPU runs out of memory in the next iteration of the training. I believe that this is because the intermediate variables from the second forward pass through the generator are not deallocated.
Can this be the reason for out-of-memory error? if yes, is there a way to manually deallocate all the variables from the second forward pass through the generator?