The two lines:
fake = torch.Tensor(fake.size()).cuda().copy_(fake.data)
fake = Variable(fake)

break the computation history. This means that when you backprop through your discriminator it won’t backprop through the generator. It might also mean that by dropping the computation history, you’re allowing some memory to be freed.
Typically, your memory usage shouldn’t go up between mini-batches. It might happen if you’re holding on to the result of some computation.  A common mistake is to do something like:
sample_loss = loss_fn(net(input), target)
sample_loss.backward()
# etc. etc.
total_loss += sample_loss

To try to average the loss over an epoch. The problem is that this may hold onto the entire computation history that computes the summed loss.  Instead do:
sample_loss = loss_fn(net(input), target)
sample_loss.backward()
# etc. etc.
total_loss += sample_loss.data

(i.e. if you’re summing the loss across an epoch, you just want to accumulate the value, but not save the computation history)