I just checked your loss function and it seems to work fine without creating a memory leak.
x = torch.randn(64, 28, requires_grad=True, device='cuda')
target = torch.empty(64, 28, device='cuda').random_(2)

criterion = LSEPLoss()

for epoch in range(10):
    loss = criterion(x, target)
    loss.backward()

Could you run it in your setup and check it?
I couldn’t test the whole script, as some objects and functions are missing, e.g. model, get_optimizer.
Also, Variables are deprecated since PyTorch 0.4.0, so you can just use tensors instead.