I printed memory_allocated.
The memory for loss1 and loss2 are accumulated in the last.
After getting loss1, Can I remove this memory before getting loss2 ?
optimizer.zero_grad()
out1 = network(input)
loss1 = criterion(out1, target1
loss += loss1.item()
del out1
print(torch.cuda.memory_allocated(0)) # 2713555968

out2 = network(input)
loss2 = criterion(out2, target)
loss += loss2.item()
del out2
print(torch.cuda.memory_allocated(0)) # 5277173248

loss.backward()