Hi,
In my case, I calculate loss as following code:
loss = 0
for A,B,C in sequence_of_tensors:
    loss += ((A - B)**2 * k1 * (A > C).float() +
             (A - B)**2 * k2 * (A < C).float() +
             (A - B)**2 * k3 * (A == C).float()).sum()


A and B are tensors. When I put all tensors on GPU, memory usage is about 5GB. But when the computation begins, I occur CUDA out of memory error. My GPU has 12GB memory, which I think is enough. The intermediate results of computation should not be larger than original tensors.
I can’t understand why such computation is so memory expensive. I’ll appreciate any help.