Hi all,
I am not posting the actual code since it is a lot and cannot be reduced that easy to the problem. But essentially I am having
output = net(input) which is a batchsize x 1 tensor.
I calculate the mean of output over the batchsize and my loss function is
loss = (mean - 10).pow(2).
So I am trying to have a mean of my network output = 10. First 2 iterations the loss goes down, in the third iteration it suddenly goes up to a million and then nan.
How can I debug such an issue (the definition of the loss was a simplification)?