Hi @ptrblck , I’m also encountering the same issue but in my case, I implemented my custom BCELoss using the code below where predicted is the output after torch.sigmoid
def custom_BCELoss(predicted,target,error_value=-100.0): 

  assert target.size()==predicted.size(), 'Size of labels is not the same!'

  term_a = torch.log(predicted)
  term_a[torch.isinf(term_a)] = error_value
  
  term_b = torch.log(1.0 - predicted)
  term_b[torch.isinf(term_b)] = error_value

  loss = -1.0 * (target * term_a + (1.0 - target) * term_b)

  loss = torch.mean(loss)

  return loss

It seems to work fine but in all cases, the predicted value will produce nan after approximately 100 epochs. I tried to use the track anomaly function to trace the error,
torch.autograd.set_detect_anomaly(True)

and it’s telling me that the error occurs at

Warning: Error detected in Log Backward torch.log(1.0 - predicted)

Since the predicted is output from torch.sigmoid, I believed the cause of error should be the isinf replace function term_b[torch.isinf(term_b)] = error_value, and I should use torch.clamp to clamp the -inf value to “error_value” instead? But if replacing -inf is a problem why would it work in the first 100 epochs?