Hi @ptrblck
I included the torch.autograd.set_detect_anomaly(True) in the script and also printed the output of focal_dice_loss_fg from the custom loss function. I guess it detects ‘nan’ in forward call:
fdloss, _, _ = focal_dice_loss(out, seg)
epoch: 1/200, subject: 1/2, batch: 31/32, avg-batch-loss: 7.7020, avg-batch-dice: 0.1117
tensor([1.0000, 1.0000, 1.0000, 1.0000, 0.9969, 0.9989, 0.9998, 1.0000],
       device='cuda:0', grad_fn=<SliceBackward>)
epoch: 1/200, subject: 1/2, batch: 32/32, avg-batch-loss: 7.7111, avg-batch-dice: 0.1084
Criteria at the end of epoch 1 subject 1 is 0.1084
criteria increased from 0.0000 to 0.1084, saving model ...

[W python_anomaly_mode.cpp:104] Warning: Error detected in PowBackward0. Traceback of forward call that caused the error:
  File "/home/banikr/PycharmProjects/HeadSeg/dummy.py", line 121, in <module>
    fdloss, _, _ = focal_dice_loss(out, seg)
  File "/home/banikr/PycharmProjects/HeadSeg/Utilities.py", line 391, in focal_dice_loss
    focal_dice_loss_fg = torch.pow((1-tversky_dice), gamma_fd)[1:] # removing 0 --> background
 (function _print_stack)
tensor([1.0000, 1.0000, 1.0000, 1.0000, 0.9292, 1.0000, 0.0000, 0.0000],
       device='cuda:0', grad_fn=<SliceBackward>)
Traceback (most recent call last):
  File "/home/banikr/PycharmProjects/HeadSeg/dummy.py", line 129, in <module>
    fdloss.backward()
  File "/home/banikr/miniconda3/envs/HeadSeg36/lib/python3.6/site-packages/torch/tensor.py", line 221, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/banikr/miniconda3/envs/HeadSeg36/lib/python3.6/site-packages/torch/autograd/__init__.py", line 132, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: Function 'PowBackward0' returned nan values in its 0th output.

But
torch.pow((1 - torch.tensor(1.0)), 0.75)) shouldnt be a nan value. I dont see a reason for inf or abnormal dice value.
I will keep checking further. Just to share you the update.