Hi, I’m trying to use PyTorch AMP after knowing that it is now out. However, I’m getting NaN values in my model output. I traced this problem to the output of matmul having some infinite values. The inputs to the matmul operation are fine.
More particularly, I’m using the non-local block from here: Non-local_pytorch/non_local_embedded_gaussian.py at master · AlexHex7/Non-local_pytorch · GitHub 2
How do I deal with this?