Based on your code snippet you are clipping the gradients in FP32 training, but not in the amp run.
Could you add it as described here 103 to get matching behavior and check, if the loss still gets NaN values?