Thanks for the data. It seems you are using a manual float16 approach, which could easily overflow.
If I wrap the loss calculation into with torch.cuda.amp.autocast() the result will be valid.
What’s the use case you are not using autocast, but a manual approach?