The activation values are not clipped or manipulated in any way, so that the FP16 bounds can yield these NaN values. I’m not familiar with your workflow and would recommend to normalize the inputs etc. as is done in the majority of the models. This would keep the parameters as well as the activations in a reasonable range and would avoid running into the overflows.
CC @yft123