Half is touchy w.r.t. numerical stability. Without further analysis of your network, it would seem much more likely that you are running into this inherent instability rather than an actual PyTorch bug. That said, if you reduce it to “this op with these inputs is NaN when it shouldn’t”, it could be a bug.
Best regards
Thomas