This is surprising…
The clip_grad_norm_ function is pretty simple and is there: https://github.com/pytorch/pytorch/blob/1c6ace87d127f45502e491b6a15886ab66975a92/torch/nn/utils/clip_grad.py#L25-L41 35
Can you try to copy paste that in your code and check it gives nan as well? Then you can add some prints there to see when the nan appears