If larger gradient magnitudes are expected and would thus create invalid values, you might clip the gradients. You could start with a max norm value of 1 or refer to any paper, which uses a similar approach.
Note however, that FloatTensors have a maximal value of:
print(torch.finfo().max)
> 3.4028234663852886e+38

so you should make sure that the NaNs are not created by an invalid operation.