I notice that the clip_grad_norm trick appears frequently in Cutting edge solutions like fastai; aside from preventing NaNs it helps to correct a fundamental weakness of any RNN, that of vanishing and exploding gradients.  This can greatly improve the models performance.