for people trying to just get an answer quickly:
torch.nn.utils.clip_grad_norm(mdl_sgd.parameters(),clip)

or with in-place clamp:
W.grad.data.clamp_(-clip,clip)


also similar Q:

  
    
    
    Gradient clipping 
  
  
    Hi everyone, 
I am working on implementing Alex Graves model for handwriting synthesis (this is is the link) 
In page 23, he mentions the output derivatives and LSTM derivatives 
 [image]
How can I do this part in PyTorch? 
Thank you, 
Omar