Proper way to do gradient clipping? 
  
  
    for people trying to just get an answer quickly: 
torch.nn.utils.clip_grad_norm(mdl_sgd.parameters(),clip)

or with in-place clamp: 
W.grad.data.clamp_(-clip,clip)


also similar Q:
  


Perhaps try this