@raulpuric In fact, your code does the same thing as torch.nn.utils.clip_grad_norm_ , as far as I can see? ie, you’re going to call this between loss.backward() and opt.step(), and clip the gradients after the full backprop has taken place?
@tom what things do you recommend applying the hook to? Are you just applying it to the state from each time step? or … ? What happens if one is using an nn.LSTM, rather than an nn.LSTMCell? Will the hook get called each timestep? Or is it obligatory to use nn.LSTMCell if we want to clip the gradients at each timestep?