hughperkins:

@tom what things do you recommend applying the hook to? Are you just applying it to the state from each time step? or … ? What happens if one is using an nn.LSTM, rather than an nn.LSTMCell? Will the hook get called each timestep? Or is it obligatory to use nn.LSTMCell if we want to clip the gradients at each timestep?


You can’t hook anything in the timesteps if you don’t see them, so currently you need to do the looping yourself, making LSTMCell a natural choice.
I hope that PR 14957 36 brings manual LSTM performance back into the “awesome” region (but didn’t time it yet), then we’d have to see how to get clipping in there.
Best regards
Thomas