Why RNN needs two biases?
Sunnydreamrain
You mean in general? For the same reason that it needs two sets of weights, one for the input and one from the previous state.
Ismail_Elezi
Sunnydreamrain
As you pointed out it doesn’t really change the definition of the model, but this is what cuDNN does, so we’ve made our RNNs consistent with this behaviour.
apaszke
OKay. Good to know. Thanks.
Sunnydreamrain