pytorch just mirrors cudnn I think; what usually happens is Wx+b non-recurrent computation is merged (W = [Wr Wz Wn]) and done first, so theoretical uses of two biases are imaginable, but generally it beats me too why cudnn has this…