Taking RNN with tanh activation for example, it follows
h_t = tanh(w_{ih} * x_t + b_{ih}  +  w_{hh} * h_{(t-1)} + b_{hh})
b_{ih} and b_{hh} are just biases (trainable constants). It does not matter for input or previous state.
Actually, let b=b_{ih} + b_{hh},
h_t = tanh(w_{ih} * x_t +  w_{hh} * h_{(t-1)} + b).
It should be the same.