Well this is how an rnn works: you combine the current input with the previous step’s hidden state to get on one hand the output of this step and the new hidden state.
Here to do so, the input and the previous layer hidden state are combined together into a single tensor. And then a linear layer is used to get the output and another to get the new state.