Hi, suppose you pile two RNN layers. Then the input to the first layer is of course input_t then the input to the second layer is the output from the first layer, that is, hidden state of the first layer.