It may be a bit confusing, but it is shifted. The reason is, before the first step, you already have one a priori hidden state, and first step looke like rnn(input[0], prior_hidden) ->hidden[0]. This way, tensor sizes match in time dimension.