That makes sense, if you’re​ initializing a random hidden state, making it random between batches, you’re gonna get different outputs for the same input.
If you were trying to make them the same you could initialize a zero hidden state (as leaving it None does) or a n_layers x 1 x hidden_dim random state that you repeat batch_size times over dimension 1. However I can’t imagine why that would be useful… if you’re building a seq2seq type model the states will be different between batches.