I don’t think you declare the hidden state assuming no batches, given that the documentation specifies that:

h_0 (num_layers * num_directions, batch, hidden_size): tensor containing the initial hidden state for each element in the batch.

This implies a hidden state of dimensions [1, batch_sz, hidden_size] for a single-layer uni-directional RNN.