Based on your code it looks like you are passing a single sequence, is this correct?
The shape of your input should be [seq_len, batch_size, input_size, so currently seq_len=1.
The shape of your hidden state should be alright, as it’s expected to be [num_layers, batch_size, hidden_size].
You don’t have to change your input regarding the number of layers.
Here is a small example taken from the docs:
seq_len, batch_size, input_size = 1, 100, 15
num_layers, hidden_size = 2, 20

rnn = nn.RNN(
    input_size=input_size,
    hidden_size=hidden_size,
    num_layers=num_layers
)

x = torch.randn(seq_len, batch_size, input_size)
h0 = torch.randn(num_layers, batch_size, hidden_size)

output, h = rnn(x, h0)