If you want to use another hidden size, you could pass the RNN output to another linear layer like in this tutorial 1.
Also, I’ve noticed another issue, which might explain the constant output and the NaN values.
Currently you are using torch.Tensor() to initialize your hidden tensor. This will use uninitialized memory, i.e. all values will be set to whatever is stored in the memory block.
You should rather use torch.zeros or torch.randn for random values.