Agree. When you check out simple RNN code result, it makes us wonder why the code in the RNN tutorial is different from the definition of RNN.
import torch.nn as nn
rnn = nn.RNN(10, 10, batch_first=True)

inp = torch.randn((1,3,10))

result = rnn(inp)

print(result[0].shape)
print(result[1].shape)
print(result[0][0])
print(result[1])

Same result of :
tensor([[[-0.0253,  0.5289, -0.2582, -0.5125, -0.4577,  0.5456, -0.4042,
-0.6476, -0.4210, -0.5473]]], grad_fn=)