After testing your script, it seems that this only occurs when using the GPU. PyTorch will throw the warning that you mentioned when running on CPU, but it will just run through the code when on GPU - with very large or very small values as output. You can try the snippet below. Switching the flag USE_GPU should show the issue that I experience. Thanks for your time!
import torch
from torch import nn

USE_GPU = True

torch.manual_seed(2809)
torch.cuda.manual_seed(2809)


device = torch.device("cuda" if USE_GPU else "cpu")
rnn = nn.GRU(10, 10).to(device)
for _ in range(100):
    input = torch.randn(1, 3, 10).to(device)
    h0 = torch.randn(1, 3, 10).double().to(device)
    print("input dtype", input.dtype)
    print("h0 dtype", h0.dtype)
    output, hn = rnn(input, h0)
    print(output.abs().max(), hn.abs().max())
    print()