Oh yes, sorry. requires_grad_() + detach() is redundant here and just having having h0 = torch.zeros(...) and passing in that would be more idomatic (in general, it seems strange to use requires_grad_ right after a factory function where could use requires_grad=True, but hey).
Best regards