BPTT isn’t anything more special than an ordinary computation graph. You can think of as unrolled if that makes it easier to understand.
You can use nn.GRU and nn.LSTM directly, and autograd will work just fine.