I’ve tried using Z.data.copy_ to store the intermediate results, in which case Zt is detached from the graph and an error is not raised.
Alternatively, I’ve used a hidden state to store the intermediate nodes and used retain_graph=True for the remaining T-1 time steps, unfortunately, the convergence is not even remotely similar to when the relation is modeled directly.