111319:

Back to the original question, is there a method that can detach a sub graph after a forward?


No there isn’t I’m afraid. We don’t allow modifying the graph.
If you just want the backward between two given Tensors, you should use the grad_input = autograd.grad(output, input) API. This will only run backward between the given Tensors.