I believe one approach would be to use the new inputs argument in the backward 2 operation, which allows you to specify the tensors, which should accumulate the gradients. In your case you could pass the parameters from f and g2 to the backward operation created by z2. Another possibility could be to detach the tensor as suggested by you, retain_grad() on x during the backward pass of z1 and pass this gradient to x.backward(grad) to continue the backpropagation through e.