Piggy:

I try to use detach_() to truncate the gradient of a layer so that it does not propagate back. However, I find that the previous layer still have gradient update.


Hello, from here it seems that you are confused about intermediate value (non-leaf node) and weight of layers (leaf node), by .detach() the weight of layer would not have any effect on the gradiant flow back to the previous layers. If you want, you could .detach() on the intermediate value and the computation graph will break down.