I am not sure what detaching l2 and l4 actually means. They are sequentially assigned to the new layer output x. I believe, they are being assigned x on the same space allocated to them initially. I don’t think any extra space is being used during reassignment.
In the case of forward hooks as well, I need the previous outputs which induce the need to sequentially find the outputs till the required layer. I might have not understood the concept of forward hooks( I only read the doubts on the discussion forum about forward hooks in pytorch) properly. Can you give a short example on the given snippet of code?