albanD:

In your example, since you don’t modify wt_new inplace, you don’t need the .clone() .


However, the original wt is going to collect gradients with respect to the original graph. So if I call .detach() wouldn’t it collect the gradients for both graphs in the same tensor? That’s definitively not what I want. I want separate gradients (or at least that’s my rationale for calling .clone() first and then .detach()). What are your thoughts master albanD?