Hi Wang-zipeng,
You can make different copies by using copy.deepcopy so that the original gradient won’t be overwrote during backpropagation. Or what do you mean that “head is shared head” ? Can you paste your code here?