Hi, but won’t this be a problem , as graph of headA will keep growing without a loss.backward() step for headA (back-propagation frees the graph created during epoch) ?  by sharing of weight do you mean manually copying of weight from one layer to another or something else.