Hi,
I think the problem is that you are updating d[bag_i] before computing backpropagation so when autograd tries to do it, the values already are modified.
Note that a = a+b is considered inplace operation.
One possible solution is to clone d[bag_i]s using .clone() function and update cloned version. Other approach is defining new tensor for concatenation.
Please see these posts. [1] 1, [2] 1
Bests