hello, i have the same problem right now, i don’t understand the right way ‘share_memory()’  work;
to now i have splitted my dataset and given every single part to a different process with the same model, at the end i want to call the optimizer (assuming that every single process have called the backpropagation on its data), so having the updated weight w.t.r. to the sum of the gradient (every process calling backward may sum the current grad with one’s).