By passing the model as an argument to say the training function, is this not creating a separate local copy  of the model for each process? I ask because I followed the Hogwild! example with my own network/data and have found that while each process tends to have the same optimization behavior, there is variance in the weights at the end of training, at each epoch that I have a checkpoint for. I do expect variance in things such as loss over time because the processes should be loading different random subsets of the same data, but it doesn’t make sense to me why the weights would be different if calling model.share_memory() should be placing the model object in a shared memory segment where there should only be one copy of weights that are manipulated. The only explanation that I can think of as to why my weights/biases would vary across processes would be if the processes are actually only using a local copy of the model…but perhaps I am seriously misunderstanding something here.