Hi, we also want to know how multiprocessing via model.share_memory() can affect our training performance or we won’t need to worry about this. @smth
I am using multiprocessing framework to speed up the CPU processing data into tensor following the best practice on
https://pytorch.org/docs/stable/notes/multiprocessing.html 6
suppose I have list  [1,2,...32]  dataset, I have 2 outer process, each process with a 4 worker data loader, so

process 1 will load

[1,2,3,4]
...
[13,14,15,16]


process 2 will load

[17,18,19,20]
...
[29,30,31,32]

so in each process, I have 4 worker data loader, so let’s see one data loader in process1 will only load
[1,2,3,4] .
all these process and multi worker dataloader is to speed up the data loading and data cooking, however, I only have one GPU and one model, so in this case, if I use  model().share_memory()  ? so each process will just process partial dataset, but their tensor will be shared inside GPU, so final model will have all trained gradient and I don’t need to worry like race condition or so?