I believe that all tensors passed into multiprocessing spawn will be converted into shared memory?
According to [1], that’s the case when using multiprocessing queue. I actually experimented with this as well before and I believe that the tensors passed as args when spawning sub-processes are indeed converted to shared memory. Otherwise, can also try to add shm.share_memory_() before appending it to the list, and I believe the result will still be the same.
Besides, thanks for taking a look into it! Do you perhaps have any thoughts on what might cause such a different in memory usage?
[1] Multiprocessing best practices — PyTorch 1.9.1 documentation 1