Hi,
Where do you set torch.set_num_threads(1) ?
You have to set it at the beginning of the Worker() function for it to have an effect on the newly created process.
Checking with @mrshenli, this seems to give the right behavior after setting this.
In particular, by default, pytorch will use all the available cores to run computations on CPU. So if you launch two processes to do this at once, then they will fight for the CPU and most likely slow each other down.