Hi! Thanks for your answer. In my current situation, without multiprocessing each epoch costs about 8 seconds. I saw other implementations used multiprocessing to speed-up training process and it really helped (but they are written with Tensorflow).
You said it’s likely that sharing cuda Tensors is not supported on windows. To verify this I tried runing a PyTorch official example program (code link 5) which uses a very similar multiprocessing structure as mine, and same error occurred if I used cuda in windows system. I believe your guess in right!
But I still want to speed-up my process. In the next try, I will use multiprocessing not torch.multiprocessing and see if it works!