While using PyHeat to analyze the time cost of my code, I find that the operation of Tensor.cuda() costs a lot, even more than that of using net to generate a result. Here is the cut of my PyHeat report:
train.jpeg1596×432 93.6 KB
 
eval.jpeg1562×266 72.2 KB

I tried using Tensor.cuda() in the function of getitem in my dataset but failed. I also tried setting torch.multiprocessing.set_start_method(‘spawn’,force=True) and setting pin_memory=True num_worker=4 in my dataloader, all not work yet. (If num_worker is 1, everything works fine, but it’s too slow)
So I searched in PyTorch doc and only find things with how to put my model in multi gpus rather than put my data on one gpu with multiprocess. In the official examples of PyTorch, I still can’t find any code that use .cuda() in getitem in dataset. I don’t know whether there is something I missed in Doc, so I create a topic here for help.
Briefly, I want to put my data on one gpu with multiprocess and train my model on that gpu too, is there any solutions?