I have done some experiments about the time cost on data load and model training.


Load data with torch.utils.data.DataLoader and train model.
Time cost on dataload:  14.0 ms
Time cost on training(dataload+model traning): 1.845 ms


Almost all time cost on data.to(device) during loading data, so I tried another way to load data. I read data to a queue in other process, then I collected those data into a batch and executed data.to(device) in other thread and push them to a queue. In main thread, I access to those data by q.get().


Time cost on dataload:  < 0.1 ms
Time cost on training(dataload+model traning): 1.885 ms
My question is, why dose method 2 cost more time on training?
I guess it is because the new thread slow down the main thread, but torch.utils.data.DataLoader load data with multithreading as well.