Actually, I’ve tested the time consumption of DataLoader.

time_per_iteration.png700×450 27.9 KB

pic1: data loading time consumption at every iteration

total_time_at_iteration.png700×450 21.9 KB

pic2: accumulation of pic1
Blue for 1 worker, and yellow for 8, green for 16, red for 32.
I only tested 128 steps with batch_size=32
Multithreading does help, but it’s unreasonable to take 1.5s on average for loading one picture.
Here’s my code:
transform = transforms.Compose([
    transforms.RandomResizedCrop(224),
    transforms.ToTensor(),
])

train_dir = '...'
train = datasets.ImageFolder(train_dir, transform)

dataloader = torch.utils.data.DataLoader(train, batch_size=32, shuffle=True, num_workers=32)

t_sum = 0
last_time = time.time()

for i, data in enumerate(dataloader):
    t = time.time()
    t_diff = t - last_time
    t_sum += t_diff
    draw_function(t_diff, t_sum)
    last_time = t

    if i >= 127:
        break