dachosen1:

What do you mean by using synthetic data?


Instead of loading real data you could initialize random data, push it to the GPU and profile the training loop without the data loading:
# for a multi-class classification
data = torch.randn(batch_size, channels, height, width, device='cuda')
target = torch.randint(0, nb_classes, (batch_size,), device='cuda')

for epoch in range(10):
    optimizer.zero_grad()
    output = model(data)
    loss = criterion(output, target)
    loss.backward()
    optimizer.step()




 dachosen1:

would increasing the number of workers in the Dataloader method increase the number workload on the GPU?


Generally yes, but too many workers might again slow down your system. The sweet spot depends on your current workstation and you would have to profile specific values.