Could you profile the model execution separately to avoid profiling the data loading as well?
To do this, you could use random CUDATenors as the input and profile the forward (and backward) calls separately:
data = torch.randn(...).cuda()

nb_iters = 100

# warmup
for _ in range(10):
    out = model(data)

torch.cuda.synchronize()
t0 = time.time()
for _ in range(nb_iters):
    out = model(data)
torch.cuda.synchronize()
t1 = time.time()
print('forward ', (t1 - t0)/nb_iters)

# warmup
for _ in range(10):
    out = model(data)
    out.mean().backward()

torch.cuda.synchronize()
t0 = time.time()
for _ in range(nb_iters):
    out = model(data)
    out.mean().backward()
torch.cuda.synchronize()
t1 = time.time()
print('backward ', (t1 - t0)/nb_iters)

or use the profiler in PyTorch.