Would it work if you set the batch_size to num_of_batch*num_of_images in your DataLoader and just reshape the batch after the loading, i.e.:
for data, target in loader:
    data = data.view(num_of_batch, num_of_images, channels, w, h)
    target = ...

In case it’s not possible, I would try to load the files sequentially and see if it’s really the bottleneck of your training procedure.