@mattrobin
I tried using itertools’s cycle function on a dataloader that loads ImageNet data to implement the same thing you are trying to do. The CPU memory usage keeps on increasing and the system crashes everytime.
Were you able to design a custom Dataset as suggested by @SimonW? If so, could you please share the code?