Hi,
been trying to speed up training time by optimizing the dataloader configuration while running inside a docker.
at first, I was limited by permission issues and the number of open file descriptors to using the ‘file system’ option. without it, my code was crashing on the first batch once using more than one worker.
later, I was able to raise the docker shm_size and also the number of open file descriptors so it was feasible to use the default ‘file descriptor’ option.
I was expecting a speed up in performance as it is the default and recommended option, but was getting and 2x slowdown.
any clues/ideas?