what kind of pre-processing are you doing on this video volumes? and are you using an efficient video loader? (maybe opencv?).
Identifying the bottlenecks in your DataLoader using line_profiler might be helpful. https://github.com/rkern/line_profiler 132
When running with line_profiler, run the program with workers=0, so that the DataLoader is in the main process and not a separate process.