@jteuwen I have a question for you.  Are you loading a single 3D file that then gets sliced and passed into training a model?  If this is the case one question I have is that when doing multiGPU training via DDP, could you not run into a situation where multiple processes would get different slices that originate from the same 3D file and would therefore want to access the file at the same time?