As you can see I need to use this kind of data during my training process, so IO has been a huge problem.
I also use DDP, which makes me tougher to adjust my code.
Is there a way in pytorch can handle this problem? Help me, please.