I see. So you have predefined batches of a set size, and you cannot change that? That’s a bit… odd. I’m sure there is a way to do what you want, but it seems too complex to spend time in. It’d be better if you do not use predefined batches and instead use a distributedsampler and DDP that automatically takes care of parallellization of the input.