@Ohm My apologies for the previous reply not being clear. The splitting automatically occurs within the DistributedSampler based on the rank that you provide. This piece of code 11 in the DistributedSampler would illustrate how this splitting is done per worker based on the rank.