Solved by rvarm1 in post #2 
              
              
                If the data is not sharded across different DDP ranks (i.e. with a distributed sampler or some custom sharding logic that you may have), then yes, DDP will use all samples on all ranks (in your example I guess there’s 2 ranks). 
This is why in general you want want to partition your data appropriate…