Actually, you cannot use ddp sampler to achieve validation. You can see DistributedSampler 35; note that the dataset has added extra samples to make it evenly divisible. Therefore, if your dataset is very small, the final result may be different. The official implementation is right.