How did you verify that all the gpus are using the same batch. Thats not how DDP works, it will take different chunks automatically.
https://pytorch.org/docs/master/generated/torch.nn.parallel.DistributedDataParallel.html 127
This container parallelizes the application of the given module by splitting the input across the specified devices by chunking in the batch dimension. The module is replicated on each machine and each device, and each such replica handles a portion of the input. During the backwards pass, gradients from each node are averaged.