My guess is pytorch is assigning complete data batch (size 8) to one GPU.

Yep, this is true. The DistributedSampler is a per-process concept. So if you spawn two processes, each process will construct its own DDP instance and DistributedSampler instance. Can you try reduce the the per-process batch_size to batch_size / world_size?