One option is to use torch.multiprocessing.Queue 48 as the shared memory. The main process can prepare multiple queues and then pass one queue to each DDP processes. The main process reads from the file and dispatch data items to the queue, while DDP processes wait on their own queue for that data item.
Another option is to split the tar file into multiple smaller pieces and let each DDP process read a different one.