mrshenli:

In your application, is it like each subprocess will produce some image embedding independently and concurrently, and then you wanna save those?


Yes, each subprocess generate embeddings from dataloader batches. I want to process all my data (generate embeddings) as fast as possible, this is why I want to use DistributedDataParallel. Process and after that save everything in one file.