HI, Thank you for your interest in my writing.
I’ve seen a lot of posts on the pytorch forum but couldn’t find a solution to what I was wondering about.
Can 3 dataloaders be applied to DistributedDataParallel??
The process I think is as follows.
GPU1 GPU2 GPU3
model model model
dataloader1 dataloader2 dataloader3
I want to backpropagation by averaging the loss values of each dataloader.
Are there any possible methods or examples??
Each data loader has different image size and batch size, so concatDataset cannot be used.
If there is a way, please comment.
Thank you.