Hi. Were you able to resolve this?
I am also using DataParallel to use multiple GPUs in Pytorch script but face a similar error with batch size 64 and 4 GPUs:
ValueError: Expected input batch_size (256) to match target batch_size (64)