DataParallel attempts to replicate single GPU training on multiple GPUs. So in your setup, your batch size of 128 using the DataParallel module is as if you used a batch size of 128 in a single use setting. If your model is sensitive to batch size then this is what you’re observing.
Try a batch size of 32.