Hi guys, when I’m training with my model with one gpu I’m getting x% performance (accuracy).
Now I’m trying to make my model a bit bigger, I don’t have enough memory so I used 2 GPUs with dataparallel of course.
My result wasn’t good, so I took again my small model and ran it on 2 GPUs with dataparallel, the same model that got on 1 GPU x% is now getting (x-3)%, and tried to run multiple times.
Does anyone have an idea?
Notice I’m talking about DataParallel and not DistributedDataParallel
Thanks!