It’s a bit unclear what setup you are using so some guesses.
Are you using nn.DataParallel or DistributedDataParallel? In the latter case, are you using SyncBatchNorm layers? Is the batch size the same on each device using the single GPU run vs. the multi-GPU one? If so, this would mean that you are basically scaling up the global batch size by 4x and might not expect to see the same results.