oh, sorry, I misread the question. I thought you mean DDP uses a little more memory than local model.

you mean it is expected that 1 of the GPUs in say a 2 GPU (single-process-single-gpu) DDP setup will use more memory because of bucketing?

no, they should be the same I think