From my experiments, it appears that for DDP using CPU processes, there is no splitting of data across the batch dimension across processes.
In the source code as well, if the model’s device_ids is None, then scattering is not performed in the forward() pass of the model.
Can someone more authoritative confirm this behavior?