Hey @Vibhatha_Abeykoon
That _sync_params call is there for two purposes:

Intra rank/process parameter sync: this is only for the legacy single-process multi-device use case, where each process operates on multiple model replicas. And this is not a recommended way to use DDP.
Inter rank/process buffer sync: this does not sync parameters, and this will be skipped if your model does not have buffers (e.g., running_mean in BatchNorm layers).


What is the main expectation of the sync call?

For many use cases, that _sync_params will be a no-op.