Hey Frank, yep, the difference is intentional. ZeroRedundancyOptimizer in PyTorch is supposed to be used in conjunction with DDP. Since DDP already holds the full model replica, it will be more memory efficient for ZeroRedundancyOptimizer to directly using those parameters as broadcast buffers instead of creating new intra-parameter shards.