Per the example here: examples/main.py at master · pytorch/examples · GitHub 3, the DDP model is created first and then it’s parameters are passed as an arg when creating the optimizer (like the first option shown).
A more specific answer would depend on what GeneratorOperation and DiscriminatorOperation do. If you want the functionality in these to be replicated across ranks and the corresponding gradients to be synchronized during the backward pass, then they should passed to DDP.