I never tried what you are trying to do, maybe others can help you with how to do it.
Though I believe it’s theoretically possible but it would be way more complicated than just using DDP with normal training(if you want to do so the way you initiate the new process seems to be very important).
What I usually do specially in GAN training is that I define my data loader and then I’ll use a training_step function which basically receives a batch in each step. Then I can have separate functions for let’s say my discriminator_step and generator_step, or in some cases I might even have another module that needs to have a separate backward.