yes I know! I’m just suggesting, since single-node multi-device training is the entry point to dist training, and will likely be enough for most users, why having it done with DDP (which is so insanely complex to get working), while torch.nn.DataParallel API is so compact and friendly. I know that DataParallel in its current form performs badly, but looking at things from the customer perspective, this is the experience the Pytorch team should aim for for DDP future. No torchrun, no MPI, no processes to spawn manually etc. It’s better to have a single-code, single process experience, like MXNet Gluon offers.. Again, I’m not saying it’s possible today in Pytorch, I’m just saying this is what you should aim for, so that the data-parallelism experience is better. Think about how easy it is to write a single-file PySpark data processing code and run it over multiple machines. Why things aren’t this simpler in the DL training world yet? We’re clearly lagging behind the big data world in terms of abstracting complexity