Hello! Yes, the memory required in this example will double when using DDP. This is because the world_size is 2 and the purpose of DDP is for data parallelism (same model, multiple data) where the model architecture is copied across multiple machines or GPUs to handle data in parallel.
In the case of a large-scale model that does not fit on a single machine or GPU, you should look into Distributed RPC 1. RPC allows for model parallelism (split model, same data) where you can split the model across machines and use RPC to communicate between the different layers. The RPC framework also handles autograd and optimizer steps internally.