Solved by H-Huang in post #2 
              
              
                Hello! Yes, the memory required in this example will double when using DDP. This is because the world_size is 2 and the purpose of DDP is for data parallelism (same model, multiple data) where the model architecture is copied across multiple machines or GPUs to handle data in parallel. 
In the case …