Thank you for your answer. The documentation does not include a working example for DDP. I have already tried many ways using map function None of which have worked so far. If you could show me a simple working example with mnist dataset to map 8 gpus to 1 or 2 or 4 gpus, or cpu with DistributedData parallel I would greatly appreciate it.