How to convert a single-GPU PyTorch script to a multi-GPU multi-node PyTorch script with DDP?
I read this 8 10 times already but honestly it’s not really helpful ; what I need is a place that lists the modification needed to convert a single-GPU code to a multi-node multi-GPU code.
Is there a place in the doc that explains how to distribute a PyTorch training script over multiple machines?