Ok many thanks @iffiX for the detailed answer.
So essentially use DistributedDataParallel for neural network stuff (which involves things like forward and backward processes) that you want to parallelize, and use torch.multiprocessing for non-neural network stuff that you want to parallelize.