I was wondering why is it not advised to use multiple GPUs using muliprocesing? As an example, http://pytorch.org/docs/master/notes/cuda.html 66 towards the end you have advise “Use nn.DataParallel instead of multiprocessing”
While there is an example to use multiple GPUs using multiprocessing http://pytorch.org/tutorials/intermediate/dist_tuto.html 129 .
I am confused as to which is the correct rule? That seems contradictory?