I have met a similar problem.And I am learning how to use ‘DistributedDataParallel’.And it’s really strange that why there are extre processes per GPU that are consuming 0 GPU ram.
To reproduce:distributed_tutorial/mnist.py at master · yangkky/distributed_tutorial · GitHub
I follow the code completely and only had three modifications.
` parser.add_argument(’-g’, ‘–gpus’, default=3, type=int,
help=‘number of gpus per node’)
os.environ[‘MASTER_ADDR’] = ‘127.0.0.1’
os.environ[‘MASTER_PORT’] = ‘23556’`
And my Command line argument was ‘python mnist.py -n 1 -g 3 -nr 0’
PID 6283 6284 6285 are my processes.And GPU 3 is being used by others
my dependencies:  pytorch == 1.6.0    torchvision == 0.7.0  CUDA 10.2           3 x RTX 2080Ti

gpu222620×594 7.77 KB