One thing is strange for me that, the code gets all gpus engaged, because their memories are a little filled up, but not that much that could work!
When I train with one gpu I set batch_size=2 and for 3gpus I should set batch_size=6, but if I set that as 6, the code gives me gpu OOM error which shows that only one gpu is engaged with the data.
[I don’t know if that is related or not but I am using docker container to run the code! with` --gpus all` of course]
Here is the output of my smi:

image642×554 52.8 KB