I am trying to apply distributed training in my project, but got a strange problem. It seems all data were loaded to one gpu, please see the figure below.
image802×843 104 KB
By the way, I am using pytorch 1.5, Ubuntu 18.04.