Science the device whose index is 0 will hold the network parameters, how to split the training batches differently to maximize the usage of GPUs? In my case, the GPUs except the device[0] has low GPU-Util.
How to fix this problem? Thank you.