Hi,
What this post says is that to have even usage, you should put everything in the DataParallel block.
In your case, if you have some part of your net that is only on one GPU (which is gpu0 by default) then it is expected that gpu0 is more used no?