If my model original batch-size is 32, when I use two gpus ,one node per gpu, I use batch-size 16(32/ngpu), but if the number of gpus is 3 or any odd number, we should keep the size like 32/3=10 or limit the number of gpus to 2?
Any help is welcome.