Thanks for your reply. The device count is 10. I also tried what you suggested and that didn’t work. @ptrblck was right, silly me, I was using a batchsize of 16. I increased it to 20 and it uses all the GPUs. Thanks guys