Frist of all, thankyou so much!
I have one more basic question.
If I use 4 devices and dim0 of my input data is 8, in this case, each GPU handle two process simultaneously or sequentially?
Why single process per  GPU is recommended?
+) the word ‘single process’ means that batch size of the data assigned to single GPU is 1?