Solved by raghavendragaleppa in post #4 
              
              
                It’s simple. 
Say your GPU has 4 cores and each core is capable of processing 64 images in a sec. And all of the 4 cores work in parallel. 
So your GPU will take the same amount of time to process 64 images and 256 images. 
So to process 256 images, it will 1 sec if you give batch size as 256 but wi…