Using multiple batches is actually the best you can get. You can also use multiple threads but make sure to use locks when using model (to keep memory requirement same). Using multithread will make sure GPU is being used even when other threads are loading image or doing pre/post processing.