Hi all,
I’m using a trained LinkNet34 model. I was expecting that doing batch inference of size n  would be faster than doing n times one single inference. Although, most of the time the batch computing shows only a slight advantage over n times single inference.
Am I missing something?

I’m only using CPU. I have applied model.eval() before. I also tried model.foward method, and with torch.no_grad() context. They didn’t affect the results.
Thanks in advance,