albanD:

So I would guess that the kernel for larger batch size are more optimized than the ones for small batch size. And this is why you see this.


Yeah, though I think it’s the other way around when trying to backprop sample-by-sample.  Maybe it’s possible to somehow fix the algorithm to the version for small batch sizes.