Well, you’re right - BUT, I don’t train my CNNs…
I use pretrained ones… so basically I only use feedforward, without gradient descent…
So once I use batch of size 1 - I get MUCH longer runtime and MUCH lower accuracy on test set, in contrast to batch of size 32/128/256.
I want to emphasize again - I don’t train, I only use pre-trained CNNs.