AFAIK Lecunn postion was that we should use minibatch of 1.
In VGG-type networks I used minibatch of 4 or 8 and result were better and obtained faster.
The reason why it could be not a choice was GPU parrallel arch, when minibatch of 1 is not computionally optimal.
But that Yann’s position was before BN paper was issed. And BN limits our minibatch size, it couldn’t be 1 anymore.
So what is the minimal minibatch size which BN-stable?
How network  behaves when minibatch became BN-unstable (when minibatch<BN_stable_size)?
Does anyone investigate this?