How AI Training Scales
So this blog post posits that there is a growing upper bound to the effectiveness batch size. Is there a similar analysis on checking whether there is also a growing lower bound? Or does batchsize=1 training work for all these tasks?