I was playing around with the MNIST example, and noticed the doing TOO big batches (10,000 images per batch) seem to be hurting accuracy scores.
This got me thinking : What are the general rules of thumb you guys have found when it comes to batch sizes?
Also, what is your favorite optimizer and activation function?
Thanks!