Generic question about batch sizes
I was playing around with the MNIST example, and noticed the doing TOO big batches (10,000 images per batch) seem to be hurting accuracy scores. This got me thinking : What are the general rules of thumb you guys have found when it comes to batch sizes? Also, what is your favorite optimizer and activation function? Thanks!
FuriouslyCurious
The batch size is usually set between 64 and 256.
vabh
Thank you for the papers!  I was trying to minimize noise through massive batches.  The papers you provided are a great explanation of why massive batches may not work. Thanks!
FuriouslyCurious