Question: I am about to train a big LSTM network with 2-3 million articles and am struggling with Memory Errors (I use AWS EC2 g2x2large).

I found out that one solution is to reduce the `batch_size`. However, I am not sure if this parameter is only related to memory efficiency issues or if it will effect my results. As a matter of fact, I also noticed that `batch_size` used in examples is usually as a power of two, which I don't understand either.

I don't mind if my network takes longer to train, but I would like to know if reducing the `batch_size` will decrease the quality of my predictions.

Thanks.

Answer: 
Oddly enough, I found that larger batch sizes with keras require more epochs to converge.

For example, the output of [this script](https://github.com/shadiakiki1986/test-ml/blob/master/t4-LSTM%20stateful/test_temporal_regression.py) based on keras' [integration test](https://github.com/fchollet/keras/blob/master/tests/integration_tests/test_temporal_data_tasks.py#L78) is

```
epochs 15   , batch size 16   , layer type Dense: final loss 0.56, seconds 1.46
epochs 15   , batch size 160  , layer type Dense: final loss 1.27, seconds 0.30
epochs 150  , batch size 160  , layer type Dense: final loss 0.55, seconds 1.74

```

Related

* Keras [issue 4708](https://github.com/fchollet/keras/issues/4708): the user turned out to be using `BatchNormalization`, which affected the results.
* [This tutorial](https://machinelearningmastery.com/tune-lstm-hyperparameters-keras-time-series-forecasting/) on LSTM, section **Tuning the Batch Size**
* search results for [keras lstm batch size effect on result](https://duckduckgo.com/?q=keras+lstm+batch+size+effect+on+result&t=canonical&ia=web)
* [My Neural Network isn't working! What should I do?](http://theorangeduck.com/page/neural-network-not-working?imm_mid=0f6562&cmp=em-data-na-na-newsltr_20170920), point 5 (You Used a too Large Batch Size) discusses exactly this

> 
> Using too large a batch size can have a negative effect on the
>  accuracy of your network during training since it reduces the
>  stochasticity of the gradient descent.
> 
> 
> 

Edit: most of the times, increasing `batch_size` is desired to speed up computation, but there are other simpler ways to do this, like using data types of a smaller footprint via the `dtype` argument, whether in *keras* or *tensorflow*, e.g. `float32` instead of `float64`
A couple of papers have been published showing – and conventional wisdom in 2020 seems to still be persuaded— that, as [Yann LeCun](https://twitter.com/ylecun/status/989610208497360896) put, large batches are bad for your health.

Two relevant papers are

* [Revisiting Small Batch Training For Deep Neural Networks, Dominic Masters and Carlo Luschi](https://arxiv.org/abs/1804.07612) which implies that anything over 32 may degrade training in SGD.

and

* [On Large-batch Training For Deep Learning: Generalization Gap And Sharp Minima](https://arxiv.org/abs/1609.04836)

which offers possible reasons. To paraphrase badly, big batches are likely to get stuck in local (“sharp”) minima, small batches not. There is some interplay with choice of learning rate.