Hi,



 amitoz:

The downside of above is one has to increase the number of epochs.


Main benefit of batching is that when dataset is too big, you cannot compute backward and forward due to different hardware or math issues, so every batch works as a representation of your whole data so that is why you see more fluctuation in your loss.
About third scenario, you can see it as a case with batch_size=1. An intuitive idea is that you still have disadvantage of increasing epochs. Another issue is that 1 sample cannot represent enough info for training so probably your model cannot fit as by viewing a new sample after 100 epochs , the learned features about first epoch may have been vanished.
Actually, this situation needs to be examined mathematically, as it can be expressed by state of optimizer, the concept of loss function and finding optima.
Bests