Yes, that is correct. I am using a batchsize of 4 due to memory limitations.
Are you suggesting that a smoother loss curve will ultimately yield superior results?
I.e. bad gradients <–> noisy loss ?
If yes, how can I achieve this on my current setup without increasing the batchsize? I am already using ~11 GB of my GTX 1080 Ti…