No, I don’t think you are seeing any overfitting here.
A model is overfitting, if the training loss is decreasing, while the validation loss is staying flat or increasing.



 aarrvv:

Can this insight be of any use when working on real but very noisy data?


Yes, as the batch size might also be considered as a hyperparameter. Also, you could e.g. increase the batch size later in training to smooth the gradient etc.