Since you shuffle your data (and hopefully your batch size is much smaller than your total dataset) I think that that’s okay.
I’d argue that a smaller batch is more problematic since your hyper parameters are tuned to your original batch size.