Correct, but in essence the model is being penalized for larger weights much more frequently if the batch size is small. Over one epoch, a model with BS=16 might get penalized 32 times, whereas BS=128 would make that only 4 times. I get that the loss considers mean and we are doing batch-gradient descent after all, but should there not be some sort of scaling for the regularization as well?