Regarding your suggestion about L2 component, shouldn’t I divide it by the batch_size instead of the number of L2 components?
EDIT: in order to copy the exact behaviour of Keras kernel_regularizer, I think that L2 regularization component should not be divided by any value at all