Machine Learning, Neural Networks, Statistics, Data Science, AI.
that the effect of the regularization term on the loss function corresponds to roughly a single training example. This ensures that the de facto effect of regularization doesn’t explode as the amount of data increases — which might explain why this scaling factor started to show up specifically