Machine Learning, Deep Learning, Artificial Intelligence, Regularization, Deep Learning Book.
any hidden unit from having a large weight. The idea here is that if we restrict the Frobenius norm, it doesn’t guarantee that the individual weights would be small, just their norm. So, we might have large weights being compensated by extremely small weights to make the overall norm small.