Machine Learning, Deep Learning, Artificial Intelligence, Regularization, Deep Learning Book.
reasons. Large penalties might cause non-convex optimization algorithms to get stuck in local minima due to small values of θ, leading to the formation of so-called dead cells, as the weights entering and leaving them are too small to have an impact. Constraints don’t enforce the weights to be near