Machine Learning, Deep Learning, Artificial Intelligence, Regularization, Deep Learning Book.
there is exactly quadratic, given by: 1.2 L¹ parameter regularization Here, the parameter norm penalty is given by: Ω(θ) = ||w||¹ This makes the gradient of the overall objective function: Now, the last term, sign(w), creates some difficulty as the gradient no longer scales linearly with w. This