A visual explanation for regularization of linear models
> The simple reason is that that illustration shows how we regularize models conceptually, with hard constraints, not how we actually implement regularization, with soft constraints!Note that these are equivalent. In particular, assuming the two problems are (for some loss "l" and regularizer "r")    minimize  l(θ) + λr(θ),

and    minimize    l(θ)
    subject to  r(θ) ≤ M,

for every λ there exists an M and for every M there exists a λ such that the resulting problems are equivalent, in the sense that a solution for one is a solution for the other. (Of course, under some fairly general regularity conditions, but these hold for all given examples). I agree that this is not often stated in many introductory texts, but the intuitive image is the same.