Regularisation Penalties/Norms I've been working with some penalty-based models. Most specifically, LASSO. But I've also been reading surrounding literature on ridge and elastic net.
Conceptually, I understand the nature of a penalty term and the implications. However, I wanted to know if someone could explain why they're also called 'norms,' and the subsequent relation with normed vector spaces etc?
Would be nice to just see a nice relatively non-technical explanation/example, if possible.

First of all, not all forms of regularization use norms. For example, dropout or early stopping do not.
As for norms, the term comes from mathematics. In $\ell_1$ and $\ell_2$ regularization, they are exactly the norms applied to the vectors of parameters. If a regular linear regression model is
$$
\underset{\beta }{\operatorname{arg\,min}} \;\| y - X\beta \|^2_2
$$
than ridge regression is
$$
\underset{\beta }{\operatorname{arg\,min}} \;\| y - X\beta \|^2_2 + \color{red}{\lambda \| \beta\|^2_2}
$$
so it is about applying the $\ell_2$ norm to the vector of parameters. Norm basically is

The norm of a mathematical object is a quantity that in some (possibly abstract) sense describes the length, size, or extent of the object. [...]

So by adding the "size" of the parameters to the total cost of the function that we minimize, we force the algorithm to find such a solution to the optimization problem where the cost is the smallest, hence where the "size" of the parameters is reduced as well, where by "size" we mean what we used as our norm.
If you ask about the name "norm" itself, the Mathematics Q&A site has two answers on this here and here, and the earliest uses of the term can be traced here. TL;DR it's about the standard units.
