Weight Decay


regularized loss is equivalent to shrinking/decaying θ  by a scalar factor of $1 - \mu \lambda$ and then apply standard gradient
and that coefficient is decaying weight
when [L2 Norm](https://texonom.com/l2-norm-38c15917350a4c82a11003474ac7d280) 
$$L_{reg} = \lambda\frac{1}{2}||w||_2^2$$








