Hi guys,
I am wondering if anyone can clearly say difference between
the learning rate decay (Learning rate scheduling) and
Weight decay (L2 regularization)?
I know the learning rate decay is for updating weights slowly or quickly and
the weight decay is to give loss function a penalty for avoid overfitting.

image761×67 2.22 KB

like the photo I’ve attached,
But is it right that those two parameters would affect on updating the weights at the same time?
Then, is it preferable to use lr schedule and weight decay at the same time?
Thanks in advance!!