Hello everyone!
I want to add as regularization term the Lipschitz constant of a neural network which I compute after each weight update. If I just modify the loss function in the training function as : loss += Lipschitz_constant , I think that this won’t affect the training as I just added a number to the loss.
How can I add the Lipschitz constant (which of course is a function with respect to the weights W) to loss function as a function of weights and not just as a number (so the optimizer does not ignore the Lipschitz constant when optimizing).
Thanks a lot.