Hello, I wanted to understand how to implement kernel regularizer(parameter in Keras/Tensorflow layer) in a layer in PyTorch. I saw examples of how to implement regularizer for overall loss, but could not find the relevant documentation for this.
Could someone point me in the right direction? I saw something along the lines like what I intend to do here: “How does one implement Weight regularization (l1 or l2) manually without optimum? 22”.
I have adapted it as follows:
l2_reg = None

for i in model.named_parameters():
   if "layer_name.weight" in i:
      if l2_reg is None:
         l2_reg = i.norm(2)**2
      else:
        l2_reg = l2_reg + i.norm(2)**2
batch_loss = some_loss_function + l2_reg * reg_lambda
batch_loss.backward()

Is this the correct way to do so?
Thanks in advance!