Sorry for question here.
It is said that when regularization L2, it should only for weight parameters, but not bias parameters.(Is it right?)
But the L2 regularization included in most optimizers in PyTorch, is for all of the parameters in the model (weight and bias).

I mean the parameters in the red box should be weight parameters only. (If what I heard of is right.)
How can I deal with it?
weight_p, bias_p = [],[]
for name, p in model.named_parameters():
  if 'bias' in name:
    bias_p += [p]
  else:
    weight_p += [p]

optim.SGD(
  [
    {'params': weight_p, 'weight_decay':1e -5},
    {'params': bias_p, 'weight_decay':0}
  ],
  lr=1e-2, momentum=0.9
)

Code here can deal with the problem above, is it right?