Nbooo:

As I understood, in Pytorch we need to set weight_decay parameter in our optimizer to achieve the same result, but I’m far from being confident here.


The weight_decay might be too aggressive in PyTorch, as it’ll add all parameters to the regularization term.
Have a look at this post 6 to exclude batch norm params etc. (or just add the conv parameters to the regularization).