I have a very nested model with many Prelu. It is not possible to specify all the parameters in the optimizer as a default and respecify all the Prelu as the specific condition.
I want to apply zero weight decay to all Prelu parameters and keep all the rest default.