albanD:

You can create one optimizer for each set of paremeters and .step() all the ones that need to be updated at that iterations.


I know your means.
However,
the number of optimizers will increase when the set of paremeters is too large, which will cause some complexity.