This stuff all worked great - thanks for the post!
I will point out something: If you want to use this with a scheduler (e.g. CyclicLR), then the lr’s of the parameter groups should also be passed to the scheduler constructor as a list of floats. Otherwise, the group dependent lr’s are lost.
Edit: note also that, for example, base_parms will not be a list of torch tensors (which it needs to be). It will be a list of tuples and one must use only the second component of each tuple.