The code looks basically alright and the per-parameter option should work, too.
However, I’m not sure, if it’s the best idea to just add another custom classifier on top on the pre-trained model. Usually you would remove the last linear layer and replace it with a new one.
PS: I’m not a big fan of tagging certain people as this might demotivate others to write an answer.