Wow very interesting!
Just to confirm, this edge case you are talking about would be in favor of the 2nd option right? (i.e. use all model parameters so you have the ability to freeze them during training, but risky due to weight decay)
Thanks for the reply