The parameters might overlap, as you are getting all parameters in param_optimizer, while also using model.roberta.parameters() and model.last_linear.parameters().
You could create dicts for all your conditions and parameter sets and check the keys for duplicates.