Ah in that case can’t you just configure warmup_steps and weight_decay directly in the TrainingArguments?
You can also change the scheduler type in case that’s what you’re after: transformers.trainer_utils — transformers 4.3.0 documentation 44
Finally, you can also implement custom callbacks in transformers - see here: Callbacks — transformers 4.3.0 documentation 11