I am pre-training RobertaMaskedForLM on my own custom dataset. I wanted to implement the layer-wise learning rate decay given in https://github.com/aws-health-ai/multi_domain_lm#learning-rate-control 12 corresponding to the paper - An Empirical Investigation Towards Efficient Multi-Domain Language
Model Pre-training 12. Is there an easy way to incorporate this decay of learning rate with layer depth towards input using transformers.Trainer?