can you please try it with SGD and see if you get the same overfitting? plus can you remove the scheduler and the learning rate update you have used?
1 - I would loose all learning rate scheduling and see if there is a difference between both optimizers with a few different parameters
2 - once you know which one is best in this case add back the LR.