Well, it’s true that Adam is able to adjust the learning rate based on the estimated moments. However, if the initial learning rate is to high, it can diverge quickly.
By monitoring the training losses you can understand if it is diverging due to high learning rate or not.