Thank you for your answer! But how does the optimizer know when the current gradient direction is not optimal (i.e. to change the hyper parameters like learning rate etc.) if it has no access to the loss value.