Hi,
sorry for my late response. I did not have the time to look into the algorithm properly to give a reason for the things I found but I found that by setting the tolerances to zero the algorithm stops when it no longer learns and be setting the tolerances to a negative number, the algorithm does not terminate although it no longer learns. The latter was clear from analyzing the running loss while the former I deduced from the latter.
The algorithm implements line search so I suppose this makes sense.