Machine Learning, Software Development, Artificial Intelligence, Technology, Data Science.
sequences of step sizes gets them faster convergence as the sequence length increases. A common pattern in their sequences is alternation between two small-ish step sizes, with an occasional huge step that’s roughly proportional to the sequence length. I don’t know if this pattern is fundamental or