Thanks for your reply, I appreciate it.
For future readers, let me just explain a bit better the intuition of the paper:
Imagine running in parallel multiple optimization algorithms, each one with a different learning rate. Now, put on top of them a master algorithm able to decide which one is giving the best performance. This would give rise to a perfect optimization algorithm, at least for convex functions. And, indeed, this is exactly the MetaGrad algorithm from NIPS’16 (https://arxiv.org/abs/1604.08740 7).
Now, image instead having an infinite number of learners in parallel and the algorithm on top doing an expectation of all of them with respect to the best distribution of learning rates. The infinity makes things easier because now the update becomes in a simple closed form. And this is exactly how my algorithm works.
The coin-betting trick is a just an easy way to design this kind of algorithms.