Online Improper Learning with an Approximation Oracle ⇤
We study the following question: given an efficient approximation algorithm for an optimization problem, can we learn efficiently in the same setting? We give a formal affirmative answer to this question in the form of a reduction from online learning to offline approximate optimization using an efficient algorithm that guarantees near optimal regret. The algorithm is efficient in terms of the number of oracle calls to a given approximation oracle -it makes only logarithmically many such calls per iteration. This resolves an open question by Kalai and Vempala, and by Garber. Furthermore, our result applies to the more general improper learning problems.
This paper mainly considered the online linear optimization problem with approximation oracle (as well as its bandit version), which had been studied for a long time. The problem is important in real world especially when we can only solve the underlying offline optimization problems approximately, such as max-cut etc. The main contribution of this paper is to propose several algorithms which are proved to be more efficient (in terms of oracle calls per round) than previous state-of-the-art algorithm while achieve nearly the same regret guarantees. In detail, authors improved previous O(\sqrt{T}) oracle complexity per round to O(log T) order (both achieve O(\sqrt{T}) regret) in full information setting, and improved previous O(T) oracle complexity per round to O(T^{2/3}*log T) order (both achieve O(T^{2/3}) regret) in bandit setting. The paper is well written and structured in a clean way. Some techniques look interesting, especially the projection and decomposition oracle.    Pros: 1. Two algorithms based on different frameworks (i.e. OMD and continual exponential weights) are proposed in full information setting. Both of them are more efficient than previous methods. OMD based algorithm looks more succinct while only holds for loss functions with PNIP property. Continual exponential weights based algorithm works for general loss functions.  2.  An algorithm is designed for bandit setting based on above OMD type algorithm with a novel regularization function, which is also more efficient than previous algorithms.   Cons: 1. A critical condition assumed in this paper is that the mean of any log-concave distribution can be exactly calculated, and all of the proposed algorithms heavily rely on calculating the gravity center of underlying constraint set. However, in practical applications, we need to approximate this gravity center, and it will cause some approximation error, which will demonstrate a trade-off between computational efficiency and final regret performance and it is exactly my main concern about this paper. According to the explanation in appendix D, this approximation error \epsilon is set as 1/T, then calculating the approximate gravity center will cost time poly(1/\epsilon) = poly(T) per round, which degenerates the computational performance of proposed algorithms. If we want to guarantee the efficiency of approximate mean calculation algorithm, then \epsilon should be set at least in 1/\log(T) order, then I am not sure whether this approximation error is enough to guarantee the final O(\sqrt{T}) regret performance; 2. Could authors explain why OMD based algorithm can achieve better efficiency than the algorithm proposed in Garber 2017 paper? I know his algorithm is based on Online Gradient Descent and ellipsoid method. Maybe the critical part is the projection and decomposition oracle constructed in this paper. It will be better if authors could provide some intuition about this oracle before stating four detailed lemmas.  Other minor comments: 1. In table 1, it will be better to add a notation about alg 6, as it only appears in the appendix; 2. In the equation under line 307 in appendix, it should be f(x) – f(y) <= \frac{1}{2\mu} …  Update: Thanks for authors' response. Though the main focus of this paper is oracle complexity, but I think computational complexity is also a critical part. Since authors mentioned using ellipsoid method could lead to a better computational complexity, which solved my main concern, I hope to see the corresponding part in the further version.