Fundamental Limits of Online and Distributed Algorithms for Statistical Learning and Estimation
Many machine learning approaches are characterized by information constraints on how they interact with the training data. These include memory and sequential access constraints (e.g. fast first-order methods to solve stochastic optimization problems); communication constraints (e.g. distributed learning); partial access to the underlying data (e.g. missing features and multi-armed bandits) and more. However, currently we have little understanding how such information constraints fundamentally affect our performance, independent of the learning problem semantics. For example, are there learning problems where any algorithm which has small memory footprint (or can use any bounded number of bits from each example, or has certain communication constraints) will perform worse than what is possible without such constraints? In this paper, we describe how a single set of results implies positive answers to the above, for several different settings.
The paper considers the effect of memory constraints on some classes of online and distributed algorithms. The main goal of the authors is to show that there exist learning problems where imposing a memory constraint provably hurts the performance of any algorithm, in the sense that the best achievable performance guarantees are worse than some known upper bound on an algorithm that operates without memory constraints. The authors also provide a lower bound on the regret of online algorithms that operate under a specific feedback constraint. The results of the paper follow from an elegant information-theoretic argument concerning "hide-and-seek problems" where a learner has to detect a biased coordinate of i.i.d. random vectors by sequentially sampling a bounded number of coordinates.The paper is written very well and the technical quality is excellent. To my best knowledge, the main results of the paper (Theorems 2 and 3) are novel, and so are their implications in the considered learning settings. My main problem is that these settings are not general enough to draw significant conclusions as done by the authors. Generally, I find that the presented results are a bit oversold, and the paper would benefit from a more modest presentation. I list my main observations considering the contributions below.First, the lower bounds on the regret achievable by memory-bounded online learning algorithms seem to directly follow from the results of Seldin, Bartlett, Crammer and Abbasi-Yadkori (ICML 2014), whose counterexample is identical to the one presented in this paper. While this alone would not be a problem, it seems that the bound presented in Theorem 4 is only meaningful when the adversary is restricted to play binary losses (or at least choose from a finite pool of loss vectors). Indeed, allowing the learner to observe \Omega(d\sqrt{T}) bits of the loss of a *fixed* arm in each round is clearly not enough for any algorithm to achieve consistency, yet the lower bound of Theorem 4 fails to capture this intuition as it only guarantees a regret of \Omega(1). Theorem 4 is also clearly loose in finite-outcome partial monitoring games with minimax regret of \Theta(T^{2/3}). That is, the claim that Theorem 4 "quantifies in a very direct way the cost of information constraints for such learning problems" (line 303) is quite misleading.The examples considered in Sections 4.2 and 4.3 theoretically support the intuition that some learning problems are difficult to solve if the learner is forced to work with a memory budget that is significantly smaller than the natural representation of the problem instance to learn. Although it is nice to see such a result proven formally, this is hardly surprising: The same kind of information vs. performance trade-off can be observed when comparing the minimax performance guarantees concerning multi-armed-bandit and online prediction problems. While this contribution is somewhat stronger than the results of Section 4.1, I don't feel that the lesson is as fundamental as the authors claim.Detailed comments-----------------159: "Let X^t be a batch of i.i.d. instances" -- instances of precisely what? 172,173: correspond -> corresponds311: It would be interesting to contrast these results to those of Van Erven, Kotlowski and Warmuth (COLT, 2014), who propose a learning algorithm based on dropping out elements of each loss vector with an arbitrary fixed probability. When setting the dropout probability sufficiently high, one can guarantee that at most b bits are retained in each round with high probability, yet their algorithm succeeds in obtaining regret guarantees of O(sqrt{T log d}). I wonder if this contradicts the results of Theorem 4...?384: 1 - exp(-\Omega(...)) -> \Omega(1-exp(-...))407: While any b-memory protocol is indeed a (b,k,[m/k]) protocol, the converse is not true, thus this reduction doesn't seem to work.678, 825: x_j is missing from the conditions of Pr_0(.|.)