On Explore-Then-Commit Strategies
We study the problem of minimising regret in two-armed bandit problems with Gaussian rewards. Our objective is to use this simple setting to illustrate that strategies based on an exploration phase (up to a stopping time) followed by exploitation are necessarily suboptimal. The results hold regardless of whether or not the difference in means between the two arms is known. Besides the main message, we also refine existing deviation inequalities, which allow us to design fully sequential strategies with finite-time regret guarantees that are (a) asymptotically optimal as the horizon grows and (b) order-optimal in the minimax sense. Furthermore we provide empirical evidence that the theory also holds in practice and discuss extensions to non-gaussian and multiple-armed case.
This paper aims at providing theoretical evidence to support the fact that Explore-Then-Commit strategies are necessarily sub-optimal in terms of regret in a multi-armed bandit setting. In this regard the paper considers a simple two armed bandit problem with Gaussian rewards with different means but same variance. In this setting, the authors show lower bounds for ETC strategies. This shows that ETC strategies are necessarily sub-optimal compared to some strategies that follow fully sequential exploration. The paper also shows that a strategy similar to UCB* is asymptotically optimal and order optimal in the minimax sense at the same time. 
1. One interesting part of the paper to me is the strategy in Algorithm 5 that is asymptotically optimal and optimal in the minimax sense. Although the strategy is not very original (a small modification of a strategy suggested by Lai), it is a new observation for a strategy to be asymptotically optimal and minimax optimal. However, it should be noted that it is optimal only in the case of two arms, which is a major drawback. The authors are encouraged to improve on this result.   2. I like the idea of using the hypothesis test result of Wald to come up with the ETC strategy in Algorithm 2 that achieves the lower bound for ETC strategy. This idea is new to the best of my knowledge. However, again it is challenging to extend this to multi-armed case.   3. I feel that Theorem 5 is rather straight forward to see from the results of the optimal BAI algorithm and there is not a lot of novelty in this result.   4. Although presented as the main result of this paper, I do not see very novel ideas in the proof of the lower bounds for these strategies. Hypothesis testing inequalities stemming from a particular inequality in Tsybakov's book " Introduction to Nonparametric Estimation" , has been used before in the bandit literature for proving lower bounds ". In fact the main step in the proof is Garivier's inequality. The parts of the proof that follows is fairly straight forward. Moreover the results have been presented only for the two arm setting.  5. Overall I like the flavor of the results presented in the paper. I like the Algorithm 5 and corresponding proof, as to the best of my knowledge this is original in the literature. The lower bounds for ETC are new results but may seem straight forward. The major drawback of the paper is the simple two arm setting and that there is not much practical significance of the paper. I would encourage the authors to generalize these results in future.   6. The presentation of the paper is very clear.