Learning with Bandit Feedback in Potential Games
This paper examines the equilibrium convergence properties of no-regret learning with exponential weights in potential games. To establish convergence with minimal information requirements on the players' side, we focus on two frameworks: the semi-bandit case (where players have access to a noisy estimate of their payoff vectors, including strategies they did not play), and the bandit case (where players are only able to observe their in-game, realized payoffs). In the semi-bandit case, we show that the induced sequence of play converges almost surely to a Nash equilibrium at a quasi-exponential rate. In the bandit case, the same result holds for ε-approximations of Nash equilibria if we introduce an exploration factor ε > 0 that guarantees that action choice probabilities never fall below ε. In particular, if the algorithm is run with a suitably decreasing exploration factor, the sequence of play converges to a bona fide Nash equilibrium with probability 1.
The authors show that exponential weight updates and its generalization FTRL, converges pointwise to a Nash equilibrium in potential games. The latter holds even when players receive bandit feedback of the game. 
Unfortunately, the main result, for the case of full feedback is already known in the economics literature, where Hedge is known as smooth fictitious play. It is presented in the paper: Hofbauer, Sandholm, "On the global convergence of smooth fictitious play", Econometrica 2002. This work is not even cited in the paper and it shows that smooth fictitious play and its extensions of FTRL (though they do not call it FTRL) does converge to Nash in potential games and in three other classes of games. 
Moreover, the result of Kleinberg, Piliouras, Tardos:"Multiplicative Updates Outperform Generic No-Regret Learning in Congestion Games", STOC'09, is presented in the paper, for the actual sequence of play (unlike what the authors claim). So that paper shows that for the class of congestion games (which is equivalent to the class of all potential games (c.f. Monderer-Shapley)), the actual play of MWU converges to Nash equilibria and in fact almost surely to Pure Nash equilibria (except for measure zero game instances), which is an even stronger result. 
In light of the above two omissions, the paper needs to be re-positioned to argue that the extension to bandit feedback is not an easy extension of the above results. It is an interesting extension and I can potentially see that extension be presented at a NIPS quality conference, but it needs to be positioned correctly with the literature and the paper needs to argue as to why it does not easily follow from existing results. In particular, given that most of these results go through stochastic approximations, such stochastic approximations are most probably robust to unbiased estimates of the payoffs, which is all that is needed for the bandit extension.