Deep Learning Games
We investigate a reduction of supervised learning to game playing that reveals new connections and learning methods. For convex one-layer problems, we demonstrate an equivalence between global minimizers of the training problem and Nash equilibria in a simple game. We then show how the game can be extended to general acyclic neural networks with differentiable convex gates, establishing a bijection between the Nash equilibria and critical (or KKT) points of the deep learning problem. Based on these connections we investigate alternative learning methods, and find that regret matching can achieve competitive training performance while producing sparser models than current deep learning strategies.
The paper reformulates learning in NNs as a game, and demonstrates a correspondence between Nash equilibria and KKT points.  I enjoyed reading the paper and am extremely sympathetic to its aims.  However, I have concerns about: (i) whether the zanni's play a significant role; (ii) whether the experiments evaluate anything meaningful, and (iii) whether the results apply to the neural nets that people actually use.   If these are suitably addressed, I will modify my scores accordingly. Detailed comments follow.  
Detailed comments  1. Line 36 introduces h(x) = \phi(\theta x) where \phi is a transfer function, which I guess is intended to model the nonlinear activations in the output layer of the (single-layer) NN. However, neither h nor \phi show up in the remainder of the section. Incorporating \phi breaks Theorems 1 and 2 since the loss \ell(\phi(\theta x), y) will not be a convex function of \theta for most choices of function \phi. I guess the authors meant to say “One linear layer learning games”?  2. Generality of results Regarding Theorems 4 and 5, the ancillary lemmas in the appendix assume that the activation functions f_v are convex and differentiable. This means that the most commonly used functions in practice (ReLU, maxout, max-pooling, sigmoid, and tanh units) are all ruled out along with many others.  The results in the paper therefore apply in much less generality than a cursory reading would suggest. The authors should clarify which NNs the results apply to, preferably in the introduction.   More broadly, can the results be extended to the neural nets used by practitioners? If not, then the title "Deep learning [x]" is a little misleading.  3. Constraints Minor point: The L1 constraint is disconnected from the rest of the paper. Meaning, the reasons for *specifically* choosing L1 are unclear to me. One could just as easily have chosen, say, L2 and used different OCO algorithms than EWA. Why highlight this particular constraint? Sparsity is mentioned a few times, but sparsity is conceptually orthogonal to the paper’s main contribution.  4. Zannis Line 225 states “the usual process of backpropagating the sampled (sub)gradients [computes] the best response actions for the zannis and the antagonist and the resulting affine utility for the protagonists.” This raises two major questions.  First question: Is it necessary to introduce zanni’s into the game in the first place? Since their best responses are given by backpropagation, one could directly construct the loss/utility for each protagonist by suitably backpropagating the loss specified by the antagonist — this recovers the path-sum gradient game introduced by Balduzzi in [2].  Continuing, line 210 states “we investigated […] independent protagonist agents at each vertex against a best response antagonist and best response zannis”. So the experiments don’t use the additional degrees of freedom introduced into the setup by introducing zannis as agents in their own right. Instead, the authors presumably just plug in backprop for the zannis. So, again, do the zannis add anything real to the setup?  I’m actually curious how the games run when all players including zannis are “alive” and update their responses using one of Algorithms 2-4. For example, does the game still converge to the same or comparable local optima to when the zanni best responses are computed via backprop? How much does convergence slow down?  Second question: What do the experiments evaluate? Line 229 states “to investigate the plausibility of applying independent expert algorithms at each vertex [of] deep learning models”. But applying SGD, RMSProp, Adam etc to backpropagated errors is *exactly* how NNs are trained. Since the zanni’s best responses are backpropagated errors, it seems the only difference between standard NN training and the experiments in the paper is that the authors impose constraints on the weights. However, the effect of the constraints (for example, changing the set's size) is not investigated. And of course the effect of constraints isn’t really what the paper is about. So what do the experiments tell the reader?  Please clarify this point. Otherwise, aside from the RM result, the experiments come down to running standard optimizers on an NN on MNIST and observing that they work.  As an aside, it is very interesting that Regret Matching (RM) is competitive with RMSProp et al in the experiments and yields sparser solutions. However, this needs to be investigated more thoroughly before it can be considered a robust result. It is also largely orthogonal to the main concerns of the paper.  Minor point: Is zanni standard terminology? I’ve never encountered it before and find it quite unpleasant. Please consider alternatives.