Tight Bounds for Collaborative PAC Learning via Multiplicative Weights *
We study the collaborative PAC learning problem recently proposed in Blum et al. [3], in which we have k players and they want to learn a target function collaboratively, such that the learned function approximates the target function well on all players' distributions simultaneously. The quality of the collaborative learning algorithm is measured by the ratio between the sample complexity of the algorithm and that of the learning algorithm for a single distribution (called the overhead). We obtain a collaborative learning algorithm with overhead O(ln k), improving the one with overhead O(ln 2 k) in [3]. We also show that an Ω(ln k) overhead is inevitable when k is polynomial bounded by the VC dimension of the hypothesis class. Finally, our experimental study has demonstrated the superiority of our algorithm compared with the one in Blum et al.
[3] on real-world datasets.

Update after rebuttal:  As mentioned in my review, the authors should clarify that using the multiplicative weight scheme was suggested by Blum et al. and its overhead was discussed (without proof) to be the same as one shown in Theorem 3 of this paper---overhead log^2(k). For a reference, see the last paragraph of section 3 in http://papers.nips.cc/paper/6833-collaborative-pac-learning.pdf. The technical insight of this paper is, in addition to using multiplicative weight update, to change the "Accuracy Test" to the "Weak Test" to further reduce the samples complexity down to the overhead of log(k). The authors in their rebuttal have ignored this fact. So it's best if the camera-ready version reflects this fact accurately.  Before rebuttal:  This paper continues the study of collaboration in the context of machine learning and obtains improved same complexity bounds.   Background: The collaborative PAC model, introduced by Blum et al (NIPS’17), considers a setting where k players with different distributions D_i's that are all consistent with some unknown function f^* want to (\epsilon, \delta)-learn a classifier for their own distribution. The question Blum et al. asks is what is the total “overhead” over the sample complexity of accomplishing one task, if the players can collaborate. As an example when players do not collaborate, the k tasks have to be performed individually leading to an overhead of O(k). Blum et al. shows that in the personalized setting, where different players can use different classifiers, the overhead is O(log(k)) with k = O(d) . They also considered a centralized setting where all players should use the same functions. In this case, they showed that overhead is O(log(k)^2) when k = O(d). They also showed a matching lower bound of Omega(log(k)) overhead for the case of k=d.  Summary: This paper obtains an improved overhead for the centralized setting. Their main result, appearing in section 3, is that one can get an overhead of O(log(k)) in the centralized setting. More precisely, the sample complexity they obtain is: O(\epsilon^-1 (d+k)(ln(k) + ln(1/delta)). There second contribution is to extend the lower bound of Blum et al. to the more general case of k = d^O(1). Lastly, they do a set of experiments comparing their algorithm to a naive algorithm and the Blum et al. algorithm on a number of datasets and show that their algorithms have better empirical performance.  Overall, I think this is a nice paper. The problem and the setting it studies is very natural. I think the subject matter can be of interest to the broader community and can be related to other areas including, federated and distributed learning, multi-task learning, and transfer learning. I appreciate that the authors have complemented their theoretical results with the experimental findings.   There are a few weaker points in the paper that I explain below.  1. The Basic algorithm: The authors spend quite a bit of space on the basic algorithm that has a log^2(k) overhead in the Centralized setting. My first concern about this is that Blum et al. discussed this algorithm briefly in their paper (see last paragraph section 3 of the Blum et al.) and the fact that it has log^2(k) overhead without going into the details. The reason is that the proof of this algorithm is too similar to the original approach of Blum et al. So, I don’t think the authors should focus so much of their efforts and presentation the paper on this basic algorithm. Secondly, their proof approach appears to be incremental over Blum et al approach. In my opinion the authors should simply state these results without proof (defer the proof to the appendix) and also mention that it was discussed by Blum et al.  2. I think the novel and interesting part of this paper is indeed the algorithm in Section 3. Unfortunately, very little information is given about this proof. I would suggest that the authors expand this part and include the proofs.  3. I'm not sure if the sample complexity in Theorem 4 is accurate. Let k =1, then this is showing a sample complexity of d/\epsilon ln(1/\delta) for one task. But, the best upper bound we know in the PAC setting is 1/\epsilon (d ln(1/\epsilon) + \ln(1/\delta)) with an additional  ln(1/\epsilon). I ask the authors to explain in the rebuttal what the correct sample complexity is in this case.


Nonlocal Neural Networks, Nonlocal Diffusion and Nonlocal Modeling
Nonlocal neural networks [25] have been proposed and shown to be effective in several computer vision tasks, where the nonlocal operations can directly capture long-range dependencies in the feature space. In this paper, we study the nature of diffusion and damping effect of nonlocal networks by doing spectrum analysis on the weight matrices of the well-trained networks, and then propose a new formulation of the nonlocal block. The new block not only learns the nonlocal interactions but also has stable dynamics, thus allowing deeper nonlocal structures. Moreover, we interpret our formulation from the general nonlocal modeling perspective, where we make connections between the proposed nonlocal network and other nonlocal models, such as nonlocal diffusion process and Markov jump process.

This paper studies the damping effect of a non-local neural network (Want et al., CVPR 2018) by performing spectrum analysis on the weight matrices obtained after well training. Given the eigenvalues of the weight matrices, the paper claimed that non-local blocks, different from local learning, have the nature of diffusion. The authors then proposed an alternative formula of the non-local blocks, which is inspired by some previous classical non-local modeling works (Buades et al. 2005, Coifman et al. 2006, Gilboa et al. 2007, Du et al. 2012). A similarity between the proposed non-local operation and other non-local models, i.e., nonlocal diffusion process and Markov jump process, has been shown in the paper to demonstrate the diffusive nature and stability of the proposed non-local blocks. The experimental results also showed that the modified non-local network not only gets higher accuracy on image classification tasks than the original one, but also is more robust to multiple blocks.  Strengths: 1) The paper is well written and organized, and technically correct. 2) I appreciate the connections with a wide range of existing non-local models. It is always an enjoy to read papers that try to find theoretical justifications for deep neural architectures. In this sense, the paper is a refreshing read. Although the spectrum analysis technique has been widely studied and applied, using it to interpret deep learning seems useful yet simple. 3) Other than the theoretical findings, another contribution of this paper is that it provides a modified non-local network architecture. Although the two formulations look similar, I am convinced by the justifications in Section 4 of the paper and Section 3 of the supplementary material that the modification makes the whole system more stable and well-defined. 4) The experiments are self-contained and kept to the point. In the original paper of non-local network (Wang et al., CVPR 2018), although it was investigated in many datasets and tasks that introducing non-local blocks can improve the learning performance, the non-local blocks are always added to different places. While in this paper, the authors studied the non-local network from a different perspective, namely stacking multiple blocks to the same place, serving for the study of stability and robustness. Then the bottom results in Table 1 also gave a comparison of the two non-local networks with non-local blocks incorporated into different places.  Weaknesses: Although this paper paid more attention to the theoretical justification of the non-local network architecture, and the empirical results are presented to validate the theoretical findings, I am curious about the application of the modified non-local network. Since in (Wang et al., CVPR 2018), the authors provided complete experiments on a few tasks and datasets, especially the video classification on Kinects. I am willing to see how the modified non-local network performs on such a task with multiple non-local blocks.  Overall, the tools used in this paper are useful and extendable. The topic raised in the paper is interesting to the community. The proposed model architecture is incremental, but the theoretical study is original and refreshing. The significance of the paper does not lie in the improvement of experimental performance, but a new angle of studying the neural network architecture. Using some mathematical tools, such as PDEs, ODEs, and stochastic processes, to interpret and explore deep learning techniques seems promising.   This is a nice paper. The rebuttal makes the research idea clearer. I keep my vote to accept this paper.


Uniform Convergence of Gradients for Non-Convex Learning and Optimization
We investigate 1) the rate at which refined properties of the empirical risk-in particular, gradients-converge to their population counterparts in standard nonconvex learning tasks, and 2) the consequences of this convergence for optimization. Our analysis follows the tradition of norm-based capacity control. We propose vector-valued Rademacher complexities as a simple, composable, and user-friendly tool to derive dimension-free uniform convergence bounds for gradients in nonconvex learning problems. As an application of our techniques, we give a new analysis of batch gradient descent methods for non-convex generalized linear models and non-convex robust regression, showing how to use any algorithm that finds approximate stationary points to obtain optimal sample complexity, even when dimension is high or possibly infinite and multiple passes over the dataset are allowed. Moving to non-smooth models we show--in contrast to the smooth case-that even for a single ReLU it is not possible to obtain dimension-independent convergence rates for gradients in the worst case. On the positive side, it is still possible to obtain dimension-independent rates under a new type of distributional assumption.

This paper studies the uniform convergence of gradients for non-convex ERM problems. The results are applied to function satisfying Kurdyka-Łojasiewicz conditions and in particular certain classes of generalized linear models, to get risk bounds based on the empirical gradient at the output points. The main tools for the analysis are normed and vector-valued Rademacher complexity. The key technical contribution is a chain rule, which bounds the normed Rademacher complexity of the gradient of composite function, using their Lipschitz parameters and the vector-valued/normed Rademacher complexities of the gradient of each functions. This serves the role of contraction lemma in the proof of uniform convergence of function values.  I think this is an okay paper with legitimate contribution, but perhaps not novel enough for NIPS. There are several minor flaws, which further weaken its publishability. The use of contraction-type arguments to bound vector norm Rademacher complexities is interesting. However, the comparison to [33] (Mei, Bai and Montanari, 2016) is not well-justified. In particular, for low-dimensional setup, this paper assumes data points to be uniformly bounded in an $\ell_2$ ball, and the bound has bad dependence on its radius; in [33], assumption on data points is sub-Gaussian on each direction, which gives a $\sqrt{d}$ radius with high probability. If the authors are going to make comparisons, they should rescale their radius parameter to $\sqrt{d}$.  Technical comments: 1. The concentration results looked weird. In Proposition 1, they get $O(\log 1/\delta/n)^\alpha$ for $alpha\in[1,2]$, and in Proposition 2, the dependence on failure probability gives $O(\log 1/\delta/n)$. This seems to be much better than standard Hoeffding or McDiarmid inequalities, but in the proof, the author seems to be using nothing more than those. So I don't think these bounds are possible in terms of concentration term. 2. If K-Ł is satisfied, is uniform convergence actually necessary? Does localized empirical process give better rates? Intuitively we can show proximity to population optimal, and only need to do localized arguments within a small ball, which could possibly lead to localized rates.


The committee machine: Computational to statistical gaps in learning a two-layers neural network
Heuristic tools from statistical physics have been used in the past to locate the phase transitions and compute the optimal learning and generalization errors in the teacher-student scenario in multi-layer neural networks. In this contribution, we provide a rigorous justification of these approaches for a two-layers neural network model called the committee machine. We also introduce a version of the approximate message passing (AMP) algorithm for the committee machine that allows to perform optimal learning in polynomial time for a large set of parameters. We find that there are regimes in which a low generalization error is information-theoretically achievable while the AMP algorithm fails to deliver it; strongly suggesting that no efficient algorithm exists for those cases, and unveiling a large computational gap.
While the traditional approach to learning and generalization follows the Vapnik-Chervonenkis [1] and Rademacher [2] worst-case type bounds, there has been a considerable body of theoretical work on calculating the generalization ability of neural networks for data arising from a probabilistic model within the framework of statistical mechanics [3,4,5,6,7]. In the wake of the need to understand the effectiveness of neural networks and also the limitations of the classical approaches [8], it is of interest to revisit the results that have emerged thanks to the physics perspective. This direction is currently experiencing a strong revival, see e.g. [9,10,11,12].
Of particular interest is the so-called teacher-student approach, where labels are generated by feeding i.i.d. random samples to a neural network architecture (the teacher) and are then presented to another neural network (the student) that is trained using these data. Early studies computed the information theoretic limitations of the supervised learning abilities of the teacher weights by a student who is given m independent n-dimensional examples with α ≡ m/n = Θ(1) and n → ∞ [3,4,7]. These works relied on non-rigorous heuristic approaches, such as the replica and cavity methods [13,14]. Additionally no provably efficient algorithm was provided to achieve the predicted learning abilities, and it was thus difficult to test those predictions, or to assess the computational difficulty.
Recent developments in statistical estimation and information theory -in particular of approximate message passing algorithms (AMP) [15,16,17,18], and a rigorous proof of the replica formula for the optimal generalization error [11]-allowed to settle these two missing points for single-layer neural networks (i.e. without any hidden variables). In the present paper, we leverage on these works, and provide rigorous asymptotic predictions and corresponding message passing algorithm for a class of two-layers networks.

The committee machine is a simple and natural model for a 2-layer neural network. (Here is the formal definition: it is a function R^n -> {+1,-1}, computed by taking K different linear combinations of the n inputs according to the weight matrix W (this is the hidden layer), then taking the sign of each hidden value, and then taking the majority vote of all these signs. The results of the paper also apply to many related models: you are allowed an arbitrary function mapping the K hidden values to the final binary output.)  This paper studies the problem of learning the weights W under a natural random model. We are given m random examples (X,Y) where the input X (in R^n) is iid Gaussian and Y (in {+1,-1}) is the associated output of the network. The unknown weights W are iid from a known prior. The authors study the asymptotic regime where the dimension n goes to infinity with the ratio alpha = m/n held constant (recall m is the number of samples) and the number of hidden layers K held constant.  Since the above is a fully-specified random model, it is amenable to study via tools from statistical physics. Prior work has rigorously analyzed the simpler case of "generalized linear models" which includes single-layer neural networks (K=1). Prior work has also analyzed the two-layer case using non-rigorous heuristics from statistical physics (which are well-established and widely believed to be reliable). The first contribution of the present work is to rigorously analyze the two-layer case (showing that the heuristics are indeed correct). Specifically, they give an exact formula for the limiting value (as n goes to infinity) of the information-theoretically optimal generalization error as a function of the sample complexity (alpha = m/n). The proof uses the "adaptive interpolation method", a recent method that has been successful for related problems.  The second contribution of the present work is to give a polynomial-time algorithm for learning the weights W based on AMP (approximate message passing). They also give an exact formula for the generalization error (as n goes to infinity) that it achieves. AMP is a well-established framework and is widely conjectured to be optimal among all polynomial-time algorithms for this type of problem. By comparing the formulas for AMP's performance and the information-theoretically optimal performance, the authors identify regimes that appear to have inherent statistical-to-computational gaps. The authors give a thorough investigation of these gaps and of the various "phase transitions" occurring in the problem.  I think this is a strong paper that builds on a lot of deep and powerful machinery from statistical physics in order to give important and fundamental advances towards a rigorous understanding of neural networks. The paper is well-written. I would be interested if the authors could comment on whether there is any hope of extending the results to more than two layers.  EDIT: I have read the other reviews and the author feedback. My opinion of the paper remains the same -- I vote to accept it.


Understanding Regularized Spectral Clustering via Graph Conductance
This paper uses the relationship between graph conductance and spectral clustering to study (i) the failures of spectral clustering and (ii) the benefits of regularization. The explanation is simple. Sparse and stochastic graphs create several "dangling sets", or small trees that are connected to the core of the graph by only one edge. Graph conductance is sensitive to these noisy dangling sets and spectral clustering inherits this sensitivity. The second part of the paper starts from a previously proposed form of regularized spectral clustering and shows that it is related to the graph conductance on a "regularized graph". When graph conductance is computed on the regularized graph, we call it CoreCut. Based upon previous arguments that relate graph conductance to spectral clustering (e.g. Cheeger inequality), minimizing CoreCut relaxes to regularized spectral clustering. Simple inspection of CoreCut reveals why it is less sensitive to dangling sets. Together, these results show that unbalanced partitions from spectral clustering can be understood as overfitting to noise in the periphery of a sparse and stochastic graph. Regularization fixes this overfitting. In addition to this statistical benefit, these results also demonstrate how regularization can improve the computational speed of spectral clustering. We provide simulations and data examples to illustrate these results.

I am familiar with spectral clustering and its relation to graph conductance, but do not know much about the references on regularized SC, such as [17, 9, 11]. So my assessment on regularized SC is based on what I learnt from this work.  Here are my comments: + The paper provides a good understanding of regularized-SC for sparse-graph partitioning based on solid theoretical reasoning. The extensive experimental results also well justify the statements. As a large fraction of social networks are sparse, so the assumptions are acceptable.  + The paper is written well and easy to follow. All theoretical results are presented well and show insight on why regularized-SC is important (Thm 4.3) and when spectral SC can work well(Corollary 4.7) - The main concern is about the usage of regularized-SC in practice. I agree that the when using spectral clustering without regularization, the first-round partition may yield a huge component and several small parts like the dangling sets mentioned in this work. However, in practice, we can easily do another round of graph partition by focusing on the obtained huge component to refine the clustering results. Regularized-SC may not require such additional effort but it introduces a parameter \tau whose tuning is not easy.   Two suggestions: -The format of 2-way-partition regularized-SC seems equivalent to global Pagerank where \tau is to control the probability of random jump. Pagerank algorithm is widely used everywhere. It would better to give some discussion on this point and review related references. Some work on Pagerank also shows connection with graph conductance such as Andersen et al.  Local graph partitioning using pagerank vectors  [FOCS2006]    - Since sparsity comes from the real social networks, it may be better to provide some real evidence on the number of dangling sets in social networks. Moreover, as shown in the appendix, Thm 3.4 requires another assumption which should be added within the statement in the main text.     Some minor typo: Line 29: sigular  -> singular Line 31: tau -> \tau Line 75:covolutional -> convolutional


Theoretical guarantees for the EM algorithm when applied to mis-specified Gaussian mixture models
Recent years have witnessed substantial progress in understanding the behavior of EM for mixture models that are correctly specified. Given that model misspecification is common in practice, it is important to understand EM in this more general setting. We provide non-asymptotic guarantees for the population and sample-based EM algorithms when used to estimate parameters of certain misspecified Gaussian mixture models. Due to mis-specification, the EM iterates no longer converge to the true model and instead converge to the projection of the true model onto the fitted model class. We provide two classes of theoretical guarantees: (a) a characterization of the bias introduced due to the mis-specification; and (b) guarantees of geometric convergence of the population EM to the model projection given a suitable initialization. This geometric convergence rate for population EM implies that the EM algorithm based on n samples converges to an estimate with 1/ √ n accuracy. We validate our theoretical findings in different cases via several numerical examples.

# After authors response  I have read the authors’ responses about the references and some intuitions about  the results and the model and the model in section 4, and I thank the authors for their explanations.  I made some detailed comments about the figures and their interpretation, that go  beyond a simple size problem. I acknowledge the will of the authors to revisit both  their display and exploitation. I also acknowledge the authors’ intention to work on  the exposition and redaction in some places, previously signaled. I thank the authors for receiving these comments, I think that the paper would highly benefit from an  improvement on theses matters.  # Summary  In this paper, the authors are interested in the effect of model specifications on the convergence of the EM algorithm. Specifically, they focus on three simple cases where the formulas are tractable, and assess the bias, as well as the convergence properties of the population and sample EM.  The three cases considered are instances of Gaussian mixtures: 1. The true model has three classes, with two classes having close means, and    the fitting model has only two classes. Variance and weights of the fitted    model are fixed according to the true values. 2. The true model has three classes, including one centered in zero and with a    small associated weight. The fitting model is the same as above. Variances and    weights of the fitting models are fixed according to the true values. 3. Both the true and fitted models have two classes. The weight and variances of the    fitted model are fixed, different from the true one.  After showing some theoretical results in each situations, the authors validate heuristically their findings with associated simulation studies.  # General comments  The problem of fitting misspecified statistical models is an important one, and this paper makes a first step toward some theoretical bounds in some specific cases. Although limited and relatively simple, the three cases considered are of practical interest, and give way to consistent developments (although I did not check the proofs in the supplementary material carefully).  As a personal opinion, it felt that the paper was missing some context on the state-of-the art. Although some references are given (mainly, Balakrishnan et al. 2017), the results are not compared to the literature, which made it difficult to get an idea of the sharpness of the bounds.  In a similar way, I found no reference to the classical model selection theory,  which deal with similar problems (trying to find the 'best' number of components to keep in the fitted model). Although the focus is a bit different, it could be interesting to make some parallels. For background on model selection, see e.g.: - Massart, P. (2007). Concentration Inequalities and Model Selection. - Baraud, Y. et al (2009). Gaussian model selection with an unknown variance. AoS.  Finally, it felt like the submission could benefit from a proof-reading for punctuation, typos, English style and grammar issues. (See below for some more specific remarks.)  # Specific remarks  ## Section 2  * Table 1: maybe adding the corresponding section for each model could help   the reader (3.1, 3.2 and 4, if I'm correct). (This is just a suggestion.)  * It could be useful to add a reference for formulas for section 2.1 (EM for    a two-component Gaussian mixture).    ## Section 3  * I found the introductory paragraph of section 3 (lines 115 - 124) confusing.   It could be worth to refactor it a bit. Specifically:   * l.118: "we do not assume any knowledge about the number of component in the     true model." Maybe I missed something, but I was under the impression that     the true model was assumed to have three or two classes. It might also be     in contradiction with the first sentence of the paragraph ("... the number     of components in the true model is *larger* than that in the fitted     model.") Maybe a clarification could help.   * l.119-121: it was not clear to me whether these scenarios were assumptions,     or statements of the general nature of misspecification.   * l.122-123: "the number of components" is repeated, maybe rephrase.     * I felt that a comparison with classical bounds of the EM (that is, when the   model is correctly specified) was maybe missing. It could be interesting to   recall these bounds, and point out the differences induced by this   misspecification.  * Similarly, it seems that more comments could be made on the links between the   results of sections 3.1 and 3.2. For instance, in seems that the bounds in   Th. 1 and Cor. 1 do not depend on the disparity parameter $\rho$, while the   ones in Th. 2 and Cor. 2 do depend on $\omega$. Could the authors provide an   intuition of why this is the case ?  * l.158: should it be $\gamma = e^{-c''\eta^2}$ to match expression l.151 ?   * l.177: By multiplying the constant of Prop. 2 by 1/\eta^{1/4}, should the   constant $C_\omega = c\omega^{1/8} / \sqrt{1 - \omega}$ ? be (without the   $\sigma$) ?  ## Section 4  * It could be interesting to comment on the specific form of model (16). In   particular, why choosing an expression where both the distance between the   two Gaussians and there variance vary conjointly, through parameter $\theta*$ ?  * It is not clear from the text whether $\sigma^2$ in the fitted model (17) is   fixed or not, and, if so, to which value.   ## Simulation studies (section 5)  It was not always clear to me how the simulations illustrated the results of the previous sections.  Also, the figures are quite small, and I had to magnify Figures 2 and 3 with a factor 3 on my screen to read them properly. I'm myopic, so maybe not representative, but it could ease the reading to leave more space to them, and/or remove some of the panels (see comments below).  * I did not understand how the results presented in the third panel of each   figure (histogram) was related to the previous theoretical results. Maybe I   missed it, but it does not seem to be exploited in the text (beside its   description l.248-250). Maybe a clarification could help the reader.  * Figure 1: Maybe it could help the reader to give the value of $\bar{\theta}$.   From the graph, it seems that it is almost equal to $\theta^*$ (4 or 3), so   that there is no bias in this case. On the second panel figure 1b for   instance, the sample EM seems to converge to $\theta^* = 3$, while $\rho=0.5$   is large, so the bias should be larger, as hinted by the text (see l.258).   Maybe I read the figure wrong, but then a clarification could help.   Could a plot of bound $C_\rho$ help ?  * Figure 3a: It looks like the fitted model is unimodal (panel 1), while   from the text (see Table 1), it should have two components. Again, I might   be missing something, but then a clarification could help.  ## Discussion  Please consider proof reading this discussions, which is difficult to read in its current state because of the many typos (see below).  # Possible typos  Below are a few possible typos that I noted along the way:  * l.52: lower case We ? * l.65: show appears two time, maybe rephrase ? * l.82: should it be "... performance of *the* EM algorithm … " (add 'the') ?   Also I think lines 89 ('the location'), 91 ('the EM'), 115 ('the EM'), 133   ('of the EM'), 168 ('the EM'), 176 ('the results'), 177 ('the bias'), 178   ('the EM'a, 270 ('the behavior of the EM')). * l.91: maybe rephrase: "... may not belong to the fit model, …" * l.91: maybe rephrase: "... the best possible estimator that can be achieved   using EM can be …" ('can' appears two times). * l.168: missing period at the end of the sentence.  * l.270: "general" appears two times. * l.271: "…for the same. illustrate..." maybe a piece is missing ? * l.273: "... is focused … " ? * l.274: "... to ask if we fit a model with scale as well." maybe rephrase ? Also: * "2 or 3 component(s) mixture": this formula appears in several places,   with 2 and 3 sometimes spelled out, and with or without an -s at "component".   It might be good to unify. * Why is there a lonely section 2.1 ?


Policy Regret in Repeated Games
The notion of policy regret in online learning is a well defined performance measure for the common scenario of adaptive adversaries, which more traditional quantities such as external regret do not take into account. We revisit the notion of policy regret and first show that there are online learning settings in which policy regret and external regret are incompatible: any sequence of play that achieves a favorable regret with respect to one definition must do poorly with respect to the other. We then focus on the game-theoretic setting where the adversary is a self-interested agent. In that setting, we show that external regret and policy regret are not in conflict and, in fact, that a wide class of algorithms can ensure a favorable regret with respect to both definitions, so long as the adversary is also using such an algorithm. We also show that the sequence of play of no-policy regret algorithms converges to a policy equilibrium, a new notion of equilibrium that we introduce. Relating this back to external regret, we show that coarse correlated equilibria, which no-external regret players converge to, are a strict subset of policy equilibria. Thus, in game-theoretic settings, every sequence of play with no external regret also admits no policy regret, but the converse does not hold.

The paper proposes a definition of bounded-memory policy regret, then defines a corresponding equilibrium concept for two-player games, and proves results on (i) convergence (in a non-standard sense) of no-policy-regret strategies to policy equilibria, and (ii) relating policy equilibria to coarse correlated equilibria.  The ideas presented in the paper are clearly motivated: a notion of regret which captures adaptive reaction of the adversary is useful to study. The technical results are also interesting in the context of learning in games.  I found the paper hard to read, however, due to presentation issues. The definitions are often preceded by ambiguous statements. For example: - In the paragraph "Algorithm of the player" line 180: Alg_t is defined as a Alg_t: (A_1 \times A_2)^t \to \Delta A_1, but then redefined as a function from A_2^t \to \Delta A_1 in the full information case, and finally as a distribution over such functions in the bandit setting. In Definition 3.3, the second definition seems to be used. In the same definition, "a possibly random algorithm" is not clearly defined, since the randomness can refer either to randomization over the action set, or to randomization over functions f_t. Line 194: what is meant by "simulating the play of players"? It is possible the reader could guess a definition of \mu, but it would be much better to give a rigorous and unambiguous definition (e.g. in terms of a distribution of an appropriate Markov chain). This entire section should be replaced with an unambiguous mathematical description of the game setting. - In Section 4, the discussion leading to the main definition (Definition 4.5) often involves imprecise statements, e.g. "under reasonable play from all parties" (line 229), "The player believes their opponent might be" (line 231). \epsilon is undefined on line 237. - When switching between distributions over the action set A and the joint policy set F, the presentation can be improved. I believe it would be better to start by defining the equilibrium concept (Definition 4.5) then discuss its interpretation. - \tilde \sigma_a and \tilde \sigma_b (line 336) are not clearly defined.  Besides presentation, several points require discussion and justification: 1) Modifying the definition of policy regret (Definition 3.1): overloading existing definitions has serious drawbacks. In this case it is even harder to justify, since the whole point of defining policy regret is to take into account the adaptivity of the adversary (a point that the paper highlighted several times). The sequence of plays in the proposed definition simply does not reflect a fixed action policy. 2) Justification of the new equilibrium concept (policy equilibrium). Policy equilibrium compares stationary distribution of Markov chains. This needs careful justification and interpretation. For example, do the players care about their stationary distributions because the game is played infinitely often? Why is this a better equilibrium concept than existing concepts for repeated games? How does it generalize to n players? The proposed concept only applies in a context of repeated play (unlike other concepts mentioned in the paper, e.g., Nash equilibria and correlated equilibria). It should be related and compared to other equilibrium concepts of repeated games. 3) The non-uniquess of stationary distributions also raises some issues (the ability of the players to select these distributions). The comment following Definition 4.5 ignores the issue by referring to "the stationary distribution", omitting that there may be more than one.  ================= Thank you for your responses and clarifications, in particular regarding Definition 3.1, this addresses one of my concerns. I agree that Definition 4.5 does not rely on the uniqueness of the stationary distribution, but the point is rather the motivation of this new equilibrium concept: when there are multiple stationary distributions, the play could converge to a different stationary distribution than the one satisfying the guarantees, and this is problematic. As raised during discussion with other reviewers, a better justification could be that one can ensure uniqueness of the stationary distribution at a relatively small cost, by mixing in a uniform distribution. A more careful discussion should be included in the revision.


Generalizing Tree Probability Estimation via Bayesian Networks
Probability estimation is one of the fundamental tasks in statistics and machine learning. However, standard methods for probability estimation on discrete objects do not handle object structure in a satisfactory manner. In this paper, we derive a general Bayesian network formulation for probability estimation on leaf-labeled trees that enables flexible approximations which can generalize beyond observations. We show that efficient algorithms for learning Bayesian networks can be easily extended to probability estimation on this challenging structured space. Experiments on both synthetic and real data show that our methods greatly outperform the current practice of using the empirical distribution, as well as a previous effort for probability estimation on trees.

The paper proposes to model the distribution over phylogenetic trees using a special kind of Bayesian networks called subsplit Bayesian networks(SBNs). SBN nodes can be considered as categorical random variables taking on values over possible subsplits which are partitions of the labels in the domain. Conditional probabilities in SBNs model the probability of a child node taking on a subsplit value conditioned on its parent(s) subsplit value(s). Given an SBN structure and a sample of rooted or unrooted trees, the authors use standard procedures (closed form MLE estimations for rooted trees and EM for unrooted trees) to estimate the parameters (conditional probabilities) of the SBN. The authors adopt a similar method of weight sharing in convolutional neural networks to share parameters in SBNs. The probabilities of phylogenetic trees approximated using SBNs have been shown to be more accurate compared to other methods on both synthetic and real data. I am not an expert on phylogenetics. I am familiar with Bayesian network representation, inference and learning (both with observed and latent variables). Hence, I could understand the idea of modeling the probability distribution over possible trees using SBNs. There is still room for improvements (see comments below), but compared to current methods like SRF and CCD, SBNs are more flexible and general. Hence, I am voting for an accept.  Quality: The more challenging and interesting task of structure learning of SBNs is yet to be explored. Simple tree like SBNs have been considered in the paper although it has been mentioned in section 3 that more complicated dependencies may be introduced.   More recent works than [LeCun et al. 1998] which aims to reduce the number of parameters in a model by parameter tying are by Chou, Li, et al. "On Parameter Tying by Quantization." (AAAI. 2016), and "Automatic Parameter Tying: A New Approach for Regularized Parameter Learning in Markov Networks." (AAAI 2018). I would suggest the authors to explore such works on parameter sharing in pgms. Also, no experimental evaluation was done to see the reduction in the number of parameters through sharing in the learned models.  The paper lacks a related works section. The authors might consider adding one to SM.  Clarity: Overall, the paper was not very difficult to understand. The following questions are related to the experimental section. It is not clear why \alpha was fixed in experiments with real datasets. Is any smoothing performed when estimating the parameters in the CCD/SBN CPTs? Also, what was the average number of iterations performed during EM? The reason behind the very low number of sampled trees for DS2 and DS3 should be explained. In [Larget, 2013] a much greater number of samples were generated for both the datasets.  General comments regarding notations: Subscript k missing for parent assignment in L(T_k) in line 130 Superscript (n) in EM subsection not clearly introduced.  Originality and significance: Learning the distribution over trees using a generative probabilistic model like SBN has advantages over simple SRF and CCDs. I believe other interesting probabilistic queries can be carried out on these distributions besides computing tree probabilities.    ---------------------------------------------------------------------------------------------------------- I thank the authors for answering to the reviewers questions. The authors have tried to answer all the questions, based on which I am updating the score for the paper. I would suggest the authors to include experimental results on parameter sharing in the current paper.


Online Learning with an Unknown Fairness Metric
We consider the problem of online learning in the linear contextual bandits setting, but in which there are also strong individual fairness constraints governed by an unknown similarity metric. These constraints demand that we select similar actions or individuals with approximately equal probability [?], which may be at odds with optimizing reward, thus modeling settings where profit and social policy are in tension. We assume we learn about an unknown Mahalanobis similarity metric from only weak feedback that identifies fairness violations, but does not quantify their extent. This is intended to represent the interventions of a regulator who "knows unfairness when he sees it" but nevertheless cannot enunciate a quantitative fairness metric over individuals. Our main result is an algorithm in the adversarial context setting that has a number of fairness violations that depends only logarithmically on T , while obtaining an optimal O( √ T ) regret bound to the best fair policy.

The authors study the problem of correcting for individual fairness in an adversarial online learning problem when the fairness constraints depend on an unknown similarity metric between the items. This captures the setting in which a quantitative fairness metric over individuals cannot be pre-specified, but is clear a posteriori. The problem is crucial as it the setting of unknown similarities is extremely natural, and keeping the objective separate from the optimization goal seems often appropriate. They show that optimal fair decisions can be efficiently learned, even in this setting that does not have the direct knowledge of the fairness metric. The setting and approach seem appropriate, and the paper opens up many interesting directions for future work.   The discussion of the related work in the supplementary material was great and I think crucial to situate this work -- I hope room can be made in the final version for it in the main body. To this I would just add two additional papers which also deal with different notions of fairness in online learning settings (https://arxiv.org/abs/1707.01875 and https://arxiv.org/abs/1802.08674).


Size-Noise Tradeoffs in Generative Networks
This paper investigates the ability of generative networks to convert their input noise distributions into other distributions. Firstly, we demonstrate a construction that allows ReLU networks to increase the dimensionality of their noise distribution by implementing a "space-filling" function based on iterated tent maps. We show this construction is optimal by analyzing the number of affine pieces in functions computed by multivariate ReLU networks. Secondly, we provide efficient ways (using polylog(1/ ) nodes) for networks to pass between univariate uniform and normal distributions, using a Taylor series approximation and a binary search gadget for computing function inverses. Lastly, we indicate how high dimensional distributions can be efficiently transformed into low dimensional distributions.

This paper examines the approximation of uniform and normal distributions using neural networks provided with a uniform or normal distribution at its input. The paper's main contributions are   1. the upper- and lower-bound for approximating higher dimensional uniform distributions with a lower dimensional one. The main idea is the use of an affine space-filling curve, which has been examined in a similar context before. 2. providing a neural network transformation to go from a uniform distribution to a normal and vice-versa.  Overall, the paper is a pleasure to read and tackles the important question of the required size/depth required for a neural network to approximate a given distribution.  Some minor notes:  Section 2.2 is not an easy read without delving into the appendix. Perhaps some intuition can be provided for the statement in line 138. Lines 145-148 are also a bit terse. An added explanation of how the lower and upper-bound compare (as is done in line 33 of the intro) would make the strength of this section more apparent.  Section 4 is somewhat trivial as it is stated. I would suggest adding a few sentences making the comparison with the result of Section 3.2 more explicit, so that the added benefit of more dimensions of uniform noise to approx. a normal becomes clear.  If more space is required, perhaps consider reducing the size of Figure 1 or placing proof of Th. 11 in the appendix.  Some typos:  l 53 This idea [h]as l 104 How easy [is it] to l 190 good behavior

Reviewer 3
This paper is a joy to read. It addresses question whether GANs can map between simpler standard distributions and how well, and the role of width and depth of the neural net in the approximation. The paper is very well organized. Deep down, the argument boils down to that the ReLU neural network can divide the input space into small convex open subsets that are locally uniform and then map them  (via affine transformations) to another small locally uniform set of the output. The measure remains the same after the mapping. The dificulty is: how small should these sets be? And how well can a given neural network do this task? I think the authors did a great job explaining the task.  It was disapointing that there was no empirical results trying to validate the bounds. I have a personal bias against papers with theoretical bounds that show no empirical evidence of their results. Specifically in this case, where the examples are simple and concrete.              Please also cite Egoroff’s theorem [Folland, 2013, Theorem 2.33] as Egorov's theorem [Kolmogorov and Fomin, 1975, pp. 290, Theorem 12]. The different spellings of the name tend to confuse readers familiar only with Kolmogorov's book. A. N. Kolmogorov and S. V. Fomin. Introductory Real Analysis. Dover, 1975.                The statement that "It stands to reason that larger neural networks or networks with more noise given as input can represent more". As R (real set) has the same cardinality as R^n, n finite, the statement seems to be connected to a notion of smoothness in the measure. It reads a little odd and a small discussion on the reason why the input needs to be high-dimensional noise could help.  ---------  Read the rebuttal and the other two reviews. This is a good contribution.


Low-Rank Tucker Decomposition of Large Tensors Using TensorSketch
We propose two randomized algorithms for low-rank Tucker decomposition of tensors. The algorithms, which incorporate sketching, only require a single pass of the input tensor and can handle tensors whose elements are streamed in any order. To the best of our knowledge, ours are the only algorithms which can do this. We test our algorithms on sparse synthetic data and compare them to multiple other methods. We also apply one of our algorithms to a real dense 38 GB tensor representing a video and use the resulting decomposition to correctly classify frames containing disturbances.
• We propose an idea for defining the sketch operators upfront. In addition to increasing accuracy and reducing run time, it allows us to make several other improvements. These include only requiring a single pass of the data, and being able to handle tensors whose elements are streamed. To the best of our knowledge, ours are the only algorithms which can do this.

In this paper, the authors present two variants of a random projection-based algorithm (TensorSketch) whose purpose is to obtain a low-rank Tucker approximation for a given multidimensional dataset. It includes the possibility that entries of the data tensor are being streamed in one specific mode. I found the topic interesting and relevant to NIPS because this kind of approach is very important for developing real world big data applications with multidimensional datasets.  However, I found the paper not well structured and lacking of clarity in the presentation among other issues. The authors recognized these issues in their response, found my comments useful to improve the quality of the paper and said they will incorporate changes in the final version. However, there is not detailed information about how the issues will be solved. The main issues I found are the following: - An introduction to “random sketching” is missing. The authors start using the term in the introduction but the it is not explained throughout the paper. They just limited themselves to give references to previous papers. Authors said, they don't want to repeat work from others. - Section 1.2 "A brief introduction to TensorSketch" is very confusing. The authors refer to previous papers where this technique was proposed but it is not explained so the reader gets lost without reading previous works. There are additional information about TensorSketch in the Supplementary information but I found it not useful and very disconnected from the main text of the article. - The usage of matrix B in lines 73 – 76 is misleading. I think there is a mistake and it should be replaced by A in lines 73 and 76. - In line 61, the method COUNTSKETCH is introduced but never explained which makes difficult to follow the article. - Equation (2) is not well motivated and its introduction in section 1.2 is confusing. There is some information about this formula in the Supplemental info but there is no link in the main text to the explanations. Moreover, the operator S^{(n)} is not defined in section 1.2. - First part of section 3, lines 113 – 122, including the classical algorithm HOOI (Algorithm 1: Tucker ALS) should be moved to the introduction section 1.1.  - Section 3.1 (Tucker-TS) is not clearly described. For example, why two operators S_1 and S_2 are needed? How dimension J1 and J2 are chosen? - Authors say that, TensorSketch algorithm meet a theoretical condition of [10] but it is not explained here. Also, they say that avoiding defining a new sketch at every iteration showed better performance in the experiments (Fig. 1) but they were not able to explain this behavior or give some interpretation of this result. - Propositions 3.1 and 3.3 are difficult to interpret without some additional explanations - In Algorithm 2:  Why two parameters J1 and J2 are needed? How these parameters are chosen in practice? Authors propose to have J1=K*R^(N-1) and J2=K*R^N as a practical choice, however it is not clear how to select an optimal value of parameter K.   Only sizes of operators T^(n) are defined in line 2 but how these matrices are generated? Are they Gaussian i.i.d matrices? This should be clearly presented and justified. These questions were not addressed by the authors in their responses. - Algorithm of section 3.2 is not well motivated. Why is this algorithm proposed?  - In the Complexity analysis, the proposed algorithms are compared only to the classical Tucker-ALS algorithm. I think they should compare also to other algorithms such as the ones used in the Experiments. For example, FSTD1 algorithm paper provided a detailed analysis of the complexity. - Figs. 2-3, 4-5 show comparison with other algorithms for sparse tensors. The results of FSTD1 and MACH show relative errors very close to 1 (100%), which, as I understand correspond to totally wrong approximations. The authors justify the poor performance of FSTD1 algorithm because of the sparsity of the tensor. I think this comparison are not useful because FSTD1 is not well defined for sparse tensors. I think comparison of algorithms should be performed also on dense tensors, as the one included in section 4.2. - In section 4.2, it would be worth to show also results applying other algorithms to this dense dataset. - Supplemental material is a set of disconnected sections without no reference in the main text.


Binary Classification from Positive-Confidence Data
Can we learn a binary classifier from only positive data, without any negative data or unlabeled data? We show that if one can equip positive data with confidence (positive-confidence), one can successfully learn a binary classifier, which we name positive-confidence (Pconf) classification. Our work is related to one-class classification which is aimed at "describing" the positive class by clustering-related methods, but one-class classification does not have the ability to tune hyper-parameters and their aim is not on "discriminating" positive and negative classes. For the Pconf classification problem, we provide a simple empirical risk minimization framework that is model-independent and optimization-independent. We theoretically establish the consistency and an estimation error bound, and demonstrate the usefulness of the proposed method for training deep neural networks through experiments.

Overview and Recommendation: This paper presents an algorithm and theoretical analysis for Pconf learning of a binary classifier, when only the positive instances and their conditional class probabilities are available. In contrast to PU learning, which has received attention recently in the ML community, Pconf learning does not require a large amount of unlabeled data or any knowledge of class priors, however it does require confidence weights p(y=+1 | x) for each example x sampled from p(x | y= +1).  The paper is well written and this solution can be useful for many real world applications, therefore, it is a clear accept. Although, the analysis is very similar to the recent analysis for PU learning published by Du Plessis et al. recently at NIPS but the problem setting is novel and interesting.  However, I think that the authors should fix some technical issues with their simulation experiment.  Summary: The main contribution of the paper is the observation that empirical risk minimization for Pconf learning can be done by importance weighting the confidence weighted risk (equation 3) using positive samples. The paper also provides an estimatior error bound for the proposed risk-estimator. The analysis only requires a bounded hypothesis class and a bounded lipschitz loss function on that hypothesis set and shows that estimation error decreases as 1/sqrt(n), which is expected and intuitive. The experimental results also support the theory.  Technical Issues: Loss of accuracy when the separation between clusters increases should simply not happen. The authors justify this by a loss-of-senstivity in the confidence values but in the end this is just a numerical issue. The gradient of the log-linear model used in the experiment can be coded directly, and then an arbitrary-precision floating-point library for python such as mpmath can be used. Since the dimensionality is just 2 and the number of examples is 1000 this experiment should not take up much resources.  ==== After Author Response ===  I am happy to see that the authors made the effort to run the experiment using the arb prec math library which made the experimental results more intuitive. I have increased my rating of the paper, because now the paper does not seem to have any technical issues.


A Simple Cache Model for Image Recognition
Training large-scale image recognition models is computationally expensive. This raises the question of whether there might be simple ways to improve the test performance of an already trained model without having to re-train or fine-tune it with new data. Here, we show that, surprisingly, this is indeed possible. The key observation we make is that the layers of a deep network close to the output layer contain independent, easily extractable class-relevant information that is not contained in the output layer itself. We propose to extract this extra class-relevant information using a simple key-value cache memory to improve the classification performance of the model at test time. Our cache memory is directly inspired by a similar cache model previously proposed for language modeling . This cache component does not require any training or fine-tuning; it can be applied to any pre-trained model and, by properly setting only two hyper-parameters, leads to significant improvements in its classification performance. Improvements are observed across several architectures and datasets. In the cache component, using features extracted from layers close to the output (but not from the output layer itself) as keys leads to the largest improvements. Concatenating features from multiple layers to form keys can further improve performance over using single-layer features as keys. The cache component also has a regularizing effect, a simple consequence of which is that it substantially increases the robustness of models against adversarial attacks.

This paper presents a cache model to be used in image recognition tasks. The authors argue that class specific information can be retrieved from earlier layers of the network to improve the accuracy of an already trained model, without having to re-train of finetune. This is achieved by extracting and caching the activations of some layers along with the class at training time. At test time a similarity measure is used to calculate how far/close the input is compared to information stored in memory. Experiments show that performance is improved in CIFAR 10/100 and ImageNet. In addition the authors show that a cache model increases resistance in adversarial attacks and show that experimentally that the cache model is more stable at larger regions around the training data.  The paper is well written and contains a thorough experimental design. I would like to see a more conventional organisation of the paper. It feels off to not read previous research and jump straight at the results.  In general I liked reading the paper. The authors hypothesize that using a cache model similar to work of Grave et al. can benefit image recognition tasks as well. The motivation is clearly explained and the contributions are easy to extract.  I liked the idea that using a cache model is like putting a prior on data so that the model prefers inputs similar to the training set. How would that work in a real world case where data is not that perfect though as in benchmark datasets? I feel that favouring similar data to training set might not be usable in datasets in the wild where some of the assumptions on data break.  I do have some concerns over computation. Very deep networks will require searching in a combinatorial fashion for number of layers, which layers, what should lambda and theta be set to etc. I think a discussion on this should be present and ways to go around it. Perhaps this can be something that is learnt concurrently to the training procedure.  Table 2 shows p_adv. This was tuned to the baseline model and tested at cache3 and cache-only using that setting. I would like to see what happens when p_adv is tuned on the other models too. The main issue I have is that by adding a cache model, essentially you have changed the model and how decisions are made. So resnet32-cache3 is a different model than vanilla resnet32. Looking at how adversarial attacks transfer with resnet20 for instance, though there is an impact it's not nearly as catastrophic as in resnet32. So I am left feeling that maybe the performance of cache models is because it is more like transfer and can't judge with certainty if the adversarial behaviour is indeed there. Perhaps tuning p_adv for cache3 and cache-only will provide more information on how large a perturbation needs to be to break the model.  Regarding references, there's a lot of arxiv work when most have been published to peer reviewed journals and conference. There should be a second pass on that to fix it.  * The rebuttal has addressed most of my concerns and cleared up a few other places.


Text-Adaptive Generative Adversarial Networks: Manipulating Images with Natural Language
This paper addresses the problem of manipulating images using natural language description. Our task aims to semantically modify visual attributes of an object in an image according to the text describing the new visual appearance. Although existing methods synthesize images having new attributes, they do not fully preserve textirrelevant contents of the original image. In this paper, we propose the text-adaptive generative adversarial network (TAGAN) to generate semantically manipulated images while preserving text-irrelevant contents. The key to our method is the text-adaptive discriminator that creates word-level local discriminators according to input text to classify fine-grained attributes independently. With this discriminator, the generator learns to generate images where only regions that correspond to the given text are modified. Experimental results show that our method outperforms existing methods on CUB and Oxford-102 datasets, and our results were mostly preferred on a user study. Extensive analysis shows that our method is able to effectively disentangle visual attributes and produce pleasing outputs.

After rebuttal comments: * readability: I trust the authors to update the paper based on my suggestions (as they agreed to in their rebuttal). * reproducibility: the authors gave clear guidelines in the rebuttal of how their produced their results. For AttrGAN, they did change the weight sweep and for SISGAN they used the same hyperparameters as they used in their method (which I would object to in general, but given that the authors took most of their hyperparameters from DCGAN, does not create an unfair advantage). I expect the additional details of the experimental results to be added in the paper (as supplementary material).   ###################  Main idea:  ------------------ Task: Manipulate images based on text instruction. Ensure that content that is not relevant to the text does not change.  Method: to avoid changing too much of the image, use local discriminators that learn the presence of individual visual attributes. The discriminator is a weighted combination of the local discriminators which determine whether a visual attribute (feature map in a convolutional image encoder) is related to a particular word. The local discriminators are combined using attention on the word vectors.  Results: substantially improving on state of the art on modifying images based on text descriptions. However, the details of how the baselines are trained are not specified (see below).  Reviewer knowledge: I am familiar with the GAN literature, but prior to reading this paper I have not been aware of work in manipulating images with text. I was not familiar with baselines (SISGAN and AttnGAN) before reading this paper.  Comment on my rating: I am willing to increase my review substantially if the paper clarity substantially increases (see below) and if more details about the baseline training are released.  Quality: -------- The idea and the experiments presented in the paper are interesting, and definitely worth noting. However, the paper suffers due to a lack of clarity (see below).   Paper not fully reproducible and it is not clear how the baselines were trained: details of data augmentation are not fully specified (lines 171-172), details of training are also lacking, especially for the baselines. Were the same architectures used, or were the ones in the original papers used? Where the hyperparameters tuned on the presented method (this would severely cause a disadvantage to the baselines)? Specifically for SISGAN, the SISGAN paper introduces two version (with and without VGG) and it is not clear to which this paper is comparing.  Originality ----------- Comparison with prior work: Not sufficient in the paper, but by reading the two baseline papers, SISGAN and AttnGAN, I believe the current work introduces the following differences: 1) Difference compared to AttnGAN * AttnGAN does not work with text, but with attributes (the goal is to generate an image with a an attribute change from the original image). This is a simpler task, so there is no need for local discriminators.  * AttnGAN also uses a reconstruction loss, and an unconditional adversarial loss (using WGAN). The difference lies in the conditional version of the model, where in the presented work a conditional GAN is used, while AttnGAN  uses an additional classifier which ensures that the attributes are preserved.  2) Difference compared to SISGAN SISGAN tackles the same problem as the one in this paper: changing images based on text instructions. The approach of combining the image and text embeddings in the encoder through residual blocks is used in both papers. in SISGAN, there is no unconditional discriminator - the discriminator sees pairs of fake data with real text as “fake”. The advantage of the current approach compared to SISGAN is that it ensures that the generated images are realistic, independent on the text association. SISGAN does not use a local discriminator that looks at individual words. The advantage of the presented method is the attention and the focus on individual words, which helps with generalization to new sentences.  The paper does introduce a novel idea, of the text adaptive discriminator. The discriminator learns association between individual words and individual feature maps of the image feature learner, and combines these into creating a global discriminator.  It is not clear to me why in equation (3), the attention is taken over the average temporal word embedding. Would using a learned attention structure here not be better?  Significance:  ------------- The results in the paper are impressive. Compared to the baselines, more content is preserved, and only text specific information is changed. The images are impressive, and the results in Table 1 show an improvement in quality based on human evaluation on a large number of samples. The paper also shows that the results are applicable for image-text similarity, as seen in Table 2. They improve or compete with state of the art in that domain as well.  One important point of the paper is that it performs analysis on the presented method. First, they look at at the text adaptive discriminator, and visualizing the class activation maps per layer. By using class activation maps, one can see to see how the model is focusing on different aspects of the image at different locations in the network. Second, they perform sentence interpolation on the text to see the different generated images, and show interpretable results. The results here are impressive too, with the model showing generalization in embedding space. Though this is limited to the particular image shown in Figure 7, it is an interesting result.  Clarity: ------ Overall: This is where the paper lacks most. The core ideas need more detail, and core concepts are not explained. Concrete example: “visual attributes" are concretely defined late in the paper, and there vaguely too. The set up of the paper is not clearly described early in the paper (what is a “positive” and what is a “negative” example text?), and the reader has to infer this from the content later on. The datasets used have (x, t) examples of images and text relevant to that image. These are the positive examples. The negative examples are obtained by sampling another t from the data. The most relevant related work is also vaguely described (I had to read the two prior relevant papers to understand the differences).  Detailed feedback: The introduction section which describes the proposed method (46-58) could have been sharper. I felt there was either not enough detail, or too much detail, depending on the view the authors wanted to take. Perhaps a rewrite would just fix this impression.  Line 94: typo: to reconstruct a latent noise vector.   Discussion of related work: This discussion could have been made clear by expanding on the related work in 93-100. This is by far the most relevant prior work, but gets less attention than the other prior work described.   Lines 111-112: “we train the RNN in the generator from scratch. “ Perhaps “we train the RNN from scratch, without pretraining.”  Lines 115-116: “Therefore, we use the reconstruction loss [27] when positive pairs of the image and the text are forwarded”. So far it is not clear what the positive examples are, so this sentence is confusing. If the problem statement was more clearly defined, this would not be an issue. One needs to state that positive examples are x and t that refer to the same content, taken from the dataset.  Lines 120-127: The definition of “visual attribute" should be made clearer here. $v$ is used to represent the visual attribute, so perhaps its definition should be moved at the end of line 127.  Line  179: “Although AttnGAN was originally designed for the text-to-image synthesis”. I think there is a not missing there.  Lines 183-185 “The inception score [30] is also not proper for this problem since the mix-up of inter-class attributes may lower the score.” This sentence is not clear - but I agree that in this case Inception score will only try to measure image quality, not relevance to text.   Lines 228-229: “the saliency map that a world classifier attends to”. Again, here it could be made more clear that this refers to the component classifiers of the discriminator, that learn to classify per word embedding  and visual map (Equation 3). When describing figure 5, it would be useful to point out that the floating point numbers are the betas referred to in equation 5.


Sketching Method for Large Scale Combinatorial Inference
We present computationally efficient algorithms to test various combinatorial structures of large-scale graphical models. In order to test the hypotheses on their topological structures, we propose two adjacency matrix sketching frameworks: neighborhood sketching and subgraph sketching. The neighborhood sketching algorithm is proposed to test the connectivity of graphical models. This algorithm randomly subsamples vertices and conducts neighborhood regression and screening. The global sketching algorithm is proposed to test the topological properties requiring exponential computation complexity, especially testing the chromatic number and the maximum clique. This algorithm infers the corresponding property based on the sampled subgraph. Our algorithms are shown to substantially accelerate the computation of existing methods. We validate our theory and method through both synthetic simulations and a real application in neuroscience.

This paper introduces sketching based hypothesis tests for some combinatorial properties of Gaussian graphical models. The authors introduce two sketching procedures --- neighborhood sketching and subgraph sketching, and establish that these procedures control FWER, while being computationally efficient. They supplement their theoretical investigations with simulation experiments and carry out a real data analysis.   The Pros:  1. The article attempts to bring together two disparate lines of research--- developments in minimax global testing and multiple testing communities in Statistics on one hand, and Property testing problems in the theoretical computer science literature on the other. I feel this line of research would be of interest to the community, and could lead to interesting future developments.   2. The authors carry out detailed simulation experiments and real data analyses to support their methods. I feel this goes a long way in making their results applicable in real problems.   The Cons.  1. The authors argue that testing for combinatorial structures in Gaussian graphical models is a central problem in many applications. While they provide some support for the max-clique problem in Section 5, they do not motivate the other problems studied in this article. I suggest the authors provide some other potential applications for the problems introduced.   2. I was left very confused by some conventions in this article--- principally due to concurrent use of terminology used in multiple testing and property testing literatures. For example, L82 introduces the general schematic for the problems to be studied, but the K-colorable problem (L137) flips the two hypotheses. Coupled with this, in Algorithm 1, the decision to ACCEPT corresponds to rejection of the null hypothesis in traditional statistics literature. This convention makes it very hard to read some sections of the paper. While I understand that the paper is interdisciplinary and that it is hard to explain all conventions due to space constraints, it would be very helpful if the authors could include some pointers for the benefit of the readers.   3. The paper would greatly improve with significant editing--- for example,  (i) L-26-- Hypothesis testing literature, L-48 --- Property testing literature, L-114- bounded degree etc.  (ii) The Section heading for Section 3 should be ``Statistical Property"  (iii) L79 -- an edge is present if $\Theta_{ij} \neq 0$.   In conclusion, I believe that the article presents the synthesis of notions introduced in two very distinct research areas, and introduces an interesting line of research for future enquiry. However, the current submission is sometimes hard to read, and the presentation could be significantly improved.   The authors have addressed the questions raised above satisfactorily.


Evolutionary Stochastic Gradient Descent for Optimization of Deep Neural Networks
We propose a population-based Evolutionary Stochastic Gradient Descent (ESGD) framework for optimizing deep neural networks. ESGD combines SGD and gradient-free evolutionary algorithms as complementary algorithms in one framework in which the optimization alternates between the SGD step and evolution step to improve the average fitness of the population. With a back-off strategy in the SGD step and an elitist strategy in the evolution step, it guarantees that the best fitness in the population will never degrade. In addition, individuals in the population optimized with various SGD-based optimizers using distinct hyperparameters in the SGD step are considered as competing species in a coevolution setting such that the complementarity of the optimizers is also taken into account. The effectiveness of ESGD is demonstrated across multiple applications including speech recognition, image recognition and language modeling, using networks with a variety of deep architectures.

The paper combines ES and SGD in a complementary way, not viewing as an alternative to each other. More intuitively, in the course of optimization, the author proposes to use evolutionary strategy to make optimiser adjust at different sections (geometry) of optimisation path. Since the geometry of the loss surface can differ drastically at different locations, it is reasonable to use different gradient based optimisers with different hyper parameters at different locations.  Then the main question becomes how to choose a proper optimiser at different locations. The paper proposes ES for that.   In terms of writing, the paper is readable and easy to follow. Simple comments for better readability are as follow. It would be better if brief explanation about ‘roullette wheel selection’ is given to be more self-contained (in line 138). It seems that psi in lines 145~147 should have superscript (k), the similar ambiguity at the last line of the algorithm1.  ***Related works The paper ‘Population Based Training of Neural Networks (PBT)’ uses the similar method as ESGD. Both uses ES but PBT aims to optimise hyperparameters of neural network while ESGD tries to optimise parameters of neural network. However, it is also possible to view that ESGD optimizes specific hyperparameters (different optimizers at different sections in the course of optimization). Moreover, it looks like that PBT can basically do the similar thing as ESGD with some minor modifications. We can try to optimise optimisers at different sections of the course of optimization as a hyperparameter using PBT.  Since in PBT, the under performing ones are replaced by well performing one together with their parameters, thus evaluation and mutation of parameters in ESGD can be done similarly in PBT. It appears that this work is a special case of PBT.  We hope that the author make the contribution over PBT more clearly. One suggestion is to compare PBT and ESGD to show the benefit of the specific choice of ES in ESGD.  ***Back-off This concept and related theorem is one thing which can differentiate ESGD from PBT. However, back off looks like a commonly used heuristic which is just using the best value in the course of optimization. The statement of the theorem is quite trivial because it basically keeps the improved value. Moreover, it seems that this theorem does not have strong practical implication since the author uses stochastic back off in one experiment, which breaks continuous improvement.  ***Robustness to the choice of ESGD hyperparameters In experiments, the configuration of a couple of ESGD hyperparameters varies for different data set. Pool of optimizers in SGD step - (in CIFAR10 - only using SGD) Back-off - in LM, stochastic back-off with probability 0.7)  Even though a fair amount of engineering hyperparameters is indispensible for a well performing ES, a question is whether ESGD is robust with respect to ESGD hyperparameters or consistent ESGD hyperparameter can be recommended. Some experiment showing the influence of ESGD hyperparameters can be given.  ***Using larger pool of optimizers in SGD step? Since various optimizers like AdaGrad, AdaDelta, RMSProp etc. behave differently, the geometry on which they can optimise well can be different and various optimiser can take care of various geometry. Contrast to the fact that, in CIFAR10, ESGD works better only with SGD, a naive conjecture is that the performance can be improved if various of optimizers are considered to tackle various geometry. If the author has not tried ESGD with various optimizers, it would be interesting to see the result with various optimiser. Or if it has been tried already, then the explanation or guideline to choose pool of optimiser can be beneficial.   ***Including random initialisation to mutation step can improve ESGD? In the proposed algorithm, random initialisation is not used and descendants are selected from parent generation with added perturbation, so there is no chance that strong mutation can happen. Can this strong mutation, which is similar to large perturbation, improve the performance?


Single-Agent Policy Tree Search With Guarantees
We introduce two novel tree search algorithms that use a policy to guide search. The first algorithm is a best-first enumeration that uses a cost function that allows us to prove an upper bound on the number of nodes to be expanded before reaching a goal state. We show that this best-first algorithm is particularly well suited for "needle-in-a-haystack" problems. The second algorithm is based on sampling and we prove an upper bound on the expected number of nodes it expands before reaching a set of goal states. We show that this algorithm is better suited for problems where many paths lead to a goal. We validate these tree search algorithms on 1,000 computer-generated levels of Sokoban, where the policy used to guide the search comes from a neural network trained using A3C. Our results show that the policy tree search algorithms we introduce are competitive with a state-of-the-art domain-independent planner that uses heuristic search.

The paper presents two planning algorithms based on tree search. The novelty of these algorithms is the use of a policy based criterion instead of a standard heuristic to guide the search. The first algorithm (LevinTS) uses a cost function d(n)/\pi(n) which ensures nodes are expanded in a best-first order. This allows the authors to derive an upper bound to the number of expansions performed before to reach a target node. The second algorithm (LubyTS) is a randomized algorithm based on the sampling of trajectories. Using a specific restarting strategy, the authors derived an upper-bound to the expected number of expansions before reaching a target state. Both the algorithms are analyzed under the assumption of a deterministic environment.  Comments ------------ The paper is overall well written and easy to follow for the part concerning the algorithms. What is less clear to me is the context and motivations.   - This paper is about planning given a baseline policy. The main issue I see here is that if the provided policy has zero or small probability of reaching the goal, the algorithms will fail. In the case of non-zero (small) probability, LevinTS is still able to find an optimal solution but with a very large number of node expansions (see Thm 3). I suppose that this is the reason why you pre-trained the policy using RL in the experiments (this allows overcoming the issue). This is a strong weakness of the proposed approach. Model-based algorithms (LevinTS is model-based) for planning do not have such requirements. On the other hand, if the goal is to refine a policy at the end of some optimization procedure I understand the choice of using a policy-guided heuristic.   - Concerning LubyTS it is hard to quantify the meaning of the bound in Thm. 6 (the easy part is to see when it fails, as mentioned before). There are other approaches based on generative models with guarantees, e.g., (Grill, Valko, Munos. Blazing the trails before beating the path: Sample-efficient Monte-Carlo planning, NIPS 2016). How do you perform compared to them?  - Your setting is very specific: you need to know the model or/and have access to a generative model (for expanding or generating trajectories),  the problem should be episodic and the reward should be given just at the end of a task (i.e., reaching the target goal). Can you extend this approach to more general settings?   - In this paper, you perform planning offline since you use a model-based approach (e.g., generative model). Is it possible to remove the assumption of the knowledge of the model?  In that case, you would have to interact with the environment trying to minimize the number of times a "bad" state is visited. Since the expansion of a node can be seen as an exploratory step, this approach seems to be related to the exploration-exploitation dilemma. Bounding the number of expansions does not correspond to having small regret in general settings. Can you integrate a concept related to the long-term performance in the search strategy? This is what is often done in MCTS.  All the proposed approaches have weaknesses that are partially acknowledged by the authors. A few points in the paper can be discussed in more details and clarified but I believe it is a nice contribution overall.  -------- after feedback I thank you for the feedback. In particular, I appreciated the way you addressed the zero probability issue. I think that this is a relevant aspect of the proposed approaches and should be addressed in the paper. Another topic should be added to the paper is the comparison with Trailblazer and other MCTS algorithms. Despite that, I personally believe that the main limitation is the assumption of deterministic transitions. While the approach can be extended to stochastic (known) models I would like to know how the theoretical results become.


Robust Learning of Fixed-Structure Bayesian Networks
We investigate the problem of learning Bayesian networks in a robust model where an -fraction of the samples are adversarially corrupted. In this work, we study the fully observable discrete case where the structure of the network is given. Even in this basic setting, previous learning algorithms either run in exponential time or lose dimension-dependent factors in their error guarantees. We provide the first computationally efficient robust learning algorithm for this problem with dimension-independent error guarantees. Our algorithm has near-optimal sample complexity, runs in polynomial time, and achieves error that scales nearly-linearly with the fraction of adversarially corrupted samples. Finally, we show on both synthetic and semi-synthetic data that our algorithm performs well in practice.

I preface this by saying that I have reviewed this paper once for NIPS 2016, and re-read it. It seems the paper has no essential changes, so my opinion is largely the same.   The paper considers the problem of learning the parameters of a Bayes net with known structure, given samples from it with potentially adversarial noise. The main goal is to get bounds on the samples that are independent of dimension. The main requirements on the Bayes net parameters are reasonable: the probability of any configuration of the parents is reasonable and the conditional probabilities on any edge are bounded away from 0 and 1.   The algorithm is quite similar morally to [DKK+16] -- on a high level it designs a quantity F(X,q), s.t. E[F(X,p)] = p, where p is the conditional probability "table" specified by the Bayes net and q is the empirical conditional probability table. The covariance matrix of F(X,q) on the other hand, "catches" the adversarial noise: if the covariance has a large eigenvalue, one can "filter" the "culprit" samples, and repeat the process. Otherwise, the empirical estimate is close in TV distance.    The paper is well-written, though the proofs are straightforward and natural (and largely follow the structure of [DKK+16]). There are some new tricks (e.g. how one "fills in" F(X,q) in the unobserved spots; why the mean of F(X,p) is diagonal; and how to use the spikes in the spectrum to produce a filter), but I feel like it's a poor fit for NIPS for two reasons.    For one, the techniques are quite similar to [DKK+16];  For another, it may be of interest to a relatively specialized crowd, both in terms of theoretical tools and practical algorithms.   As mentioned a previous time I reviewed this paper: the more interesting question is structure-learning Bayes nets, even under "benign" noise assumptions.  _______________________ Edit after rebuttal: after the rebuttal/discussions with other reviewers, my opinion is largely the same, though I may have scored a bit harshly. Correspondingly, I raised my score up by a point.


Support Recovery for Orthogonal Matching Pursuit: Upper and Lower bounds
We study the problem of sparse regression where the goal is to learn a sparse vector that best optimizes a given objective function. Under the assumption that the objective function satisfies restricted strong convexity (RSC), we analyze orthogonal matching pursuit (OMP), a greedy algorithm that is used heavily in applications, and obtain a support recovery result as well as a tight generalization error bound for the OMP estimator. Further, we show a lower bound for OMP, demonstrating that both our results on support recovery and generalization error are tight up to logarithmic factors. To the best of our knowledge, these are the first such tight upper and lower bounds for any sparse regression algorithm under the RSC assumption.

##Summary##  The paper studies the support recovery performance of orthogonal matching pursuit, where the goal is to estimate the position of the non-zero elements of a high-dimensional signal from its compressed measurements. While results under the L_2 metric are well-known, and sharp RIP conditions have been established, there are still some theoretical gaps in understanding support recovery: what is the sharp condition? The work follows this line and present lower and upper bounds under natural assumprtions.  ##Detailed Comments##  I am satisfied with most of the discussion in the paper. Below are a few comments that I hope the authors could address in the feedback.  - I am curious how did the authors obtain better dependence on the condition number than [20,21,25]. Is it possible to sketch the main technique/idea right after Remark 3?  - The relaxed sparsity idea for OMP was used in [28]. What is the major difference between Theorem 3.1 and [28]?  - line 244: I did not really follow why noise helps recovery. In 1-bit matrix completion, noise does help distinguish the model. But what is the intuition here?  - The equation numbering is way strange. Authors need to improve the typeset in the revision. Authors should also keep the style of bib entries consistent.  ---Updates After Author Feedback---  Authors addressed my concerns and I feel this is a nice work. In particular, authors draw more careful analysis on the progress of OMP, and tightened the old bounds [28] that have been used for a decade. Personally I like their theoretical results. I hope they will sketch the proof technique and add more comparison to [20,21,25,28] in the main body, say list the bounds in a table. I believe it is a significant contribution for the community, and I increase my rating from "7" to "8".


Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs
The loss functions of deep neural networks are complex and their geometric properties are not well understood. We show that the optima of these complex loss functions are in fact connected by simple curves over which training and test accuracy are nearly constant. We introduce a training procedure to discover these high-accuracy pathways between modes. Inspired by this new geometric insight, we also propose a new ensembling method entitled Fast Geometric Ensembling (FGE). Using FGE we can train high-performing ensembles in the time required to train a single model. We achieve improved performance compared to the recent state-of-the-art Snapshot Ensembles, on CIFAR-10, CIFAR-100, and ImageNet.

Update after author response:  Thank you for the response. Additional details that the curves between the local optima are not unique would be also interesting to see. Nice work!  Summary: This paper first shows a very interesting finding on the loss surfaces of deep neural nets, and then presents a new ensembling method called Fast Geometric Ensembling (FGE). Given two already well trained deep neural nets (with no limitations on their architectures, apparently), we have two sets of weight vectors w1 and w2 (in a very high-dimensional space). This paper states a (surprising) fact that for given two weights w1 and w2, we can (always?) find a connecting path between w1 and w2 where the training and test accuracy is nearly constant. Figure 1 demonstrates this, and Left is the training accuracy plot on the 2D subspace passing independent weights w1, w2, w3 of ResNet-164 (from different random starts); whereas Middle and Right are the 2D subspace passing independent weights w1, w2 and one bend point w3 on the curve (Middle: Bezier, Right: Polygonal chain). Learning the curves with one bend w3 for given two weights w1, w2 is based on minimizing the expected loss over a uniform distribution on the curve with respect to w3. Investigating the test accuracy of ensembles of w1 and w(t) on the curve by changing t suggests that even the ensemble of w1 and w(t) close to w1 have better accuracy than each of them. So by perturbing w1 around w1 by gradient descent with cyclical learning rate scheduling, we can collect weights at lower learning rate points, and the ensemble of these weights (FGE) have much better accuracy. The experiments also provide enough evidences that this idea work using a set of the-state-of-the-art architectures such as ResNet-110 and WRN-28-10 as well as more standard VGG-16.  Strength: - The finding that the loss surfaces have the connecting curves between any converged weights w1 and w2 where the training and test accuracy is nearly constant is very interesting. This phenomena is presented clearly with some illustrative examples for the modern CNN architectures such as ResNet and WRN.   - It is also interesting to confirm how far we need to move along a connecting curve to find a point nice to be mixed into the ensemble. This well motivates a proposed ensembling method FGE.   - Compared to the other ensembling such as Snapshot Ensembles [9], the proposed FGE would be nicely presented, and shows good empirical performance.  Weakness: - The uniqueness of connecting curves between two weights would be unclear, and there might be a gap between the curve and FGE. A natural question would be, for example, if we run the curve findings several times, we will see many different curves? Or, those curves would be nearly unique?   - The evidences are basically empirical, and it would be nice if we have some supportive explanations on why this curve happens (and whether it always happens).  - The connections of the curve finding (the first part) and FGE (the second part) would be rather weak. When I read the first part and the title, I imagined that take random weights, learn curves between weights, and find nice wights to be mixed into the final ensemble, but it was not like that. (this can work, but also computationally demanding)   Comment: - Overall I liked the paper even though the evidences are empirical. It was fun to read. The reported phenomena are quite mysterious, and interesting enough to inspire some subsequent research.  - To be honest, I'm not sure the first curve-finding part explains well why the FGE work. The cyclical learning rate scheduling would perturb the weight around the initial converged weight, but it cannot guarantee that weight is changing along the curve described in the first part.