Data-dependent PAC-Bayes priors via differential privacy
The Probably Approximately Correct (PAC) Bayes framework (McAllester, 1999) can incorporate knowledge about the learning algorithm and (data) distribution through the use of distribution-dependent priors, yielding tighter generalization bounds on data-dependent posteriors. Using this flexibility, however, is difficult, especially when the data distribution is presumed to be unknown. We show how an e-differentially private data-dependent prior yields a valid PAC-Bayes bound, and then show how non-private mechanisms for choosing priors can also yield generalization bounds. As an application of this result, we show that a Gaussian prior mean chosen via stochastic gradient Langevin dynamics (SGLD; Welling and Teh, 2011) leads to a valid PAC-Bayes bound given control of the 2-Wasserstein distance to an e-differentially private stationary distribution. We study our datadependent bounds empirically, and show that they can be nonvacuous even when other distribution-dependent bounds are vacuous.

%% Summary %%  This paper develops new PAC-Bayes bounds with data-dependent priors. Distribution-dependent PAC-Bayesian bounds (especially ones with distribution-dependent priors) are by now well-known. However, as far as I am aware, the authors' bound is the first non-trivial PAC-Bayesian bound where the prior is allowed to depend on the data. The key idea to accomplish this is the observation that a differentially private prior can be related to a prior that does not depend on the data, thereby allowing (as the authors do in their first main result) the development of a PAC-Bayesian bound that uses a data-dependent-yet-differentially-private prior based on the standard one that uses a data-independent prior. The proof of the new bound depends on the connection between max information and differential privacy (versions of the latter imply bounds on versions of the former). A second key result of the authors, more relevant from the computational perspective, is that if a differentially private prior is close in 2-Wasserstein distance to some other (not necessarily differentially private) prior, then a PAC-Bayesian bound can again be developed using this latter data-dependent-but-not-differentially-private prior, where we pick up some additional error according to the 2-Wasserstein distance. This result allows the authors to leverage a connection between stochastic gradient Langevin dynamics (which is computationally friendly) and Gibbs distributions (which are differentially private but computational nightmares).  In addition to the above bounds, the authors also perform an empirical study. I have to admit that I focused more on the theoretical guarantees, as I believe they already are interesting enough to warrant publication of the paper. Also, the authors did a poor job in writing the Section 6, leaving vital details in the appendix, like figures which they constantly refer to. I feel that this was not in the spirit of abiding by the 8 page limit, as the paper was not self-contained as a result. This also was confusing, as the authors clearly had extra space throughout the paper to include these two figures.   %% Reviewer's Expertise **  I am an expert in PAC-Bayesian inequalities and also well-versed in recent developments related to differential privacy. I am less familiar with stochastic gradient Langevin dynamics but know the basics.    %% Detailed Comments %%  Aside from Section 6, I found this paper to be remarkably well-written and clear. I especially appreciated the key insight in the paragraph immediately after equation (1.1), for why the Lever et al. bound often must be vacuous for large values of $\tau$.  I believe that Theorem 4.2 is a very interesting (and not so complicated) result which really makes for a significant and original contribution. It should open up future directions in developing new PAC-Bayesian guarantees, and so this result alone I feel makes a compelling argument for accepting this paper. In addition, Theorem 5.3, which allows us to only require a prior that is close in 2-Wasserstein distance to a differentially private one, further broadens the PAC-Bayesian toolbox in a significant and original way. That said, Theorem 5.3 has a major weakness, which is that it forces us to give up on obtaining ``very high probability'' bounds, since (5.3) has a term which grows as $1/\delta'$ as the failure probability $\delta'$ decreases. This is a crucial weakness which the authors should mention after Theorem 5.3, including discussion of whether or not this issue is fundamental.  I looked through the proof of the main results Theorems 4.2 and 5.3 and I believe the analysis is sound. I do think the authors made a typo in the definition of $g$ in Theorem 5.3; you should remove the negative sign (indeed the logarithm in (5.4) is not well-defined if the negative sign stays!). I would have liked a more useful/detailed version of what is currently Corollary 5.4. It currently glosses over too many details to really say much. Since you have extra space in the paper, I recommend including a more explicit version of this corollary. You should also include around this point a citation to the Raginsky / Rakhlin / Telgarsky (2017) paper that you refer to in the appendix. I also recommend giving, at least in the appendix, a precise citation of which result from their paper you are using.     Minor comments:  You never explain the notation for the squiggly arrow in the main paper. Please fix this. It is too important / widely used in the main text to be left to the appendix.  On page 4, line 2, you say ``the above result''. I think you are referring to Definition 3.1, which is of course a definition, not a result. So, you should change the text accordingly.  In Theorem 3.2, you have one instance of $n$ which should be replaced by $m$ (see the inline math following the word ``Then'')  In the proof of Lemma D.1, you should mention that the upper bound of total variation by KL divergence is from Pinsker's inequality. ``Standard results'' is a bit vague to inform the uninformed reader.   %% UPDATE AFTER AUTHOR'S RESPONSE %%  I've read the author's rebuttal and their responses are satisfactory. I do hope you will highlight the weakness of the high probability bound involving the 1/\delta' dependence.


Identification and Estimation Of Causal Effects from Dependent Data
The assumption that data samples are independent and identically distributed (iid) is standard in many areas of statistics and machine learning. Nevertheless, in some settings, such as social networks, infectious disease modeling, and reasoning with spatial and temporal data, this assumption is false. An extensive literature exists on making causal inferences under the iid assumption [17,11,26,21], even when unobserved confounding bias may be present. But, as pointed out in [19], causal inference in non-iid contexts is challenging due to the presence of both unobserved confounding and data dependence. In this paper we develop a general theory describing when causal inferences are possible in such scenarios. We use segregated graphs [20], a generalization of latent projection mixed graphs [28], to represent causal models of this type and provide a complete algorithm for nonparametric identification in these models. We then demonstrate how statistical inference may be performed on causal parameters identified by this algorithm. In particular, we consider cases where only a single sample is available for parts of the model due to full interference, i.e., all units are pathwise dependent and neighbors' treatments affect each others' outcomes [24]. We apply these techniques to a synthetic data set which considers users sharing fake news articles given the structure of their social network, user activity levels, and baseline demographics and socioeconomic covariates.

The work presented in the paper is clearly of value.  The existing theory for identification and estimation of causal parameters in DAGs for IID data has been central to our understanding of causal inference, and developing analogous results for data under interference would be useful both to apply directly to data in which we know interference occurs and to better understand the potential impacts of violations of the IID assumption.  While the paper should be accepted, the current version could be substantially improved in both organization and in its discussion of several key issues, including generality, assumptions, temporal effects, and prior work.  Organization and Presentation  Some aspects of the organization make the paper challenging for readers.  Some sections do not provide a “roadmap” to the basic logic before plunging into the details, others do not present a high-level intuition for why a given theoretical result is being presented,  The entire paper would be substantially improved if the authors provided readers with a high-level roadmap to the overall reasoning of the paper, making clear the basic logic that allows an identification theory to be developed under interference (before plunging into the details of sections 2 and 3).  As I understand it, the outline of that logic is: 1) Represent models as latent variable chain graphs, 2) Divide the model into blocks, 3) Assume no interference between blocks, 4) Express identification theory by using truncated nested factorization of latent projection ADMGs.  This logic differs from the current abstract because it helps readers see how the paper achieves what it does, rather than just what the paper achieves.  The paper provides a great deal of theory with relatively little intuition to help readers understand the underlying concepts.  Such intuition would be relatively easy to provide.  For example, the fixing operation will be novel to most readers, and the paper would be improved by providing an intuition for what fixing corresponds to (if such an intution exists).  The intuition for conditioning, for example, is often presented in terms of partitioning of possible worlds.  If no such intuition exists, then it would be worth explaining up front that it is only a method for producing a nested factorization of an ADMG (or even relegating this to the supplemental material).  The abtract implies that unobserved confounding bias is a necessary (or at least likely) consequence of non-iid data, but the paper doesn’t make clear why this is so.  A quick description would be useful, so that readers would not have to consult outside papers, such as reference 14.  In contrast, Section 1 merely invokes latent confounding as a likely situation that appears in conjunction with interference.  More clarity on this point would help readers.  Generality  It is very difficult to assess the generality of the results presented in the paper.  Specifically, the details of the motivating example provided in Section 2 makes the results appear very narrow, but the more expansive langauge in the introduction and conclusion make the results seem much more general (“…complete identification theory, and estimation for dependent data.”).  Specifically, the work presented here applies to data consisting of a single object type and stable, symmetric relationships, such as friendship ties.  This is worth emphasizing in the introduction, because interference can occur in cases in which asymmetric relationships also occur (such as social network data, communications data, and citation data) and where multiple types of objects occur.  Assumptions  The method outlined in the paper appears to rely strongly on the presence of blocks (maximal sets of vertices in the model which each pair is connected by an undirected path) that lack interference.  This seems unlikely in many reasonable scenarios.  Interblock non-interference may be a common assumption, but that doesn’t mean that it is realistic.  The authors should either provide evidence that this is a plausible assumption in real applications or provide evidence that moderate violations of this assumption do not greatly affect the results.  Section 2 states that “a crucial assumption in our example is that…community membership does not directly influence sharing, except as mediated by social network activity of the user.”  This seems a moderately strong assumption, and it would be good to expand somewhat on both the likelihood of being able to gather data that meet such an assumption and the reasons why this assumption is made.  Temporal effects  The motivating example (and other plausible examples) seems extremely difficult to analyze without explicit consideration of the time-series of behavior.  This is not just a simple statement that temporal information aids causal identification.  Instead, the point here is that the system described almost certainly has substantial feedback effects and time-varying exogenous influences, and it may never reach any form of equilibrium.  Thus, it is unclear what a data sample corresponds to here.  The paper would be improved greatly by describing in more detail the underlying assumptions of data collection.  Prior work  The paper ignores a moderately large literature on inferring causal models of non-iid data that has appeared in the graphical models literature over the past decade.  Relevant papers include:  Marc Maier (2013). Reasoning about independence in probabilistic models of relational data. arXiv:1302.4381.  Lee & Honavar (2016). On learning causal models from relational data. AAAI 2016.  Maier et al. (2013). A sound and complete algorithm for learning causal models from relational data.  UAI 2013.  Arbour et al. (2016). Inferring network effects from observational data. KDD 2016.  These papers do not subsume or contradict the current paper, but the current paper would be improved by contrasting the assumptions of these papers with the approach taken here.  Summarize the strengths and weaknesses of the submission, focusing on each of the following four criteria: quality, clarity, originality, and significance.


REFUEL: Exploring Sparse Features in Deep Reinforcement Learning for Fast Disease Diagnosis
This paper proposes REFUEL, a reinforcement learning method with two techniques: reward shaping and feature rebuilding, to improve the performance of online symptom checking for disease diagnosis. Reward shaping can guide the search of policy towards better directions. Feature rebuilding can guide the agent to learn correlations between features. Together, they can find symptom queries that can yield positive responses from a patient with high probability. Experimental results justify that the two techniques in REFUEL allow the symptom checker to identify the disease more rapidly and accurately.

Dear Authors. Thank you for your response. I still have concerns about your explanation of the discount factor. I can clarify that your policies with gamma=1 shown in rebuttal in Figures 2 and 3 don't ask more questions because your final reward (+1 for correct prediction) is too small. If the final reward was larger (you would need to tune it or just set it to a "very" large value), the agent would ask more questions without a problem. This means that gamma=1 would be sufficient to solve this problem. If the neural network requires bounding rewards, all the rewards could be scaled down, and the step penalty would be proportionally smaller than -1. As I said in my original review, your paper is a nice and well-written work, but the discount factor is problematic in this paper, and I feel that I should not argue for acceptance, although I would not be upset if the paper was accepted. If your paper was a purely applied paper, I could support it, but since you are claiming technical contributions, you need to provide convincing arguments that they are necessary. This is something that is missing in your paper.  -----  A medial application is addressed using reinforcement learning. The goal of this work is to compute a policy that will detect a disease, minimising the number of features tested. The authors introduce two distinct algorithmic contributions. The first one shows how reward shaping can be applied to this domain. The second contribution is an extended neural network for the policy gradient method that predicts the unobserved features in a dedicated output layer. The experimental results on simulated data indicate that the baseline method combined with reward shaping and learning with additional output layer lead to the best performance with respect to both convergence speed and the quality of the final policy.  This paper presents good research, but I am not sure if it can be accepted in this form.  Writing is very good in general, but the introduction fails to provide a clear problem statement. For example, the term "query budget" is not explained in the introduction, but budgeted learning in machine learning often assumes fixed budget, and it is not the case in this paper. A precise definition of budget is required because now, after reading section 1, it is not clear what the real, technical challenge is.  The authors cite tree-based greedy methods. I believe that to demonstrate benefits of reinforcement learning, it would be useful if the authors could add results of such approaches. Since the most simple baseline is quite weak, I would argue that the greedy methods can beat it.  Between lines 71 and 72, the uniform distribution is used twice for the initial state. What is the reason for that? Does it mean that the initial state is randomly sampled? If so, this is probably to deal with the exploration problem?  Considering the existing reinforcement learning literature, the reward defined in lines 79-80 is very unusual. To punish the agent for asking too many questions, reward of -1 should be given for every query. With this, the problem becomes a standard stochastic-shortest path problem. With that, the analysis in Fig. 4 would not be required because the alg. would be penalised for repeated queries.  The dual neural network shown in Fig. 1 is interesting and it is in line with the current developments in various applications of deep learning. This is probably the strongest contribution in this paper. The authors were quite clever here, and intuitively, this idea makes a lot of sense in this application. The authors could provide a better explanation of this design because I am pretty sure that people who have not seen this trick in other deep learning papers will be confused.  Theorem 2 indicates that the authors were clever observing the relationship between the discount factor and the absolute values of the potential function. However, this problem was previously discussed in "Marek Grzes, Daniel Kudenko: Theoretical and Empirical Analysis of Reward Shaping in Reinforcement Learning. ICMLA 2009: 337-344". It seems to me that the authors have a more general approach here than the past reference.  Even though Theorem 2 and the past work on the same problem show an important problem related to the discount factor, I don't think that the authors of this paper have to worry about this problem. The authors could set the discount factor gamma to 1, and then the entire investigation in Theorem 2 would not be necessary. Right now, Theorem 2 can  be used to determine the discount factor (which is a bit weird because Rich Sutton would surely argue that the discount factor is part of the domain), so why not to set it to the value of 1. The problem is episodic so the value of 1 would be justified or even preferred. This particular observation makes me believe that Theorem 2 represents a contribution that is not fully justified in this particular paper.  The potential function used in this work is informative, and I am not surprised that reward shaping leads to better performance.  This model assumes that every patient suffers from at most one disease. If two diseases were present, the algorithm would identify incorrect correlations. The authors should explain that in the paper.  In Tab. 1 and 3, the baseline has fewer steps than the RE^2. Why is that? This is probably related to my comment about stochastic-shortest paths that is above.  The paper is interesting, but I am not sure if the presentation and contributions are good enough to guarantee acceptance.


The Global Anchor Method for Quantifying Linguistic Shifts and Domain Adaptation
Language is dynamic, constantly evolving and adapting with respect to time, domain or topic. The adaptability of language is an active research area, where researchers discover social, cultural and domain-specific changes in language using distributional tools such as word embeddings. In this paper, we introduce the global anchor method for detecting corpus-level language shifts. We show both theoretically and empirically that the global anchor method is equivalent to the alignment method, a widely-used method for comparing word embeddings, in terms of detecting corpus-level language shifts. Despite their equivalence in terms of detection abilities, we demonstrate that the global anchor method is superior in terms of applicability as it can compare embeddings of different dimensionalities. Furthermore, the global anchor method has implementation and parallelization advantages. We show that the global anchor method reveals fine structures in the evolution of language and domain adaptation. When combined with the graph Laplacian technique, the global anchor method recovers the evolution trajectory and domain clustering of disparate text corpora.

After the author response: I'm upvoting the paper. Looking forward to seeing experiment extensions in the future.  Summary: The paper proposes a "global anchor method" for detecting corpus-level language shifts as an alternative to the more common "alignment method", both leveraging distributed word representations. A proof is provided for the theoretical equivalence of the two methods, but the advantages of the former are highlighted as it allows to straightforwardly compare word embeddings of different dimensionality, along with some downstream advantages in uncovering the finer details of language shift.  In my view, this is a very nice, albeit not well-rounded contribution, where the method exposition gets slightly more prominence than the experiments.  Still, I vote accept: it is a strong, important contribution to the body of work in uncovering language shift and evolution.  Strengths: - Very clear exposition, especially the motivation and the method equivalence proof. - The method is clean and simple, yet expressive. - The experiments are for the most part convincing.  Weaknesses: - With limited space, the paper leaves quite some to wish for in terms of demonstrating the method application in actually uncovering (sets of interesting examples of) linguistic shift. Related work does a much better job at that, and the appendix of this draft does not contribute in this direction. A lot of space is dedicated to method equivalence, which is admirable, but it also shifts the weight of the contribution towards the first part of the writeup.  - Along similar lines, only Google Books and arXiv are used, while related work does more. Altogether this indicates a slightly unbalanced contribution, where the empirical work slightly suffers.  Questions: - Isn't the treatment of the "alignment method" slightly unfair with respect to dimensionality? If the corpora are available for building the embeddings, it seems a triviality to make the embeddings dimensions uniform.


Adaptive Sampling Towards Fast Graph Representation Learning
Graph Convolutional Networks (GCNs) have become a crucial tool on learning representations of graph vertices. The main challenge of adapting GCNs on largescale graphs is the scalability issue that it incurs heavy cost both in computation and memory due to the uncontrollable neighborhood expansion across layers. In this paper, we accelerate the training of GCNs through developing an adaptive layer-wise sampling method. By constructing the network layer by layer in a top-down passway, we sample the lower layer conditioned on the top one, where the sampled neighborhoods are shared by different parent nodes and the over expansion is avoided owing to the fixed-size sampling. More importantly, the proposed sampler is adaptive and applicable for explicit variance reduction, which in turn enhances the training of our method. Furthermore, we propose a novel and economical approach to promote the message passing over distant nodes by applying skip connections. Intensive experiments on several benchmarks verify the effectiveness of our method regarding the classification accuracy while enjoying faster convergence speed.

In this paper, the authors solve the scalability issue of GCNs by developing a layer-by-layer sampling method for network construction. There are two benefits for this: first, it enables the neighborhood sharing in upper layers; second, the parameters of the sampler and the network can be finetuned to reduce the variance explicitly.  Moreover, to enable message passing across distant nodes, this paper proposes to apply skip connections to reuse to nodes in the (l-1)-th layer as the 2-hop neighborhoods of the (l+1)-th layer. Finally, remarkable improvements in accuracy and speedups are demonstrated for the proposed methods on four benchmarks.    1) Clarity: This is a Well-written paper. It is well organized and easy to understand.  2) Novelty: Compared to GraphSage and FastGCN that have been developed previously, the layer-wise sampling scheme by this paper is novel and efficient. Also, the idea of applying skip-connections to preserve second-order proximity is interesting.   3) Significance: Experimental evaluations verify the fast convergence speed of the proposed method compared to other sampling counter-parts. The importance of the variance reduction is also justified by Table 1.    4) Minor things: - Line 252 in the paper says the window-size of early stopping is 10, while in the supplementary the number becomes 30 in line 22. Which is correct?  - To justify the importance of the variance reduction, the authors set \lambda=0. However, without the variance term, how can we train the parameters of the self-dependent function, since the gradient of the classification loss is always zero? - In Line 120, the computational complexity for Eq.(1) should be O(|E|D^lD^{l-1}), not O(N^2 D^lD^{l-1}).    In all, a rather thorough paper that derives an efficient way to perform convolutions on graphs using layer-wise sampling and skip-connections. At one hand, this leads to improvements over other competing methods in terms of accuracy and convergence speed. Hence, I recommend accepting this paper. ###################After Rebuttal########################### I think the authors have addressed my previous concerns, which make me not to change my mind and recommend the acceptance.


Learning to Decompose and Disentangle Representations for Video Prediction
Our goal is to predict future video frames given a sequence of input frames. Despite large amounts of video data, this remains a challenging task because of the highdimensionality of video frames. We address this challenge by proposing the Decompositional Disentangled Predictive Auto-Encoder (DDPAE), a framework that combines structured probabilistic models and deep networks to automatically (i) decompose the high-dimensional video that we aim to predict into components, and (ii) disentangle each component to have low-dimensional temporal dynamics that are easier to predict. Crucially, with an appropriately specified generative model of video frames, our DDPAE is able to learn both the latent decomposition and disentanglement without explicit supervision. For the Moving MNIST dataset, we show that DDPAE is able to recover the underlying components (individual digits) and disentanglement (appearance and location) as we intuitively would do. We further demonstrate that DDPAE can be applied to the Bouncing Balls dataset involving complex interactions between multiple objects to predict the video frame directly from the pixels and recover physical states without explicit supervision.

The paper addresses the problem of predicting future frames in videos from previously seen frames. This task has been gaining a lot on popularity due to it's applications in different areas [8, 22], but also because it aims at modeling the underlying video representations in terms of motion and appearance. The paper proposes a new Decompositional Disentangled Predictive Auto-Encoder (DDPAE), a framework that combines structured probabilistic models and deep networks to automatically (i) decompose the high-dimensional video that we aim to predict into components, and (ii) disentangle each component to have low-dimensional temporal dynamics that are easier to predict. The main novelty is the automatic decomposition of the video into components that are easier to predict, as well as the disentanglement of each component into static appearance and 2D motion.         Positive sides:       1. Well written and easy to read. The paper is well structured, the language maintains high standards and is easy to read.        2. Relevant and interesting task. The paper is tackling the problem of learning generative models of object dynamics and appearance in videos which is a challenging task with strong potential impact.        3. Novelty. Although the paper re-uses a lot of existing ideas and works, still the combination is novel and relevant to consider.        4. Experimental evaluation. The proposed model outperforms existing works with decomposition and disentanglement.         Negative sides:       1. Clarity. The paper is really hard to follow and fully understand. Curently the paper spends little time in explaining the relevant details of this work: how is the video decomposed? What is the initial starting point when the video starts? How do you set the number of components in the video? How do you learn the content part of the disentangled representation? What is the parameterization of the pose part? How do you predict the betas? Why using a seq2seq model? Is the order between components arbitrary? If not (suggested by Fig 2) then how do you choose the order at training and test time? How do you ensure that the components correspond to objects (digits and balls)? How does the inference work? It is hard to grasp such a comlex part from only 3 lines of text.        2. Experiments. The experiments are done only in a very controlled setup with small and toy datasets. While that is fine as a debugging step to understand the model, ultimately the paper should tackle real-world datasets and showcase potential usability in the real world. Most of the maade assumptions (constant latent part) is fine on these toy datasets, but it brakes in real life. It is easy to define a priori the number of components in these toy datasets, because it is very constrained. In the real world the objects have much more complex pose dynamics, especially non-rigid objects. The datasets considered have black backgrounds only, which is a significant downside. You might hope that the proposed model in the real world might allocate a single (or several) components to the background?    After reading the authors rebuttal, I remain with my decision to accept this paper. The rebuttal has been helpful to further understand the paper, which indicates that the paper is hard to thoroughly understand, however, the contribution is significant and this work will be definitely well accepted in the computer vision world.


TETRIS: TilE-matching the TRemendous Irregular Sparsity
Compressing neural networks by pruning weights with small magnitudes can significantly reduce the computation and storage cost. Although pruning makes the model smaller, it is difficult to get a practical speedup in modern computing platforms such as CPU and GPU due to the irregularity. Structural pruning has attracted a lot of research interest to make sparsity hardware-friendly. Increasing the sparsity granularity can lead to better hardware utilization, but it will compromise the sparsity for maintaining accuracy. In this work, we propose a novel method, TETRIS, to achieve both better hardware utilization and higher sparsity. Just like a tile-matching game 2 , we cluster the irregularly distributed weights with small value into structured groups by reordering the input/output dimension and structurally prune them. Results show that it can achieve comparable sparsity with the irregular element-wise pruning and demonstrate negligible accuracy loss. The experiments also show ideal speedup, which is proportional to the sparsity, on GPU platforms. Our proposed method provides a new solution toward algorithm and architecture co-optimization for accuracy-efficiency trade-off.

This paper deals with sparsifying the weights of a neural net to reduce memory requirements and speed up inference. Simply pruning small weights yields unstructured sparsity, which is hard to exploit with standard libraries and hardware. This paper imposes block sparsity, where each weight tensor is divided into fixed blocks (of size 32 x 32, for example) and non-zero weights are specified in only a fraction of the blocks. The paper's innovation is an iterative algorithm for reordering the rows and columns of a tensor to group together the large weights, reducing the accuracy loss from block pruning. Experiments on the VGG16 network for ImageNet show that this method achieves better speed-accuracy trade-offs than either unstructured weight pruning or block pruning without reordering.  Update: I appreciate the authors' responses. I've added some more comments in the body of my review, marked with the word "Update".  Summary of evaluation: The paper shows solid empirical results for pruning network weights in a way that doesn't lose much accuracy and actually speeds up inference, rather than slowing it down due to indexing overhead. The proposed reordering algorithm could use more analysis and could be better related to previous work. On balance, though, I think the paper is worth publishing.  Originality: The paper builds on a preprint by Narang et al. (2017) that proposes block-level pruning for neural nets without row/column reordering. There also seems to be a large literature in the computational linear algebra community on reordering rows and columns to make better use of sparsity. Examples of that work include:  P. R. Amestoy, T. A. Davis, I. S. Duff (1996) "An approximate minimum degree ordering algorithm". SIAM J. Matrix Analysis & Applic. 17(4):886-905.  A. Pinar, M. T. Heath (1999) "Improving performance of sparse matrix-vector multiplication". Proc. ACM/IEEE Conf. on Supercomputing.  Some of that literature is summarized with diagrams at https://www.mathworks.com/help/matlab/examples/sparse-matrices.html.  I didn't find any prior work tackling the exact problem being addressed here, but it would be helpful to consider how insights from that literature might be useful for the current problem. Update: The authors' response mentions that their algorithm was inspired by a 2010 paper (on a different problem) by Goreinov et al. I couldn't find a reference to that paper in the current manuscript; it would be good to include it.  Clarity: For the most part, the paper is easy to follow. A few points could be made clearer. First, it's hard to tell exactly what base pruning method was used in the experiments. Based on a comment at the end of the introduction (p. 2, line 54), I assume it's the method of Narang et al. (2017). It would be helpful to describe that method explicitly. It would also be helpful to say exactly what is meant by a block-sparse matrix here: a key point is that a regular grid of blocks is fixed ahead of time (if I understand correctly from the blocksparse library documentation), in contrast to, say, a general block-diagonal matrix, where the blocks can have arbitrary sizes and alignments. Update: I appreciate the discussion of these points in the response.  In the experiment section, Fig. 6 makes it easy to see the comparison to the unstructured-pruning baseline, but it's harder to compare to the baseline of block pruning without reordering -- I needed to cross-reference with the accuracy numbers in Fig. 5(b). It would be good to include the no-reordering curves in Fig. 6 as well, at least for 32x32 blocks.  Quality: The empirical results are good, showing a dramatic speed-up over the unstructured sparse baseline and a significant accuracy improvement compared to the baseline without reordering. The results are clearest for VGG16 on ImageNet. The paper also gives results for MNIST and CIFAR-10, suggesting that the improvement is not specific to one network. It would be even more persuasive to show wall time speed-up numbers for those networks as well (in addition to the sparsity-accuracy curves). Update: the authors' response reports 10x and 3x speed-ups for MNIST and CIFAR; it will be good to include those numbers in the paper.  The quality of the algorithm section is mixed. On one hand, the row/column reordering algorithm makes intuitive sense. It's interesting to see the optimizations in Section 4.2 (p. 5-6) that reduced the running time of reordering from over 2 hours to under 1 minute for the VGG16 model.  On the other hand, there's no discussion of consistency between the objective being optimized by the base pruning method P and the one being optimized by the reordering steps in the algorithm. It looks like the pruning method of Narang et al. (2017) minimizes the maximum magnitude among the pruned weights. Update: The authors' response includes the crucial fact that they switched to an L1 version of the Narang et al. pruning algorithm, minimizing the sum of absolute values of the pruned weights. This definitely needs to be stated in the paper.  It's less clear what the reordering steps are optimizing, because Eq. 4 (p. 4) uses norm notation on the matrix of pruned weights without specifying what norm is intended. Based on Eq. 7, which sums the absolute values of entries, it seems to be the l1 norm. (Update: the authors' response confirms this.) So the base pruning method is optimizing a max over weight magnitudes, while the reordering steps are optimizing a sum. Could the algorithm end up oscillating, with the two steps undoing each others' work? Update: The response clarifies that, in fact, both parts of the algorithm are optimizing a sum, so there's no risk of oscillation. The paper should mention this explicitly.  Conversely, if the base pruning objective is consistent with the reordering objective, is it really necessary to re-run pruning after the reordering steps? The reordering steps are taking the block choices from the previous pruning step and trying to make them as good as possible, so it's not clear whether another choice of blocks could end up even better. Update: The authors' response says that re-pruning is indeed necessary. It's obvious that an arbitrary reordering of rows and columns could necessitate re-pruning, but it's still not obvious to me that this can happen when both the reordering and the pruning are optimizing the L1 objective. Would it be possible to provide a small example where this happens, maybe in the Supplement?  Significance: There is a lot of interest in running neural nets on devices with limited memory and computation resources. This work seems to make a useful contribution on that front. It is complementary to other methods such as discretization of weights; I'm not sure if there are other sparsification techniques intended for standard hardware that should be included as additional baselines for comparison.


Automatic Program Synthesis of Long Programs with a Learned Garbage Collector
We consider the problem of generating automatic code given sample input-output pairs. We train a neural network to map from the current state and the outputs to the program's next statement. The neural network optimizes multiple tasks concurrently: the next operation out of a set of high level commands, the operands of the next statement, and which variables can be dropped from memory. Using our method we are able to create programs that are more than twice as long as existing state-of-the-art solutions, while improving the success rate for comparable lengths, and cutting the run-time by two orders of magnitude. Our code, including an implementation of various literature baselines, is publicly available at https: //github.com/amitz25/PCCoder

This paper presents an approach to use neural networks to guide the search of programs in a DSL to perform program synthesis from few input-output examples. In particular, it learns two networks f and g to predict the next statement and which variables to drop from the memory, given the current state of the program together with the desired output. During training, an auxiliary task of predicting operators (with lambda functions) is used to improve the training. The environment state is represented as a set of fixed dimensional embeddings, which are then passed through a dense block and pooled to obtain example order-independent representation. This representation is then used to compute three predictions: a multiclass statement prediction f, a multi-label binary classification g, and an auxiliary task of predicting statement operators and the lambda functions. The search is performed using the CAB extension for beam search that iteratively weakens the pruning heuristics to perform new beam search runs. The technique is evaluated on the DeepCoder dataset and is shown to significantly outperform DeepCoder and baselines both in terms of synthesis time and also in terms of the lengths of the programs that can be synthesized.  Overall, this paper presents a nice idea to use program states (dynamic environments) for predicting next functions. The technique is well-presented and a comprehensive evaluation is performed with many different ablation settings to evaluate the usefulness of different design choices. The results in comparison to previous techniques like DeepCoder are quite impressive and I also liked various ablation experiments.  The paper is mostly well written and clear, but the model description parts can be presented more clearly. The evaluation is described in detail with reasonable design choices. The idea of using dynamic state information and variable dropping for learning to search in synthesis is novel. The technique is evaluated on toy-ish integer benchmarks (unlike RobustFill or Karel with real-world synthesis benchmarks), but still can be useful for making progress on hard synthesis tasks.  Given the current progress in neural program synthesis approaches, DeepCoder is quite a weak baseline now. Note that DeepCoder trains only example embeddings and doesn’t train the search component, and relies on an external search. In particular, with systems like RobustFill, where one trains the search as well during training would be a better baseline method to compare here. Having said that the paper makes a key contribution here of using dynamic program states for making predictions, unlike RobustFill that only uses the syntactic partial programs. One way to perform such a comparison can be to train PCCoder with environment embeddings consisting of only (input-output) states without the additional intermediate variables. That would clarify precisely the contribution of having dynamic program states during the prediction process.  The model description for dense blocks is also not described in great detail and I have several questions about the model’s design choices. Since the dense blocks seem to have a rather significant impact on performance (73% vs 83%), it would be good to understand the key properties of such a network compared to previous models used for synthesis.  As the model is performing pooling over the environment state embeddings, is it the case that it can be provided an arbitrary number of input-output examples at test time?  What is the size of the possible statements |S|? With increasing number of choices for operator arguments in a more complex DSL, this set can easily blow up. Why not use different networks for predicting functions and operands? It seems the auxiliary task of predicting functions is anyways being helpful. What happens if one adds other auxiliary task of predicting operands?  It was also not clear what is the complexity of the programs that PCCoder can currently synthesize. In the author response, it would be great if the authors can provide a few samples of length 12 and length 14 programs that PCCoder can synthesize together with the corresponding input-output examples. In figure 4, what are the 8 different programs and the environments?  For embedding program states, the current approach uses a relatively simple concatenation of variable embeddings. It might be interesting to explore other ways to embed program states such as the ones suggested in [1].   1. Dynamic Neural Program Embedding for Program Repair. Ke Wang, Rishabh Singh, Zhendong Su. ICLR 2018


Unsupervised Learning of Artistic Styles with Archetypal Style Analysis
In this paper, we introduce an unsupervised learning approach to automatically discover, summarize, and manipulate artistic styles from large collections of paintings. Our method is based on archetypal analysis, which is an unsupervised learning technique akin to sparse coding with a geometric interpretation. When applied to neural style representations from a collection of artworks, it learns a dictionary of archetypal styles, which can be easily visualized. After training the model, the style of a new image, which is characterized by local statistics of deep visual features, is approximated by a sparse convex combination of archetypes. This enables us to interpret which archetypal styles are present in the input image, and in which proportion. Finally, our approach allows us to manipulate the coefficients of the latent archetypal decomposition, and achieve various special effects such as style enhancement, transfer, and interpolation between multiple archetypes.

Summary:  The authors introduce an unsupervised approach to learn structure in the space of artistic styles, which allows to represent an arbitrary artistic image as a combination of “basis”  or “archetypal” styles, as well as change the input image (resynthesize it) by modifying a contribution of each of the archetypal styles. Following Gatys et al., an artistic style is defined as a set of first and second order statistics (concatenated in a vector) in layers of a pre-trained deep network (authors use VGG-19). The authors compute such parametric styles representations for a collection of artistic images, apply SVD for dimensionality reduction (to make vectors 4096-dim), and use archetypal analysis on the resulting data set of 4096-dim style representations. Given a data set X, the archetypal analysis method allows to learn a dictionary Z of vectors, allowing to approximately represent each x = \sum{\alpha_i * z_i}, where \alpha_i sum up to one (elements of a simplex). Dictionary vectors can themselves be decomposed z = \sum{\beta_i * x_i} in terms of data vectors as well. This allows to compute corresponding mean features maps and covariance matrices of conv layers for vectors in Z (by linearly combining those of x_i), which makes Z a dictionary of styles.  For synthesis the authors suggest an extension to the method of Li et al., consisting of whitening and color transforms in the feature space of the DNN (using the means and covariance matrices of styles images), and trained encoder / decoder pairs for fast mapping to and from features spaces. The proposed extension add a parameter to explicitly model a trade-off between style and content lacking in the original model.  Overall, for any image the method allows to find its decomposition into archetypal styles, visualize (using the synthesis) each of them as textures, have input image stylized using archetypal styles, and modify the style of the input image by changing the decomposition coefficients and resynthesizing it.  Confidence score:  I’m fairly familiar with the literature and practical implementations of style transfer, and I think I understood the proposed method relatively well.  Quality:  The paper is good quality containing thorough description of the method and quite extensive experimental results.  Clarity:  The paper is clearly written and easy to follow.  Originality:  The paper addresses the problem of decomposing and visualizing artistic styles, combing ideas from style transfer literature (universal style transfer) and unsupervised learning (archetypal analysis). Such analysis of styles is novel as it gives a way to discover structure in the space of styles, while prior work mostly focused on synthesis and not on decompositions of styles.  Significance:  The problem of decomposition of styles, or finding factors of variation in styles, is difficult. There are methods allowing to transfer only certain aspects of the style image (e.g. color, fine- or coarse-grained details, …), but a principled description of what the elementary components of a style are is missing. This submission is a step in this direction. Though the archetypal styles are themselves quite complex and could be difficult to interpret (it is not that each archetypal texture is either a color blob or a repeated brush stroke, or something else basic enough to form a “linearly independent basis“ in the space of styles), but nevertheless they provide a new way to describe styles, and the proposed methods allows for fine-grained modifications of input images, which is interesting for applications. Therefore I believe this submission should be accepted for publication.


The Everlasting Database: Statistical Validity at a Fair Price
The problem of handling adaptivity in data analysis, intentional or not, permeates a variety of fields, including test-set overfitting in ML challenges and the accumulation of invalid scientific discoveries. We propose a mechanism for answering an arbitrarily long sequence of potentially adaptive statistical queries, by charging a price for each query and using the proceeds to collect additional samples. Crucially, we guarantee statistical validity without any assumptions on how the queries are generated. We also ensure with high probability that the cost for M non-adaptive queries is O(log M ), while the cost to a potentially adaptive user who makes M queries that do not depend on any others is O( p M ).

This paper studies the problem of answering a (possibly infinite) sequence of (adaptive and non-adaptive) statistical queries without overfitting. Queries trigger the acquisition of fresh data when the mechanism determines that overfitting is likely, so adaptive queries necessitate new data. By continually acquiring fresh data as needed, the mechanism can (whp) guarantee accuracy in perpetuity. Moreover, by passing on the "cost" of data acquisition to queries that trigger it, the mechanism guarantees that (whp) non-adaptive queries pay cost O(log(# queries)) while adaptive queries pay cost O(sqrt(# queries)). Suggested applications are the normal ones for adaptive data analysis: ML competition leaderboards and scientific discovery.  ---Positive points (no order)--- 1. This paper is clear throughout: the introduction sets up the (natural) problem well, and the descriptions of the mechanisms and their guarantees are easily understandable. The comparison to known work is reasonably comprehensive while remaining brief, as is the sketch of optimality. Reading this paper is pleasant. This is partly due to quality writing and partly due to the relatively simple (or, more charitably, elegant) ideas used.  2. The varying cost guarantees for adaptive and non-adaptive users, arising as a natural outgrowth of being adaptive or non-adaptive, are nice.  3. As somebody who's generally more familiar with the differential privacy literature than the adaptive data analysis literature, I'm happy to see concentrated differential privacy working its way into adaptive data analysis -- as far as I know this is the first adaptive data analysis paper to use these tools.  ---Negative points (in order of decreasing importance)--- 1. I think it ultimately falls on the right side of this divide, but this paper is close to being "too" simple. By my count there is only one result (Lemma 6) in this paper that takes more than a minute or so to grok, and I’m by no means an expert in either differential privacy or adaptive data analysis. Indeed the protocol as a whole is very close to well-known basic adaptive data analysis protocols. The addition of data acquisition is a nice wrinkle, but the result feels a bit like the first naive protocol someone familiar with adaptive data analysis would provide if given this problem. It's silly to say "I am rejecting this paper because it is not long or hard enough!", but the paper does feel "thin" and even a little obvious. Of course, this obviousness may only be in retrospect, but I feel this is a paper that could move from "marginal accept" to "accept" with just a little more work. Even just writing up the "more formal treatment" of the cost-spreading mechanism mentioned in the conclusion, shaving down the cost guarantee constants, or incorporating growing data acquisition costs (which feels a little more realistic) would suffice.  2. The use of "minus current capital" in the EverlastingValidation pseudocode is somewhat vague, it would make more sense to me to actually write this as a variable that's modified with charges.  ---Overall--- This paper does a good job of introducing its problem, outlining a clear solution and analysis, and situating its contribution in the existing literature. The same simplicity that makes for enjoyably clear reading also makes it hard to wholeheartedly recommend this paper for acceptance: while reading, my immediate reactions were primarily “yup, that’s what you’d do there” rather than “oh, I see, I now understand that's unfamiliar technique”. As a non-expert in both differential privacy and adaptive data analysis, this gives me pause even if I like the paper – I’m just not sure it’s making a strong contribution to the state of the art. Again, I really wish the authors had just chased down one of the extensions they mention!  As a result, I marginally recommend acceptance.   ---Update After Response--- After reading the authors' response, I am increasing my score from a 6 to a 7. My broad perception of the paper is essentially the same: I think it's a useful result, but a little thin as a NIPS paper. I'm willing to defend the paper a bit.


ResNet with one-neuron hidden layers is a Universal Approximator
We demonstrate that a very deep ResNet with stacked modules that have one neuron per hidden layer and ReLU activation functions can uniformly approximate any Lebesgue integrable function in d dimensions, i.e. ℓ 1 (R d ). Due to the identity mapping inherent to ResNets, our network has alternating layers of dimension one and d. This stands in sharp contrast to fully connected networks, which are not universal approximators if their width is the input dimension d [21,11]. Hence, our result implies an increase in representational power for narrow deep networks by the ResNet architecture.

In this paper, the authors demonstrated the universal approximation of ResNet with one-neuron hidden layer. In more details, ResNet with one single neural per hidden layer can approximate any Lebesgue-integrable functions to any accuracy as the depth goes to infinity. This also shows that ResNet has more representation power than narrow deep networks which are not universal approximators if the width is the input dimension. This result seems interesting since universal approximation is a fundamental property for any models. However, my comment may not be correct since I am not familiar with approximation theory of neural networks. I also did not check the involved proof.  The authors shows that the universal approximation of ResNet with one-neuron hidden layer implies the width-d+1 fully connected network are universal approximators. I did not fully understand this. From my understanding, it seems that ResNet with one-neuron hidden layers can not be understood as width-d+1 fully connected networks since there is no activation function applied to the identity map. Furthermore, there is a linear transformation applied to the activation function in the ResNet architecture.  After rebuttal: The authors' response addresses my concerns. I am in favor of its acceptance.


Efficient nonmyopic batch active search
Active search is a learning paradigm for actively identifying as many members of a given class as possible. A critical target scenario is high-throughput screening for scientific discovery, such as drug or materials discovery. In these settings, specialized instruments can often evaluate multiple points simultaneously; however, all existing work on active search focuses on sequential acquisition. We bridge this gap, addressing batch active search from both the theoretical and practical perspective. We first derive the Bayesian optimal policy for this problem, then prove a lower bound on the performance gap between sequential and batch optimal policies: the "cost of parallelization." We also propose novel, efficient batch policies inspired by state-of-the-art sequential policies, and develop an aggressive pruning technique that can dramatically speed up computation. We conduct thorough experiments on data from three application domains: a citation network, material science, and drug discovery, testing all proposed policies with a wide range of batch sizes. Our results demonstrate that the empirical performance gap matches our theoretical bound, that nonmyopic policies usually significantly outperform myopic alternatives, and that diversity is an important consideration for batch policy design.

This work investigates the different batch-mode extensions of an active search method called efficient nonmyopic policy (ENS) [12]. ENS achieves good performance efficiently because it assumes the sample selections are independent after a step [12]. This paper proposes two strategies: 1) converting the batch active search problem to sequential one by guessing the hidden labels of selected samples 2) try to enumerate all possible hidden labels of selected samples by Monte Carlo.  Strength:  Allowing to sample batches is important in practice. The work addresses several theoretical and practical challenges in many aspects of batch active search such as how difficult batch active search could be, why pessimistic oracle works well, and how to make the methods more efficient by pruning. The explanation is clear, and the literature review is comprehensive. The experiment results seem to show significant improvement as well.  Weakness:  The ideas of extension seem to be intuitive and not very novel (the authors seem to honestly admit this in the related work section when comparing this work with [3,8,9]). This seems to make the work a little bit incremental. In the experiments, Monte Carlo (batch-ENS) works pretty well consistently, but the authors do not provide intuitions or theoretical guarantees to explain the reasons.    Questions: 1. In [12], they also show the results of GpiDAPH3 fingerprint. Why not also run the experiment here? 2. You said only 10 out of 120 datasets are considered as in [7,12]. Why not compare batch and greedy in other 110 datasets? 3. If you change the budget (T) in the drug dataset, does the performance decay curve still fits the conclusion of Theorem 1 well (like Figure 1(a))? 4. In the material science dataset, the pessimistic oracle seems not to work well. Why do your explanations in Section 5.2 not hold in the dataset?    Suggestions: Instead of just saying that the drug dataset fits Theorem 1 well, it will be better to characterize the properties of datasets to which you can apply Theorem 1 and your analysis shows that this drug dataset satisfies the properties, which naturally implies Theorem 1 hold and demonstrate the practical value of Theorem 1.  Minor suggestions: 1. Equation (1): Using X to denote the candidate of the next batch is confusing because it is usually used to represent the set of all training examples 2. In the drug dataset experiment, I cannot find how large the budget T is set 3. In section 5.2, the comparison of myopic vs. nonmyopic is not necessary. The comparison in drug dataset has been done at [12].  In supplmentary material 4. Table 1 and 2: why not also show results when batch size is 1 as you did in the drug dataset? 5. In the material science dataset experiment, I cannot find how large the budget T is set  After rebuttal: Thanks for the explanation. It is nice to see the theorem roughly holds for the batch size part when different budgets are used. However, based on this new figure, the performance does not improve with the rate 1/log(T) as T increases. I suggest authors to replace Figure 1a with the figure in the rebuttal and address the possible reasons (or leave it as future work) of why the rate 1/log(T) is not applied here. There are no major issues found by other reviewers, so I changed my rate from tending to accept to accepting.


Unsupervised Adversarial Invariance
Data representations that contain all the information about target variables but are invariant to nuisance factors benefit supervised learning algorithms by preventing them from learning associations between these factors and the targets, thus reducing overfitting. We present a novel unsupervised invariance induction framework for neural networks that learns a split representation of data through competitive training between the prediction task and a reconstruction task coupled with disentanglement, without needing any labeled information about nuisance factors or domain knowledge. We describe an adversarial instantiation of this framework and provide analysis of its working. Our unsupervised model outperforms state-of-the-art methods, which are supervised, at inducing invariance to inherent nuisance factors, effectively using synthetic data augmentation to learn invariance, and domain adaptation. Our method can be applied to any prediction task, eg., binary/multi-class classification or regression, without loss of generality.

The paper presents a general procedure to disentangle the factors of variation in the data that contribute to the prediction of a desired target variable and nuisance factors of variation. The authors achieve this by partitioning the hidden representations learned by neural network “h” into “e1” that contains factors that contribute directly to predicting the outcomes “y” and “e2” that contains nuisance factors that are not useful in predicting “y”. An adversarial disentanglement objective is used to ensure that e1 and e2 capture different pieces of information and a data reconstruction objective (with e1 noised) is used to ensure that all of the information is not present solely in e1 while e2 learns nothing.  The overall approach is clever, sound and well thought out. The motivation of disentanglement and learning nuisance factors in data is well presented as well.  My main concerns with this paper is the experimental work:  1) The paper claims to outperform methods where the nuisance factors are known in advance and can be used learn representations that are “explicitly” invariant to these factors via data augmentation or adversarial training. However, I think the baselines presented that use this information are not strong enough, for example in MNIST where different rotations or other affine transformations are applied to the data, how does this compare to the spatial transformer network? (Jaderberg et. al 2016).  2) I found almost all presented tasks to be toyish in nature, while it is certainly useful to experiment in settings where the nuisance factors are known in advance, I would encourage the authors to also work with richer datasets where an adversarial disentanglement objective, a classification objective and a reconstruction objective, working together can be hard to train. Specifically, I see the reconstruction objective becoming harder to optimize with high dimensional inputs. Demonstrating the capability of this model to work with richer datasets will help with adoption. Simple MNIST and YALE-B experiments are a good start but are insufficient to demonstrate clear improvements.  3) It would be informative to also report in Table 1 the accuracy of predicting y from e2 (by “post-fitting” a classifier after training) and z from e2. Along the same lines, more ablations that add different amounts of noise to e1 during reconstruction, alter the coefficients on the disentanglement versus reconstruction objectives would be useful to report.  In conclusion I think that this is a promising piece of work that currently lacks strong experimental backing. I'd be more convinced by experiments on harder and richer datasets.


Interpreting Neural Network Judgments via Minimal, Stable, and Symbolic Corrections
We present a new algorithm to generate minimal, stable, and symbolic corrections to an input that will cause a neural network with ReLU activations to change its output. We argue that such a correction is a useful way to provide feedback to a user when the network's output is different from a desired output. Our algorithm generates such a correction by solving a series of linear constraint satisfaction problems. The technique is evaluated on three neural network models: one predicting whether an applicant will pay a mortgage, one predicting whether a first-order theorem can be proved efficiently by a solver using certain heuristics, and the final one judging whether a drawing is an accurate rendition of a canonical drawing of a cat.

This work proposes a novel method that can potentially provide actionable insight to the user when a neural network makes a less than favorable decision.  The paper is interesting in that it provides stable and hence potentially actionable insight that can help the target user change an undesired outcome in the future. The work focuses on asymmetric insight in the sense that insight or suggestions are provided only when the classification is for a certain class. So it is mainly applicable to specific kind of binary classification problems where being classified into one class is more undesirable and requires justification. Some hand wavy arguments are provided in the supplement for extension to multiple classes (one vs all), however it would be good to see experiments on those in practice as it is not at all obvious how the solution extends when you have more than one undesirable class.  Technically, they solve a large linear optimization problem to obtain the region of interest based on which the insight is conveyed based on minimal perturbation. Although, they provide some tricks to scale their approach the method seems applicable mainly to smallish or medium sized neural networks (symbolic correction with 8 features is not really that high dimensional). More large scale experiments need to be conducted to justify the solutions they propose.  The related work section also is kind of thin. Relevant works on interpretability such as S Lundberg et. al. 2017, Koh et. al. 2017, Kindermans 2018 etc. are not referred to. Also a very relevant work on contrastive explanations which looks for minimal perturbations Dhurandhar et. al. 2018 is not discussed.  Focus is on actionability however, its possible that the features they select may not be actionable for every person or instance. For example, interest rate is out of control of the individual or property type is something the individual may be inflexible about depending on his intent (a person wanting to buy a commercial building may not be happy with a recommendation telling to buy a residential condo even if he can afford it as he wants to start a business). So a dicussion with such constraints would be important to have.  Minor comment:  The text refers to some green lines in figure 1c, but I fail to see any.  Post Rebuttal:  I thank the authors for their response. I still think this is a borderline paper based on my comments.


Submodular Field Grammars: Representation, Inference, and Application to Image Parsing
Natural scenes contain many layers of part-subpart structure, and distributions over them are thus naturally represented by stochastic image grammars, with one production per decomposition of a part. Unfortunately, in contrast to language grammars, where the number of possible split points for a production A → BC is linear in the length of A, in an image there are an exponential number of ways to split a region into subregions. This makes parsing intractable and requires image grammars to be severely restricted in practice, for example by allowing only rectangular regions. In this paper, we address this problem by associating with each production a submodular Markov random field whose labels are the subparts and whose labeling segments the current object into these subparts. We call the resulting model a submodular field grammar (SFG). Finding the MAP split of a region into subregions is now tractable, and by exploiting this we develop an efficient approximate algorithm for MAP parsing of images with SFGs. Empirically, we show promising improvements in accuracy when using SFGs for scene understanding, and demonstrate exponential improvements in inference time compared to traditional methods, while returning comparable minima.

I thank the authors for their response.  I felt the authors gave an good response to my questions.  I encourage the authors to more carefully clarify the task since it is a non-standard task if the paper is accepted. ----------- Original review ---------- Summary: Similar to text parsing based on formal grammars, this paper proposes a way to parse images based on formal visual grammars that have a part, subpart structure such as Scene -> Sky, Ground.  The key problem is that splitting the image into *arbitrarily-shaped* pixel regions to associate with the production rules is computationally difficult in general.  This paper proposes to associate formal grammar production rules with submodular Markov random fields (MRF).  The submodular structure of the associated MRF allows for fast inference for a single rule into arbitrarily-shaped subregions and a dynamic-programming-like algorithm for parsing the entire image structure.  The experimental results show that the method is indeed much faster than previous methods.  Pros: 1) Well-written and easy to read even though some of the details are fairly technical. 2) Excellent speed results with competitive or better accuracy. 3) Novel combination of MRFs and formal grammars for images  Cons: 1) The specific task (i.e. input data/parameters and output) seemed a little confusing (note that I am not particularly familiar with the parsing task especially for images).  Is the task a hierarchical version of semantic segmentation?  Are the raw image pixels given to the algorithm or just semantic features like segmentation label?  In one place the energy seems to depend on the raw pixels (Sec. 2.1) but later the energy is denoted to only depend on y (Sec. 4.1); maybe this is just a notation thing but I wanted to check. 2) The experiments are also a little confusing but maybe this is just because the task is not a classic task.  How are the DeepLab unary features used?  Are they used instead of the raw pixel values? 3) What would be an example of a non-tree-structured grammar (see lines 66-68)?  It would be good to more clearly describe the difference from [6,7 ref in paper] that are similar but for tree-structured grammars.  Minor comments: The illustrations were very helpful in making the ideas concrete.  I would suggest presenting the theorem/propositions first and then giving comments on them instead of defining them after giving comments.


Is Q-learning Provably Efficient?
Model-free reinforcement learning (RL) algorithms, such as Q-learning, directly parameterize and update value functions or policies without explicitly modeling the environment. They are typically simpler, more flexible to use, and thus more prevalent in modern deep RL than model-based approaches. However, empirical work has suggested that model-free algorithms may require more samples to learn [7,22]. The theoretical question of "whether model-free algorithms can be made sample efficient" is one of the most fundamental questions in RL, and remains unsolved even in the basic scenario with finitely many states and actions. We prove that, in an episodic MDP setting, Q-learning with UCB exploration achieves regretÕ( √ H 3 SAT ), where S and A are the numbers of states and actions, H is the number of steps per episode, and T is the total number of steps. This sample efficiency matches the optimal regret that can be achieved by any model-based approach, up to a single √ H factor. To the best of our knowledge, this is the first analysis in the model-free setting that establishes √ T regret without requiring access to a "simulator."

Is Q-learning Provably Efficient? =================================  This paper studies the problem of efficient exploration in finite episodic MDPs. They present a variant of optimistic initialization + tuned learning rates for Q-learning that recover a UCB-style algorithm. The main contribution of this work is a polynomial regret bound for perhaps one of the most iconic "model-free" algorithms.    There are several things to like about this paper:  - Q-learning is perhaps the classic intro to RL algorithms, so it's nice to see that we can recover sample efficient guarantees for a variant of this algorithm.  - The writing/structure of the paper is clear and appealing; particularly given a potentially dry/analytical subject matter - good job!  - There is some genuine new result in this paper, the first "worst-case" sqrt{T} regret bounds for a model-free RL algorithm + a clear exposition. The computational time is also particularly appealing  compared to existing model-free algorithms with sqrt{T} *expected* (Bayesian) regret (such as RLSVI), which have much higher computational and memory requirements.  - Trading off the statistical complexity (regret) with the computational complexity (time/space) is a clearly valuable element of the discussion in this literature, and one that is often overlooked. The concise presentation in Table 1 is laudable and will help to clarify the field for many.    However, the paper also has a few shortcomings:  - The bounds are *not* the first sqrt{T} bounds for model-free reinforcement learning... at least not according to the definition offered in the first paragraph of this paper. That distinction goes to "randomized least squares value iteration" from "Generalization and Exploration via Randomized Value Functions" see https://arxiv.org/abs/1402.0635 or more complete presentation/analysis in https://arxiv.org/abs/1703.07608. It should be noted that that analysis is given in terms of the Bayesian Expected regret, not the worst case regret as studied here... but if you want to claim the first sqrt{T} bounds then I think it is very important to qualify this technical distinction. It is interesting to provide a similar algorithm with worst-case guarantees... but I don't think it's as big a jump. Note too that the quality of these bounds esentially matches that of RLSVI.... but unlike RLSVI this algorithm does not naturally extend to generalization.  - The point above greatly affects the novelty and impact of this paper. In addition, most of the technical results / analysis approach is really derived from a similar line of work, that begins with RLSVI -> UCBVI -> This Q-learning analysis.  - Note, on the positive side here, that there are potential other definitions of memory/runtime that do distinguish this analysis and mean that, within *that* class of models this is the first provably-efficient application of UCB and this is a great contribution especially given the central status of Q-learning!  - The distinction between model-free and model-based in this tabular context is not really as big a distinction as the paper makes out. In many ways this paper feels like a way of relabelling the memory requirements of UCBVI [5] by separating each estimation problem to be independent at each timestep. Most of the tools seem to be repurposed from the "standard library". The real benefit seems to come from computational savings that are afforded by the extra H-dependence.  - Actually this issue of learning a distinct Q for each "timestep" and particular NOT SHARING DATA across timesteps sidesteps one of the key difficulties in the analysis of UCBVI + related algorithms. That's OK, but it feels like the paper is hiding some of that, whereas I think it should take center stage to help explain what is going on here.  - Q-learning is interesting to study particularly because of its connection to DQN... but I'm not convinced that this paper really gives us much insight that we can carry over to RL with function approximation. I think "practioners" already knew that UCB is theoretically better than e-greedy and I'm really not convinced by the suggestion that practitioners use a learning rate O(H) at the start... I think that only applies when you have a tabular Q separated at each timestep... this "key takeaway" could even be damaging if it is wrong! The interesting setting is Q learning with function approximation, but this presentation is misleading if it makes people think Q-learning with function approximation is efficient. Furthermore, it is well known that *everyone* using neural networks typically tunes the learning rate over orders of magnitude, so I'm not sure that this claim of O(H/t) vs O(1/t) really affords anything interesting.  - The other observation that UCB affords better treatment of uncertainty is better than epsilon-greedy is also essentially vacuous... epsilon-greedy doesn't maintain any treatment of uncertainty!    Overall I think that this is a reasonable and even good paper in many parts. I've gone back to this review more than any other paper in my batch of reviews and in many ways it does leave me torn:  There are significant issues in the novelty, particularly in their claim to be the first sqrt{T} regret bound model-free RL algorithm - which is incorrect according to the paper's definitions and thus cannot be accepted in its current state. Many of the other claims are overblown or even misleading to "practitioners", where generalization should of course be a primary concern.  On the other hand, if the authors can address some of the objections then I think it could be further improved and it will be a great candidate for another conference or potentially here. The existence of previous "model-free" algorithms does not negate the contribution of this work and, with respect to time and space, this is a clear improvement for the tabular finite horizon setting.  Due to the issues I have raised I have sided with a rejection... I think there are just too many issues and too essential a rewrite to recommend acceptance without access to the camera-ready version. However, I feel confident that it *will* be possible to address these issues for a future version and that this will help to make it a better paper. I've indicated a 5/10 as hope to reflect this dual-sided view.  ========================================== Amended post-rebuttal  Thank you to the authors for your thoughtful rebuttal. I really agree with the points that you make... and I just want to point out that this type of discussion is *not* present in the version of the paper you submitted. My worries are not based in whether or not this paper can/should be a great contribution - I think so, more that the way it is currently presented is potentially misleading.  For example, the proposed definition of model-free differs greatly from what is currently in the paper and that's important. I think that a discussion on this topic is particularly valuable and helps to clarify the contribution of this work. - For RLSVI I think it would be helpful to mention the complexity in terms of D=dimension of approximating basis functions, but of course the only case with proven regret bounds are for D=SA.  On the topic of distinct Q and RL with function approximation, it's not so much that I think it's unworthy to study this simpler situation - I think it's a great first step. More, that I worry the claims for "practitioner" who applies Q-learning with neural network and discounted problem might be overstated. I'm not sure that the same conclusions apply in this setting, what does it mean to do optimistic initialization for neural networks + SGD?  My worry following the rebuttal is how many of these points / amendments would actually make it into a final camera ready version. However, under the understanding that the area chair would enforce this requirement I will upgrade my review to a 6.


Plug-in Estimation in High-Dimensional Linear Inverse Problems: A Rigorous Analysis
Estimating a vector x from noisy linear measurements Ax + w often requires use of prior knowledge or structural constraints on x for accurate reconstruction. Several recent works have considered combining linear least-squares estimation with a generic or "plug-in" denoiser function that can be designed in a modular manner based on the prior knowledge about x. While these methods have shown excellent performance, it has been difficult to obtain rigorous performance guarantees. This work considers plug-in denoising combined with the recentlydeveloped Vector Approximate Message Passing (VAMP) algorithm, which is itself derived via Expectation Propagation techniques. It shown that the mean squared error of this "plug-and-play" VAMP can be exactly predicted for highdimensional right-rotationally invariant random A and Lipschitz denoisers. The method is demonstrated on applications in image recovery and parametric bilinear estimation.

Update: After reading the rebuttal of the authors, I think the authors have addressed my comments satisfactorily. I've increased my score of the paper to 7.  ---------------------------------- Original comments:  Summary:   This paper provides a proof for the state evolution of plug-in denosing-based VAMP algorithm. The results extend the analyses in a previous paper by a subset of the authors. It is well written and the numerical results look nice.  Here are my detailed comments:  1. Is it fair to say that the main difference between the current paper and [21] is related to the “convergent under Gaussian noise” assumption (in Definition 2), and other parts of the proof are parallel to those of [21]?  2. The main contribution of the paper is a proof of state evolution under certain conditions. The authors then provide simulations to show the performance advantage of the proposed algorithm over other algorithms. However, in my view, there seem to be some disconnections between theory and simulations: (1) In the analysis, it is assumed that the denoiser is Lipschitz continuous. However, the denoisers under study (such as the BM3D denoiser and the rank one AMP denoiser for bilinear estimation) doesn’t seem to satisfy such continuity assumptions. The possible discontinuity can have an impact on how you compute the derivatives required by the algorithm. This issue is discussed in [10] and a Monte Carlo method was proposed to address the discontinuity. I’m wondering what’s the impact of discontinuity on the performance of the proposed algorithm? Did you use similar ideas?  (2) It is assumed that $A$ is right rotationally invariant. However, the matrixes involved in the numerical results (particularly the one in the bilinear estimation example) are quite structured. How does such structure affect the accuracy of the state evolution analysis?   (3) The main contribution of the paper is a state evolution theory under certain assumptions. However, the simulations are carried out under conditions (seem (1) (2) above) different from the analysis. It’s not clear whether the theory can still provide good prediction of the empirical performance. To make the story complete, instead of comparing the empirical performance of the algorithm relative to others, I think it’s better to add numerical results that verify the SE predictions.   3. In theorem 1, it is assumed that $\bar{\alpha_{1k}}\in(0,1)$. This assumption seems quite strong. How can one check whether it holds or not? For example, does it hold for the denoisers involved in the numerical results of this paper?  4. The assumption in (13) seem to be extremely strong: it assumes that the initial estimate is an observation of the true signal correction by iid Gaussian noise. Is this practical?  5. The authors cite both [18] and [21]. Sometimes the authors refer to [18] and sometimes to [21]. But it seems that [18] is basically a short version of [21]. It is necessary to include both references?


Paraphrasing Complex Network: Network Compression via Factor Transfer
Many researchers have sought ways of model compression to reduce the size of a deep neural network (DNN) with minimal performance degradation in order to use DNNs in embedded systems. Among the model compression methods, a method called knowledge transfer is to train a student network with a stronger teacher network. In this paper, we propose a novel knowledge transfer method which uses convolutional operations to paraphrase teacher's knowledge and to translate it for the student. This is done by two convolutional modules, which are called a paraphraser and a translator. The paraphraser is trained in an unsupervised manner to extract the teacher factors which are defined as paraphrased information of the teacher network. The translator located at the student network extracts the student factors and helps to translate the teacher factors by mimicking them. We observed that our student network trained with the proposed factor transfer method outperforms the ones trained with conventional knowledge transfer methods.

This paper aims at transferring information between a complex network and a simpler one by using the so called paraphrase. The idea is having a bridge to transfer knowledge (activations) even when the student (smaller network) has different number of features / feature dimensions compared to the 'teacher'  This paraphaser is trained in an unsupervised manner to map the existing feature maps / information into the desired shape via an autoencoder. Once trained, this information will be included in the loss function when training the student one via a translator (to adapt sizes). This translator is jointly trained with the student and aims at helping the student to "assimilate the information".   - I wonder about the variations with respect k. Seems like this parameter is crucial and fixing this to a value does not consistently provide the best results. What would be a good way to find this value in a different application? For instance table 1 and 3 show a large variation depending on this number.  - the text claims several times on the relevance of the factors extracted. How can this be verified? - Some tables contain a combination of FT+KD and this combination does not always result in better results. What is the take home message here? Why this numbers are not always included? - How does  this compare to FitNets and the idea of helping with intermediate layers? - If the dimensions are the set equal, (line 155) I guess there could be an ablation study showing the need for those translators, right? Would be nice to see those numbers. - In the experimental setup it is mentioned about l2 reg but then the algorithm seems to use l1. The way is written is very confusing.  - In summary, experiments are not really convincing    Other details: - My understanding is table represents a reimplementation of state of the art methods, right? Would be great if the comparison could be directly on published numbers for fair comparison and understanding the strength of the contribution.   - Tables contain redundant information. For instance Table 6 two bottom rows. As the space is limited, this could be omitted.  - Text suggest the proposal also works for object detection. However, numbers do not really support this (marginal increment of 0. 3 - The way results are presented seem to be overclaiming. For instance, Table 2, in my opinion, does not show "much better accuracy" but some improvement.     ------------------------------- I appreciate author's feedback for addressing my concerns. I have increased my score accordingly.


Analytic solution and stationary phase approximation for the Bayesian lasso and elastic net
The lasso and elastic net linear regression models impose a double-exponential prior distribution on the model parameters to achieve regression shrinkage and variable selection, allowing the inference of robust models from large data sets. However, there has been limited success in deriving estimates for the full posterior distribution of regression coefficients in these models, due to a need to evaluate analytically intractable partition function integrals. Here, the Fourier transform is used to express these integrals as complex-valued oscillatory integrals over "regression frequencies". This results in an analytic expansion and stationary phase approximation for the partition functions of the Bayesian lasso and elastic net, where the non-differentiability of the double-exponential prior has so far eluded such an approach. Use of this approximation leads to highly accurate numerical estimates for the expectation values and marginal posterior distributions of the regression coefficients, and allows for Bayesian inference of much higher dimensional models than previously possible.

Summary An approximation to the posterior distribution from a Bayesian lasso or Bayesian elastic net prior is developed. The method uses a saddle-point approximation to the partition function. This is developed by writing the posterior distribution in terms of tau = n / sigma^2 and uses an approximation for large tau. The results are illustrated on three data sets: diabetes (n=442, p=10), leukaemia (n=72, p=3571) and Cancer Cell Line Encyclopedia (n=474, p=1000). These demonstrate some of the performance characteristics of the approximation.  Strengths The paper is clearly written and shows a new, useful method for approximation posterior distributions in sparse regression problems. This is an original approach and there is a wealth of background information in the appendix. There is strong evidence that the method works well on a range of problems. Posterior inference with sparsity priors in linear regression is an important problem and a good approximation would be very useful. As the author notes Gibbs sampling can be slow (although there have been advances in this area for large p, small n problems, see e.g. Johndrow et al, 2018). This paper provides a big step in providing a fast optimisation based approach.  Weaknesses My reading of the paper is that the method is limited to linear regression with prior distributions whose log density is convex. It would interesting to know whether there is any scope to extend to logistic regression models (or other GLMs) or to other sparse priors (such as the horseshoe). I found it difficult to interpret the results in Figure 1. What is a small error in this context? The results suggest that tau needs to be at least 100000 for an extremely small error but, presumably, the method works well with smaller values of tau (for example, the diabetes data uses tau=682.3). A discussion would be useful. For example, a small n (say approx n=100) and low signal-to-noise ratio could easily lead to tau in 100's. Would the method work well in these situations? Page 7 discusses prediction using the posterior predictive mean. How easily could the posterior predictive density be calculated? Also could other summaries such as posterior variance be calculated using this approach?  References J. E. Johndrow, P. Orenstein and A. Bhattacharya (2018): Scalable MCMC for Bayes Shrinkage Priors, arXiv:1705.00841


Representer Point Selection for Explaining Deep Neural Networks
We propose to explain the predictions of a deep neural network, by pointing to the set of what we call representer points in the training set, for a given test point prediction. Specifically, we show that we can decompose the pre-activation prediction of a neural network into a linear combination of activations of training points, with the weights corresponding to what we call representer values, which thus capture the importance of that training point on the learned parameters of the network. But it provides a deeper understanding of the network than simply training point influence: with positive representer values corresponding to excitatory training points, and negative values corresponding to inhibitory points, which as we show provides considerably more insight. Our method is also much more scalable, allowing for real-time feedback in a manner not feasible with influence functions.

The manuscript proposes a method to measure the effect that training examples have on the performance of a deep neural network (DNN). Towards this goal, a method is proposed to identify "representer" points that measure the importance that a training image has on the performance of a given model. The paper proposes to identify representer points of two types: i) positive points; which correspond to examples that support the decisions made by the network, and ii) negative points; which inhibit the believe of the model towards certain predictions. Experiments with the VGG-16 and ResNet-50 architectures on the MNIST, CIFAR-10, and Animals with Attributes (AWA) datasets show the strenghts of the proposed method.  The proposed method is sound and well motivated. A detailed formal presentation of the method is provided. In addition, aspects related to the computational cost and numerical properties of the proposed method are adequately discussed.  I find the dataset debugging experiment (Sec.4.1) an interesting means to asses the effect of training examples on the learning of the model being trained.  My main concerns with the manuscript are the following:  i) I consider there is some room for improvement regarding the formal presentation of the method since it can be a bit complex to go through it at the begining. For instance, initially I had troubles understanding what the text "the last intermediate layer feature in the neural network" in l.82 refers to.  ii) While positive/negative representers could be good indicators of the performance of the model, in general, I find hard to find it as a good explanatory means for a specific given test example. For instance, in Fig.4 images of influential examples are presented. However, it is difficult for me to make the direct connection on how these examples "explain" the prediction made for the test image. In my opinion the debugging experiment (Sec.4.1), combined with its related discussion in Sec.4.3 seem to be a more effective use of the proposed method, and perhaps a potentially stronger contribution,  than that of prediction explainability that is claimed by the paper.  iii) Finally, the presented results (Fig. 4,5,6) for the model explanations are mostly qualitative which can be to some extent subjective. In this regard, I find the amount of provided qualitative examples quite reduced. Therefore, I encourage the authors to add supplementary material in order to show extended qualitative results of the explanations produced by their method. Perhaps this can further demonstrate the explanation potential of the proposed method.  I would appreciate if my concerns are addressed in the rebuttal.  ======================== Post rebuttal update ========================  Having read the reviews and the provided rebuttal, my impression of the manuscript is the following.   I do appreciate that a significant part of the rebuttal is focused on addressing my comments. However, I am still not convinced regarding the usefulness of the positive/negative representers as means for explanation. At the end of Sec.1 there is an example highlighting the un-annotated class instances, e.g. antilopes, as negative representers. Later in Sec. 4.2 more examples where the negative representers are images with instances of other classes with similar shape to that of the class of interest are discussed. Similarly, in the rebuttal an example regarding bears was provided. Still very similar to what was presented on the paper.   I would be more convinced about the explanation capabilities of the method if, on top of these representers, there would be an overlaid visualization, e.g. a grad-CAM (Selvaraju et a., arXiv:1610.02391), CNN-Fixation (Reddy et al.,  arXiv:1708.06670 ), DeconvNet (Springenberg et al. ICLR'15), etc. heatmap, highlighting the visual features that make this image an actual negative representer, e.g. highlighting the un-annotated antilopes. At this point, it is the person visualizing the representer negative image, the one coming out with the explanation.  Having said this, and as stated on my initial review, I do find quite valuable the use of these representer points as means for debugging (Sec. 4.1 and Sec 4.3). I share the opinion of the other reviewers that the proposed method holds good promise as a model debugging tool. Moreover, I motivate the authors to update the manuscript to present their method as a debugging method instead of as a tool for explanation. In this regard, I still feel the explanation capability of the proposed method not fully justified.  Finally, I appreciate the additional qualitative examples. I will encourage the authors to enrich their manuscript by adding supplementary material with a good sample of these examples.