Stochastic Nonparametric Event-Tensor Decomposition
Tensor decompositions are fundamental tools for multiway data analysis. Existing approaches, however, ignore the valuable temporal information along with data, or simply discretize them into time steps so that important temporal patterns are easily missed. Moreover, most methods are limited to multilinear decomposition forms, and hence are unable to capture intricate, nonlinear relationships in data. To address these issues, we formulate event-tensors, to preserve the complete temporal information for multiway data, and propose a novel Bayesian nonparametric decomposition model. Our model can (1) fully exploit the time stamps to capture the critical, causal/triggering effects between the interaction events, (2) flexibly estimate the complex relationships between the entities in tensor modes, and (3) uncover hidden structures from their temporal interactions. For scalable inference, we develop a doubly stochastic variational Expectation-Maximization algorithm to conduct an online decomposition. Evaluations on both synthetic and real-world datasets show that our model not only improves upon the predictive performance of existing methods, but also discovers interesting clusters underlying the data.
To address these issues, we first formulate a new data abstraction, event-tensor, to preserve all the time stamps for multiway data. In an event-tensor, each entry comprises a sequence of interaction events rather than a numerical value. Second, we propose a powerful Bayesian nonparametric model to decompose event-tensors (Section 3). We hybridize latent Gaussian processes and Hawkes processes to capture various excitation effects among the observed interaction events, and the underlying complex relationships between the entities that participated in the events. Furthermore, we design a novel triggering function that enables discovering clusters of entities (or latent factors) in terms of excitation strengths. Besides, the triggering function allows us to flexibly specify the triggering range (say, via domain knowledge) to better capture local excitations and to control the trade-off to the computational cost. Finally, to handle data where both the tensor entries and interaction events are many, we derive a fully decomposed variational model evidence lower bound by using Poisson process superposition theorem and the variational sparse Gaussian process framework (Titsias, 2009). Based on the bound, we develop a doubly stochastic variational Expectation-Maximization algorithm to fulfill a scalable, online decomposition (Section 4).
For evaluation, we examined our model in both predictive performance and structure discovery. On three real-world datasets, our model often largely improves upon the prediction accuracy of the existing methods that use Poisson processes and/or time factors to incorporate temporal information. Simulation shows the latent factors estimated by our model clearly reflect the ground-truth clusters while by the competing methods do not. We further examined the structures discovered by our model on the real-world datasets and found many interesting patterns, such as groups of 911 accidents with strong associations, locations of townships that are apt to have consecutive accidents, and UFO shapes that are more likely to be sighted together (Section 6).

Post rebuttal: thank you for your answers! I would encourage authors to consider experiments with matrix completion with time stamps in the camera ready version and compare to baselines such as Factorization Machines or Neural Networks where time information is ignored. In my opinion this could help to draw the attention of researchers outside of the Bayesian community and increase the significance of the work.  This paper considers the problem of tensor factorization across time. Proposed model combines Hawkes process to accommodate temporal dependencies, with base rate dependent on a function of latent factors equipped with Gaussian Process prior. Gaussian Process has been previously considered in the tensor factorization domain and showed some advantages over more classical multilinear functions. However the addition of Hawkes process while maintaining advantages of GP appear novel to me and more elegant than previous discrete time approaches. Authors take Variational Inference pass, but deriving ELBO is not trivial for the proposed model and as a workaround authors decompose Hawkes process into multiple Poisson processes and introduce additional categorical ("cause") latent variable to arrive at manageable ELBO.  I think this is a good paper addressing a fairly sophisticated problem of stochastic tensor decomposition with a new model and some interesting inference decisions.  Combination of Hawkes process and GP was new to me and interesting to see. Experimental baselines appear strong and experiments demonstrate the advantage of modeling time component and using GP. Overall paper is written well.  On the downside, method is quite slow, but nonetheless experimental results are shown on several relatively big datasets. The other downside is the practical usefulness of the method. Clustering analysis does not appear particularly meaningful (looking at the Tables in the supplement, I can't really relate to the interpretation suggested in the paper), although perhaps the data is to blame (method seems to identify some structure in the toy simulation experiment). Have you tried matrix completion? For example, 100k MovieLens data (for predicting user-movie interactions instead of actual ratings)? I believe timestamp is often discarded, but maybe your model can perform well by taking it into account. It would be interesting to know if the method can compete with black box prediction approaches (i.e. neural networks or Factorization Machines). It is ok if not, since there are other merits, but it could make the story more complete and show limitations of the approach (overall, authors did not really discuss what are the limitations besides run-time).  Experiments show that GP + Hawkes process are better than GP or multilinear factorization with discrete time. What about using discrete time approach and GP instead of multilinear factorization. Is there any prior work that uses GP (for tensor decomposition) in temporal setting?  Typos in lines 55, 377.


On Controllable Sparse Alternatives to Softmax
Converting an n-dimensional vector to a probability distribution over n objects is a commonly used component in many machine learning tasks like multiclass classification, multilabel classification, attention mechanisms etc. For this, several probability mapping functions have been proposed and employed in literature such as softmax, sum-normalization, spherical softmax, and sparsemax, but there is very little understanding in terms how they relate with each other. Further, none of the above formulations offer an explicit control over the degree of sparsity. To address this, we develop a unified framework that encompasses all these formulations as special cases. This framework ensures simple closed-form solutions and existence of sub-gradients suitable for learning via backpropagation. Within this framework, we propose two novel sparse formulations, sparsegen-lin and sparsehourglass, that seek to provide a control over the degree of desired sparsity. We further develop novel convex loss functions that help induce the behavior of aforementioned formulations in the multilabel classification setting, showing improved performance. We also demonstrate empirically that the proposed formulations, when used to compute attention weights, achieve better or comparable performance on standard seq2seq tasks like neural machine translation and abstractive summarization.

Summary -------  This paper introduces several variants of the sparsemax mapping. Experiments are conducted on multi-label classification and attention mechanisms for neural machine translation / summarization.  With some exceptions (see below), the paper is overall clearly written. The authors put great effort in explaining their ideas with figures.  Experiments on two different tasks (multi-label classification and NMT) is also very positive.  That said, the technical contribution seems very limited. The proposed "unifying" framework is just sparsemax with different inputs. As a result, many of the proofs in the Appendix are just existing proofs with different inputs. Sparseflex is just sparsemax with a regularization parameter, which was already proposed in [16]. Sparsecone and sparsehourglass feel quite adhoc and it is not clear why they should be any better than sparsemax.  In the experiments, I am surprised sparsemax performed so much worse than sparseflex (sometimes up to 10 point difference!). As I explain below, sparsemax and sparseflex should perform exactly the same in theory. Therefore, I suspect there are issues in the empirical validation.  Overall, I feel the paper is quite below the acceptance bar for NIPS.  Major comments --------------  * I am sorry to be a bit blunt but the current title is really bad.  I would really avoid bad analogies or bad puns in a title.  My recommendation is to keep titles formal and informative.  How about just "On Sparse Alternatives to Softmax"?   * [16] is a very relevant work and I was surprised not to see it mentioned until the experiment section. I think it deserves a mention in the introduction, since it also provides a unifying framework.  * Theorem 1 is a trivial result. I wouldn't call it a theorem.  * Section 3.2: So is sparseflex just sparsemax with a regularization parameter? The fact that tunning lambda allows to control sparsity was already mentioned in [16].  * Introducing too many new names (sparseflex, sparsecone, etc) is not a good idea, as it confuses the reader. My recommendation is to pick only one or two proposed methods. You don't need to introduce new names for every special case.  * I am surprised that sparsemax performs so much worse than sparseflex in Figure 5 and 6.  Indeed, if the vectors of scores z is produced by a linear model, i.e., z = W x, the regularization parameter can be absorbed into W as is obvious from Eq. (2). Given this fact, how do you explain why sparseflex performs so much better?  * Appendices A.1 to A.4 are essentially the same proof again and again with different inputs. Since sparsegen reduces to the projection on the simplex, there is no need for all this...  * For multi-label classification, experiments on real data would be welcome. Also it'd be great to report the average sparsity per sample on the test set.  Detailed comments -----------------  * Line 62: Z_+ is commonly denoted N.  * Line 63: {a, b, c} usually defines the elements of a set, not a vector  * There are many works tackling the projection onto the simplex before [15]. It would great to give them proper credit. Some examples:  P.Brucker. An O(n) algorithm for quadratic knapsack problems. Operations Research Letters,3(3):163–166, 1984.  C. Michelot.  A finite algorithm for finding the projection of a point onto the canonical simplex of Rn.  Journal of Optimization Theory and Applications, 50(1):195–200, 1986.  * Line 209: multiple labels can be true => multiple labels are allowed  ----------  > However, A.3 follows directly from A.1 and can be removed.  I still think that A.1 is basically a known result since Eq. (6) (supp) can be reduced to the projection on the simplex.  > [On empirical evaluation]  Thank you for the clarification between L_sparsemax and L_sparseflex. This addressed my concern about Fig. 5.


Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation
Generating novel graph structures that optimize given objectives while obeying some given underlying rules is fundamental for chemistry, biology and social science research. This is especially important in the task of molecular graph generation, whose goal is to discover novel molecules with desired properties such as drug-likeness and synthetic accessibility, while obeying physical laws such as chemical valency. However, designing models to find molecules that optimize desired properties while incorporating highly complex and non-differentiable rules remains to be a challenging task. Here we propose Graph Convolutional Policy Network (GCPN), a general graph convolutional network based model for goaldirected graph generation through reinforcement learning. The model is trained to optimize domain-specific rewards and adversarial loss through policy gradient, and acts in an environment that incorporates domain-specific rules. Experimental results show that GCPN can achieve 61% improvement on chemical property optimization over state-of-the-art baselines while resembling known molecules, and achieve 184% improvement on the constrained property optimization task.

The paper proposes combining graph neural nets, RL, and adversarial learning to train a molecule generation procedure optimized according to specific task requirements. RL is used to optimize non-differentiable objectives. Adversarial learning is used to encourage the policy to generate molecules that are similar to known ones. Graph neural nets are used to capture the graph structure of the molecules. Results are shown for three different tasks -- optimizing a specific property of the generated molecule, generating molecules that satisfy constraints on property scores, and generating molecules that satisfy constraints on their composition. ZINC dataset is used for training. Results show the proposed approach outperforming other learning based approaches.  Strengths: - The paper is well-written and easy to read. - Although the proposed approach is a straightforward combination of existing ideas, it is still interesting to explore the performance of such a combination on the kind of tasks considered in the paper.  Weaknesses: - In the evaluation, there is no comparison to simple baselines with no learning, like Genetic Algorithms, Hill Climbing, random search, etc. which makes it difficult to evaluate how much benefit does sophisticated learning-based approaches provide.  - In the optimization tasks, the running time, computational budget, and the number of function evaluations should all be controlled across the various algorithms (at least approximately) so that the comparisons are fair. There is no mention of these factors in the paper. Since each optimization algorithm represents a trade-off between solution quality and time/computation/evaluation budget, it is not clear where on that trade-off curve each algorithm being compared is and whether they are even comparable. (If one only considers solution quality, even brute force search will do well.) This makes it hard to decide whether the results are interesting.  Comments/Questions: - How are the potentially unstable learning dynamics of adversarial learning and RL handled? Insights into the challenges can be very useful.  - Lines 171-172 mention using "small" positive and negative rewards to discourage infeasible actions. What value is determined to be "small" and how is it determined?  - If the policy becomes deterministic, then it will always generate the same molecule. How is the policy kept stochastic so that it generates diverse molecules?  - There is a missing reference in the Supplementary Material.


Variational Learning on Aggregate Outputs with Gaussian Processes
While a typical supervised learning framework assumes that the inputs and the outputs are measured at the same levels of granularity, many applications, including global mapping of disease, only have access to outputs at a much coarser level than that of the inputs. Aggregation of outputs makes generalization to new inputs much more difficult. We consider an approach to this problem based on variational learning with a model of output aggregation and Gaussian processes, where aggregation leads to intractability of the standard evidence lower bounds. We propose new bounds and tractable approximations, leading to improved prediction accuracy and scalability to large datasets, while explicitly taking uncertainty into account. We develop a framework which extends to several types of likelihoods, including the Poisson model for aggregated count data. We apply our framework to a challenging and important problem, the fine-scale spatial modelling of malaria incidence, with over 1 million observations.

PAPER SUMMARY:  This paper proposes an interesting variational inference framework for a weakly supervised learning scenario (with counting data) where the corresponding output to each input datum is not directly observable. Instead, the training data is partitioned into subsets & a noisy observation of the aggregate output of each subset is provided. The task is to learn a mapping from each (unseen) input datum to its corresponding output.  SIGNIFICANCE & NOVELTY:  Addressing the regression task using aggregate observations instead of individual observations using GP with Poisson likelihood is interesting & fairly new to me. The technical development also appears correct and feasible.  The authors should, however, provide a more in-depth discussion to highlight the fundamental differences (in terms of both model formulation & solution  technique) between their proposed framework & the previous work of Gaussian process modulated Poisson process [17] (VBPP). This is only mentioned very briefly in the related work section that VBPP did not handle aggregation of output. While this is true, it is also interesting to know why it is non-trivial to extend VBPP to handle aggregate output & how did the authors' new formulation & solution technique address(or sidestep)  this challenge (hence, the significance of the proposed work).  EXPERIMENT:   A related work [17] (VBPP) was also tested on the same domain of Malaria prediction task. Is the experiment setting in [17] amenable to the proposed framework (e.g., one datum per bag)? It will be interesting to compare the performance of VBAgg and VBPP in such setting.   The authors may also want to include further experiments with varying the no. of inducing points. As we increase the no. of inducing points, the prediction quality should improve at the cost of extra processing time. For a specific task, I imagine this would help us determining the "right" no. of inducing points (i.e., too few inducing points will degrade the prediction quality but too many inducing point will incur extra computing cost).  CLARITY:  The paper is clearly written. I do, however, find the notations are a bit too dense & hard to follow. If possible, please consider simplifying the notations to improve readability.  REVIEW SUMMARY:  This is an interesting paper with novel contributions. The proposed theoretical framework is new to me (to the best of my knowledge) & is also demonstrated empirically in an important real-life problem with promising results.  ----  I have read the rebuttal. Thank you for the clarifications.


Porcupine Neural Networks: Approximating Neural Network Landscapes
Neural networks have been used prominently in several machine learning and statistics applications. In general, the underlying optimization of neural networks is non-convex which makes analyzing their performance challenging. In this paper, we take another approach to this problem by constraining the network such that the corresponding optimization landscape has good theoretical properties without significantly compromising performance. In particular, for two-layer neural networks we introduce Porcupine Neural Networks (PNNs) whose weight vectors are constrained to lie over a finite set of lines. We show that most local optima of PNN optimizations are global while we have a characterization of regions where bad local optimizers may exist. Moreover, our theoretical and empirical results suggest that an unconstrained neural network can be approximated using a polynomially-large PNN.
conditions for a local minimum to be global in the case of having regularization on the network weights.
We review these prior work in SM 1 Section 2. In this paper, we study a key question: can an unconstrained neural network be approximated with a constrained one whose optimization landscape has good theoretical properties? For two-layer neural networks, we provide an affirmative answer to this question by introducing a family of constrained neural networks which we refer to as Porcupine Neural Networks (PNNs) (Figure 1-a). In PNNs, incoming weight vectors to hidden neurons are constrained to lie on a fixed set of lines. For example, a neural network with multiple inputs and multiple neurons where each neuron is connected to one input is a PNN since input weight vectors to neurons lie on lines parallel to the standard axes (SM Section 3).
Designing new objective functions with good landscape properties for non-convex problems has been explored previously in [19,15,24]. Our work provides an alternative procedure to obtain a new objective function for the underlying non-convex optimization. Our framework can be extended naturally to deeper networks as well even though in our analysis in this paper, we focus on two-layer networks. Moreover, a PNN can be viewed as a neural network whose feature vectors (i.e., input weight vectors to neurons) are fixed up to scalings due to the PNN optimization. This view can relate a random PNN (i.e., a PNN whose lines are random) to the application of random features in kernel machines [25,26,27,28,29,30,6]. Although our results in Sections 4 and 5 are for general PNNs, we study them for random PNNs in Section 6 as well.
We analyze population risk landscapes of two-layer PNNs with jointly Gaussian inputs and rectified linear unit (ReLU) activation functions at hidden neurons. We show that under some modeling assumptions, most local optima of PNN optimizations are also global optimizers. Moreover, we characterize the parameter regions where bad local optima (i.e., local optimizers that are not global) may exist.
Next, we study whether one can approximate an unconstrained (fully-connected) two-layer neural network function with a PNN whose number of neurons is polynomially-large in dimension. Our empirical results offer an affirmative answer to this question. For example, suppose the output data is generated using an unconstrained two-layer neural network with d = 15 inputs and k * = 20 hidden neurons. Using this data, we train a random two-layer PNN with k hidden neurons. We evaluate the PNN approximation error as the mean-squared error (MSE) normalized by the L 2 norm of the output samples in a two-fold cross validation setup. As depicted in Figure 1-b, by increasing the number of PNN hidden neurons k, the PNN approximation error decreases. Notably, to obtain a relatively small approximation error, PNN's number of hidden neurons does not need to be exponentially large in dimension. We explain details of this experiment in Section 7.
In Section 6, we study a characterization of the PNN approximation error with respect to the input dimension and the complexity of the unconstrained neural network function. We show that under some modeling assumptions, the PNN approximation error can be bounded by the spectral norm of the generalized Schur complement of a kernel matrix. We analyze this bound for random PNNs in the high-dimensional regime when the ground-truth data is generated using an unconstrained neural network with random weights. For the case where the dimension of input and the number of hidden neurons increase with the same rate, we compute the asymptotic limit. Finally, in Section 8, we discuss how the proposed PNN framework can potentially be used to explain the success of local search methods such as gradient descent in solving the unconstrained neural network optimization.
In summary, PNNs provide three main advantageous. The first is that its loss landscape has nice properties with provably few bad locals. Thus, we can have some guarantees on the performance of gradient descent. Second, its approximation power is good, i.e. our experimental and theoretical results suggest that one can approximate unconstrained networks arbitrarily closely with polynomiallylarge PNNs. Third, PNN uses fewer parameters than unconstrained neural networks, thus leading to better generalization and also benefits in terms of storage. Finally, we discuss using the PNN framework in general deep neural networks in Section 8.

This paper presents a new type of neural network where the weights are constrained to lie over a fixed set of lines (called PPN lines). Although this reduces the capacity of the network, it still yields a high-capacity network and more importantly, it “simplifies” the energy landscape making it easier for an optimizer to find the optimum weights. I do have some comments, mostly regarding the relation to previous work or some technical part in the analysis. I will consider raising my scores if these questions are addressed in the rebuttal.   1) Comparison to existing approaches I think the approach presented by the authors is interesting as it offers some improvements over past approaches. As pointed out by the authors, most existing work assume the number of neurons is less than the dimensions. However, there are exceptions and I think it’s worth pointing this out. For instance, Soltanolkotabi et al. (2017) already focused on the (more reasonable) over-parametrized regime where n < O(d).  2) Comparison to other architectures such as resnet or densenet The authors essentially suggest a way to reduce the capacity of the network. There are already many ways to modify the architecture of a neural network to achieve similar effects such as using residual or skip connections. See e.g. https://arxiv.org/pdf/1712.09913.pdf for a discussion of the effect of these connections on the landscape of a neural network.  3) Characterization global minima Vidal et al. (https://arxiv.org/pdf/1712.04741.pdf) gave conditions for global optimality of a neural network (related to the number of zero weights). This seems to relate to the conditions given in Theorem 2, can you comment on this?  4) Choice of PNN lines How important is the choice of PNN lines on which your project the weights? For instance, would you get similar guarantees with random projections? Would this simply affect the eigenvalues of the kernel function?  5) Gaussian input “We analyze population risk landscapes of two-layer PNNs with jointly Gaussian inputs” It’s not clear to me where in the analysis you require the Gaussian input? Can you please comment on this?  6) Choice of activation function The analysis relies on the a rectified linear unit (ReLU) activation function. How important is this choice? Can you generalize it to other types of activation functions?  7) Extension to >2 hidden layers Can the authors comment on extending their approach to more than 2 layers?  8) Proof of Theorem 2 for characterization of global optima You seem to only consider the fact that the norm of the gradient should be zero. Aren’t you only consider first-order critical points? What about the eigenvalues of the Hessian? In theorem 3, you said the Hessian is positive semidefinite which does not rule out zero-eigenvalues (i.e. degenerate points).  9) Global optimizer The authors seem to assume the existence of one global minimizer in all their theorems, even in the case where every local optimizer it not a global optimizer. For instance, take Theorem 1 in the case where the conditions of Theorem 2 are not satisfied.  10) Effect of variables d (input dimension) and r (number of PNN lines) As pointed out by the authors, Theorem 5 does not characterize the rate of the decrease as a function of d (and r). I wonder if the authors have conducted some experiments to see the empirical effect of d and r. From a theory point of view, only the regime d->infty is analyzed in the paper. From an practitioner point of view, I think it would really be interesting to understand how large r should be to reach a good level of performance.  11) MNIST experiments: The authors only use a subset of MNIST (they only consider images of 1’s and 2’s). Can you elaborate on this choice? Where is the difficulty in extending this to >2 classes?  12) Appendix The numbering scheme used in the appendix is different from the main paper, sometime making it difficult to find the proof of a theorem. a) For instance, where is the proof of Theorem 1 in the appendix? The closest seems to be theorem 4 where the form of the Kernel C is much simpler that given in the main paper. Can you explain how you derive the form of the kernel giving in Theorem 1 in the main paper? b) Similar remarks for other theorems. I think you should consider re-organizing the supplementary material to make it easier for the reader to find the proofs of the theorems cited in the paper.


Memory Replay GANs: learning to generate images from new categories without forgetting
Previous works on sequential learning address the problem of forgetting in discriminative models. In this paper we consider the case of generative models. In particular, we investigate generative adversarial networks (GANs) in the task of learning new categories in a sequential fashion. We first show that sequential fine tuning renders the network unable to properly generate images from previous categories (i.e. forgetting). Addressing this problem, we propose Memory Replay GANs (MeRGANs), a conditional GAN framework that integrates a memory replay generator. We study two methods to prevent forgetting by leveraging these replays, namely joint training with replay and replay alignment. Qualitative and quantitative experimental results in MNIST, SVHN and LSUN datasets show that our memory replay approach can generate competitive images while significantly mitigating the forgetting of previous categories.
While previous works study forgetting in discriminative tasks, in this paper we focus on forgetting in generative models (GANs in particular) through the problem of generating images when categories are presented sequentially as disjoint tasks. The closest related work is [24], that adapts elastic weight consolidation (EWC) [4] to GANs. In contrast, our method relies on memory replay and we describe two approaches to prevent forgetting by joint retraining and by aligning replays. The former includes replayed samples in the training process, while the latter forces to synchronize the replays of the current generator with those generated by an auxiliary generator (a snapshot taken before starting to learn the new task). An advantage of studying forgetting in image generation is that the dynamics of forgetting and consolidation can be observed visually through the generated images themselves.

Update following the author rebuttal:  I would like to thank the authors for their thoughtful rebuttal. I feel like they appropriately addressed the main points I raised, namely the incomplete evaluation and the choice of GANs over other generative model families, and I'm inclined to recommend the paper's acceptance. I updated my review score accordingly.  -----  The paper presents a conditional GAN framework to combat catastrophic forgetting when learning new image categories in a sequential fashion. The paper is well-written and its exposition of the problem, proposed solution, and related work is clear.  Starting from the AC-GAN conditional generative modeling formulation, the authors introduce the notion of a sequence of tasks by modeling image classes (for MNIST, SVHN, and LSUN) in sequence, where the model for each class in the sequence is initialized with the model parameters for the previous class in the sequence. They note that in this setup the AC-GAN’s classifier cannot be used, as for any given task there is only one class being trained on.  The idea put forward is to use the generator as a memory replay mechanism to prevent forgetting when training for the current task. Two approaches are considered:  - A joint replay approach (MerGAN-JTR), where upon completing a training task the model is sampled from, and the samples for previous tasks are combined with the dataset for the current task to form a hybrid real/synthetic class-conditional dataset. In this setup the AC-GAN formulation is used. - A replay alignment approach (MeRGAN-RA), where class-conditional samples for previous tasks are drawn continuously throughout training using the current value of the generator’s class-independent parameters and are compared with the class-conditional samples of the hybrid dataset using an L2 pixelwise loss (using the same latent samples z for both). In this setup the AC-GAN’s classifier is not used.  The two approaches, along with several baselines -- naive sequential fine-tuning (SFT), elastic weight consolidation (EWC), deep generative replay (DGR), and joint training (JT) -- are trained on sequences of classes from MNIST, SVHN, and LSUN and evaluated qualitatively and quantitatively.  For MNIST and SVHN, qualitative evaluation consists of examining class-conditional model samples after completing the last training task in the sequence and shows that both proposed approaches perform better than the competing EWC and DGR approaches (especially for SVHN). For LSUN, qualitative evaluation compares both proposed approaches with the naive SFT baseline and shows samples drawn after training on each task in the sequence. I would have liked to see a comparison with EWC and DGR here as well; can the authors tell how they compare?  In terms of quantitative evaluation, the accuracy of an external classifier on class-conditional samples is presented for MNIST, SVHN, and LSUN, showing that the proposed approaches outperform the naive SFT baseline as well as the competing approaches. Once again, I would have liked to see a comparison with EWC and DGR for LSUN. I find it interesting that for MNIST the proposed approaches do better than the upper-bound JT baseline; do the authors have an idea why? I would like to see confidence intervals on the accuracies, as it may help decide whether the improved performance over JT is statistically significant or not. Models trained on LSUN are also evaluated in terms of Fréchet Inception Distance, showing again that the two proposed approaches perform much better than the naive SFT baseline.  One criticism I would have to voice about the evaluation procedure is that the sequence of class-conditional tasks setup is a little contrived. I agree with the premise that in real-world scenarios the full training data is often not immediately available, but at the same time the sort of input distribution drifts we are likely to encounter is probably smoother than the discrete shifts in input distribution corresponding to switching the object class (an example would be the day/night cycle a camera pointed at a highway experiences). Moreover, information about the input distribution -- be it a discrete class label or some continuous attribute representation -- probably won’t be available in real-world scenarios.  Having said that, I am aware this is an issue not unique to the proposed method, and I am okay with the use of the proposed evaluation setup given that most previous work on the topic also uses a similar evaluation setup -- and also given that the search for a better setup is still an open question to me and I don’t have a better idea to propose.  Another criticism I have is that the choice of GANs as the training framework makes it unnecessarily difficult to compare and evaluate different methods: the right way of evaluating GANs is still very much an open problem, which means that any choice of GAN evaluation metric comes with its own set of caveats. For instance, one problem with the classifier accuracy metric is that the model could produce class-conditional samples achieving a high accuracy by collapsing the generator to only a few exemplars per class.  Given that the proposed method appears to be applicable to any class-conditional generative model, I think the choice of an explicit likelihood model family along with a log-likelihood metric would have been easier to defend from an evaluation point of view.  In summary, the paper presents a clear exposition of an idea to solve a very relevant research problem, namely catastrophic forgetting in generative models. I am a bit less enthusiastic about the empirical evaluation, mostly because I feel it is incomplete (as I said, I would like to see comparisons with competing methods for LSUN), and because the choice of model family and evaluation metric makes it harder to compare and evaluate competing methods.


Evolved Policy Gradients
We propose a metalearning approach for learning gradient-based reinforcement learning (RL) algorithms. The idea is to evolve a differentiable loss function, such that an agent, which optimizes its policy to minimize this loss, will achieve high rewards. The loss is parametrized via temporal convolutions over the agent's experience. Because this loss is highly flexible in its ability to take into account the agent's history, it enables fast task learning. Empirical results show that our evolved policy gradient algorithm (EPG) achieves faster learning on several randomized environments compared to an off-the-shelf policy gradient method. We also demonstrate that EPG's learned loss can generalize to out-of-distribution test time tasks, and exhibits qualitatively different behavior from other popular metalearning algorithms.

The authors consider a meta learning framework where in the outer loop a parametrized loss function is learned from environment reward with a simple ES approach, and in the inner loop standard policy gradient optimization is performed with the (meta-)learned loss function. The method is demonstrated on two continuous control tasks with body randomization and results show that (during test time) using loss functions learned by the proposed framework, the agent is able to achieve better performance compared to PPO, without observing environment rewards.  The idea of this paper is interesting from multiple perspectives: 1. Meta-learning useful and sharable priors for RL agent from a distribution of randomized tasks. 2. Reducing the reliance of environment rewards during test time (or deployment) 3. How to combine evolution training (ES) with RL, in a hierarchical way.  The use of ES can be better motivated. A question might be: why not using RL with temporal abstraction (such as many Hierarchical RL papers) for the outer loop to learn the parameterized loss? ES may (or may not) be a better approach to RL since the optimization without BP is easier? And it can address longterm credit assignment problem better than RL with temporal abstraction?  The authors claim that one contribution is the learned loss function reduces the reliance of external reward during testing time. This is also a very important message but without much support from experiments or relevant discussions. I think the main problem is the tasks are too simple, thus to me the results in this paper do not imply that this idea could work on real tasks (such sim2real transfer).  This also raises the concern of if the loss function here is really a "loss", in a traditional sense, since the loss function here is actually a parameterized model with additional MLP and temporal convolutions, which is also "transferred" in the evaluation time. This could change the story from learning loss functions to reusing a pretrained hierarchical reward encoding module, which has been well studied before. This also explains why there's no need for external reward using the learned loss function -- the external reward could be already encoded in the additional parameters. One argument could be the evolved loss function can be generalizable to more diverse tasks while simple transfer learning by modulation would fail.  Overall I think the paper has many interesting ideas even though the presentation could be further improved. I recommend acceptance.


Semidefinite relaxations for certifying robustness to adversarial examples
Despite their impressive performance on diverse tasks, neural networks fail catastrophically in the presence of adversarial inputs-imperceptibly but adversarially perturbed versions of natural inputs. We have witnessed an arms race between defenders who attempt to train robust networks and attackers who try to construct adversarial examples. One promise of ending the arms race is developing certified defenses, ones which are provably robust against all attackers in some family. These certified defenses are based on convex relaxations which construct an upper bound on the worst case loss over all attackers in the family. Previous relaxations are loose on networks that are not trained against the respective relaxation. In this paper, we propose a new semidefinite relaxation for certifying robustness that applies to arbitrary ReLU networks. We show that our proposed relaxation is tighter than previous relaxations and produces meaningful robustness guarantees on three different foreign networks whose training objectives are agnostic to our proposed relaxation.

Summary =============== The paper proposes a method to certify the adversarial robustness of a given ReLU neural network. The main difficulty in certifying these networks is dealing with the non-convex ReLU constraints. The proposed method formulates these as quadratic constraints to obtain a quadratic program that is then relaxed to an SDP. The idea is natural and well-executed. An important feature of this approach is that the verified network does not need to be "verification-aware" (i.e. trained towards the goal of being verifiable by this procedure). The experimental results are compelling, constructing the first certificate of robustness for verification-oblivious MNIST models w.r.t. moderate sized L_infty perturbations. This is a clear step of progress towards robust and verifiable modes, thus I recommend acceptance.  Quality =============== The problem studied is important and has received significant attention recently. The method is natural and effective. All necessary experiments are conducted. There is a good amount of analysis explaining how the method differs in terms of approximation tightness from the corresponding LP relaxations qualitatively and quantitatively.  Clarity =============== The paper is well-written. The approach and analysis is clear. There are a few places where writing looks sloppy (e.g. beginning of Section 3.2) but this has no effect on the overall readability.  Originality =============== The approach is novel and is clearly differentiated from prior work. While SDP formulations for NN verification where also proposed in Raghunathan et al [17] the approach is essentially completely different. In fact, it's unclear if these SDP relaxation could scale reasonably to more than a single layer.  Significance =============== Training adversarially robust models is an important topic. There are currently three approaches towards evaluating these models: a) empirical, b) exact verification, c) efficient upper bounds. This paper makes a significant step in direction c). In fact it is the first method to verify independently trained networks on MNIST against moderate-sized adversaries (eps=0.1).  Additional comments =============== I don't understand the choice of using a 2-layer neural network for the LP-NN. The paper of Kolter and Wong [11] uses larger models and achieves a provable error rate of at most 8% with LP-cert (half the error rate you report in Table 1). Why wasn't the original model from their paper used?  Comments to the authors =============== 66. "..and is likely intractable.": In fact the ReLUplex paper shows that verifying ReLU networks is NP-hard in general. 274. "tighte", typo References: you list Madry et al. twice.  ** UPDATE ** ============== While I stand behind my original score based on the technical contributions of the paper, I agree with Reviewer 2 that several improvements to the exposition are in order. I believe that the authors should implement these improvements independent of the final decision. I personally would like to see experimental results for LP-NN applied to large models.


Deep Network for the Integrated 3D Sensing of Multiple People in Natural Images
We present MubyNet -a feed-forward, multitask, bottom up system for the integrated localization, as well as 3d pose and shape estimation, of multiple people in monocular images. The challenge is the formal modeling of the problem that intrinsically requires discrete and continuous computation, e.g. grouping people vs. predicting 3d pose. The model identifies human body structures (joints and limbs) in images, groups them based on 2d and 3d information fused using learned scoring functions, and optimally aggregates such responses into partial or complete 3d human skeleton hypotheses under kinematic tree constraints, but without knowing in advance the number of people in the scene and their visibility relations. We design a multi-task deep neural network with differentiable stages where the person grouping problem is formulated as an integer program based on learned body part scores parameterized by both 2d and 3d information. This avoids suboptimality resulting from separate 2d and 3d reasoning, with grouping performed based on the combined representation. The final stage of 3d pose and shape prediction is based on a learned attention process where information from different human body parts is optimally integrated. State-of-the-art results are obtained in large scale datasets like Human3.6M and Panoptic, and qualitatively by reconstructing the 3d shape and pose of multiple people, under occlusion, in difficult monocular images.

This paper presents a multi-stage deep network for recovering 3D poses and shapes of multiple people from a single image. Unlike the top-down approaches that use human detectors to decompose the problem into single-person subtasks, this paper follows a recent bottom-up method that detects all the necessary local features so that the multi-person sensing problem can be solved jointly by graph matching. Given extracted features from the input image, the first stage predicts 2D keypoint heatmaps and 3D pose hypotheses on those limb pixels. The second stage first extracts local features from all the limb candidates and predicts their scores of being valid limbs or connections, and then solves a binary integer programming problem based on limb scores and the skeleton kinematic tree to group keypoints into 2D skeletons. In the third stage, the predicted 3D pose hypotheses on the predicted limb pixels are combined to synthesis the final 3D pose of each skeleton. The body shape is based on the SMPL model. The fitted shape and pose parameters (alpha and beta) are predicted from 3D skeletons via a differentiable mesh animation function. As the alpha and beta are unsupervised, GMM and L2 priors are used to regularize them. Results are first reported on a subset of Human3.6M with a single person in each image. Multi-person results are reported on CMU Panoptic datasets with comparisons to previous optimization-based methods.   +This paper presents a complex deep network system for single-image 3D human sensing. All the components are clearly explained and corresponding chosen techniques are also well motivated, so the overall pipeline seems pretty reasonable. All the math notations are also well defined.  +The limb scoring network is pretty interesting. Previous bottom-up approaches either predict dense limb vectors (PAFs) or some forms of per-pixel hough votes. The limb scoring network instead simplifies the problem into a light-weighted limb hypothesis verification problem based on sparsely sampled features.  +The unsupervised SMPL modeling fitting is also interesting.  +The good quantitative results on Human3.6M and CMU Panoptic datasets verify the effectiveness of the proposed method.  -As it is a system-level work, such individual component may lack sufficient evaluations. For example, the limb scoring network bears some similarities with the part affinity fields in OpenPose. Although this paper argues that PAFs have some artificial parameters, e.g. the width of each PAF and need greedy grouping, it will be great to show some examples to compare the grouping quality and also present quantitative evaluations on 2D pose detection. I think it will strengthen the contribution and shed light on how better 2D pose detection can improve the overall results. The in-network SML fitting also needs some justifications. Experimental comparisons are need to demonstrate the claimed advantages over direct alpha-beta regression in [14]. -Some technical details may be missing. How is the person’s scene translation t being predicted? It is quite an important parameter. If the scene translations of two adjacent people are too close, there will be mesh penetration. But I didn’t see such results in the paper. I’m curious how this paper solved it or just didn’t present such results. Related to this question, since it is multi-person 3d pose detection, so how the 3D pose of each person is represented? The joint locations relative to root or joint locations in the scene? What is the value of N_samples? It is not clear from Fig. 3(b) that how the samples are extracted. A better figure is needed.  -It will be great if this paper can present some failure cases which will inspire future works.


GLoMo: Unsupervised Learning of Transferable Relational Graphs
Modern deep transfer learning approaches have mainly focused on learning generic feature vectors from one task that are transferable to other tasks, such as word embeddings in language and pretrained convolutional features in vision. However, these approaches usually transfer unary features and largely ignore more structured graphical representations. This work explores the possibility of learning generic latent relational graphs that capture dependencies between pairs of data units (e.g., words or pixels) from large-scale unlabeled data and transferring the graphs to downstream tasks. Our proposed transfer learning framework improves performance on various tasks including question answering, natural language inference, sentiment analysis, and image classification. We also show that the learned graphs are generic enough to be transferred to different embeddings on which the graphs have not been trained (including GloVe embeddings, ELMo embeddings, and task-specific RNN hidden units), or embedding-free units such as image pixels.

This paper presents a method to transfer graph structures learned on unlabeled data to downstream tasks, which is a conceptual shift from existing research that aims to transfer features (e.g., embeddings). The method consists of jointly training a feature and graph predictor using an unsupervised objective (which are decoupled) and then extracting only the output of the graph predictor for downstream tasks, where it is multiplicatively applied to arbitrary features. The method yields small improvements on a variety of NLP and vision tasks, and the qualitative analysis of the learned graphs does not convince me that it learns "meaningful" substructures. Overall, however, the paper has a compelling and promising idea (graph transfer), and it seems like there is room to improve on its results, so I'm a weak accept.  Detailed comments: - Is "unsupervisedly" a word? It sounds weird...  - The objective function in eq 3 is interesting and could have potential uses outside of just graph induction, as it seems especially powerful from the ablations in table 2... Even for feature learning (e.g., ELMo), predicting the next D words instead of just the next word may give gains. However, I'd like more details on how exactly the RNN decoder was implemented. Did the authors feed the ground-truth previous word/pixel at every step (making the decoder a language model) or try and predict the next D words/pixels directly from the hidden state at time t (e.g., no input feeding)? The latter method seems like it would result in more powerful representations, as the former gives the decoder a lot of leeway to ignore parts of the original context after a couple predictions.  - Did you try experimenting with the features learned by your feature predictor as embeddings (instead of GloVe or ELMo)? Perhaps those features are even better because they were jointly trained with the graph predictor?  - The decoupling between feature and graph predictor is not very clearly described. It appears like they are not entirely decoupled, as the feature predictor takes the learned graph as input, and the graph predictor is only trained with one type of feature predictor... Fig 2 is also confusing, as there are arrows from the graph predictor to the feature predictor but the objective is computed over the "mixed graph". It would be easier to understand if the graph and feature predictor diagrams were switched, and the "mixed graph" explicitly shown as a combination of graph * weighted feature sum at each layer.   - Instead of performing transfer with just a single graph M as in sec 2.2, it seems like you could also try using each layer-specific affinity matrix individually (e.g., if your downstream model also had multiple layers).  - Fig 3 is very confusing. Are these visualizations of M (the mixture of graphs)? How does each row sum to 1 if many of them are completely blank? A color scale would be very helpful here, and also a more informative caption.


Learning long-range spatial dependencies with horizontal gated recurrent units
Progress in deep learning has spawned great successes in many engineering applications. As a prime example, convolutional neural networks, a type of feedforward neural networks, are now approaching -and sometimes even surpassing -human accuracy on a variety of visual recognition tasks. Here, however, we show that these neural networks and their recent extensions struggle in recognition tasks where co-dependent visual features must be detected over long spatial ranges. We introduce a visual challenge, Pathfinder, and describe a novel recurrent neural network architecture called the horizontal gated recurrent unit (hGRU) to learn intrinsic horizontal connections -both within and across feature columns. We demonstrate that a single hGRU layer matches or outperforms all tested feedforward hierarchical baselines including state-of-the-art architectures with orders of magnitude more parameters.

Summary   The authors introduce an image classification model designed for images which have non-local features, i.e. features with long spatial distance. Their recurrent neural network, called hGRU, is said to be inspired by neuroscience theories on cortical circuits, but based on an LSTM variant called GRU [17]. They provide empirical results on the Pathfinder dataset, where two white dots on a black background are either connected by white dashed lines or not. The authors compare to a variety of other models. While several models seem to achieve near 100% test accuracy, hGRU does so with 200 times fewer trainable parameters than the nearest competitor. Finally, they compare and, to some extent, analyse human scores and a trained hGRU on a contour detection task in a natural scene.  Disclaimer: This review will focus mostly focus on the Machine Learning aspect since we have little expertise in visual cortical circuits.   The paper correctly identifies known limitations of CNNs and proposes a solution given the current state of knowledge regarding visual cortical circuits. This may be a strength of the paper as it links RNN machine learning models to such theories. For the Pathfinder task, the hGRU seems to show good empirical results.   One weakness of this paper is its lack of clarity and aspects of the experimental evaluation. The ResNet baseline seems to be just as good, with no signs of overfitting. The complexity added to the hGRU model is not well motivated and better baselines could be chosen. What follows is a list 10 specific details that we would like to highlight, in no particular order:  1. Formatting: is this the original NIPS style? Spacing regarding sections titles, figures, and tables seem to deviate from the template. But we may be wrong.  2. The general process is still not 100% clear to us. The GRU, or RNNs in general, are applied to sequences. But unlike other RNNs applied to image classification which iterate over the pixels/spatial dimensions, the proposed model seems to iterate over a sequence of the same image. Is this correct?  2.1 Comment: The general high-level motivation seems to be map reading (see fig 1.c) but this is an inherently sequential problem to which we would apply sequential models so it seems odd that one would compare to pure CNNs in the first place.  3. Section 2 begins with a review of the GRU. But what follows doesn't seem to be the GRU of [17]. Compare eq.1 in the paper and eq.5 in [7]. a) there doesn't seem to be a trained transformation on the sequence input x_i and b) the model convolves the hidden state, which the standard GRU doesn't do (and afaik the convolution is usually done on the input stream, not on the hidden state). c) Since the authors extend the GRU we think it would make section 2 much more readable if they used the same/similar nomenclature and variable names. E.g., there are large variations of H which all mean different things. This makes it difficult to read.  4. It is not clear what horizontal connections are. One the one hand, it seems to be an essential part of the model, on the other hand, GRU is introduced as a method of learning horizontal connections. While the term certainly carries a lot of meaning in the neuroscience context, it is not clear to us what it means in the context of an RNN model.  5. What is a feed forward drive? The equations seem to indicate that is the input at every sequence step but the latter part of the sentence describes it as coming from a previous convolutional layer.   6. The dimensions of the tensors involved in the convolution don't seem to match. The convolution in a ConvNet is usually a 2D discrete convolution over the 2 spatial dimensions. If the image is WxHxC (width, height, and, e.g., the 3 colour channels), and one kernel is 1x1xC (line 77) then we believe the resulting volume should be WxHx1 and the bias is a scalar. The authors most certainly want to have several kernels and therefore several biases but we only found this hyper-parameter for the feed forward models that are described in section 3.4. The fact that they have C biases is confusing.  7. Looking very closely at the diagram, it seems that the ResNet architectures are as good if not even slightly better than the hGRU. Numerical measurements would probably help, but that is a minor issue. It's just that the authors claim that "neural networks and their extensions" struggle in those tasks. Since we may include ResNets in that definition, their own experiment would refute that claim. The fact that the hGRU is using many fewer parameters is indeed interesting but the ResNet is also a more general model and there is (surprisingly) no sign of overfitting due to a large model. So what is the motivation of the authors of having fewer parameters?   8. Given the fact that ResNets perform so well on this task, why didn't the authors consider the earlier and closely related highway (HW) networks [high1]? HWs use a gating mechanism which is inspired by the LSTM architecture, but for images. Resnets are a special case of HW, that is, HW  might make an even stronger baseline as it would also allow for a mix and gain-like computation, unlike ResNets.  9. In general, the hGRU is quite a bit more complex than the GRU. How does it compare to a double layer GRU? Since the hGRU also introduces a two-layer like cell (inhibiton part is seperated by a nonlinearity from the exhibition part) it seems unfair to compare to the GRU with fewer layers (and therefore smaller model complexity)  10. Can the authors elaborate on the motivation behind using the scalars in eq 8-11? And why are they k-dimensional? What is k?   11. Related work:  The authors focus on GRU, very similar to LSTM with recurrent forget gates [lstm2], but GRU cannot learn to count [gru2] or to solve context-free languages [gru2] and also does not work as well for translation [gru3]. So why not use "horizontal LSTM" instead of "horizontal GRU"? Did the authors try?   What is the difference to PyramidLSTM [lstm3], the basis of PixelRNNs? Why no comparison? Authors compare against ResNets, a special case of the earlier highway nets [high1]. What about comparing to highway nets? See point 8 above.    [gru2] Weiss et al. On the Practical Computational Power of Finite Precision RNNs for Language Recognition. Preprint arXiv:1805.04908.  [gru3] Britz et al (2017). Massive Exploration of Neural Machine Translation Architectures. Preprint arXiv:1703.03906   [lstm2] Gers et al. “Learning to Forget: Continual Prediction with LSTM.“ Neural Computation, 12(10):2451-2471, 2000.   [lstm3] Stollenga et al. Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation. NIPS 2015. Preprint: arxiv:1506.07452, June 2015.  [high1] Srivastava et al. Highway networks. Preprints arXiv:1505.00387 (May 2015) and arXiv:1507.06228 (Jul 2015). Also at NIPS'2015.    After the rebuttal phase, this review was edited by adding the following text:  Thanks for the author feedback. However, we remain unconvinced. The baseline methods used for performance comparisons (on a problem on which few compete) are not the state of the art methods for such tasks - partially because they throw away spatial information the deeper they get, while shallower layers cannot connect the dots (literally) due to the restricted field of view. Why don't the authors compare to a state of the art  baseline method that can deal with arbitrary distances between pixels - standard CNNs cannot, but the good old multi-dimensional (MD) RNN can (https://arxiv.org/abs/0705.2011). For each pixel, a 2D-RNN implicitly uses the entire image as a spatial context (and a 3D-RNN uses an entire video as context). A 2D-RNN should be a natural competitor on this simple long range 2D task. The RNN is usually LSTM (such as 2D-LSTM) but could be something else. See also MD-RNN speedups through parallelization (https://arxiv.org/abs/1506.07452).  The submission, however, seems to indicate that the authors don’t even fully understand multi-dimensional RNNs, writing instead about "images transformed into one-dimensional sequences” in this context, although the point of MD-RNNs is exactly the opposite. Note that an MD-RNN in general does have local spatial organization, like the model of the authors. For any given pixel, a 2D-RNN sees this pixel plus the internal 2D-RNN states corresponding to neighbouring pixels (which already may represent info about lots of other pixels farther away). That’s how the 2D-RNN can recursively infer long range information despite its local 2D spatial neighbourhood wiring. So any good old MD-RNN is in fact strongly spatially organised, and in that sense even biologically plausible to some extent, AFAIK at least as plausible as the system in the present submission. The authors basically propose an alternative local 2D spatial neighbourhood wiring, which should be experimentally compared to older wirings of that type. And to our limited knowledge of biology, it is not possible to reject one of those 2D wirings based on evidence from neuroscience - as far as we can judge, the older 2D-RNN wiring is just as compatible with neurophysiological evidence as the new proposal.   Since the authors talk about GRU: they could have used a 2D-GRU as a 2D-RNN baseline, instead of their more limited feedforward baseline methods. GRU, however, is a variant of the vanilla LSTM by Gers et al 2000, but lacking one gate, that’s why it has those problems with counting and with recognising languages. Since the task might require counting, the best baseline method might be a 2D-LSTM, which was already shown to work on challenging related problems such as brain image segmentation where the long range context is important (https://arxiv.org/abs/1506.07452), while I don’t know of similar 2D-GRU successes.   We also agree with the AC regarding negative weights. Despite some motivation/wording that might appeal to neuroscientists, the proposed architecture is a standard ML model that has been tweaked to work on this specific problem. So it should be compared to the most appropriate alternative ML models (in that case 2D-RNNs). For now, this is a Machine Learning paper slightly disguised as a Computational Neuroscience paper.  Anyway, the paper has even more important drawbacks than the baseline dispute. Lack of clarity still makes it hard to re-implement and reproduce, and a lot of complexity is added which is not well motivated or empirically evaluated through, say, an ablation study.   Nevertheless, we encourage the authors to produce a major revision of this interesting  work and re-submit again to the next conference!


Learning to Repair Software Vulnerabilities with Generative Adversarial Networks
Motivated by the problem of automated repair of software vulnerabilities, we propose an adversarial learning approach that maps from one discrete source domain to another target domain without requiring paired labeled examples or source and target domains to be bijections. We demonstrate that the proposed adversarial learning approach is an effective technique for repairing software vulnerabilities, performing close to seq2seq approaches that require labeled pairs. The proposed Generative Adversarial Network approach is application-agnostic in that it can be applied to other problems similar to code repair, such as grammar correction or sentiment translation.

Update based on author rebuttal: The authors address some of my criticisms and promise to improve some of the motivation in subsequent drafts. The rebuttal doesn't change my impression of accepting this paper.  This paper proposes a system for correcting sequences, with a target application of fixing buggy source code. They use a sequence-to-sequence model within a GAN framework, which allows the model to be trained without paired source/target data. Some additional new tricks are proposed to make the model output consistent translations of the input. The model is tested on two synthetic tasks and a source code correction benchmark.  Overall, I thought this paper provided a compelling model which seems to function well on sequence-to-sequence tasks even without explicit source/target pairs. The authors focus only on datasets where explicit source/target pairs are available in order to compare their method to standard seq2seq models. This was a disappointment to me. As the authors say, "The major advantage of our approach is that it can be used in the absence of paired data, opening up a wide set of previously unusable data sets for the sequence correction task."  While the experiments suggest the model can still perform reasonably well without explicitly using the source-target pairs, the model itself was not tested in this regime. In particular, a truly realistic test of the model would be in a setting where there is no guarantee that a given source sequence has a paired target sequence anywhere in the training data. I think this paper would be a lot stronger if it was shown to work in this setting - perhaps by being deployed in a source code vulnerability detection system and proving to be genuinely useful. That being said, I am not familiar with the source code correction problem domain so I may be missing the significance of the reported results. Apart from this, I found some of the presentation confusing or otherwise amenable to improvement; some specific comments are below.   > "only a limited number of relatively small paired sequence correction datasets exist" In the context (and via the citation), it sounds like you are discussing machine translation datasets, not "sequence correction" datasets, but in that case this statement is false - there are many translation datasets, and many of them are large. From later context, I think you mean generic datasets which deal with correcting sequences (like fixing grammar errors, or source code correction), but I was thrown off by this sentence in the midst of a paragraph about MT.  - In your description of WGAN, you don't mention that in order to actually estimate Wasserstein distance the function class must be constrained to 1-Lipschitz functions, for example by weight clipping (as originally proposed in WGAN) or via a gradient penalty (as proposed by Gulrajani et al.)  You mention this later, but it's inaccurate to introduce the WGAN the way that you do without mentioning this.  - You state that you feed the discriminator "soft" network outputs directly rather than sampling. Do you still sample from the generator's output and feed the sample back in? If you don't, your model is not very powerful because it no longer has a stochastic variable and all possible future outcomes must be encoded in the deterministic hidden state. If you do, then you can't propagate gradients though the sampling operation so your model is only learning one-step dependencies. Which did you do and why? From later discussion (4.2) it sounds like the latter; I'd suggest addressing this early on.  - Your discussion of the weak discriminator feels a little incomplete to me. It is nice that with a weak discriminator you obtain useful gradients, but there is a trade-off between discriminator strength and the generator's abilities, namely because the generator only ever learns how to solve the task through the discriminator. Being able to utilize discriminator gradients is not enough - the discriminator has to be able to provide a good model of the data distribution in order to be useful. Of course, as you point out, useful gradients are a prerequisite, but my point is that the generator's performance is limited in the case of a weak generator. I would suggest adding a few sentences to comment on this.  - More common is to call the "Improved WGAN" loss the "WGAN-GP" loss, due to the gradient penalty.


Di erentially Private Testing of Identity and Closeness of Discrete Distributions
We study the fundamental problems of identity testing (goodness of fit), and closeness testing (two sample test) of distributions over k elements, under di erential privacy. While the problems have a long history in statistics, finite sample bounds for these problems have only been established recently. In this work, we derive upper and lower bounds on the sample complexity of both the problems under (Á, ")-di erential privacy. We provide sample optimal algorithms for identity testing problem for all parameter ranges, and the first results for closeness testing. Our closeness testing bounds are optimal in the sparse regime where the number of samples is at most k. Our upper bounds are obtained by privatizing non-private estimators for these problems. The non-private estimators are chosen to have small sensitivity. We propose a general framework to establish lower bounds on the sample complexity of statistical tasks under di erential privacy. We show a bound on di erentially private algorithms in terms of a coupling between the two hypothesis classes we aim to test. By carefully constructing chosen priors over the hypothesis classes, and using Le Cam's two point theorem we provide a general mechanism for proving lower bounds. We believe that the framework can be used to obtain strong lower bounds for other statistical tasks under privacy.

This paper studies two basic testing problems for distributions over finite domains. The first is the identity testing problem, where given a known distribution q and i.i.d. samples from p, the goal is to distinguish between the case where p = q and the case where the distributions are far in total variation distance. A second, closely related problem, is the closeness testing problem where q is also taken to be an unknown distribution accessed via i.i.d. samples. This submission studies the sample complexity of solving these problems subject to differential privacy. For identity testing, it gives sample complexity upper and lower bounds which match up to constant factors. For closeness testing, it gives bounds which match in the "sparse" regime where the data domain is much larger than the accuracy/privacy parameters, and which match up to polynomial factors otherwise.  The private identity testing problem that the authors consider had been studied in prior work of Cai, Diakonikolas, and Kamath, and this submission gives an improved (and simplified) upper bound. Private closeness testing appears not to have been studied before. (Independent work of Aliakbarpour, Diakonikolas, and Rubinfeld gives similar upper bounds which are complemented by experimental results, but does not study lower bounds.)  The upper bounds are obtained by a) privatizing a reduction from identity testing to uniformity testing, due to Goldreich, and b) modifying recent uniformity and closeness testers to guarantee differential privacy. These modifications turn out to be relatively straightforward, as the test statistics introduced in these prior works have low sensitivity. Hence, private testers can be obtained by making randomized decisions based on these test statistics, instead of deterministically thresholding them.  Lower bounds follow from a new coupling technique. The idea is to construct two distributions on finite samples that an accurate tester must be able to distinguish. However, if there is a coupling between these distributions where the samples have low expected Hamming distance, then differentially private algorithms will have a hard time telling them apart. This is a nice trick which is powerful enough to prove tight lower bounds, and extends generically to give lower bounds for (eps, delta)-DP.  To summarize, the results are fairly interesting, deftly leverage recent advances in distribution testing, and introduce some nice new techniques. I think this is a solid paper for NIPS.  After reading author feedback: I still believe this is a strong paper and should be accepted.


When do random forests fail?
Random forests are learning algorithms that build large collections of random trees and make predictions by averaging the individual tree predictions. In this paper, we consider various tree constructions and examine how the choice of parameters affects the generalization error of the resulting random forests as the sample size goes to infinity. We show that subsampling of data points during the tree construction phase is important: Forests can become inconsistent with either no subsampling or too severe subsampling. As a consequence, even highly randomized trees can lead to inconsistent forests if no subsampling is used, which implies that some of the commonly used setups for random forests can be inconsistent. As a second consequence we can show that trees that have good performance in nearest-neighbor search can be a poor choice for random forests.

This paper studies necessary conditions for (infinite) ensembles of randomized regression trees to be inconsistent (in regression). The authors first establish that locality and diversity are necessary conditions for the consistency of forests. From this result, they show that several types of forests are inconsistent: forests of extremely randomized deep trees satisfying the nearest-neighbor preserving property (which includes two random projection tree algorithms as instances), forests of fully-grown trees with fast shrinking cells, and finally forests grown with too severe subsampling.   The paper contains interesting and, to the best of my knowledge, original theoretical results about (in)consistency of forests of randomized trees. The originality of the paper is to try to characterize conditions under which generic random forests models are inconsistent, while other papers focus on consistency proof of some simplified algorithms. The paper is also very well written and pleasant to read. The authors very well summarize their main theoretical results in the paper, while detailed proofs are given in the supplementary material.   Although the paper is well written, I find however that several statements or implications that are drawn from the theorems in the paper are too strong or not strictly correct. The authors overstate a bit the surprising nature of their results and how much they contradict statements from the literature. Just to give two examples of statements that I found not well argued:  1) About the importance of subsampling: "we show that subsampling of data points during the tree construction phase is critical: forests can become inconsistent with either no subsampling or too severe subsampling", "without subsampling, forests of deep trees can become inconsistent due to violation of diversity", "In summary, our results indicate that subsampling plays an important role in random forests and may need to be tuned more carefully than other parameters". I have two problems with these statements: First, I don't see where it is formally proven (in this paper or in another) that adding subsampling to the different tree models for which inconsistency is proven in the paper will lead to consistency. Actually, page 6, the authors say "we speculate that sufficient subsampling can make the forests consistent again...We leave this for future work". In the absence of a proof, I think the previous statements are premature. Second, even if subsampling would lead to consistency, I don't think subsampling is really crucial per se. Forests of deep trees are inconsistent because the trees are deep. Subsampling is thus clearly not the only or even the most obvious solution. Another solution to make them consistent could be for example to prune the trees (in a way that depends on learning sample size). As subsampling is not the only solution, it is not critical per se, what is critical is the effect that it has on diversity, not its added randomness. 2) About randomness and overfitting: "Theorem 2 implies that forests of extremely randomized trees can still overfit. This results disagrees with the conventional belief that injecting more "randomness" prevents tree from overfitting" and in the conclusion: "Another implication is that even extremely randomized tree can lead to overfitting forests, which disagrees with the conventional belief that injecting more "randomness" will prevent trees from overfitting". I don't understand this statement and the link with the theorem provided in the paper. The conventional belief is not that injecting more randomness implies NO overfitting but rather than injecting more randomness implies LESS overfitting. And I see no violation of this common belief in the paper: forests of extremely randomized deep trees have a low variance but not a zero variance (that's why they are not consistent), and adding subsampling is a way to add even more randomness that can lead to less overfitting and thus consistency, as suggested in the paper. So, I don't see any contradiction or surprising results here.  Overall, I think that the authors should be more cautious when discussing the implications of their theorems. This requires in my opinion a significant rewrite of some parts of the paper.  The paper topic does not fit also very well to the format of a conference paper. It does not focus on a single clear idea, as the contribution is a series of theorems that remains incremental with respect to previous results in this domain and also does not end any story. Their practical impact is also limited as they concern simplified models (UW trees) that are not really competitive in practice. The paper has also a 11-pages supplementary material that is difficult to check within the time allocated for reviews at conferences. A statistical journal might be a better target for this work.  Minor comments: * All theorems refer to a data model in Eq (2) and the authors mention that restrictions on the data model are more important for Lemma 2 than for Lemma 1. Actually, I could not find any Eq (2), neither in the paper nor in the appendix. What is this data model?  * The use of the term extremely randomized trees in the paper does not seem to match the use of this name in (Geurts et al., 2006) and is therefore a bit misleading. Extremely randomized trees in (Geurts et al., 2006) are not UW trees because the best split is still selected among K inputs selected at random, which is thus selected on the basis of the labels. Extremely randomized trees are UW trees only when K=1 and in this case, they are called Totally randomized trees in (Geurts et al., 2006). Furthermore, extremely randomized trees in (Geurts et al., 2006) are grown in a specific manner, while Definition 5 is generic. Why not calling them simply "deep UW trees"?  Update after the author's reponses  I thank the authors for their response. They do not totally address my concerns however: - Importance of subsampling: "The reviewer is also correct in that we did not formally prove but only speculate that with proper subsampling, our RF models shown inconsistent in our paper could become consistent again". Then, I reiterate my comment that you can not yet claim in the paper that subsampling is critical. - Randomness and overfitting: "We agree with the reviewer that the conventional belief is that “injecting more randomness implies less overfitting".This is exactly what we would like to argue against". "We argue (here and in paper) that our Theorem 2 suggests that this type of thinking is incorrect, since highly randomized trees without subsampling can still overfit." Again, to me adding subsampling means injecting more randomness, which implies less overfitting. I still don’t see how the result in theorem 2 really contradicts this statement. I agree however that different ways of injecting randomness are certainly not equivalent and that the RF variant of [Geurts et al., 2006] might overfit more than Breiman’s original Random Forests. But this is not strictly shown in the paper and I don’t really see how this could be proven as it is difficult to compare the levels of randomness of different randomization schemes. Note that the link between overfitting and consistency is not totally clear to me. At finite sample size, a model might overfit less than another, even if only the latter is consistent. Overall, I still believe that the discussion about randomness and overfitting in the paper is confusing and needs to be clarified.  Note that I acknowledge the value of the theoretical results in the paper. I still believe however that the authors should be more cautious in the conclusions they draw from these results.


Implicit Bias of Gradient Descent on Linear Convolutional Networks
We show that gradient descent on full width linear convolutional networks of depth L converges to a linear predictor related to the 2/L bridge penalty in the frequency domain. This is in contrast to fully connected linear networks, where regardless of depth, gradient descent converges to the 2 maximum margin solution.

The paper considers the problem of formalizing the implicit bias of gradient descent on fully connected linear/convolutional networks with an exponential loss. Building on the recent work by Soudry et al. which considered a one layer neural network with no activation the paper generalizes the analysis to networks with greater depth (with no activations) and the exponential loss. The two main networks considered by the authors and the corresponding results are as follows.  Linear Fully Connected Networks - In this setting the authors show that gradient descent in the limit converges to a predictor which in direction is the max margin predictor. This behaviour is the same as what was established in the earlier paper of Soudry et al for one layer neural networks. In particular the paper shows that depth has no effect in this setting on the implicit bias of gradient descent.   Convolutional Network - The paper further considers the case of convolutional networks and works in the model where every layer has the same width as the input and the convolutional filter has width equal to that of the layer. Note that they work with the one dimensional convolutional filter. For this model the authors show that the solution converges in direction to a critical point of the problem - minimize the 2/L norm of the fourier coefficients of the predictor given the predictor classifies perfectly. In particular this says that as the depth of the network increases the classifier learned is encouraged to be sparse in the fourier domain.   Characterizing implicit bias of gradient descent has attracted a lot of recent attention as it seems to be the source for the auto regularization effect observed in neural net training. While the work of Soudry et al was the first step in this direction, this paper makes the above two very interesting generalizations of that work. In particular the characterization obtained for linear convolutional networks is illuminating and could be of great significance breeding further research.   The paper achieves the result by first showing that for any network which implements a homogenous polynomial in terms of its parameters, we have that the solution converges in direction to a perfect classifier with the smallest l_2 norm. Note that this result is in the parameter space. The two results then follow by understanding the minimum l2 norm solution in terms of the predictor it corresponds to. The interesting thing here is that if the network is assumed to be diagonal then the depth plays a role in the implicit bias. The result for convolutional case follows by noting that colvolutional networks correspond to diagonal networks in the setup considered by the authors.   The paper is well written (modulo a couple of typos), considers an interesting problem and provides an illuminating result. The discussion section of the paper is strong with multiple future directions. I think the paper would be a great addition to the program.    Typos - I believe there are typos in the definition of fourier coefficients (line 52 and line 140)  and the definition of the convolution line 70.


Fast Estimation of Causal Interactions using Wold Processes
We here focus on the task of learning Granger causality matrices for multivariate point processes. In order to accomplish this task, our work is the first to explore the use of Wold processes. By doing so, we are able to develop asymptotically fast MCMC learning algorithms. With N being the total number of events and K the number of processes, our learning algorithm has a O(N ( log(N ) + log(K))) cost per iteration. This is much faster than the O(N 3 K 2 ) or O(K 3 ) for the state of the art. Our approach, called GRANGER-BUSCA, is validated on nine datasets. This is an advance in relation to most prior efforts which focus mostly on subsets of the Memetracker data. Regarding accuracy, GRANGER-BUSCA is three times more accurate (in Precision@10) than the state of the art for the commonly explored subsets Memetracker. Due to GRANGER-BUSCA's much lower training complexity, our approach is the only one able to train models for larger, full, sets of data.

Summary: This work studies the Granger causality in a special type of point processes called Wold processes.  This type of processes is defined in terms of a Markovian transition on the inter-event times.  The Markovian transition between increments is established by the transition density functions. The intensity of the point processes considered in this work consists of two components. The first one is constant value that represents the exogenous events in the process and the other one is a summation of rational terms that capture the influences of the other processes including self-excitements. Learning the Granger causality in such processes boils down to learning the parameters of the second component. This work proposes an MCMC-based Algorithm for learning these parameters called Granger-Busca.  The complexity of the proposed algorithm is O(N(log N + log K)), where N being the total number of events and K the number of processes. They also validate their algorithm’s performance via several experiments.   Pros:  Studying the Granger causality in Wold point processes is novel and can be interesting to several disciplines.  The proposed algorithm is faster that the state of the art (for Multivariate Hawkes processes).   Cons:  The proposed method is designed for a special sub-class of Wold processes with particular form of the intensity function, Equation (3). Unlike the proposed methods for Hawkes processes, the proposed method in this work cannot be extended to processes more general forms of the transition functions. The comparison between the complexity of the proposed method and the previous algorithms for Hawkes is not a fair comparison, since there are dealing with different classes of point processes with different complexity. For instance, as also mentioned in the paper, Hawkes processes do not have finite memory.  The MemeTracker dataset was previously modeled via Hawkes processes and in this work using Busca processes. What is the justification for this modeling? What is their model validation criterion? It seems the authors merely refer the work of Bacry  and Song groups. Some very relevant references are missing. For instance: Learning network of multivariate hawkes processes: A time series approach Etesami et al UAI 2016.  Online learning for multivariate Hawkes processes Yang et al NIPS 2017.  Hawkes Process Modeling of Adverse Drug Reactions with Longitudinal Observational Data Bao et al. Machine Learning and Healthcare 2017.  The multivariate hawkes process in high dimensions: Beyond mutual excitation Chen et al.  ArXiv   Overall, this interesting work can be strengthened by generalizing the proposed algorithm to a broader class of transition functions. I rate this work as “marginally above acceptance” because their algorithm reduces the complexity extensively compared to the state of the art. However, as I mentioned the state of the art algorithms are considering Hawkes processes that have infinite memory.


Blind Deconvolutional Phase Retrieval via Convex Programming
We consider the task of recovering two real or complex m-vectors from phaseless Fourier measurements of their circular convolution. Our method is a novel convex relaxation that is based on a lifted matrix recovery formulation that allows a nontrivial convex relaxation of the bilinear measurements from convolution. We prove that if the two signals belong to known random subspaces of dimensions k and n, then they can be recovered up to the inherent scaling ambiguity with m >> (k + n) log 2 m phaseless measurements. Our method provides the first theoretical recovery guarantee for this problem by a computationally efficient algorithm and does not require a solution estimate to be computed for initialization. Our proof is based Rademacher complexity estimates. Additionally, we provide an ADMM implementation of the method and provide numerical experiments that verify the theory.

This paper considers blind deconvolution from phaseless measurements, i.e., recovering w and x from the measurements y = |F(w \cconv x)|, where \cconv is the circular convolution. The problem is motivated by an application in visible light communication. When w and x lie on two known random subspaces, the paper proves that w and x can be recovered (up to certain intrinsic ambiguities) via solving a convex program, provided that there are enough phaseless measurements (i.e. y is long enough) that approximately match the intrinsic degrees of freedom, i.e., length(w) + length(x).   As acknowledged by the authors in Sec 1.4, the problem encompasses many difficult problems and hence the result (Theorem 1) and its implications should be interpreted carefully. For example, without the random subspace assumption on x and assuming x = 1 (all one vector or matrix), then the problem reduces to the classic Fourier phase retrieval problem, which does not warrant a provable solution yet. The randomness in the assumption seems to alleviate many aspects of difficulties associated with the original problem. As indicated in the paper, the derived convex relaxation does not seem to work well on more deterministic setting for the subspaces.   As a theoretical paper that sheds light on the problem, there is a couple of interesting ideas contained in the paper.   1) The convex relaxation is novel, especially in dealing with the quadratic constraints. Exploiting positiveness of products of PSD matrices enables restriction of the hyperbola constraint into one particular branch, which in turn enables the convex relaxation. Similar strategy was used in the BranchHull paper (Aghasi et al 2017a), although there restriction was possible at the expense of additional side information on signs.     2) To show recovery performance of the convex relaxation in Eq (5), some nontrivial technicalities are dealt with. First, the constraint set is linearized near the optimal set, i.e., Lemma 1, leading to a relaxed convex program Eq (8). Second, it is shown the descent cone does not meet the subspace of nontrivial perturbation when m (# measurements) is sufficiently large, which implies that any nontrivial feasible perturbation strictly increases the objective value. Similar proof ideas also appear in the BranchHull paper. The matrix version entailed here is technically more involved.   Overall, there convex relaxation and the analysis are interesting, despite the precursors that are very relevant. The practicality of the result is arguable, nevertheless.    Additional Comments:   * Line 44 -- 52: Another motivation the current reviewer is aware of is some ill-posed Fourier phase retrieval problem as discussed in   -- Elser, Veit, Ti-Yen Lan, and Tamir Bendory. "Benchmark problems for phase retrieval." arXiv preprint arXiv:1706.00399 (2017).   When the 2D signal can be modeled as a \cconv x --- say in the above paper where the data are simulated as sparse superposition of basic patterns over the image (e.g., Fig 1 there), the 2D signal is not injective if treated in the classic Fourier phase retrieval framework and hence fundamentally not recoverable. Instead, if one model the process as y = |F(a \cconv x)|, there may still be hope of recovery due to the explicit account of the structures present.   * Around Line 93: It may help to formally show why the relaxed constraint set in Eq (5) is convex.   * Around Line 93: The relaxation idea is very similar to the BranchHull paper. I think it is important to note this precursor here.   * How easy/hard/tricky is it to obtain a result if only one of the two subspaces is random, say one of w and x is sparse with known support? This might be more relevant to practice, in view of the comments in Sec 1.4 and the other motivation I discussed above.   * Page 5: What is the circled sum notation?