Word embeddings are widely used in machine learning based natural language processing systems. It is common to use pre-trained word embeddings which provide benefits such as reduced training time and improved overall performance. There has been a recent interest in applying natural language processing techniques to programming languages. However, none of this recent work uses pretrained embeddings on code tokens. Using extreme summarization as the downstream task, we show that using pre-trained embeddings on code tokens provides the same benefits as it does to natural languages, achieving: over 1.9x speedup, 5% improvement in test loss, 4% improvement in F1 scores, and resistance to over-fitting. We also show that the choice of language used for the embeddings does not have to match that of the task to achieve these benefits and that even embeddings pre-trained on human languages provide these benefits to programming languages.

THE EFFECTIVENESS OF PRE-TRAINED CODE EMBEDDINGS
Summary:
This work shows how pre-training word vectors using corpuses of code leads to representations that are more suitable than randomly initialized (and trained) representations for function/method name prediction (here called extreme summarization). This paper applies a standard language model to several collected corpuses of code written in different programming languages in order to pretrain the embeddings. It then uses a standard model for the extreme summarization task that takes pre-trained language model embeddings. This leads to speedups in training, improvements in validation loss, and less overfitting. I worry that these results didn't really need much proving given that we have already seen the exact same methods work with natural languages. It would actually be more surprising if they didn't work for programming languages, which suggests that the real question is whether code embeddings are actually more effective than natural language embeddings for this problem given that the authors show syntax of the code is far less important than the semantics of the words in the vocabulary. It is not clear that embeddings pre-trained with much more data on natural languages wouldn't work just as well.
Pros:
This paper takes the time to clearly explain the effectiveness of pre-training words vectors in a new setting. It is easy to follow and understand thanks to the clear organization and exposition.
Speedup and validation loss improvements are demonstrated for a variety of programming languages despite the final evaluation being only in Java, which is quite surprising.
The authors discover that overlap in actual programming language syntax is less important than overlap in semantic representation.
Because the models are out-of-the-box, it is easy to focus on the actual contributions of this paper related to pre-training.
Does seem to clearly demonstrate that pre-trained embeddings of some form should be used for this extreme summarization task.
Cons:
It is not clear how big the collected corpus is. This is important because the work that this paper cites on pre-trained embeddings (Mikolov et al 2013, Pennington et al 2014, McCann et al 2017, Peters et al 2018) typically use fairly large datasets for pre-training. All of these results might be watered down by insufficient pre-training data for the language model when in fact the results could be much stronger with more data. It would be nice to show the effects of pre-training dataset size as is done in the aforementioned previous works. Without this comparison, it is hard to tell whether the paper sufficiently explores this idea.
The models are both standard, out-of-the-box models. There is no novelty on the modeling side of this paper.
The pre-training methods are also not novel. They are methods that have already been shown to work applied in a slightly different setting.
It is not clear that the setting is actually different enough to require this pre-training. Comparing to randomly initialized embeddings is fine, but I would also like to see a comparison to other pre-trained embeddings like GloVe, GloVe+CoVe, or ELMo (Pennington et al 2014, McCann et al 2017, Peters et al 2018). Since the authors find that it is the semantics of the words that matter more than the syntax of any particular programming language, then perhaps it would actually be better to use pre-trained embeddings that tap into much larger amounts of data. At the very least, it seems it would make sense to perhaps supplement a standard pre-trained embedding with those suggested by the authors since so many of the words in the code must be English words. If this is too farfetch'd, then I would suggest that the authors provide some statistics showing why GloVe, GloVe+CoVe, and ELMo are not appropriate starting points for comparison, but the overlap from the pre-training corpuses is already so low that it seems supplementing with standard pre-trained embeddings should only help.
The evaluation dataset detailed in Allamanis et al 2016 uses two metrics: an F1 metric and an exact match metric. This paper only compares on validation loss. What's more, it reports everything in relative terms so that the raw improvement is masked until Figure 1 makes it somewhat possible to deduce. The problem here is that we don't know how a 0.0-0.5 raw improvement in validation loss translates to the metrics established for the dataset by Allamanis et al 2016. If those are no longer the standard metrics, the authors should explain how validation loss came to supplant the original metrics proposed by Allamanis et al 2016.
What's more, there is no context for how well models typically do on this evaluation task. Without any comparisons it is impossible to tell whether any of the experiments are using models in a reasonable realm of performance on this task.
Overall:
All these effects have already been shown for pre-trained embeddings in the past, and the experiments involve running standard methods on newly collected datasets. This means there is no novelty in the pre-training method or the extreme summarization method. Little is known about the newly collected datasets, it is not clear how to interpret the relative improvements in validation loss compared to the original metrics of Allamanis et al 2016, and the paper lacks necessary comparisons to othef pre-trained embeddings, so though the overall claim that pre-trained embeddings should be used for this task seems to hold up, it is not clear that this is a complete argument for the method chosen by the authors.


Despite its empirical success, the theoretical underpinnings of the stability, convergence and acceleration properties of batch normalization (BN) remain elusive. In this paper, we attack this problem from a modeling approach, where we perform a thorough theoretical analysis on BN applied to a simplified model: ordinary least squares (OLS). We discover that gradient descent on OLS with BN has interesting properties, including a scaling law, convergence for arbitrary learning rates for the weights, acceleration effects, as well as insensitivity to the choice of learning rates. We then demonstrate numerically that these findings are not specific to the OLS problem and hold qualitatively for more complex supervised learning problems. This points to a new direction towards uncovering the mathematical principles that underlies batch normalization.

The author analyze the convergence properties of batch normalization for the ordinary least square (OLS) objective. They also provide experimental results on the OLS objective as well as small scale neural networks. First of all, understanding the properties of batch normalization is an important topic in the machine learning community so in that sense, contributions that tackle this problem are of interest for the community. However, this paper has a significant number of problems that need to be addressed before publication, perhaps the most important one being the overlap with prior work. Please address this point clearly in your rebuttal.
1) Overlap with Kolher et al. 2018: The authors erroneously state that Kolher et al. considered the convergence properties of BNGD on linear networks while after taking a close look at their analysis, they first derive an analysis for least-squares and then also provide an extension of their analysis to perceptrons. The major problem is that this paper does not correctly state the difference between their analysis and Kolher et al who already derived similar results for OLS. I will come back to this aspect multiple times below.
2) Properties of the minimizer
The authors should clearly state that Kolher et al. first proved that a^* and w^* have similar properties to Eq. 8. If I understand correctly, the difference seem to be that the algorithm analyzed in Kohler relies on the optimal a^* while the analysis presented here alternates between optimizing a and w. Is this correct? Is there any advantage in not using a^*? I think this would be worth clarifying.
3) Scaling property
I find this section confusing. Specifically,
a) The authors say they rely on this property in the proof but it is not very clear why this is beneficial. Can you please elaborate?
b) It seems to me this scaling property is also similar to the analysis of Kolher et al. who showed that the reparametrized OLS objective yields a Rayleigh quotient objective. Can you comment on this?
c) The idea of “restarting” is not clear to me, are you saying that one the magnitude of the vector w goes above a certain threshold, then one can rescale the vector therefore going back to what you called an equivalent representation? I don’t see why the text has to make this part so unclear. Looking at the proof of Theorem 3.3, this “property” seem to be used to simply rescale the a and w parameters.
d) The authors claim that “the scaling law (Proposition 3.2) should play a significant role” to extend the analysis to more general models. This requires further explanation, why would this help for say neural networks or other more complex models?
4) Convergence rate
It seems to me that the results obtained in this paper are weaker than previous known results, I would have liked to see a discussion of these results. Specifically,
a) Theorem 3.3 is an asymptotic convergence result so it is much weaker than the linear rate of convergence derived in Kolher et al. The authors require a sufficiently small step size. Looking at the analysis of Kolher et al., they show that the reparametrized OLS objective yields a Rayleigh quotient objective. Wouldn’t a constant step size also yield convergence in that case?
b) Proposition 3.4 also only provides a local convergence rate. The authors argue BNGD could have a faster convergence. This does seem to again be a weaker result. So again, I think it would be very beneficial if the authors could clearly state the differences with previous work.
5) Saddles for neural nets
The authors claim they “have not encountered convergence to saddles” for the experiments with neural networks. How did you check whether the limit point reached by BNGD was not a saddle point? This requires computing all the eigenvalues of the Hessian which is typically expensive. How was this done exactly?
6) Extension of the analysis to deep neural networks
The analysis provided in this paper only applies to OLS while Kolher et al. also derived an analysis for neural networks. Can the authors comment on extending their own analysis to neural nets and how this would differ from the one derived in Kolher et al.?
7) Experiments
How would you estimate the range of suitable step sizes (for both a and w) for BNGD for a neural network?


SOM-VAE: INTERPRETABLE DISCRETE REPRESENTATION LEARNING ON TIME SERIES
High-dimensional time series are common in many domains. Since human cognition is not optimized to work well in high-dimensional spaces, these areas could benefit from interpretable low-dimensional representations. However, most representation learning algorithms for time series data are difficult to interpret. This is due to non-intuitive mappings from data features to salient properties of the representation and non-smoothness over time.
To address this problem, we propose a new representation learning framework building on ideas from interpretable discrete dimensionality reduction and deep generative modeling. This framework allows us to learn discrete representations of time series, which give rise to smooth and interpretable embeddings with superior clustering performance. We introduce a new way to overcome the non-differentiability in discrete representation learning and present a gradient-based version of the traditional self-organizing map algorithm that is more performant than the original. Furthermore, to allow for a probabilistic interpretation of our method, we integrate a Markov model in the representation space. This model uncovers the temporal transition structure, improves clustering performance even further and provides additional explanatory insights as well as a natural representation of uncertainty. We evaluate our model in terms of clustering performance and interpretability on static (Fashion-)MNIST data, a time series of linearly interpolated (Fashion-)MNIST images, a chaotic Lorenz attractor system with two macro states, as well as on a challenging real world medical time series application on the eICU data set. Our learned representations compare favorably with competitor methods and facilitate downstream tasks on the real world data.

This work addresses the problem of learning latent embeddings of high-dimensional time series data. The paper emphasises the need of interpretable representations accounting for the correlated nature of temporal data. To this scope, the study proposes to cluster the data in a latent space estimated through an auto-encoder. The clustering is obtained by leveraging on the idea of self-organising maps (SOM). Within this setting, the data is mapped into a 2D lattice where each coordinate point represents the center of an inner cluster.
This construction motivates the formulation of the auto encoder through the definition of several cost terms promoting reconstruction, clustering, and consistency across latent mappings.
This definition of the problem allows an heuristic for circumventing the non-differentiability of the discrete mapping. The enhance consistency over time, the model is further equipped with an additional cost term enforcing transition smoothness across data points and latent embeddings.
The experiments are carried out with respect to synthetic 2D time-series, chaotic time-series from dynamical systems, and clinical data. In each case the proposed method shows promising results with respect to the proposed benchmark.
The study presents some interesting methodological and technical ideas. On the other hand the manuscript presentation is quite convoluted, at the expense of a lacks of clarity in the details about the implementation of the methodology. Moreover, motivated by practical aspects, the model optimisation relies on computational strategies not completely supported from the theoretical point of view (such as the zeroing of the gradient in backpropagation, or the approximation of the clustering function to overcome non-differentiability). The impact of these modeling choices would deserve more investigation and discussion.
Detailed comments:
- As also stated by the authors, the use of a 2D latent representation is completely arbitrary. It may be true that a 2D embedding provides a simple visualisation, however interpretability can be obtained also with much richer representations in a number of different ways (e.g. sparsity, parametric representations, …). Therefore the feeling is that the proposed structure may be quite ad-hoc, and one may wonder whether the algorithm would still generalise to more complex latent representations.
- Related to the previous comment, the number of latent points seems to be crucial to the performance of the method. However this aspect is not discussed in detail, while it would be beneficial to provide experiment about the sensitivity and accuracy with respect to the choice if this parameters.
- The method relies on several cost terms plugged together. While each of them takes care of specific consistency aspects of the model, their mutual relation and balance may be very critical. This is governed by a series of trade-off parameters whose effect is not discussed  nor explored throughout the study. I guess that the optimisation stability may be also quite sensitive to this trade-off, and it would be important to provide more details about this aspect.
- Surprisingly, k-means seems to perform quite well in spite of its simplicity. Also, there is no mention about initialisation and choice of the parameter “k”. The authors may want to better discuss the performance of this algorithm, especially compared to its much lower modeling complexity with respect to the proposed method.
- Still related to the comparison with respect to the state-of-art, interpretability in time series analysis can be achieved with much lesser assumptions and parameters by using standard approaches such as independent component analysis. I would expect this sort of comparison, especially in case of long-term data such as the one provided in the Lorenz system.
- Clustering of short-term time series, such as the clinical ones, is a challenging task. The feeling is that a highly parametrised model, such as the proposed one,  may still not be superior with respect to classical methods, such as the mixture of linear regressions. This sort of comparison would be quite informative to appreciate the real value of the proposed methodology.


ACTIVE LEARNING WITH PARTIAL FEEDBACK
While many active learning papers assume that the learner can simply ask for a label and receive it, real annotation often presents a mismatch between the form of a label (say, one among many classes), and the form of an annotation (typically yes/no binary feedback). To annotate examples corpora for multiclass classification, we might need to ask multiple yes/no questions, exploiting a label hierarchy if one is available. To address this more realistic setting, we propose active learning with partial feedback (ALPF), where the learner must actively choose both which example to label and which binary question to ask. At each step, the learner selects an example, asking if it belongs to a chosen (possibly composite) class. Each answer eliminates some classes, leaving the learner with a partial label. The learner may then either ask more questions about the same example (until an exact label is uncovered) or move on immediately, leaving the first example partially labeled. Active learning with partial labels requires (i) a sampling strategy to choose (example, class) pairs, and (ii) learning from partial labels between rounds. Experiments on Tiny ImageNet demonstrate that our most effective method improves 26% (relative) in top-1 classification accuracy compared to i.i.d. baselines and standard active learners given 30% of the annotation budget that would be required (naively) to annotate the dataset. Moreover, ALPF-learners fully annotate TinyImageNet at 42% lower cost. Surprisingly, we observe that accounting for per-example annotation costs can alter the conventional wisdom that active learners should solicit labels for hard examples.

The paper considers a multiclass classification problem in which labels are grouped in a given number M of subsets c_j, which contain all individual labels as singletons. Training takes place through an active learning setting in which all training examples x_i are initially provided without their ground truth labels y_i. The learner issues queries of the form (x_i,c_j) where c_j is one of the given subsets of labels. The annotator only replies yes/no according to whether the true label y_i of x_i belongs to c_j or not. Hence, for each training example the learner maintains a "version space" containing all labels that are consistent with the answers received so far for that example. The active learning process consists of the following steps: (1) use the current learning model to score queries (x_i,c_j); (2) query the best (x_i,c_j); (3) update the model.
In their experiments, the authors use a mini-batched version, where queries are issued and re-ranked several times before updating the model. Assuming the learner generates predictive models which map examples to probability distributions over the class labels, several uncertainty measures can be used to score queries: expected info gain, expected remaining classes, expected decrease in remaining classes. Experiments are run using the Res-18 neural network architecture over CIFAR10, CIFAR100, and Tiny ImageNet, with training sets of 50k, 50k, and 100k examples. The subsets c_j are computed using the Wordnet hierarchy on the label names resulting in 27, 261, and 304 subsets for the three datasets. The experiments show the advantage of performing adaptive queries as opposed to several baselines: random example selection with binary search over labels, active learning over the examples with binary search over the labels, and others.
This paper develops a natural learning strategy combining two known approaches: active learning and learning with partial labels. The main idea is to exploit adaptation in both choosing examples and queries. The experimental approach is sound and the results are informative. In general, a good experimental paper with a somewhat incremental conceptual contribution.
In (2) there is t+1 on the left-hand side and t on the right-hand side, as if it were an update. Is it a typo?
In 3.1, how is the standard multiclass classifier making use of the partially labeled examples during training?
How are the number of questions required to exactly label all training examples computed? Why does this number vary across the different methods?
What specific partial feedback strategies are used by AQ for labeling examples?
EDC seems to consistently outperform ERC for small annotation budgets. Any intuition why this happens?


DEEP LAYERS AS STOCHASTIC SOLVERS
We provide a novel perspective on the forward pass through a block of layers in a deep network. In particular, we show that a forward pass through a standard dropout layer followed by a linear layer and a non-linear activation is equivalent to optimizing a convex objective with a single iteration of a τ -nice Proximal Stochastic Gradient method. We further show that replacing standard Bernoulli dropout with additive dropout is equivalent to optimizing the same convex objective with a variance-reduced proximal method. By expressing both fully-connected and convolutional layers as special cases of a high-order tensor product, we unify the underlying convex optimization problem in the tensor setting and derive a formula for the Lipschitz constant L used to determine the optimal step size of the above proximal methods. We conduct experiments with standard convolutional networks applied to the CIFAR-10 and CIFAR-100 datasets and show that replacing a block of layers with multiple iterations of the corresponding solver, with step size set via L, consistently improves classification accuracy.

This paper theoretically verifies an equivalence between stochastic solvers on a particular class of convex optimization problems and a forward pass through a dropout layer followed by a linear layer and a non-linear activation. Experiments show that replacing a block of layers with multiple iterations of the corresponding solver improves classification accuracy. My detailed comments are as follows.
*Positive points:
1. The perspective is novel and interesting, i.e., training a forward pass through a dropout layer followed by a linear layer and a non-linear activation is equivalent to optimizing a convex problem by a Proximal Stochastic Gradient method. More importantly, this perspective has been theoretically verified.
2. In the experiments, training networks with solvers replacing deep layers is able to improve accuracy significantly.
*Negative points:
1. Some technical details are not clear and many notations are used without clear explanations. Specifically, many notations based on (Bibi & Ghanem, 2017) make the paper hard to follow. Moreover, there are many mistakes in proofs. Please revise the paper according to the following comments.
2. There are many limitations for the proposed method. Specifically, the theoretical results are hard to be extended to more general neural networks (e.g., ResNet) with Batch Normalization which are widely used.
3. The experiment section should be significantly improved. There are only two datasets (i.e., CIFAR-10, CIFAR-100). It would be convincing that more baselines are compared on other datasets, such as ImageNet.
*Detailed comments:
**Comments on technical issues.
1. In Problem (1), the definition of $g(x)$ and $f¬_i()$ should be provided for clarity.
2. The motivation and some details of Function (2) should be provided since $F(x^l)$ is important for proving the equivalence between stochastic solvers and a forward network. In addition, $x$ should be corrected as $x^l$.
3. Is Equation (3) wrong? Based on the definition of Prox-GD in (Xiao & Zhang, 2014), it should be $x^l=Prox(x^{l-1} – 1/L \nabla F(x^l)) = Prox((I-1/L A)x^{l-1} + 1/L (AA^T x^l + b))$ which is different from Equation (3). Moreover, the Lipschitz constant w.r.t. maximal eigenvalue should be proved.
4. In Definitions D.1 and D.2, what is the definition of $fold_{H0}$? Is the dimensionality of $bdiag(D)$ wrong? Why is $bdiag(D)$ an identity mapping when $n_3=n_4$?
5. There are some issues on Equation (7) and its proofs. Is $A(:, i, :, :)$ and $\vec{X}(i, :, :, :) $ wrong? It affects the results of Equation (8). Does Equation (25) miss the operator $fold_{HO}$ in Appendix G? Please check the proofs of Proposition 1.
6. There are some issues on proofs of Lemma 2. Why are $F_H \otimes F_W \otimes I_{n_1}$ and $F_H \otimes F_W \otimes I_{n_2}$ orthogonal? Is the third and fourth equality in (24) wrong? For the fourth equality in (24), Eigen decomposition seems to be for a matrix, not a tensor.
**Comments on Experiments
1. Training Networks is equivalent to optimizing proximal solvers. Why can training networks with solvers replacing blocks of layers improve accuracy? Reasonable explanations should be provided.
2. Optimizing a convex optimization problem can easily obtain the optimal solution. What happens if solvers are used to replace more blocks of layers? Complexity analysis for these should be provided.
3. The experiments are only conducted on two datasets (i.e., CIFAR-10, CIFAR-100). It would be better to compared more baselines on other datasets, such as ImageNet.


BOUNCE AND LEARN: MODELING SCENE DYNAMICS WITH REAL-WORLD BOUNCES
We introduce an approach to model surface properties governing bounces in everyday scenes. Our model learns end-to-end, starting from sensor inputs, to predict post-bounce trajectories and infer two underlying physical properties that govern bouncing -restitution and effective collision normals. Our model, Bounce and Learn, comprises two modules -a Physics Inference Module (PIM) and a Visual Inference Module (VIM). VIM learns to infer physical parameters for locations in a scene given a single still image, while PIM learns to model physical interactions for the prediction task given physical parameters and observed pre-collision 3D trajectories. To achieve our results, we introduce the Bounce Dataset comprising 5K RGB-D videos of bouncing trajectories of a foam ball to probe surfaces of varying shapes and materials in everyday scenes including homes and offices. Our proposed model learns from our collected dataset of real-world bounces and is bootstrapped with additional information from simple physics simulations. We show on our newly collected dataset that our model out-performs baselines, including trajectory fitting with Newtonian physics, in predicting post-bounce trajectories and inferring physical properties of a scene.

The authors present both a dataset of videos of a real-world foam ball bouncing and a model to learn the trajectory of the ball at collision (bounce) points in these videos.  The model is comprised of a Physics Inference Module (PIM) and a Visual Inference Module (VIM).  The PIM takes in both a vector of physical parameters (coefficient of restitution and collision normal) and a point cloud representation of the pre-bounce trajectory, and produces a point cloud representation of the post-bounce trajectory (or, rather, an encoded version of such).  The VIM takes in an image and ground-truth bounce location and produces the physical parameters of the surface at that location.
I find the paper well-written and clear.  The motivation in the introduction is persuasive and the related work section is complete.  However, the authors are introducing both a new training paradigm (to my knowledge unused in the literature) and a new model, and without any existing baselines to compare against I find it a bit difficult to understand how well the model works.
Overall, the authors’ model is somewhat complicated and not as general as it initially seems.  To justify this complication I would like to see more convincing results and benchmarking or application to more than one single dataset (e.g. non-spheres bouncing).
Here are some specific concerns:
1)  I could not find a link to an open-sourced version of the dataset(s).  Given that the authors emphasize the dataset as a main contribution of the paper, they should open-source it and make the link prominent in the main text (apologies if I somehow missed it).
2)  The authors claim in multiple places that the model is trained end-to-end, but this does not seem to be the case.  Specifically, the PIM is pre-trained on an auxiliary dataset from simulation.  The trajectory encoder also seems to be pre-trained (though I could be wrong about that, see my question below).  Furthermore, there is a bit of hand-holding:  The PIM uses ground-truth state for pre-training, and the VIM gets the ground-truth bounce location.  In light of this, the model seems a lot less general and end-to-end than implied in the abstract and introduction.
3)  No comparison to existing baselines.  I would like to see how the authors’ model compares to standard video prediction algorithms.  The authors could evaluate their model with respect to pixel loss (after ground-truth rendering) and compare to a video prediction algorithm (such as PredNet by Lotter, Kreiman, & Cox, 2016).  Given that the authors’ method uses some extra “privileged” information (as described in point 2), it should far out-perform algorithms that train only on video data, and such a result would strengthen the paper a lot.
4)  Table 1 is not a very convincing demonstration of performance.  Regardless of baselines, the table does not show confidence intervals.  I would love to see training curves with errorbars of the models on the most important metrics (e.g. Dist and COR Median Absolute Error).
I also was confused about a couple of things:
1)  How was the PointNet trajectory encoder trained?  I did not see this mentioned anywhere.  Were gradients passed through from the PIM?  Was the same network used for both the simulation and real-world data?
2)  The performance of the center-based model in Table 1 seems surprisingly low.  The center-based model should be as good at the Train core, Fix traj. enc. model, since it has access to the ball’s position.  Why is it worse?  Is the VIM at fault?  Or is the sphere-fitting sub-optimal?  How does it compare on the simulated data with ground truth physical parameters?
3)  Lastly, the color-scheme is a bit confusing.  It looks like the foam ball in the videos was rainbow-colored.  However, in the model outputs in trajectory figures time is also rainbow-colored.  This was initially a bit confusing.  Perhaps grayscale for the model outputs would be clearer.


DETECTING EGREGIOUS RESPONSES IN NEURAL SEQUENCE-TO-SEQUENCE MODELS
In this work, we attempt to answer a critical question: whether there exists some input sequence that will cause a well-trained discrete-space neural network sequence-to-sequence (seq2seq) model to generate egregious outputs (aggressive,  malicious, attacking, etc.). And if such inputs exist, how to find them efficiently. We adopt an empirical methodology, in which we first create lists of egregious output sequences, and then design a discrete optimization algorithm to find input sequences that will cause the model to generate them. Moreover, the optimization algorithm is enhanced for large vocabulary search and constrained to search for input sequences that are likely to be input by real-world users. In our experiments, we apply this approach to dialogue response generation models trained on three real-world dialogue data-sets: Ubuntu, Switchboard and OpenSubtitles, testing whether the model can generate malicious responses. We demonstrate that given the trigger inputs our algorithm finds, a significant number of malicious sentences are assigned large probability by the model, which reveals an undesirable consequence of standard seq2seq training.

This paper explores the task of finding discrete adversarial examples for (current) dialog models in a post hoc manner (i.e., once models are trained). In particular, the authors propose an optimization procedure for crafting inputs (utterances) that trigger trained dialog models to respond in an egregious manner.
This line of research is interesting as it relates to real-world problems that our models face before they can be safely deployed. The paper is easy to read, nicely written, and the proposed optimization method seems reasonable. The study also seems clear and the results are fairly robust across three datasets. It was also interesting to study datasets which, a priori, seem like they would not contain much egregious content (e.g., Ubuntu "help desk" conversations).
My main question is that after reading the paper, I'm not sure that one has an answer to the question that the authors set out to answer. In particular, are our current seq2seq models for dialogs prone to generating egregious responses? On one hand, it seems like models can assign higher-than-average probability to egregious responses. On the other, it is unclear what this means. For example, it seems like the possibility that such a model outputs such an answer in a conversation might still be very small. Quantifying this would be worthwhile.
Further, one would imagine that a complete dialog system pipeline would contain a collection of different models including a seq2seq model but also others. In that context, is it clear that it's the role of the seq2seq model to limit egregious responses?
A related aspect is that it would have been interesting to explore a bit more the reasons that cause the generation of such egregious responses. It is unclear how representative is the example that is detailed ("I will kill you" in Section 5.3). Are other examples using words in other contexts? Also, it seems reasonable that if one wants to avoid such answers, countermeasures (e.g., in designing the loss or in adding common sense knowledge) have to be considered.
Other comments:
- I am not sure of the value of Section 3. In particular, it seems like the presentation of the paper would be as effective if this section was summarized in a short paragraph (and perhaps detailed in an appendix).

- Section 3.1, "continuous relaxation of the input embedding", what does that mean since the embedding already lives in continuous space?

- I understand that your study only considers (when optimizing for egregious responses)) dialogs that are 1-turn long. I wonder if you could increase hit rates by crafting multiple inputs at once.

- In Section 4.3, you fix G (size of the word search space) to 100. Have you tried different values? Do you know if larger Gs could have an impact of reported hit metrics?
- In Table 3, results from the first column (normal, o-greedy) seem interesting. Wouldn't one expect that the model can actually generate (almost) all normal responses? Your results indicate that for Ubuntu models can only generate between 65% and 82% of actual (test) responses. Do you know what in the Ubuntu corpus leads to such a result?

- In Section 5.3, you seem to say that the lack of diversity of greedy-decoded sentences is related to the low performance of the "o-greedy" metric. Could this result simply be explained because the model is unlikely to generate sentences that it has never seen before?
 You could try changing the temperature of the decoding distribution, that should improve diversity and you could then check whether or not that also increases the hit rate of the o-greedy metric.
- Perhaps tailoring the mal lists to each specific dataset would make sense (I understand that there is already some differences in between the mal lists of the different datasets but perhaps building the lists with a particular dataset in mind would yield "better" results).


TIME-AGNOSTIC PREDICTION: PREDICTING PREDICTABLE VIDEO FRAMES
Prediction is arguably one of the most basic functions of an intelligent system. In general, the problem of predicting events in the future or between two waypoints is exceedingly difficult. However, most phenomena naturally pass through relatively predictable bottlenecks-while we cannot predict the precise trajectory of a robot arm between being at rest and holding an object up, we can be certain that it must have picked the object up. To exploit this, we decouple visual prediction from a rigid notion of time. While conventional approaches predict frames at regularly spaced temporal intervals, our time-agnostic predictors (TAP) are not tied to specific times so that they may instead discover predictable "bottleneck" frames no matter when they occur. We evaluate our approach for future and intermediate frame prediction across three robotic manipulation tasks. Our predictions are not only of higher visual quality, but also correspond to coherent semantic subgoals in temporally extended tasks.

The authors present a method on prediction of frames in a video, with the key contribution being that the target prediction is floating, resolved by a minimum on the error of prediction. The authors show the merits of the approach on a synthetic benchmark of object manipulation with a robotic arm.
Quality: this paper appears to contain a lot of work, and in general is of high quality.
Clarity: some sections of the paper were harder to digest, but overall the quality of the writing is good and the authors have made efforts to present examples and diagrams where appropriate. Fig 1, especially helps one get a quick understanding of the concept of a `bottleneck` state.
Originality: To the extent of my knowledge, this work is novel. It proposes a new loss function, which is an interesting direction to explore.
Significance: I would say this work is significant. There appears to be a significant improvement in the visual quality of predictions. In most cases, the L1 error metric does not show such a huge improvement, but the visual difference is remarkable, so this goes to show that the L1 metric is perhaps not good enough at this point.
Overall, I think this work is significant and I would recommend its acceptance for publication at ICLR. There are some drawbacks, but I don’t think they are major or would justify rejection (see comments below).
I’m curious as to why you called the method in section 3.2 the “Generalized minimum”? It feels more like a weighted (or preference weighted) minimum to me and confused me a few times as I was reading the paper (GENerative? GENeralized? what’s general about it?). Just a comment.
What results does figure 4 present? Are they only for the grasping sequence? Please specify.
In connection with the previous comment, I think the results would be more readable if the match-steps were normalized for each sequence (at least for Figure 4). There would be a clearer mapping between fixT methods and the normalized matching step (e.g., we would expect fix0.75 to achieve a matching step of 0.75 instead of 6 / ? ).
Section 4, Intermediate prediction. The statement “the genmin w(t) preference is bell-shaped” is vague. Do you mean a Gaussian? If so, you should say “a Gaussian centered at T/2 and tuned so that …”
Section 4, Bottleneck discovery frequency. I am not entirely convinced by the measuring of bottleneck states. You say that a distance is computed between the predicted object position and the ground-truth object position. If a model were to output exactly the same frame as given in context, would the distance be zero? If so, doesn’t that mean that a model who predicts a non-bottleneck state before or after the robotic arm moves the pieces is estimated to have a very good bottleneck prediction frequency? I found this part of the paper the hardest to follow and the least convincing. Perhaps some intermediate results could help prospective readers understand better and be convinced of the protocol’s merits.
Typos:
Appendix E, 2nd paragraph, first sentence: “... generate an bidirectional state” --> “generate A bidirectional state”


INTERPOLATION-PREDICTION NETWORKS FOR IRREGULARLY SAMPLED TIME SERIES
In this paper, we present a new deep learning architecture for addressing the problem of supervised learning with sparse and irregularly sampled multivariate time series. The architecture is based on the use of a semi-parametric interpolation network followed by the application of a prediction network. The interpolation network allows for information to be shared across multiple dimensions of a multivariate time series during the interpolation stage, while any standard deep learning model can be used for the prediction network. This work is motivated by the analysis of physiological time series data in electronic health records, which are sparse, irregularly sampled, and multivariate. We investigate the performance of this architecture on both classification and regression tasks, showing that our approach outperforms a range of baseline and recently proposed models.

I have mixed feelings about this submission, and as such, I look forward to discussing it with both the authors and my fellow reviewers. In short, I like the simplicity of the idea, but I am uncertain about the degree to which it satisfies ICLR's novelty criterion ("present substantively new ideas or explore an underexplored or highly novel question"); I do feel confident that some ICLR readers would (perhaps unfairly) describe this approach as "obvious." The paper's presentation suffers, and it fails to communicate essential details clearly. Finally, for folks familiar with healthcare data and MIMIC-III specifically, the results are underwhelming: yes, the proposed approach beats (the authors' own implementations of) baselines, but it underperforms other published results on the MIMIC-III 48-hour mortality task ([1][2] report AUCs of 0.87 or higher). As such, I am assigning the paper a "weak accept" to communicate my ambivalence and reserve the right to adjust it up or down after discussion.
SUMMARY
This paper proposes an "interpolation layer" to resample irregularly sampled time series before feeding them into a neural net architecture. The interpolation layer consists of parametric kernels, e.g., radial basis functions, configured to estimate the values of input time series at reference time points based on univariate temporal and then multivariate correlations. The outputs include smooth and transient interpolated values (controlled by kernel bandwidth) and counts (referred to as intensity) at each reference point. As far as I understand, this model can be trained end-to-end. The paper also proposes a simple strategy for combatting overfitting (add an autoencoder and reconstruction error term to the objective in combination with a heuristic in which some points are masked as inputs and must be interpolated from non-masked points). In experiments on two data sets (UWaveGesture and a medical data set) and two tasks (classification and regression) this approach outperforms the main competing approaches [3][4][5][6] in most contexts.
Below I provide a list of strengths, weaknesses, and general questions or feedback.
STRENGTHS
- I applaud the simplicity of the idea: this much simpler framework leverages many of the intuitions behind the GP adapter framework (GP-GRU) [4][5] with comparable performance and appears to train orders of magnitude faster (caveat: on one data set and task)
- It likewise outperforms both commonly used preprocessing (GRU-F) [2][3] and the much more complicated neural net architecture (GRU-HD) from [6] (across two datasets and tasks)
- The simplicity of this approach probably lends itself to additional customization and innovation
- The literature review seems quite thorough and does an especially nice job of covering recent work on RNNs for multivariate time series and irregular sampling or missing values
- The experiments are thorough and well-designed overall. The authors use two data sets and two tasks (classification and regression). More data sets and tasks is always nice, but even two is pretty laudable (many authors might settle on just one given the experimental and computational effort required for these experiments). They include and beat or outperform two baselines that can justifiably be called state-of-the-art (GP-GRU and GRU-HD).
I think a relatively safe takeaway is that for irregularly sampled data, this approach is is preferable to both heuristic preprocessing and more complex models. That seems like a not insignificant finding in empirical machine learning for messy time series data.
WEAKNESSES
- Section 3 is possibly the most critical section (since it describes the contribution) but is hard to follow: I don't envy the authors the task of explaining a variable with two superscripts and three subscripts (Equation 1), but it IS their paper, so it's on them to do it. See feedback section for other examples.
- Although I consider the related work well done, I can't help but wonder if there isn't older work on RBFs, etc., that might have been missed (I mostly want to encourage the authors to look once more and then come back and tell me I'm wrong).
- The MIMIC-III experiments omit the GP-GRU model, which weakens the results by leaving the reader to imagine how it might compare (I would expect it to outperform the proposed approach by an even wider margin than it did for UWave).
- I am sympathetic to the idea of fixing certain architectural choices, e.g., layers and units in the GRU and number of inducing points, across all models because it (a) gives the appearance of a "fair comparison" and (b) reduces burden of effort, but I do not agree that it yields a truly fair comparison. The GRU-* model performance on UWave is suspiciously bad, suggesting severe overfitting and the possibility that the models are overparameterized. It leaves the reader wondering if the architectural choices happen to be optimal for the proposed model only (whether by accident or design). A truly fair comparison requires independently tuning hyperparameters for each model.
- Although the proposed approach outperforms baselines in these experiments, the overall results are underwhelming in the wider context of recent work using MIMIC-III. Multiple publications have reported AUCS of 0.87 [1][2] or higher for 48-hour risk of mortality (it is difficult to compare the LOS results since different papers use different units). Of course, the experiments use different cohorts and variables so they're not directly comparable, but it nonetheless diminishes the potential impact of the results presented here.
FEEDBACK AND QUESTIONS
- I had to read 3.2.1 multiple times to understand the relationships between the different "layers" in the interpolator, and I'm still not sure what the relationship is between the smooth and transient kernels or exactly how the intensity values are estimated (are they just windowed counts or weighted sums?).
- I'm also not 100% clear on (a) which parameters (if any) in the interpolator are optimized during end-to-end learning and which are just fixed or tuned as hyperparameters. This should be stated clearly and even better, I'd recommend writing down the gradient update rules for the interpolator parameters (you can put them in the appendix).
- Since the model uses global structure for interpolation and requires pre-specifying the number of inducing points, could it be used to make continuous predictions (and how?), e.g., forecast mortality at each hour?
- On a related note, if the number of inducing points is pre-specified, can the model be applied to sequences of different length?
- How does performance depend on choice of number of inducing points?
- How does the proposed approach handle time series that are missing entirely, e.g., if no pH values are measured?
- What does Table 3 in the appendix mean by "missingness?" Given that the paper is concerned with irregular sampling (not missing data), I would expect statistics on sampling rates, not missingness...
- Why derive your own MIMIC-III subset and tasks rather than use one of several pre-existing benchmarks (both of which include more variables and tasks) [1][2]?
- FYI: the Che, et al., 2016, paper on missing values [6] has been published in JBIO, so you should cite that version.
REFERENCES
[1] Purushotham, et al. "Benchmark of Deep Learning Models on Large Healthcare MIMIC Datasets." arXiv preprint arXiv:1710.08531 (2017)
[2] Harutyunyan, et al. "Multitask learning and benchmarking with clinical time series data." arXiv preprint arXiv:1703.07771 (2017)
[3] Lipton, Kale, and Wetzel, 2016
[4] Li and Marlin, 2016.
[5] Futoma, et al., 2017.
[6] Che, et al., 2016. <-- new JBIO 2018 version!


Each training step for a variational autoencoder (VAE) requires us to sample from the approximate posterior, so we usually choose simple (e.g. factorised) approximate posteriors in which sampling is an efficient computation that fully exploits GPU parallelism. However, such simple approximate posteriors are often insufficient, as they eliminate statistical dependencies in the posterior. While it is possible to use normalizing flow approximate posteriors for continuous latents, there is nothing analogous for discrete latents. The most natural approach to model discrete dependencies is an autoregressive distribution, but sampling from such distributions is inherently sequential and thus slow. We develop a fast, parallel sampling procedure for autoregressive distributions based on fixed-point iterations which enables efficient and accurate variational inference in discrete state-space models. To optimize the variational bound, we considered two ways to evaluate probabilities: inserting the relaxed samples directly into the pmf for the discrete distribution, or converting to continuous logistic latent variables and interpreting the K-step fixed-point iterations as a normalizing flow. We found that converting to continuous latent variables gave considerable additional scope for mismatch between the true and approximate posteriors, which resulted in biased inferences, we thus used the former approach. We tested our approach on the neuroscience problem of inferring discrete spiking activity from noisy calcium-imaging data, and found that it gave accurate connectivity estimates in an order of magnitude less time.
The development of variational auto-encoders (VAE) (Kingma & Welling, 2014;Rezende et al., 2014) has enabled Bayesian methods to be applied to very large-scale data, by leveraging neural networks, trained by stochastic gradient ascent, to approximate the posterior. Importantly, to perform each stochastic gradient ascent step, we need to sample from the current approximate posterior, restricting us to approximate posteriors in which sampling can be performed rapidly by leveraging parallel GPU computations. As such, most work on VAE's has used factorised approximate posteriors (e.g. Kingma & Welling, 2014;Rezende et al., 2014;Blundell et al., 2015;Higgins et al., 2017;Aitchison et al., 2017), but in many domains of interest we expect the posterior over latents to be highly correlated, not only because the posterior inherits correlations from the prior (e.g. as we might find in a dynamical system), but also because the likelihood itself can induce correlations due to effects such as explaining away (Pearl, 1988a;b). One approach to introducing correlations into the approximate posterior is normalizing flows (Rezende & Mohamed, 2015;Kingma et al., 2016) which transforms variables generated from a simple, often factorised distribution into a complex correlated distribution, in such a way that the determinant of the Jacobian (and thus the probability of the transformed variables) can easily be computed.
However, the normalizing flow approach can only be applied to continuous latents, and there are important problems which require discrete latent variables and correlated posteriors, making efficient and accurate stochastic variational inference challenging. In particular, we consider the neuroscience problem of inferring the correlated spiking activity of neural populations recorded by calcium imaging. Due to the indirect nature of calcium imaging, spike inference algorithms must be used to infer the underlying neural spiking activity (Friedrich et al., 2017;Speiser et al., 2017). Not only does the data naturally give strong explaining away induced anticorrelations (as a spike in nearby timebins produces very similar data), but there are prior correlations induced by synaptic connectivity which induces similar correlations in the approximate posterior.
To address these challenging tasks with discrete latent variables, and correlated approximate posteriors, we considered two approaches. First, we considered applying normalizing flows by transforming our discrete latents into continuous latents, which are thresholded to recover the original discrete variables (Maddison et al., 2016). However, we found that working with continuous variables gave rise to far more scope for mismatch between the approximate and true posteriors than working with discrete variables, and that this mismatch resulted in biased inferences. Instead, we developed a fast-sampling procedure for discrete autoregressive posteriors. In particular, we considered an autoregressive approximate posterior and circumvent the requirement for slow sequential sampling by developing flow-like fixed-point iterations that are guaranteed to sample the true posterior after T iterations, but in practice converge much more rapidly (in our simulations, ∼ 5 iterations), and efficiently exploit GPU parallelism.
Applying the flow-like fixed point iterations to simulated neuroscience problems, we were able to sample from autoregressive approximate posteriors in almost the same time required for factorised posteriors, and at least an order of magnitude faster than sequentially sampling from the underlying autoregressive process, allowing us to realize the benefits of correlated posteriors in large-scale settings.

Normalizing flows provide a way for variational posteriors to go beyond the mean-field assumption and introduce correlations in the posterior for latent variables. Typically, normalizing flows are only defined for continuous distributions, and the authors tackle the issue of creating flexible variational posteriors for discrete latent models. They posit a general autoregressive posterior family for discrete variables or their continuous relaxations. In order to perform variational inference with reparameterization gradients, one needs to have a sample from their variational family and be able to evaluate the density at the sample. The obvious way to sample from this autoregressive variational family is O(T) for T the number of autoregressive time-steps. Instead, they propose a method based off fixed point iterations to compute logits in parallel based off the results of previous iterations to generate an approximate sample. Moreover, they can interpret each iteration as a volume-preserving flow, so they don't have to add any terms to the density. They also use a continuous relaxation of Bernoulli random variables so they can back-propagate through a recognition network. They use their method on synthetic calcium spike data, and show that the correlated posterior is better-suited to handle uncertainty.
This is certainly original work, and it presents it in a general way. The authors' formulation of the probabilistic model and variational family in equations 6a-7b can be extended to any autoregressive family. As the authors mention, they also do not need to work only with continuous relaxations of Bernoulli random variables, as they can extend it to categorical random variables using something like the Gumbel-Softmax/Concrete distributions.   Although the idea behind parallelizing updates is not inherently elaborate, the authors provide an interesting interpretation as a normalizing flow.
My main criticism of the paper is the experiments section. The experiments are only performed on synthetic data sets. How do the methods scale to larger data sets? The authors state that iterations of the fixed point procedure converge rapidly in practice, but it seems like it's only been evaluated on these synthetic data sets. It seems like performing K fixed point iterations fixes the dependencies to be in a window of size K, so this may perform worse in practice for models that have long temporal dependencies (or for non-temporal latent discrete random variables where the ordering is not important).
The experiments also do not seem to evaluate any held-out metrics. The experiments would be stronger by e.g. approximating the marginal held-out loss (perhaps using IWAE or otherwise), since it seems almost guaranteed that more flexible variational families should achieve a tighter bound on the training set (it's possible that there were actually held-out metrics but I missed them, in which case please let me know).
Another criticism of the paper is with the clarity. The authors sometime use notation before/without defining it. For example, T is used without definition it at the beginning of page 2 and N is used without definition at the beginning of section 2.1. It makes it difficult to have intuition for the math as a result of not knowing the definitions. Even things like explicitly stating that k is a timestep index before equation 9 would be helpful.
More minor notes:
-I got a lot out of Figure 1A. For Figure 1B-E, what is beta? Is there intuition for how these results change as a function of beta?
-When you say Equation 8 in section 2.2 I believe you mean equation 7b.
Overall, there has not been much (if any) work for correlated posterior families for discrete latent variable models, and the authors have provided a promising first step. The next step would be seeing more experimental results for a larger variety of models.
PROS
-Idea is very interesting and novel, with a nice connection to normalizing flows
-Underexplored area of research, promising first steps.
CONS
-Experiments should be more thorough
-Lack of clarity made it hard to understand at certain points


AUTOMATICALLY COMPOSING REPRESENTATION TRANSFORMATIONS AS A MEANS FOR GENERALIZATION
A generally intelligent learner should generalize to more complex tasks than it has previously encountered, but the two common paradigms in machine learning -either training a separate learner per task or training a single learner for all tasks -both have difficulty with such generalization because they do not leverage the compositional structure of the task distribution. This paper introduces the compositional problem graph as a broadly applicable formalism to relate tasks of different complexity in terms of problems with shared subproblems. We propose the compositional generalization problem for measuring how readily old knowledge can be reused and hence built upon. As a first step for tackling compositional generalization, we introduce the compositional recursive learner, a domaingeneral framework for learning algorithmic procedures for composing representation transformations, producing a learner that reasons about what computation to execute by making analogies to previously seen problems. We show on a symbolic and a high-dimensional domain that our compositional approach can generalize to more complex problems than the learner has previously encountered, whereas baselines that are not explicitly compositional do not.

==== Summary ====
This paper proposes a model for learning problems that exhibit compositional and recursive structure, called Compositional Recursive Learner (CRL). The paper approaches the subject by first defining a problem as a transformation of an input representation x from a source domain t_x to a target domain t_y. If t_x = t_y then it is called a recursive problem, and otherwise a translational problem. A composite problem is the composition of such transformations. The key observation of the paper is that many real-world problems can be solved iteratively by either recursively transforming an instance of a problem to a simpler instance, or by translating it to a similar problem which we already know how to solve (e.g., translating a sentence from English to French through Spanish). The CRL model is essentially composed of two parts, a set of differential functions and a controller (policy) for selecting functions. At each step i, the controller observes the last intermediate computation x_i and the target domain t_y, and then selects a function and the subset of x_i to operate on. For each instance, the resulting compositional function is trained via back-propagation, and the controller is trained via policy gradient. Finally, the paper presents experiments on two synthetic datasets, translating an arithmetic expression written in one language to its outcome written in another language, and classifying MNIST digits that were distorted by an unknown random sequence of affine transformations. CRL is compared to RNN on the arithmetic task and shown to be able to generalize both to longer sequences and to unseen language pairs when trained on few examples, while RNN can achieve similar performance only using many more examples. On MNIST, it is qualitatively shown that CRL can usually (but not always) find the sequence of transformations to restore the digit to its canonical form.
==== Detailed Review ====
I generally like this article, as it contains a neat solution to a common problem that builds on and extends prior work. Specifically, the proposed CRL model is a natural evolution of previous attempts at solving problems via compositionally, e.g. Neural Programmer [1] that learns a policy for composing predefined commands, and Neural Module Networks [2] that learns the parameters of shared differential modules connected via deterministically defined structure (found via simple parse tree). The paper contains a careful review of the related works and highlights the similarities and differences from prior approaches. Though the experiments are mostly synthetic, the underlying method seems to be readily applicable to many real-world problems.
However, the true contributions of the paper are somewhat muddied by presenting CRL as more general than what is actually supported by the experiments. More specifically, the paper presents CRL as a general method for learning compositional problems by decomposing them into simpler sub-problems that are automatically discovered, but in practice, a far more limited version of CRL is used in the experiments, and the suggested translational capabilities of CRL, which are important for abstract sub-problem discovery, are not properly validated:
1. In both experiments, the building-block functions are hand-crafted to fit to the prior knowledge on the compositionally of the problem. For the arithmetic task, the functions are limited to operate each step just on a single window of encompassing 3 symbols (e.g., <number> <op> <number>,  <op> <number> <op>) and return a distribution over the possible symbols, which heavily forces the functions to represent simple evaluators for simple expressions of the form <number> <op> <number>. For the distorted MNIST task, the functions are limited to neural networks which choose the parameter of predetermined transformations (scaling, translation, or rotation) of the input. In both cases, CRL did not *found* sub-problems for reducing the complexity of the original instance but just had to *fine tune* loosely predefined sub-problems. Incorporating expert knowledge into the model like so is actually an elegant and useful trick for solving real problems, and it should be emphasized far clearly in the article. The story of “discovering subproblems” should be left for the discussion / future research section, because though it might be a small step towards that goal, it is not quite there yet.
2. The experiments very neatly show how recursive transformations offer a nice framework for simplifying an instance of a problem. However, the translation capabilities of the model are barely tested by the presented experiments, and it can be argued that all transformations used by the model are recursive in both experiments. First, only the arithmetic task has a translation aspect to it, i.e., the task is to read an expression in one language and then output the answer in a different language. Second, this problem is only weakly related to translation because it is possible to translate the symbols independently, word by word, as opposed to written language that has complex dependencies between words. Third, the authors report that in practice proper translation was only used in the very last operation for translating the computed value of the input expression to the requested language, and not as a method to translate one instance that we cannot solve into another that we can. Finally, all functions operate and return on all symbols and not ones limited to a specific language, and so by the paper’s own definition, these are all recursive problems and not translational ones.
In conclusion, I believe this paper should be accepted even with the above issues, mostly because the core method is novel, clearly explained, and appears to be very useful in practice. Nevertheless, I strongly suggest to the authors to revise their article to focus on the core qualities of their method that can be backed by their current experiments, and correctly frame the discussion on possible future capabilities as such.
[1] Reed et al. Neural Programmer-Interpreters. ICLR 2016.
[2] Andreas et al. Neural Module Networks. CVPR 2016.


CHARACTERIZING AUDIO ADVERSARIAL EXAMPLES USING TEMPORAL DEPENDENCY
Recent studies have highlighted adversarial examples as a ubiquitous threat to different neural network models and many downstream applications. Nonetheless, as unique data properties have inspired distinct and powerful learning principles, this paper aims to explore their potentials towards mitigating adversarial inputs. In particular, our results reveal the importance of using the temporal dependency in audio data to gain discriminate power against adversarial examples. Tested on the automatic speech recognition (ASR) tasks and three recent audio adversarial attacks, we find that (i) input transformation developed from image adversarial defense provides limited robustness improvement and is subtle to advanced attacks; (ii) temporal dependency can be exploited to gain discriminative power against audio adversarial examples and is resistant to adaptive attacks considered in our experiments. Our results not only show promising means of improving the robustness of ASR systems but also offer novel insights in exploiting domain-specific data properties to mitigate negative effects of adversarial examples.

This paper investigates adversarial examples for audio data. The standard defense techniques proposed for images are studied in the context of audio. It is shown that these techniques are somewhat robust to adversarial attacks, but fail against adaptive attacks. A method exploiting the temporal dependencies of the data is then presented and shown to be robust to adversarial examples and to adaptive attacks.
The paper addresses an important issue, and the two main findings of the paper, the transformation methods used in computer vision are not useful against audio adversarial example and using temporal dependencies improves the defense capability are significant. The proposed TD method is novel.
The first part of the paper is easy to read (Section 1-3), but Section 4 is hard to follow, for the following reasons:
* Section 4.1 presents the metrics used in the evaluation, which is nice. But in the following subsections, other metrics are used: effectiveness ratio, detection rate and relative perturbation. They should be clearly defined in 4.1, and the authors should discuss why they used these metrics.
* Section 4.2 should be reorganized as it is hard to follow: there are three types of attack, so one subsection per attack should make the section clearer.
* It's not always clear what each attack is doing and why it is used. I suggest the authors to have a separate subsection with the description of each attack and the motivation of why it is used.
Because of the above, it's hard to clearly assess the performance of each method for each attack, it would be better to have a Table that summarizes the results for the transformation methods. Also, I don't understand the statement in 4.2: "We report that the autoencoder works fine for transforming benign instances (57.6% WER in Common Voice compared to 27.5%)": given that it's not an attack, the PER should be the same with and without transform, as we don't want the transform to affect non-adversarial examples ? Please clarify that.
The experiments on the proposed TD method are clear enough to show the viability of the approach.
Overall, the findings of this paper are significant and it is good step towards audio adversarial examples defense. But the experimental part is hard to follow and does not bring a clear picture. I am still willing to accept the paper if the authors improve and clarify Section 4.
Revision after rebuttal:
The new version is definitely clearer and easier to read, hence I support the paper for acceptance and change my rating to 7.
There are still minor improvements that can be done in Section 4 to improve the overall clarity:
* About the metrics, the "Average attack success rate" and the "Target command recognition rate" should be clearly defined, probably under the description of the attack methods.
* The Adaptive attack approach could be introduced unter "Attack methods" in 4.1.
* Table 4 is not easy to read, the authors should improve it.
* The first paragraph in Section 4 ("The presentation flows ...") is very interesting, but almost reads like a conclusion, so maybe the authors could move that to the end of Section 4 or to Section 5.


In this paper, we focus on the supervised learning problem with corrupt training data. We assume that the training dataset is generated from a mixture of a target distribution and other unknown distributions. We estimate the quality of each data by revealing the correlation between the generated distribution and the target distribution. To this end, we present a novel framework referred to here as ChoiceNet that can robustly infer the target distribution in the presence of inconsistent data. We demonstrate that the proposed framework is applicable to both classification and regression tasks. Particularly, ChoiceNet is evaluated in comprehensive experiments, where we show that it constantly outperforms existing baseline methods in the handling of noisy data in synthetic regression tasks as well as behavior cloning problems. In the classification tasks, we apply the proposed method to the MNIST and CIFAR-10 datasets and it shows superior performances in terms of robustness to different types of noisy labels.

The paper presents a framework, called ChoiceNet, for learning when the
supervision outputs (e.g., labels) are corrupted by noise. The method relies on
estimating the correlation between the training data distribution and a
target distribution, where training data distribution is assumed to be a mixture
of that target distribution and other unknown distributions. The paper also
presents some compelling results on synthetic and real datasets, for both
regression and classification problems.
The proposed idea builds on top of previously published work on Mixture Density
Networks (MDNs) and Mixup (Zhang et al, 2017). The main difference is the MDN
are modified to construct the Mixture of Correlated Density Network (MCDN)
block, that forms the main component of ChoiceNets.
I like the overall direction and idea of modelling correlation between the
target distribution and the data distribution to deal with noisy labels. The
results are also compelling and I thus lean towards accepting this paper. My
decision on "marginal accept" is based primarily on my unfamiliarity with this
specific area and that some parts of the paper are not very easy or intuitive
to read through.
== Related Work ==
I like the related work discussion, but would emphasize more the connection to
MDNs and to Mixup. Only one sentence is mentioned about Mixup but reading
through the abstract and the introduction that is the first paper that came to
my mind and thus I believe that it may deserve a bit more discussion.
Also, there are a couple more papers that felt relevant to this work but are
not mentioned:
  - Estimating Accuracy from Unlabeled Data: A Bayesian Approach, Platanios et al., ICML 2016.
    I believe this is related in how noisy labels are modeled (i.e., section 3
    in the reviewed paper) and in the idea of correlation/consistency as a means
    to detect errors. There are couple more papers in this line of work that
    may be relevant.
  - ADIOS: Architectures Deep In Output Space, Al-Shedivat et al., ICML 2016.
    I believe this is related in learning some structure in the output space,
    even though not directly dealing with noisy labels.
== Method ==
I believe the methods section could have been written in a more
clear/easy-to-follow way, but this may also be due to my unfamiliarity with this
area. Figure 1 is hard to parse and does not really offer much more than section
3.2 currently does. If the figure is improved with some more text/labels on
boxes rather than plain equations, it may go a long way in making the methods
section easier to follow.
I would also point out MCDN as the key contribution of this paper as ChoiceNet
is just any base network with an MCDN block stacked on top of this. Thus, I
believe this should be emphasized more to make your key contribution clear.
== Experiments ==
The experiments are nicely presented and are quite thorough. A couple minor
comments I have are:
  - It would be nice to run regression experiments for bigger real-world
    datasets, as the ones used seem to be quite small.
  - I am a bit confused at the fact that in table 3 you compare your method to
    mixup and in table 4 you also show results when using both your method and
    mixup combined. Up until that point I thought that mixup was posed as an
    alternative method, but here it seems it's quite orthogonal and can be used
    together, which I think makes sense, but would be good to clarify. Also,
    given that you show combined results in table 4, why not also perform
    exactly the same analysis for table 3 and also show numbers for CN + Mixup?
It would also be nice to use the same naming scheme for both tables. I would
use: ConvNet, ConvNet + CN, ConvNet + CN + Mixup, and the same with WRN for
table 4. This would make the tables easier to read because currently the first
thing that comes to mind is what may be different between the two setups given
that they are presented side-by-side but use different naming conventions.
One question that comes to mind is that you make certain assumptions on the
kinds of noise your model can capture, so are there any cases where you have
good intuition as to why your model may fail? It would be good to present a
short discussion on this to help readers understand whether they can benefit by
using your model or not.


ON THE LOSS LANDSCAPE OF A CLASS OF DEEP NEURAL NETWORKS WITH NO BAD LOCAL VALLEYS
We identify a class of over-parameterized deep neural networks with standard activation functions and cross-entropy loss which provably have no bad local valley, in the sense that from any point in parameter space there exists a continuous path on which the cross-entropy loss is non-increasing and gets arbitrarily close to zero. This implies that these networks have no sub-optimal strict local minima.

This paper presents a class of neural networks that does not have bad local valleys. The “no bad local valleys” implies that for any point on the loss surface there exists a continuous path starting from it, on which the loss doesn’t increase and gets arbitrarily smaller and close to zero. The key idea is to add direct skip connections from hidden nodes (from any hidden layer) to the output.
The good property of loss surface for networks with skip connections is impressive and the authors present interesting experimental results pointing out that
* adding skip connections doesn’t harm the generalization.
* adding skip connections sometimes enables training for networks with sigmoid activation functions, while the networks without skip connections fail to achieve reasonable performance.
* comparison of the generalization performance for the random sampling algorithm vs SGD and its connection to implicit bias is interesting.
However, from a theoretical point of view, I would say the contribution of this work doesn’t seem to be very significant, for the following reasons:
* In the first place, figuring out “why existing models work” would be more meaningful than suggesting a new architecture which is on par with existing ones, unless one can show a significant performance improvement over the other ones.
* The proof of the main theorem (Thm 3.3) is not very interesting, nor develops novel proof techniques. It heavily relies on Lemma 3.2, which I think is the main technical contribution of this paper. Apart from its technicality in the proof, the statement of Lemma 3.2 is just as expected and gives me little surprise, because having more than N hidden nodes connected directly to the output looks morally “equivalent” to having a layer as wide as N, and it is known that in such settings (e.g. Nguyen & Hein 17’) it is easy to attain global minima.
* I also think that having more than N skip connections can be problematic if N is very large, for example N>10^6. Then the network requires at least 1M nodes to fall in this class of networks without bad local valleys. If it is possible to remove this N-hidden-node requirement, it will be much more impressive.
Below, I’ll list specific comments/questions about the paper.
* Assumption 3.1.2 doesn’t make sense. Assumption 3.1.2 says “there exists N neurons satisfying…” and then the first bullet point says “for all j = 1, …, M”. Also, the statement “one of the following conditions” is unclear. Does it mean that we must have either “N satisfying the first bullet” or “N satisfying the second bullet”, or does it mean we can have N/2 satisfying the first and N/2 satisfying the second?
* The paper does not describe where the assumptions are used. They are never used in the proof of Theorem 3.3, are they? I believe that they are used in the proof of Lemma 3.2 in the appendix, but if you can sketch/mention how the assumptions come into play in the proofs, that will be more helpful in understanding the meaning of the assumptions.
* Are there any specific reasons for considering cross-entropy loss only? Lemma 3.2 looks general, so this result seems to be applicable to other losses. I wonder if there is any difficulty with different losses.
* Are hidden nodes with skip connections connected to ALL m output nodes or just some of the output nodes? I think it’s implicitly assumed in the proof that they are connected to all output nodes, but in this case Figure 2 is a bit misleading because there are hidden nodes with skip connections to only one of the output nodes.
* For the experiments, how did you deal with pooling layers in the VGG and DenseNet architectures? Does max-pooling satisfy the assumptions? Or the experimental setting doesn’t necessarily satisfy the assumptions?
* Can you show the “improvement” of loss surface by adding skip connections? Maybe coming up with a toy dataset and network WITH bad local valleys will be sufficient, because after adding N skip connections the network will be free of bad local valleys.
Minor points
* In the Assumption 3.1.3, the $N$ in $r \neq s \in N$ means $[N]$?
* In the introduction, there is a sentence “potentially has many local minima, even for simple models like deep linear networks (Kawaguchi, 2016),” which is not true. Deep linear networks have only global minima and saddle points, even for general differentiable convex losses (Laurent & von Brecht 18’ and Yun et al. 18’).
* Assumption 3.1.3 looked a bit confusing to me at first glance. You might want to add some clarification such as “for example, in the fully connected network case, this means that all data points are distinct.”


Disentangling underlying generative factors of a data distribution is important for interpretability and generalizable representations. In this paper, we introduce two novel disentangling methods. Our first method, Unlabeled Disentangling GAN (UD-GAN, unsupervised), decomposes the latent noise by generating similar/dissimilar image pairs and it learns a distance metric on these pairs with siamese networks and a contrastive loss. This pairwise approach provides consistent representations for similar data points. Our second method (UD-GAN-G, weakly supervised) modifies the UD-GAN with user-defined guidance functions, which restrict the information that goes into the siamese networks. This constraint helps UD-GAN-G to focus on the desired semantic variations in the data. We show that both our methods outperform existing unsupervised approaches in quantitative metrics that measure semantic accuracy of the learned representations. In addition, we illustrate that simple guidance functions we use in UD-GAN-G allow us to directly capture the desired variations in the data.

Summary
The paper presents a novel approach for learning a generative model where different factors of variations can be independently manipulated. The method is build upon  the GAN framework where the latent variables are divided into different subsets (chunks) which are expected to encode information about high-level factors of variation. To this end, a Siamese Network for each chunk is trained with a contrastive loss minimizing the distance between generated images sharing the same factor (the latent variables in the chunk are equal), and maximizing the distance between pairs where the latent variables differ. Given that the proposed model fails in this fully-unsupervised setting, the authors propose to add weak-supervision into the model by forcing the Siamese networks to  focus only on particular aspects of generated images (e.g, color, edges, etc..). This is achieved by applying  a basic transformation  over the input images in order to remove specific information. The evaluation of the  proposed model is carried out using the MS-Celeb dataset where the authors provide qualitative results.
Methodology
*Disentangling generative factors without explicit labels is a challenging and interesting problem. The idea of dividing the latent representation in different subsets and using a proxy task involving triplets of images has been already explored in [3]. However, the use of Siamese networks in this context is novel and sound.
*As shown in the reported results, the proposed method fails to learn meaningful factors in the unsupervised setting. However, the authors do not provide an in-depth discussion of this phenomena. Given that previous works [1,2,3] have successfully addressed this problem using a completely unsupervised approach, it would be necessary to give more insights about: (i) why the proposed method is failing (ii) why this negative result is interesting and (iii) if the method could be useful in other potential scenarios.
*The strategy proposed to introduce weak-supervision is too ad-hoc. I agree that using cues such as the average color of an image can be useful if we want to model basic factors of variation. However, it is unclear how a similar strategy could be applied if we are interested in learning variables with higher-level semantics such as the expression of a face or its pose.
*As far as I understand, the transformations applied to the input images (e.g, edge detection) must be differentiable (given that it is necessary to backpropagate the gradient of the contrastive loss through the generator network). If this is the case, this should be properly discussed in the paper. Moreover, given that the amount of differentiable transformations is reduced, this also limits the application of the proposed method for more interesting scenarios.
*It is not clear why the latent variables modelling the generative factors are defined using a Gaussian prior. How the case where two images have a very similar latent factor is avoided while generating pairs of images for the Siamese network?  Have the authors considered to use categorical or binary variables? The use of the contrastive loss sounds more appropriate in this case.
Experimental results
*The experimental section is too limited. First of all, only a small number of qualitative results are reported and, therefore, it is very difficult to assess the proposed method and draw any conclusion. For example, when the edge extractor is used, what kind of information is modeled by the latent variables? Is it consistent across different samples?
Moreover, it is not clear why the authors have limited the evaluation to the case where only two “chunks” are used. In principle, the method could be applied with many more subsets of latent variables and then manually inspect them to check it they are semantically meaningful (see [2])
*As previously mentioned, there are many recent works addressing the same problem from a fully-unsupervised perspective [1,2,3]. All these works provide quantitative results evaluating the learned representations by using them to predict real labels (e.g, attributes in the CelebA data-set). The authors could provide a similar evaluation for their method by using the feature representations learned by the siamese networks in order to evaluate how much information they convey about real factors of variation. This could clarify the advantages of the weakly-supervised strategy compared to unsupervised approaches.
Review summary
+The addressed problem (learning disentangled representations without explicit labeling) is challenging and interesting.
+The idea of using a proxy task (contrastive loss with triplets of generated images) is somewhat novel and promising.
- The authors report only negative results for the fully-unsupervised version of UD-GAN The paper lacks and in-depth discussion about why this negative result is interesting.
-The strategy proposed to provide weak-supervision to the model is too ad-hoc and it is not clear how to apply it in general applications
-The experimental section do not clarify the benefits of the proposed approach. In particular, the qualitative results are too limited and no quantitative evaluations is provided.
[1] Variational Inference of Disentangled Latent Concepts from Unlabelled Observations (Kumar et al, ICLR 2018)
[2] Beta-vae: Learning basic visual concepts with a constrained variational framework. (Higgins et. al, ICLR 2017)
[3] Disentangling Factors of Variation by Mixing Them. (Hu et. al, CVPR  2018)


DIFFERENTIABLE PERTURB-AND-PARSE: SEMI-SUPERVISED PARSING WITH A STRUCTURED VARIATIONAL AUTOENCODER
Human annotation for syntactic parsing is expensive, and large resources are available only for a fraction of languages. A question we ask is whether one can leverage abundant unlabeled texts to improve syntactic parsers, beyond just using the texts to obtain more generalisable lexical features (i.e. beyond word embeddings). To this end, we propose a novel latent-variable generative model for semi-supervised syntactic dependency parsing. As exact inference is intractable, we introduce a differentiable relaxation to obtain approximate samples and compute gradients with respect to the parser parameters. Our method (Differentiable Perturb-and-Parse) relies on differentiable dynamic programming over stochastically perturbed arc weights. We demonstrate effectiveness of our approach with experiments on English, French and Swedish.

This paper proposed a variational autoencoder-based method for semi-supervised dependency parsing. Given an input sentence s, an LSTM-based encoder generates a sentence embedding z, and a NN of Kiperwasser & Goldberg (2016) generates a dependency structure T. Gradients over the tree encoder are approximated by (1) adding a perturbation matrix over the weight matrix and (2) relax dynamic programming-based parsing algorithm to a differentiable format. The decoder combines standard LSTM and Graph Convolutional Network to generate the input sentence from z and T. The authors evaluated the proposed method on three languages, using 10% of the original training data as labeled and the rest as unlabeled data.
Pros
1. I like the idea of this sentence->tree->sentence autoencoder for semi-supervised parsing. The authors proposed a novel and nice way to tackle key challenges in gradient computation. VAE involves marginalization over all possible dependency trees, which is computationally infeasible, and the proposed method used a Gumbel-Max trick to approximate it. The tree inference procedure involves non-differentiable structured prediction, and the authors used a peaked-softmax method to address the issue. The whole model is fully differentiable and can be thus trained end to end.
2. The direction of semi-supervised parsing is useful and promising, not only for resource-poor languages, but also for popular languages like English. A successful research on this direction could be potentially helpful for lots of future work.
Cons, and suggestions on experiments
My main concerns are around experiments. Overall I think they are not strong enough to demonstrate that this paper has sufficient contribution to semi-supervised parsing. Below are details.
1. The current version only used 10% of original training data as labeled and the rest as unlabeled data. This makes the reported numbers way below existing state-of-the-art performance. For example, the SOTA UAS on English PTB has been >95%. Ideally, the authors should be able to train a competitive supervised parser on full training data (English or other languages), and get huge amount of unlabeled data from other sources (e.g. News) to further push up the performance. The current setting makes it hard to justify how useful the proposed method could be in practice.
2. The best numbers from the proposed model is lower than baseline (Kipperwasser & Goldberg) on English, and only marginally better on Swedish. This probably means the supervised baseline is weak, and it's hard to tell if the gains from VAE will retain if applied to a stronger supervised.
3. A performance curve with different amount of labeled and unlabeled data would be useful to better understand the impact of semi-supervised learning.
4. What's the impact of perturbation? One could simply use T=Eisner(W) as approximation. Did you observe any significant benefits from sampling?
Other questions
1. What's the impact of keeping the tree constraint on dependencies during backpropagation?  Have you tried removing the tree constraint like previous work?
2. Are sentence embedding and trees generated from two separate LSTM encoders? Are there any parameter sharing between the two?


EMERGENT COORDINATION THROUGH COMPETITION
We study the emergence of cooperative behaviors in reinforcement learning agents by introducing a challenging competitive multi-agent soccer environment with continuous simulated physics. We demonstrate that decentralized, populationbased training with co-play can lead to a progression in agents' behaviors: from random, to simple ball chasing, and finally showing evidence of cooperation. Our study highlights several of the challenges encountered in large scale multi-agent training in continuous control. In particular, we demonstrate that the automatic optimization of simple shaping rewards, not themselves conducive to co-operative behavior, can lead to long-horizon team behavior. We further apply an evaluation scheme, grounded by game theoretic principals, that can assess agent performance in the absence of pre-defined evaluation tasks or human baselines.

This paper introduces a new multiagent research environment---a simplified version of 2x2 RoboSoccer using the MuJoCo physics engine with spherical players that can rotate laterally, move forwards / backwards, and jump.
The paper deploys a fine-tuned version of population-based sampling on top of a stochastic value gradient reinforcement learning algorithm to train the agents.  Some of the fine-tunings used include deploying different discount factors on multiple different reward channels for reward shaping.
The claimed novel contributions of the paper are (1) a new multiagent testbed, (2) a decentralized training procedure, (3) fine-tuning reward shaping, and (4) highlighting the challenges in evaluation in novel multiagent competitive environments.
Overall, my judgment is that the paper is fine, but the authors have not helped me to understand the significance of their contributions.
Taking each in turn:
(1) What is the significance of the new environment?  What unique characteristics make it difficult?  What makes this environment an importantly different testbed or development environment?  The connection to RoboSoccer is motivating but tenuous. The new environment should have particular characteristics that expose problems with past algorithms or offer new challenges existing algorithms have not addressed at all.
(2) Why is it important to have a decentralized training procedure when the authors have control over all the agents?  If it will allow faster training, has the authors' algorithm been demonstrated to accomplish that goal?
(3) It's hard to evaluate new algorithms when the domain studied is also new. We have no sense for state-of-the-art performance on this domain across a range of algorithms.  The authors conduct a careful ablation study on their new algorithm but do not compare their approach to other classes of algorithms.
(4) The authors indicate that evaluating the quality of an algorithm for a competitive context is hard in absence of established benchmarks---whereas in single-agent or cooperative environments progress can be measured against the goal of the environment, progress in competitive environments requires comparison to approaches that are thought to be good.  Here the authors are themselves pointing out a fundamental problem with introducing new competitive multiagent testbeds, and the authors don't resolve this tension.  Since the main contribution of the work is the environment, it's hard to see how this point the authors themselves make doesn't undermine that central contribution.
Besides other comments mentioned above, a couple other ways to improve the paper would be:
- Clarify why this environment is important to be introducing---what are the unique things that can be studied with this new environment?
- Hold an open competition to get benchmarks created by other teams of researchers
Some minor comments:
- $n_r$ is not defined explicitly in the text as far as I have found
- The authors state: "The specific shaping rewards use for soccer are detailed in Section 4.2" but I couldn't find them there.
---
Post-rebuttal
My main concern was assessing the value of the overall contribution of the paper. The other reviewers seem to appreciate both the new environment being offered and the combination of techniques deployed in the authors' solution. If there is an audience that will appreciate this work at ICLR as seems to be indicated by those reviews, then I would increase my score to marginally above the acceptance threshold.