This work investigates the performance of a composite neural network, which is composed of pre-trained neural network models and non-instantiated neural network models, connected to form a rooted directed graph. A pre-trained neural network model is generally a well trained neural network model targeted for a specific function. The advantages of adopting such a pre-trained model in a composite neural network are two folds. One is to benefit from other's intelligence and diligence, and the other is saving the efforts in data preparation and resources and time in training. However, the overall performance of composite neural network is still not clear. In this work, we prove that a composite neural network, with high probability, performs better than any of its pre-trained components under certain assumptions. In addition, if an extra pre-trained component is added to a composite network, with high probability the overall performance will be improved. In the empirical evaluations, distinctively different applications support the above findings.

The paper aims at justifying the performance gain that is acquired by the use of "composite" neural networks (e.g., composed of a pre-trained neural network and additional layers that will be trained for the new task).
I found the paper lacking in terms of writing and in terms of clarity in expressing scientific/mathematical ideas especially for a theory paper.
Example from the Abstract:
"The advantages of adopting such a pre-trained model in a composite neural network are two folds. One is to benefit from other’s intelligence and diligence, and the other is saving the efforts in data preparation and resources
and time in training"
The main results of the paper (Theorem 1,2,3) are of the following nature: if you use more features (i.e., "components") in the input of a network then you have "more information", and this cannot be bad. Here are the corresponding claims in the Abstract:
"we prove that a composite neural network, with high probability, performs better than any of its pre-trained components under certain assumptions."
"if an extra pre-trained component is added to a composite network, with high probability the overall performance will be improved."
However, this argument seems to be just about expressiveness; adding more features can be statistically problematic.
Furthermore, why is it specific to pre-trained components? Essentially the theorems are about adding any features.
Finally, the assumption that the pre-trained components are linearly independent is invalid and the makes the whole analysis somewhat simplistic.
The motivating Example 1 just shows that the convex hull of a class of hypotheses can include more hypotheses than the class itself. I don't see any connection between this and the use of pre-training.
Other examples unclear statements from the intro:
"One of distinctive features of the complicated applications is their applicable data sources are boundless. Consequently, their solutions need frequent revisions."
"Although neural networks can approximate arbitrary functions as close as possible (Hornik, 1991), the major reason for not existing such competent neural networks for those complicated applications is their problems are hardly fully understood and their applicable data sources cannot be identified all at once."
There are many typos in the paper including this one about X for the XOR function:
"Assume there is a set of locations indexed as X = {(0; 0); (0; 1); (1; 0); (1; 0)} with the corresponding values Y = (0; 1; 1; 0). Obviously, the observed function is the XOR"


SUPERVISED COMMUNITY DETECTION WITH LINE GRAPH NEURAL NETWORKS
Community detection in graphs can be solved via spectral methods or posterior inference under certain probabilistic graphical models. Focusing on random graph families such as the stochastic block model, recent research has unified both approaches and identified both statistical and computational detection thresholds in terms of the signal-to-noise ratio. By recasting community detection as a node-wise classification problem on graphs, we can also study it from a learning perspective. We present a novel family of Graph Neural Networks (GNNs) for solving community detection problems in a supervised learning setting. We show that, in a data-driven manner and without access to the underlying generative models, they can match or even surpass the performance of the belief propagation algorithm on binary and multiclass stochastic block models, which is believed to reach the computational threshold in these cases. In particular, we propose to augment GNNs with the non-backtracking operator defined on the line graph of edge adjacencies. The GNNs are achieved good performance on real-world datasets. In addition, we perform the first analysis of the optimization landscape of using (linear) GNNs to solve community detection problems, demonstrating that under certain simplifications and assumptions, the loss value at any local minimum is close to the loss value at the global minimum/minima.

This paper presents a study of the community detection problem via graph neural networks. The presented results open the possibility that neural networks are able to discover the optimal algorithm for a given task. This is rather convincingly demonstrated on the example of the stochastic block model, where the optimal performance is known (for 2 symmetric groups) or strongly conjectured (for more groups). The method is rather computationally demanding, and also somewhat unrealistic in the aspect that the training examples might not be available, but for a pioneering study of this kind this is well acceptable.
Despite my overall very positive opinion, I found a couple of claims that are misleading and overall hurt the quality of the paper, and I would strongly suggest to the authors to adjust these claims:
** The method is claimed to "even improve upon current computational thresholds in hard regimes." This is misleading, because (as correctly stated in the body of the paper) the computational threshold to which the paper refers apply in the limit of large graph sizes whereas the observed improvements are for finite sizes. It is shown here that for finite sizes the present method is better than belief propagation. But this clearly does not imply that it improves the conjectured computational thresholds that are asymptotic. At best this is an interesting hypothesis for future work, not more.
** The energy landscape is analyzed "under certain simplifications and assumptions". Conclusions state "an interesting transition from rugged to simple as the size of the graphs increase under appropriate concentration conditions." This is very vague. It would be great if the paper could offer intuitive explanation of there simplifications and assumptions that is between these unclear remarks and the full statement of the theorem and the proof that I did not find simple to understand. For instance state the intuition on in which region of parameters are those results true and in which they are not.
** "multilinear fully connected neural networks whose landscape is well understood (Kawaguchi, 2016)." this is in my opinion grossly overstated. While surely that paper presents interesting results, they are set in a regime that lets a lot to be still understood about landscape of fully connected neural networks. It is restricted to specific activation functions, and the results for non-linear networks rely on unjustified simplifications, the sample complexity trade-off is not considered, etc.
Misprint: Page 2: cetain -> certain.


Memorization in over-parameterized neural networks can severely hurt generalization in the presence of mislabeled examples. However, mislabeled examples are to hard avoid in extremely large datasets. We address this problem using the implicit regularization effect of stochastic gradient descent with large learning rates. By the loss statistics, we are able to identify mislabeled examples with remarkable success. Then, we discard the mislabeled examples on the fly and continue training with the rest of the examples. This leads to ON-THE-FLY DATA DENOISING (ODD), a simple yet effective algorithm that is robust to mislabeled examples, while introducing almost zero computational overhead. Empirical results demonstrate the effectiveness of ODD on several datasets containing artificial and real-world mislabeled examples.

Thanks for the rebuttal. But, I am still not very convinced with the proposed results. For CIFAR-100 (0%), you get about 0.2% gain, for ImageNet (0%), you get about 0.2% loss in top-5 accuracy, and for WebVision, you get about 0.3% gain. I am not sure whether you can call these as statistically significant gains. I believe such gain/loss can be obtained with many other tweaks, such as the learning rate scheduling, as the authors have done.
I believe extensive testing the proposed method on many real noisy datasets, not the synthetically generated ones, and showing the consistent gains would much strengthen the paper. But, at the current version, the only such result is Table 5, which is, again, not very convincing to me.
So, I still keep my rating.
=======
Summary:
The authors propose a simple empirical method for cleaning the dataset for training. By using the implicit regularization property of SGD-based optimization method, the authors come up with a method of setting a threshold for the training loss statistics such that the examples that show losses above the threshold are regarded as noisy examples and are discarded. Their empirical results show that ODD (their method) can outperform other baselines when artificial random label noise is injected. They also show ablation studies on the hyperparameters and show the final result seems to be robust to those parameters.
Pros:
- The method is very simple
- The empirical results, particularly on the synthetic noisy training data, seems to be encouraging.
- The ablation study argues that the method is robust to the hyperparameters, p, E, and h.
Cons:
- I think the results remains to be highly empirical. While it is interesting to see the division of the loss statistics in Figure 2, I am not very convinced about the real usage of the proposed method. The result in Table 5 shows that ODD can outperform ERM for real world datasets, but the improvement seems to be marginal. Moreover, the hyperparameter p was set to 30 for that experiment, but how did the authors choose that parameter? Clearly, if you choose wrong p, I think the performance will degrade, and it is not clear how you can choose p in real applications. The ablation studies are only with synthetic noisy label data, so I think the result is somewhat limited.
-
I think the paper shows interesting results, but my concern is that it seems to be quite empirical. The positive results are particularly on the synthetic data case.


Dynamic Channel Pruning: Feature Boosting and Suppression
Making deep convolutional neural networks more accurate typically comes at the cost of increased computational and memory resources. In this paper, we reduce this cost by exploiting the fact that the importance of features computed by convolutional layers is highly input-dependent, and propose feature boosting and suppression (FBS), a new method to predictively amplify salient convolutional channels and skip unimportant ones at run-time. FBS introduces small auxiliary connections to existing convolutional layers. In contrast to channel pruning methods which permanently remove channels, it preserves the full network structures and accelerates convolution by dynamically skipping unimportant input and output channels. FBS-augmented networks are trained with conventional stochastic gradient descent, making it readily available for many state-of-the-art CNNs. We compare FBS to a range of existing channel pruning and dynamic execution schemes and demonstrate large improvements on ImageNet classification. Experiments show that FBS can respectively provide 5× and 2× savings in compute on VGG-16 and ResNet-18, both with less than 0.6% top-5 accuracy loss.

Summary:
This paper proposed a feature boosting and suppression method for dynamic channel pruning. To be specific, the proposed method firstly predicts the importance of each channel and then use an affine function to amplify/suppress the importance of different channels. However, the idea of dynamic channel pruning is not novel. Moreover, the comparisons in the experiments are quite limited.
My detailed comments are as follows.
Strengths:
1. The motivation for this paper is reasonable and very important.
2. The authors proposed a new method for dynamic channel pruning.
Weaknesses:
1. The idea of dynamic channel pruning is not novel. In my opinion, this paper is only an extension to Network Slimming (Liu et al., 2017). What is the essential difference between the proposed method and Network Slimming?
2. The writing and organization of this paper need to be significantly improved. There are many grammatical errors and this paper should be carefully proof-read.
3. The authors argued that the importance of features is highly input-dependent. This problem is reasonable but the proposed method still cannot handle it. According to Eqn. (7), the prediction of channel saliency relies on a data batch rather than a single data. Given different inputs in a batch, the selected channels should be different for each input rather than a general one for the whole batch. Please comment on this issue.
4. The proposed method does not remove any channels from the original model. As a result, both the memory and the computational cost will not be reduced. It is confusing why the proposed method can yield a significant speed-up in the experiments.
5. The authors only evaluate the proposed method on shallow models, e.g., VGG and ResNet18. What about the deeper model like ResNet50 on ImageNet?
6. It is very confusing why the authors only reported top-5 error of VGG. The results of top-1 error for VGG should be compared in the experiments.
7. Several state-of-the-art channel pruning methods should be considered as the baselines, such as ThiNet (Luo et al., 2017), Channel pruning (He et al., 2017) and DCP (Zhuang et al., 2018)
[1] Channel pruning for accelerating very deep neural networks. CVPR 2017.
[2] Thinet: A filter level pruning method for deep neural network compression. CVPR 2017.
[3] Discrimination-aware Channel Pruning for Deep Neural Networks. NIPS 2018.


Knowledge extraction techniques are used to convert neural networks into symbolic descriptions with the objective of producing more comprehensible learning models. The central challenge is to find an explanation which is more comprehensible than the original model while still representing that model faithfully. The distributed nature of deep networks has led many to believe that the hidden features of a neural network cannot be explained by logical descriptions simple enough to be understood by humans, and that decompositional knowledge extraction should be abandoned in favour of other methods. In this paper we examine this question systematically by proposing a knowledge extraction method using M-of-N rules which allows us to map the complexity/accuracy landscape of rules describing hidden features in a Convolutional Neural Network (CNN). Experiments reported in this paper show that the shape of this landscape reveals an optimal trade off between comprehensibility and accuracy, showing that each latent variable has an optimal M-of-N rule to describe its behaviour. We find that the rules with optimal tradeoff in the first and final layer have a high degree of explainability whereas the rules with the optimal tradeoff in the second and third layer are less explainable. The results shed light on the feasibility of rule extraction from deep networks, and point to the value of decompositional knowledge extraction as a method of explainability.

The paper proposes to rewrite each neuron of a neural network as a M-of-N decision rule. An measure of rule complexity (which takes into the account the number of terms in the rule) is proposed, and an approximate rule induction algorithm which binarizes the neurons using an information gain criterion is provided.
The paper gives no evaluation of the accuracy of extracted rules on a test set. Instead, the fidelity to network is evaluated on the train set for individual neurons (see below).
Since even for moderately-sized networks the method would result in lots of rules, the authors propose instead to use the proposed algorithm and complexity criterion as a tool for understanding the complexity of concepts detected by a layer. The results, gathered in Figure 1, suggest that for some layers complex rules do approximate the behavior of neurons, while for other layers a neuron can't be replaced with a single but complex M-of-N rule.
The ideas presented in this paper seem rudimentary and require further exploration before being publishable. First of all, the main result of complexity-vs-layer doesn't differentiate between failures of the approximate rule induction algorithm (the terms in rules are considered for inclusion in a single order, reducing the search space) and the genuine complexity of the rules - this can be verified by evaluating a more exhaustive algorithm on at leas a few neurons (not necessarily on all of them).
Second, if the rules are extracted in a layerwise fashion,  their errors accumulate for deeper layers. However, Algorithm 1 suggests that each neuron is replaced by a rule independently from others, and moreover that it requires the true value of the neurons in the layer below, not of their rule counterparts. This means that the rules can't be combined and explains why the paper doesn't provide any measure for aggregate rule accuracy.
Similarly, the robustness of rules to adversarial examples is meaningless - it seems that applying the rules results in a system which is less accurate (only 8/10 of rules mimick their neurons with no rule complexity penalty) overall, but also makes fewer adversarial examples.
Minor remarks.
Please don't ever produce Figures such as Figure 1: no legend (description in text), color selection is not black and white friendly, font is so small that the axis labels are hard to read... In fact, the poor quality of the Figure by itself should be sufficient  to reject the paper for not abiding to the author guidelines (sec. 4.3: All artwork must be neat, clean, and legible.)


LOCAL SGD CONVERGES FAST AND COMMUNICATES LITTLE
Mini-batch stochastic gradient descent (SGD) is state of the art in large scale distributed training. The scheme can reach a linear speedup with respect to the number of workers, but this is rarely seen in practice as the scheme often suffers from large network delays and bandwidth limits. To overcome this communication bottleneck recent works propose to reduce the communication frequency. An algorithm of this type is local SGD that runs SGD independently in parallel on different workers and averages the sequences only once in a while. This scheme shows promising results in practice, but eluded thorough theoretical analysis. We prove concise convergence rates for local SGD on convex problems and show that it converges at the same rate as mini-batch SGD in terms of number of evaluated gradients, that is, the scheme achieves linear speedup in the number of workers and mini-batch size. The number of communication rounds can be reduced up to a factor of T 1/2 -where T denotes the number of total steps-compared to mini-batch SGD. This also holds for asynchronous implementations. Local SGD can also be used for large scale training of deep learning models. The results shown here aim serving as a guideline to further explore the theoretical and practical aspects of local SGD in these applications.

The authors of this paper analyze a well known technique for parallel training, where each compute node locally trains a model with SGD, and once in a while the K compute nodes average their models. Local SGD, although not as widely used as mini-batch SGD, can provide some gains in terms of the cost of communication. This can be achieved by decreasing the frequency of synchronization, while locally also increasing the minibatch.
To the best of my knowledge, the authors are the first to provide a complete theoretical analysis of local SGD for strongly convex functions. They prove that under strong convexity, and the bounded gradients assumption, local SGD will (in the worst case) achieve a linear speedup over vanilla SGD, as long as the parallel models are averaged frequently enough. They show that although frequent averaging is important for speedup, the overall communication cost can be lower than minibatch SGD that may require smaller batches and hence more frequent communication.
The authors extend their results to the asynchronous case, where a similar convergence bound is derived. The overall theory seems to be partly inspired by the perturbed iterates framework of Mania et al., however the application is novel and interesting.
The authors include some limited experimental results that validate their bounds.
This is a well-written paper, that will certainly be of interest to researchers working on stochastic optimization, and distributed learning. The results are interesting and clearly stated. The proofs seem complete and correct, and are easy to follow.
I have two minor comments:
1) In a recent paper, Dong et al. [1] suggest that for any problem (convex or nonconvex), the largest possible batch size in minibatch SGD that allows for linear speedups will be proportional to “gradient diversity”, i.e., a measure of similarity between the concurrently processed gradients. For example, when all gradient are identical, there is no speedup to be extracted. This diversity term does not seem to appear in the main theorem, as one may expect. For example, the presented bounds still seem to provide speedup gains for the case where all individual n functions are identical (eg minimum grad. diversity). This should not be possible, as there are no parallel speedups to be extracted in this case. I’m wondering how that fact is reflected in the presented bounds (maybe it’s one of the extreme parameter cases that are not covered by the main theorem).
2) The authors do not provide details of their experimental setup. For example it would be useful to know what hardware they implemented their algorithms on. It seems that they run experiments for up to 1K workers. Are these individual cores, or was this the result of hyper-threading? Finally, it’s unclear if Fig 1 is a theoretical, or an experimental curve.
[1] http://proceedings.mlr.press/v84/yin18a/yin18a.pdf


DO DEEP GENERATIVE MODELS KNOW WHAT THEY DON'T KNOW?
A neural network deployed in the wild may be asked to make predictions for inputs that were drawn from a different distribution than that of the training data. A plethora of work has demonstrated that it is easy to find or synthesize inputs for which a neural network is highly confident yet wrong. Generative models are widely viewed to be robust to such mistaken confidence as modeling the density of the input features can be used to detect novel, out-of-distribution inputs. In this paper we challenge this assumption. We find that the density learned by flow-based models, VAEs, and PixelCNNs cannot distinguish images of common objects such as dogs, trucks, and horses (i.e. CIFAR-10) from those of house numbers (i.e. SVHN), assigning a higher likelihood to the latter when the model is trained on the former. Moreover, we find evidence of this phenomenon when pairing several popular image data sets: FashionMNIST vs MNIST, CelebA vs SVHN, ImageNet vs CIFAR-10 / CIFAR-100 / SVHN. To investigate this curious behavior, we focus analysis on flow-based generative models in particular since they are trained and evaluated via the exact marginal likelihood. We find such behavior persists even when we restrict the flows to constant-volume transformations. These transformations admit some theoretical analysis, and we show that the difference in likelihoods can be explained by the location and variances of the data and the model curvature. Our results caution against using the density estimates from deep generative models to identify inputs similar to the training distribution until their behavior for out-of-distribution inputs is better understood.

Pros:
- The finding that SVHN has larger likelihood than CIFAR according to networks is interesting.
- The empirical and theoretical analyses are clear, seem thorough, and make sense.
- Section 5 can provide some insight when the model is too rigid and too log-concave (e.g. Gaussian).
Cons:
- The premises of the analyses are not very convincing, limiting the significance of the paper.
- In particular, Section 4 is a series of empirical analyses, based on one dataset pair. In 3/4 of the pairs the author tried, this phenomenon is not there. Whether the findings generalize to other situations where the phenomenon appears is uncertain.
- It is good that Section 5 has some theoretical analysis. But I personally find it very disturbing to base it on a 2nd order approximation of a probability density function of images when modeling something as intricate as models that generate images. At least this limitation should be pointed out in the paper.
- Some parts of the paper feel long-winded and aimless.
[Quality]
See above pros and cons.
A few less important disagreement I have with the paper:
- I don't think Glow necessarily is encouraged to increase sensitivity to perturbations. The bijection needs to map training images to a high-density region of the Gaussian, and that aspect would make the model think twice before making the volume term too large.
- Figure 6(a) clearly suggests that the data mean for SVHN and CIFAR are very different, instead of similar.
[Clarity]
In general, the paper is clear and easy to understand given enough reading time, but feels at times long-winded.
Section 2 background takes too much space.
Section 3 too much redundancy -- it just explains that SVHN has a higher likelihood when trained on CIFAR, and a few variations of the same experiment.
Section 4 seems to lack a high-level idea of what it want to prove -- the hypothesis around the volume term is dismissed shortly after, and it ultimately proves that we do not know what is the reason behind the high SVHN likelihood, making it look like a distracting side-experiment.
A few editorial issues:
- On page 4 footnote 2, as far as I know the paper did not define BPD.
- There are two lines of text between Fig. 4 and Fig. 5, which is confusing.
[Originality]
I am not an expert in this specific field (analyzing generative models), but I believe this analysis is novel.
However, there are papers empirically analyzing novelty detection using generative model -- should analyze or at least cite:
    Vít Škvára et al. Are generative deep models for novelty detection truly better?
    ^ at first glance, their AUROC is never under 0.5, indicating that this phenomenon did not appear in their experiments although a lot of inlier-novelty pairs are tried.
A part of the paper's contribution (section 5 conclusion) seem to overlap with others' work. The section concludes that if the second dataset has small variances, it will get higher likelihood. But this is too similar to the cited findings on page 6 (models assign high likelihood to constant images).
[Significance]
The paper has a very interesting finding; pointing out and in-depth analysis of negative results should benefit the community greatly.
However, only 1 dataset pair is experimented -- there should be more to ensure the findings generalize, since Sections 3 and 4 rely completely on empirical analysis. According to the conclusions of the paper, such dataset pairs should be easy to find -- just find a dataset that "lies within" another. Did you try e.g. CIFAR-100 train and CIFAR-10 test?
Section 5 is based on a 2nd order expansion on the $log p(x)$ given by a deep network -- I shouldn't be the judge of this, but from a realistic perspective this does not mean much.


In high-dimensional reinforcement learning settings with sparse rewards, performing effective exploration to even obtain any reward signal is an open challenge. While model-based approaches hold promise of better exploration via planning, it is extremely difficult to learn a reliable enough Markov Decision Process (MDP) in high dimensions (e.g., over 10 100 states). In this paper, we propose learning an abstract MDP over a much smaller number of states (e.g., 10
5 ), which we can plan over for effective exploration. We assume we have an abstraction function that maps concrete states (e.g., raw pixels) to abstract states (e.g., agent position, ignoring other objects). In our approach, a manager maintains an abstract MDP over a subset of the abstract states, which grows monotonically through targeted exploration (possible due to the abstract MDP). Concurrently, we learn a worker policy to travel between abstract states; the worker deals with the messiness of concrete states and presents a clean abstraction to the manager. On three of the hardest games from the Arcade Learning Environment (MONTEZUMA'S RE-VENGE, PITFALL!, and PRIVATE EYE), our approach outperforms the previous state-of-the-art by over a factor of 2 in each game. In PITFALL!, our approach is the first to achieve superhuman performance without demonstrations.

This paper considers how to effectively perform exploration in the setting where a difficult, high-dimensional MDP can be mapped to a simpler, lower-dimensional MDP. They propose a hierarchical approach where a model of the abstract MDP is incrementally learned, and then used to train sub-policies to transition between abstract states. These sub-policies are trained using intrinsic rewards for transitioning to the correct state, and the transition probabilities in the abstract MDP reflect how well a sub-policy can perform the transition.
The approach is evaluated on three difficult Atari games, which all require difficult exploration: Montezuma's Revenge, Pitfall and Private Eye, and is shown to achieve good performance in all of them. Furthermore, the model can be used to generalize to new tasks by changing the rewards associated with different transitions.
The main downside with this paper is that the mapping from original state (i.e. pixels) to the abstract state is assumed to be known beforehand, which requires prior knowledge. The authors hardcode this mapping for each of the games by fetching the relevant bits of information from RAM. This prevents fair comparison to many other methods which only use pixels, and makes this paper borderline rather than strong accept.
Quality: the method is evaluated on difficult problems and shown to perform well. The experiments are thorough and explore a variety of dimensions such as robustness to stochasticity, granularity of the abstract state and generalization to new tasks. The approach does strike me as rather complicated though - it requires 19 (!) different hyperparameters as shown in table 2. The authors do mention that many of these did not require much tuning and they intend on making their code public. Still, this suggests that re-implementation or extensions by others may be challenging. Are all of these moving parts necessary?
Clarity: the paper is well-written, for the most part clear, and the details are thoroughly described in the appendix.
Originality: this approach in the context of modern deep learning is to my knowledge novel.
Significance: This paper provides a general approach for hierarchical model-based planning when the mapping from the hard MDP to the easy one is known, and in this sense is significant. It is limited by the assumption that the mapping to abstract states is known. I suspect the complexity of the approach may also be a limiting factor.
Pros:
+ good results on 3 challenging problems
+ effective demonstration of hierachical model-based planning
Cons:
- requires significant prior knowledge for state encoding
- complicated method
Minor:
- in the intro, last paragraph: "Our approach significantly outperforms previous non-demonstration SOTA approaches in all 3 domains". Please specify that you use extra knowledge extracted from RAM, otherwise this is misleading.
- Algorithm 1: nagivate -> navigate
- Section 4, last sentence: broken appendix link.
- Bottom of page 6: "Recent work on contextual MDPs...as we do here" is not a sentence.
- In related work, it would be nice to mention some relevant early work by Schmidhuber on subgoal generation: http://people.idsia.ch/~juergen/subgoals.html
*** Updated ***
After reading the updated paper, responses, other reviews, and looking at related works more closely, I have changed my score to a 5. This is due to several factors.
Although the paper's core idea is definitely interesting, the fact that they use hardcoded features, rather the standard setup which uses pixels, makes comparison to other methods much more complicated. In particular, I think that the comparison to DQN-PixelCNN is unfair, as this other method makes very few assumptions about the inputs (only that they are pixels). The authors sort of point this out in the main text, but this is somewhat misleading. They say "PixelCNN uses less prior knowledge than our approach". In fact, it uses as much prior knowledge as any RL method which operates on pixels. Granted, this is nonzero, but it's vastly less than what this paper's method assumes. The other comparison is to SOORL (which uses a different state encoding altogether). The comparison to SmartHash is fairer, although the variant of SmartHash they compare against is not the main method the paper proposes (a generic autoencoder-based state encoding which makes minimal assumptions about the input). It would have been better if the authors included experiments for their method using such a learned state encoding.
Reporting SOTA results on very hard tasks using extra hardcoded features or other domain knowledge is potentially misleading to the community as to how far along we are in solving these tasks, and extra care should be taken to put these results in context. Otherwise, for those not familiar with the subtleties, this makes it seem like these tasks are being solved when in fact they are not. My concern is that other works may then be asked to be compared against these artificially high results. Having many different task setups also makes comparison between different published works confusing in general. Other works (such as Ostrovski et al) have been able to make progress on these tasks while staying within the standard pixel-based framework.
These concerns would have been partially mitigated had the authors made it *very* clear that they were assuming substantial prior knowledge, which makes their method non-comparable to others which do not make this assumption. This could have been done in the introduction (which was one of my comments, but this was not included in the updated draft). I.e., something to the effect of "We emphasize that our approach assumes substantially more prior knowledge than other approaches which operate only on pixels, and as such is not directly comparable with these approaches". In addition, I would have liked if the authors had followed the suggestion of Reviewer 1 to include results in pixel space, even if negative, but this was not done either (using a simple autoencoder-based representation, like the one in the SmartHash paper, would have also been fine). As it is, statements such as "Our approach achieves more than 2x the reward of prior non-demonstration SOTA approaches" and "our approach relies on some prior knowledge in the state abstraction function, although we compare against SOTA methods using a similar amount of prior knowledge in our experiments" are quite misleading and unfair to other methods which do not assume access to prior knowledge (the second statement is untrue for the case of DQN-PixelCNN).
Another point which I had not noticed previously is the very high sample complexity (2 billion). One of the motivations behind model-based approaches is that they are supposed to be more sample efficient, but that does not seem to be the case here.


A Deep learning has achieved astonishing results on many tasks with large amounts of data and generalization within the proximity of training data. For many important real-world applications, these requirements are unfeasible and additional prior knowledge on the task domain is required to overcome the resulting problems.
In particular, learning physics models for model-based control requires robust extrapolation from fewer samples -often collected online in real-time -and model errors may lead to drastic damages of the system. Directly incorporating physical insight has enabled us to obtain a novel deep model learning approach that extrapolates well while requiring fewer samples. As a first example, we propose Deep Lagrangian Networks (DeLaN) as a deep network structure upon which Lagrangian Mechanics have been imposed. DeLaN can learn the equations of motion of a mechanical system (i.e., system dynamics) with a deep network efficiently while ensuring physical plausibility. The resulting DeLaN network performs very well at robot tracking control. The proposed method did not only outperform previous model learning approaches at learning speed but exhibits substantially improved and more robust extrapolation to novel trajectories and learns online in real-time.

This paper looks at system identification for a multi-link robot based upon combining a neural network with the manipulator equations.  Specifically, the authors propose to model the robot dynamics using the typical manipulator equations, but have a deep neural network parameterize the H(q) and g(q) matrices.  They illustrate that the method can control the systems of a simualted 2-dof robot and real Barrett WAM arm, better than a pure neural network modeling approach, PID control, or an analytic model.
Overall, I think there is a genuinely nice application in this paper, but it's not sufficiently compared to existing approaches nor put in the proper context.  There is a lot of language in the paper about encoding the prior via a PDE, but really what the authors are doing is quite simple: they are doing system identification under the standard robot manipulator equations but using a deep network to model the inertia tensor H(q) and the gravity term g(q).  Learning the parameters that make up H(q) and g(q) is completely standard system identification in robotics, but it's interesting to encode these as a generic deep network (I'm somewhat surprised this hasn't been done before, though a quick search didn't turn up any obvious candidates).  However, given this setting, there are several major issues with the presentation and evaluation, which make the paper unsuitable in its current from.
1) Given the fact that the authors are really just in the domain of system identification and control, there are _many_ approaches that they should compare to.  At the very least, however, the authors should compare to standard system identification techniques (see e.g., Wu et al., "An overview of dynamic parameter identification of robots", 2010, and references therein).  This is especially important on the real robot case, where the authors correctly mention that the WAM arm cannot be expressed exactly by the manipulator equations; this makes it all the more important to try identify system parameters via a data-driven approach, not with the hope of finding the exactly "correct" manipulator equations, but with finding some that are good enough to outperform the "analytical" model that the authors mention.  It's initially non-obvious to me that a generic neural network to model the H and g terms would do any better than some of these standard approaches.
2) A lot of the derivations in the text are frankly unnecessary.  Any standard automatic differentiation toolkit will be able to compute all the necessary derivatives, and for a paper such as this the authors can simply specify the architecture of the system (that they use a Cholesky factorization representation of H, with diagonals required to be strictly positive) and let everything else be handled by TensorFlow, or PyTorch, etc.  The derivations in Sections 4.2 and 4.3 aren't needed.
3) The authors keep referring to the Lagrangian equations as a PDE, and while this is true in general for the actual form here it's just a second order ODE; see e.g. https://en.wikipedia.org/wiki/Lagrangian_mechanics.  Moreover, these are really just the standard manipulator equations for multi-link systems, and can just be denoted as such.
Despite these drawbacks, I really do like the overall idea of the approach presented here, it's just that the authors would need to _substantially_ revise the presentation and experiments in order to make this a compelling paper.  Specifically, if they simply present the method as a system identification approach for the manipulator equations, with the key terms parameterized by a deep network (and compare to relevant system identification approaches), I think the results here would be interesting, even if they would probably be more interesting to a robotics audience rather than a core ML audience.  But as it is, the paper really doesn't situation this work within the proper context, making it quite difficult to assess its importance or significance.


We propose a model that learns to perform zero-shot classification using a metalearner that is trained to produce a correction to the output of a previously trained learner. The model consists of two modules: a task module that supplies an initial prediction, and a correction module that updates the initial prediction. The task module is the learner and the correction module is the meta-learner. The correction module is trained in an episodic approach whereby many different task modules are trained on various subsets of the total training data, with the rest being used as unseen data for the correction module. The correction module takes as input a representation of the task module's training data so that the predicted correction is a function of the task module's training data. The correction module is trained to update the task module's prediction to be closer to the target value. This approach leads to state-of-the-art performance for zero-shot classification on natural language class descriptions on the CUB and NAB datasets.

Summary: This paper proposes a “meta-learning” approach for zero-shot learning. There is a Task Module that works in a conventional zero-shot way: Training to predict a class prototype using the auxiliary/text data description of that task. The new part is the added Correction Module that inputs both the target/zero-shot task description, the training task description, and the current prediction of the task module, and then outputs a correction vector that is added to the output of the task-module to produce the final output. The resulting system achieves state of the art results on zero-shot fine-grained classification (CUB and NAB).
Assessment: Overall this might be a good idea worthy of publication at some point. But despite the good results, the current realisation is not well analysed about exactly how and why it works, with no insight being provided; and leaves some doubt about the validity of the comparative experiments. The writing is also very rushed. It is not ICLR standard yet.
Strengths:
+ Interesting idea overall.
+ Good results.
Weaknesses:
- Poor clarity.
- Some experimental evaluation questions.
- Poor analysis.
Comments:
1. The correction module inputs the full set of training features T_s (Alg1-L13). However the training dataset is fixed, therefore this input is effectively a constant. So its not clear how a constant input can possibly be useful.
1.1 Possibly this has something to do with the episodic training, but this is exactly the kind of thing that should be analysed and explained, but is not discussed at all.
2. The paper is sold as a meta-learning paper, but it’s not clearly explained what is the “meta” part of the algorithm.
3. Its not explained anywhere how exactly the T_s, T_s^u, etc are fed into the correction network. Is it average pooling? It seems that simple average pooling is unlikely to be adequate given the large number (150) of classes in CUB.
4. There are no experimental details such as hyper parameters, network architecture, etc.
5. Based on the ablation study (Tab 2), the baseline task network without correction network already achieves state of the art results. Conceptually the task-network alone is a very standard “regression” based approach to ZSL of the type that people tried almost 10 years ago. So what is the explanation for why its so good? This makes the comparison to all the competitors in Tab1 suspect. If there is some reason (E.g., better image feature extractor or pre/post-processing) that makes the ultra simple baseline there already outperform SotA, then you have to ask how all the prior methods would perform if they were run with the same tweak.
6. Overall no insight provided about what kind of corrections are made, when they are useful, etc. This is important to provide insight about how/why correcting outputs can work.
7. There is nothing particularly unique about this setup for ZSL. It could equally be applied to correct outputs in the case of few-shot learning (CF: Prototype Networks). It would be more convincing if it was applied to both settings and analysed better for both.
8. The writing is very rushed. There are lots of writing and editorial errors. To name a few: P4 Extra “Task module is trained to minimise.” P4 “\mu_u” Is repeated. Citation style “Mohamed Elhoseiny & Elgammal” is wrong, check the bibtex.


LEARNING REPRESENTATIONS OF SETS THROUGH OPTIMIZED PERMUTATIONS
Representations of sets are challenging to learn because operations on sets should be permutation-invariant. To this end, we propose a Permutation-Optimisation module that learns how to permute a set end-to-end. The permuted set can be further processed to learn a permutation-invariant representation of that set, avoiding a bottleneck in traditional set models. We demonstrate our model's ability to learn permutations and set representations with either explicit or implicit supervision on four datasets, on which we achieve state-of-the-art results: number sorting, image mosaics, classification from image mosaics, and visual question answering.

Update: From the perspective of a "broader ML" audience, I cannot recommend acceptance of this paper. The paper does not provide even a clear and concrete problem statement due to which it is difficult for me to appreciate the results. This is the only paper out of all ICLR2019 papers that I have reviewed / read which has such an issue. Of course for the conference, the area chair / program chairs can choose how to weigh the acceptance decisions between interest to the broader ML audience and the audience in the area of the paper.
----------------------------------------------------------------------------------------------------------------------------------
 This paper addresses the problem that often features are obtained as a set, whereas certain orders of these features are known to allow for easier learning. With this motivation the goal of this paper is to learn a permutation of the features. This paper makes the following three main contributions:
1. The idea of using pairwise comparison costs instead of position-based costs
2. The methodological crux of how to go from the pairwise comparison costs to the permutation (that is, solving Eqn. (2) using  Eqn. (1) )
3. An empirical evaluation
I like the idea and the empirical evaluations are promising. However, I have a major concern about the second contribution on the method. There is a massive amount of literature on this very problem and a number of algorithms are proposed in the literature. This literature takes various forms including rank aggregation and most popularly the (weighted) minimum feedback arc set problem.  The submitted paper is oblivious to this enormous literature both in the related work section as well as the empirical evaluations. I have listed below a few papers pertaining to various versions of the problem (this list is by no means exhaustive). With this issue, I cannot give a positive evaluation of this submitted paper since it is not clear whether the paper is just re-solving a solved problem. That said, I am happy to reconsider if the related work and the empirical evaluations are augmented with comparisons to the past literature on the methodological crux of the submitted paper (e.g., why off-the-shelf use of previously proposed algorithms may or may not suffice here.)
Unweighted feedback arc set:
A fast and effective heuristic for the feedback arc set problem, Eades et al.
Efficient Computation of Feedback Arc Set at Web-Scale, Simpson et al.
How to rank with few errors, Kenyon-Mathieu et al.
Aggregating Inconsistent Information: Ranking and Clustering, Ailon et al.
Hardness results:
The Minimum Feedback Arc Set Problem is NP-hard for Tournaments, Charbit et al.
Weighted feedback arc set:
A branch-and-bound algorithm to solve the linear ordering problem for weighted tournaments, Charon et al.
Exact and heuristic algorithms for the weighted feedback arc set problem: A special case of the skew‐symmetric quadratic assignment problem, Flood
Approximating Minimum Feedback Sets and Multicuts in Directed Graphs, Even et al.
Random inputs:
Noisy sorting without resampling, Braverman et al.
Stochastically transitive models for pairwise comparisons: Statistical and computational issues, Shah et al.
On estimation in tournaments and graphs under monotonicity constraints, Chatterjee et al.
Survey (slightly dated):
An updated survey on the linear ordering problem for weighted or unweighted tournaments, Charon et al.
Convex relaxation of permutation matrices:
On convex relaxation of graph isomorphism, Afalo et al.
Facets of the linear ordering polytope, Grotschel


Intuitively, unfamiliarity should lead to lack of confidence. In reality, current algorithms often make highly confident yet wrong predictions when faced with unexpected test samples from an unknown distribution different from training. Unlike domain adaptation methods, we cannot gather an "unexpected dataset" prior to test, and unlike novelty detection methods, a best-effort original task prediction is still expected. We compare a number of methods from related fields such as calibration and epistemic uncertainty modeling, as well as two proposed methods that reduce overconfident errors of samples from an unknown novel distribution without drastically increasing evaluation time: (1) G-distillation, training an ensemble of classifiers and then distill into a single model using both labeled and unlabeled examples, or (2) NCR, reducing prediction confidence based on its novelty detection score. Experimentally, we investigate the overconfidence problem and evaluate our solution by creating "familiar" and "novel" test splits, where "familiar" are identically distributed with training and "novel" are not. We discover that calibrating using temperature scaling on familiar data is the best single-model method for improving novel confidence, followed by our proposed methods. In addition, some methods' NLL performance are roughly equivalent to a regularly trained model with certain degree of smoothing. Calibrating can also reduce confident errors, for example, in gender recognition by 95% on demographic groups different from the training data.

This paper introduces two methods of adjusting the overconfidence error for predictions on novel data. The ensemble distillation approach is to penalize the distillation loss on a potentially unlabeled general dataset.  The second approach (NCR) detects the novelty first and reweigh the prediction based on the familiarity to training data.
*stationarity*
From the statistical perspective, the overconfidence of extrapolation can kick in from two sources: a)the epistemic uncertainty.  The point estimation of softmax ignores the uncertainty of prediction at all.  A full Bayesian approach will remedy this though computationally impractical.  b) the generative distribution p(y|x) might not be identical on training data and test data.  To see the difference, if the training sample size goes to infinity, the uncertainty in a) will go to zero, but b) may still exist.  Section 3 assumes the invariant p(y|x) in novel data.  But theoretically, both methods do not require such invariance?
Slightly related here, there can be novel data for classification, and in principle, there can also be novel data for novelty detection?  That will make NCR fail.
*why the distillation helps uncertainty adjustment*
I am not convinced how the g-distillation works for this task.  In the extreme case if the ensemble model itself is totally wrong for novel data and the unlabeled general data used in training, how can I learn any extra uncertainty information from that noise? To be fair, when the temperature goes high enough, the ensemble will make uniform prediction and then the distillation loss is merely a loss function that enforces uniformity.  If I replace the ensemble softmax by a uniform prior for unlabeled general data, do I achieve the same effect?  That is essentially the same regularization as method 2, except g-distillation is on logit scale.
*robustness-accuracy tradeoff*
The experiments do not reveal too much robustness-efficiency conflict, as the new methods still perform good enough on familiar dataset. Indeed they can be even better than the baseline in E99 loss. Does it suggest the over-confidence is even a concern for familiar data/ iid data?
In general, the paper is well-written and well-motivated. It would be more interesting to make some theoretical explanation why/when this simple approach works.   I would recommend a weak accept at this point.


We propose a new batch mode active learning algorithm designed for neural networks and large query batch sizes. The method, Discriminative Active Learning (DAL), poses active learning as a binary classification task, attempting to choose examples to label in such a way as to make the labeled set and the unlabeled pool indistinguishable. Experimenting on image classification tasks, we empirically show our method to be on par with state of the art methods in medium and large query batch sizes, while being simple to implement and also extend to other domains besides classification tasks. Our experiments also show that none of the state of the art methods of today are clearly better than uncertainty sampling, negating some of the reported results in the recent literature.

This paper presents a new approach to an active learning problem where the idea is to train a classifier to distinguish labeled and unlabeled datapoints and select those that look the most like unlabeled.
The paper is clearly written and easy to follow. The idea is quite novel and evokes interesting thoughts. I appreciated that the authors provide links and connections to other problems. Another positive aspect is that evaluation methodology is quite sound and includes comparison to many recent algorithms for AL with neural networks. The analysis of Section 5.5 is quite interesting.
However, I have a few concerns regarding the methodology. First of all, I am not completely convinced by the fact that selecting the samples that resemble the most unlabeled data is beneficial for the classifier. It seem that in this case just the data from under-explored regions will be selected at every new iteration. If this is the purpose, some simpler methods, for example, relying on density sampling, can be used. Could you elaborate how you method would compare to them? I can see this method as a way to measure the representativeness of datapoints, but I would see it as a component of AL, not an AL alone. What would happen it is combined with Uncertainty and you use it to labeled the points that are both uncertain and resemble unlabeled data?
Besides, the proposed approach does not take the advantage of all the information that is available to AL, in particular, it does not use at the information about labels. I believe that labels contain a lot of useful information for making an informed selection decision and ignoring it when it is available is not rational.
Next, I have conceptual difficulties understanding what would happen to a classifier at next iteration when it is trained on the data that was determined by the previous classifier. Seems that the training data is non-iid and might cause some strange bias. In addition to this, it sounds a bit strange to use classification where overfitting is acceptable.
Finally, the results of the experimental evaluation do not demonstrate a significant advantage of the proposed method and thus it is unclear is there is a benefit of using this method in practice.
Questions:
- Could you elaborate why DAL strategy does not end up doing just random sampling?
- Nothing restrict DAL from being applied with classifiers other than neural networks and smaller problems. How do you think DAL would work on simpler datasets and classifiers?
- How does the classifier (that distinguished between labeled and unlabeled data) deal with very unbalanced classes? I suppose that normally unlabeled set is much bigger than labeled. What does 98% accuracy mean in this case?
- How many experiments were run to produce each figure? Are error bars of most experiments so small that are almost invisible?
Small comments:
- I think in many cases citep command should be used instead of cite.
- Can you explain more about the paragraph 3 of related work where you say that uncertainty-based approach would be different from margin-based approach if the classifier is neural network?
- Last sentence before 3.1: how do you guarantee in this case that the selected examples are not similar to each other (that was mentioned as a limitation for batch uncertainty selection, last paragraph on page 1)?
- It was hard to understand the beginning of 5.5, at first it sounds like the ranking of methods is going to be analysed.
- I am not sure "discriminative" is a good name for this algorithm. It suggested that is it opposite to "generative" (query synthesis?), but then all AL that rank datapoints with some scoring function are "discriminative".


We consider visual domains in which a class label specifies the content of an image, and class-irrelevant properties that differentiate instances constitute the style. We present a domain-independent method that permits the open-ended recombination of style of one image with the content of another. Open ended simply means that the method generalizes to style and content not present in the training data. The method starts by constructing a content embedding using an existing deep metric-learning technique. This trained content encoder is incorporated into a variational autoencoder (VAE), paired with a to-be-trained style encoder. The VAE reconstruction loss alone is inadequate to ensure a decomposition of the latent representation into style and content. Our method thus includes an auxiliary loss, leakage filtering, which ensures that no style information remaining in the content representation is used for reconstruction and vice versa. We synthesize novel images by decoding the style representation obtained from one image with the content representation from another. Using this method for data-set augmentation, we obtain state-of-the-art performance on few-shot learning tasks.
In any domain involving classification, entities are distinguished not only by class label but also by attributes orthogonal to class label. For example, if faces are classified by identity, within-class variation is due to lighting, pose, expression, hairstyle; if masterworks of art are classified by the painter, within-class variation is due to choice of subject matter. Following tradition (Tenenbaum & Freeman, 2000), we refer to between-and within-class variation as content and style, respectively. What constitutes content is defined with respect to a task. For example, in a face-recognition task, identity is the content; in an emotion-recognition task, expression is the content. There has been a wealth of research focused on decomposing content and style, with the promise that decompositions might provide insight into a domain or improve classification performance. Decompositions also allow for the synthesis of novel entities by recombining the content of one entity with the style of another. Recombinations are interesting as a creative exercise (e.g., transforming the musical composition of one artist in the style of another) or for data set augmentation.
We propose an approach to content-style decomposition and recombination. We refer to the method as STOC, for Style Transfer onto Open-Ended Content. Our approach is differentiated from past work in the following ways. First, STOC can transfer to novel content. In contrast, most previous work assumes the content classes in testing are the same as those in training. Second, STOC is general purpose and can be applied to any domain. In contrast, previous work includes approaches that leverage specific domain knowledge (e.g., human body pose). Third, STOC has an explicit objective, leakage filtering, designed to isolate content and style. No such explicit objective is found in most previous work, and as a result, synthesized examples may fail to preserve content as style is varied and vice versa. Fourth, STOC requires a labeling of entities by content class, but explicit style labels are not required. In contrast, some previous work assumes supervised training of both style and content representations. Figure 1 shows examples of content-style recombination using STOC on the VGG-Face (Parkhi et al.,  2015)  data set. In each column, the content of the image in the top row is combined with the style of the image on the bottom row to synthesize a novel image, shown in the middle row. The images in the top and bottom rows are of identities (content) held out from the training set. Style is well maintained, and content is fairly well transferred, at least to the degree that the faces in the middle row are more similar to top-row than bottom-row faces. The training faces are labeled by identity, but style is induced by the training procedure.
In any domain involving classification, entities are distinguished not only by class label but also by attributes orthogonal to class label. For example, if faces are classified by identity, within-class variation is due to lighting, pose, expression, hairstyle; if masterworks of art are classified by the painter, within-class variation is due to choice of subject matter. Following tradition (Tenenbaum & Freeman, 2000), we refer to between-and within-class variation as content and style, respectively. What constitutes content is defined with respect to a task. For example, in a face-recognition task, identity is the content; in an emotion-recognition task, expression is the content. There has been a wealth of research focused on decomposing content and style, with the promise that decompositions might provide insight into a domain or improve classification performance. Decompositions also allow for the synthesis of novel entities by recombining the content of one entity with the style of another. Recombinations are interesting as a creative exercise (e.g., transforming the musical composition of one artist in the style of another) or for data set augmentation.
We propose an approach to content-style decomposition and recombination. We refer to the method as STOC, for Style Transfer onto Open-Ended Content. Our approach is differentiated from past work in the following ways. First, STOC can transfer to novel content. In contrast, most previous work assumes the content classes in testing are the same as those in training. Second, STOC is general purpose and can be applied to any domain. In contrast, previous work includes approaches that leverage specific domain knowledge (e.g., human body pose). Third, STOC has an explicit objective, leakage filtering, designed to isolate content and style. No such explicit objective is found in most previous work, and as a result, synthesized examples may fail to preserve content as style is varied and vice versa. Fourth, STOC requires a labeling of entities by content class, but explicit style labels are not required. In contrast, some previous work assumes supervised training of both style and content representations. Figure 1 shows examples of content-style recombination using STOC on the VGG-Face (Parkhi et al., 2015) data set. In each column, the content of the image in the top row is combined with the style of the image on the bottom row to synthesize a novel image, shown in the middle row. The images in the top and bottom rows are of identities (content) held out from the training set. Style is well maintained, and content is fairly well transferred, at least to the degree that the faces in the middle row are more similar to top-row than bottom-row faces. The training faces are labeled by identity, but style is induced by the training procedure.

SUMMARY
The paper considers several methods for building generative models that disentangle image content (category label) and style (within-category variation). Experiments on MNIST, Omniglot, and VGG-Faces demonstrate that the proposed methods can learn to generate images combining the style of one image and the content of another. The proposed method is also used as a form of learned data augmentation, where it improves one-shot and low-shot learning on Omniglot.
Pros:
- The paper is well-written and easy to follow
- The proposed methods CC, CE, PM, and LF are all simple and intuitive
- Improving low-shot learning via generative models is an interesting and important direction
Cons:
- No comparison to prior work on generation results
- Limited discussion comparing the proposed methods to other published alternatives
- No ablations on Omniglot or VGG-Faces generation
- Low-shot results are not very convincing
COMPARISON WITH PRIOR WORK
There have been many methods that propose various forms of conditional image generation in generative models, such as conditional VAEs in [Sohn et al, 2015]; there have also been previous methods such as [Siddharth et al, 2017] which disentangle style and content using the same sort of supervision as in this paper. Given the extensive prior work on generative models I was a bit surprised to see no comparisons of images generated with the proposed method against those generated by previously proposed methods. Without such comparisons it is difficult to judge the significance of the qualitative results in Figures 3, 5, and 6. In Figure 3 I also find it difficult to tell whether there are any substantial differences between the four proposed methods.
The proposed predictiability minimization is very related to some recent approaches for domain transfer such as [Tzeng et al, 2017]; I would have liked to see a more detailed discussion of how the proposed methods relate to others.
OMNIGLOT / VGG-FACES ABLATIONS
The final model includes several components - the KL divergence term from the VAE, two terms from LF, and a WGAN-GP adversarial loss. How much do each of these terms contribute the quality of the generated results?
LOW-SHOT RESULTS
I appreciate low-shot learning as a testbed for this sort of disentangled image generation, but unfortunately the experimental results are not very convincing. For one-shot performance on Omniglot, the baseline Histogram Embedding methods achieves 0.974 accuracy which improves to 0.975 using STOC. Is such a small improvement significant, or can it be explained due to variance in other factors (random initializations, hyperparameters, etc)?
For low-shot learning on Omniglot, the proposed method is outperformed by [Antoniou et al, 2017] at all values of k. More importantly, I’m concerned that the comparison between the two methods is unfair due to the use of different dataset splits, as demonstrated by the drastically different baseline accuracies. Although it’s true that the proposed method achieves a proportionally larger improvement over the baseline compared with [Antoniou et al, 2017], the differences in experimental setup may be too large to draw a conclusion one way or the other about which method is better.
OVERALL
Although the paper is well-written and presents several intuitive methods for content/style decomposition with generative models, it’s hard to tell whether the results are significant due to incomplete comparison with prior work. On the generation side I would like to see a comparison especially with [Siddharth et al, 2017]. For low shot learning I think that the proposed method shows some promise, but it is difficult to draw hard conclusions from the experiments. For these reasons I lean slightly toward rejection.
MISSING REFERENCES
Siddharth et al, “Learning Disentangled Representations with Semi-Supervised Deep Generative Models”, NIPS 2017
Sohn, Lee, and Yan, “Learning structured output representation using deep conditional generative models”, NIPS 2015
Tzeng, Hoffman, Darrell, and Saenko, “Adversarial Discriminative Domain Adaptation”, CVPR 2017


TOP-DOWN NEURAL MODEL FOR FORMULAE
We present a simple neural model that given a formula and a property tries to answer the question whether the formula has the given property, for example whether a propositional formula is always true. The structure of the formula is captured by a feedforward neural network recursively built for the given formula in a topdown manner. The results of this network are then processed by two recurrent neural networks. One of the interesting aspects of our model is how propositional atoms are treated. For example, the model is insensitive to their names, it only matters whether they are the same or distinct.

Cons
1.	There is no study of the representations developed by the model, which is unfortunate because this is a conference on learning representations and because there is little light shed on how the network achieves its rather high level of performance.
2.	It seems less generally useful to have such a special-purpose network for computing global properties like tautologicality than to have a network that produces actual vector encodings of propositions, as typical of the bottom-up tree-structured models.
Pros
3.	The paper is quite clear.
4.	The problem is important.
5.	The paper pursues the familiar path of a tree-structured network isomorphic to the parse tree of a propositional-calculus formula, but with the original twist of passing information top-down rather than bottom-up.
6.	The results are impressively strong. In particular, it improves by 10% absolute over the special-purpose and highly performant PossibleWorldNet on the most difficult category of problems, the ‘massive’ category, achieving 83.6% accuracy.
Pro/Con mix
7.	Although the paper did not provide much insight into what was going on in the network to allow it to perform well (point 1 in ‘Cons’), I was able to convince myself I could understand a way the architecture *could* succeed (whether this possible approach matches the actual processing in the model I have no way of assessing). In brief, the vector that is passed down the network can be thought of as a list of truth values across multiple possible worlds of the tree node at which the vector resides. To search for a counterexample to tautologicalhood, the original input vector to the root node could be the zero (false) vector. If the kth value in the vector at a parent node labeled ‘or’ is 0 (the disjunction is false in world k) then in the two children the kth value must also be 0. If the kth value of the vector at an XOR node is 0, the kth value of the two children must both be 0 or both be 1; actually these values need not reside in position k so the children could both have value 0 at some position i and both have value 1 at another position j. Then in the RNN-Var component of the network, which checks for consistency across multiple tokens of the same proposition variable, each position k in all vectors for the same variable can be checked for equality, producing a value 1 in the output vector if all have value 1, producing 0 if all have value 0, and producing value -1 if the values do not all agree. Then RNN-All checks across all vectors for proposition variable types to see if there’s a position k in which no value -1 occurs; if so, the values of the variable vectors at position k give the truth values for all variables such that the overall proposition has the desired value 0: a counterexample exists. If no such position k exists, the proposition is a tautology. This seems roughly right, at least.


REASONING ABOUT PHYSICAL INTERACTIONS WITH OBJECT-ORIENTED PREDICTION AND PLANNING
Object-based factorizations provide a useful level of abstraction for interacting with the world. Building explicit object representations, however, often requires supervisory signals that are difficult to obtain in practice. We present a paradigm for learning object-centric representations for physical scene understanding without direct supervision of object properties. Our model, Object-Oriented Prediction and Planning (O2P2), jointly learns a perception function to map from image observations to object representations, a pairwise physics interaction function to predict the time evolution of a collection of objects, and a rendering function to map objects back to pixels. For evaluation, we consider not only the accuracy of the physical predictions of the model, but also its utility for downstream tasks that require an actionable representation of intuitive physics. After training our model on an image prediction task, we can use its learned representations to build block towers more complicated than those observed during training.

A method is proposed, which learns to reason on physical interactions of different objects (solids like cuboids, tetrahedrons etc.). Traditionally in related work the goal is to predict/forecast future observations, correctly predicting (and thus learning) physics. This is also the case in this paper, but the authors explicitly state that the target is to evaluate the learned model on downstream tasks requiring a physical understanding of the modelled environment.
The main contribution here lies in the fact that no supervision is used for object properties. Instead, a mask predictor is trained without supervision, directly connected to the rest of the model, ie. to the physics predictor and the output renderer. The method involves a planning phase, were different objects are dropped on the scene in the right order, targeting bottom objects first and top objects later. The premise here is that predicting the right order of the planning actions requires understanding the physics of the underlying scene.
I particularly appreciated the fact, that object instance renderers are combined with a global renderer, which puts individual images together using predicted heatmaps for each object. With a particular parametrization, these heatmaps could be related to depth maps allowing correct depth ordering, but depth information has not been explicitly provided during training.
Important issues:
One of the biggest concerns is the presentation of the planning algorithm, and more importantly, a proper formalization of what is calculated, and thus a proper justification of this part. The whole algorithm is very vaguely described in a series of 4 items on page 4. It is intuitively almost clear how these steps are performed, but the exact details are vague. At several steps, calculated entities are “compared” to other entities, but it is never said what this comparison really results in. The procedure is reminiscent of particle filtering, in that states (here: actions) are sampled from a distribution and then evaluated through a likelihood function, resulting in resampling. However, whereas in particle filtering there is clear probabilistic formalization of all key quantities, in this paper we only have a couple of phrases which describe sampling and “comparisons” in a vague manner.
Since the procedure performs planning by predicting a sequence of actions whose output at the end can be evaluated, thus translated into a reward, I would have also liked a discussion (or at least a remark) why reinforcement learning has not been considered here.
I am also concerned by an overclaim of the paper. As opposed to what the paper states in various places, the authors really only evaluate the model on video prediction and not on other downstream tasks. A single downstream task is very briefly mentioned in the experimental section, but it is only very vaguely described, it is unclear what experiments have been performed and there is no evaluation whatsoever.
Open questions:
Why is the proposed method better than one of the oracles?
Minor remarks:
It is unclear what we see in image 4, as there is only a single image for each case (=row) and method (=column).
The paper is not fully self-contained. Several important aspects are only referred to by citing work, e.g. CEM sampling and perceptual loss. These are concepts which are easy to explain and which do not take much space. They should be added to the paper.
A threshold is mentioned in the evaluation section. A plot should be given showing the criterion as a function of this threshold, as is standard in, for instance, pose estimation literature.
I encourage the authors to use the technical terms “unary terms” and “binary terms” in the equation in section 2.2. This is the way how the community referred to interactions in graphical models for relational reasoning long before deep learning showed up on the horizon, let’s be consistent with the past.
I do not think that the physics module can be reasonable be called a “physics simulator” as has been done throughout the paper. It does not simulate physics, it predicts physics after learning, which is not a simulation.
A cube has not been confused with a rectangle, as mentioned in the paper, but with a rectangular cuboid. A rectangle is a 2D shape, a rectangular cuboid is a 3D polyhedron.


VISUAL REASONING BY PROGRESSIVE MODULE NETWORKS
Humans learn to solve tasks of increasing complexity by building on top of previously acquired knowledge. Typically, there exists a natural progression in the tasks that we learn -most do not require completely independent solutions, but can be broken down into simpler subtasks. We propose to represent a solver for each task as a neural module that calls existing modules (solvers for simpler tasks) in a functional program-like manner. Lower modules are a black box to the calling module, and communicate only via a query and an output. Thus, a module for a new task learns to query existing modules and composes their outputs in order to produce its own output. Our model effectively combines previous skill-sets, does not suffer from forgetting, and is fully differentiable. We test our model in learning a set of visual reasoning tasks, and demonstrate improved performances in all tasks by learning progressively. By evaluating the reasoning process using human judges, we show that our model is more interpretable than an attention-based baseline.

[Summary]
This paper presents a multi-task learning approach for VQA that represent a solver for each task as a neural module that calls existing modules in a program manner. The authors manually design the task hierarchy and propose a progressive module network to recursive calls the lower modules and gather the information by soft-attention. The final prediction uses all the states and question to infer the final answers. The authors verify the effectiveness of the proposed method on the performance of different tasks and modules. Experiment on VQA shows the proposed model benefits from utilizing different modules. The authors also qualitatively show the model's reasoning process and human study on judging answering quality.
[Strength]
1. The proposed method is novel and explores to use the existing modules as a black box for visual question answering.  This is different from most existing work.
2: By examing different modules, the proposed method is more interpretable compare to canonical methods.
3: The experiment results are good, especially for the counting problem.
[Weakness]
1. The title of the paper is "visual reasoning by progressive module networks." The title may be a little overstated since the major task is focused on visual question answering (VQA).
2. Annotation is not clear in this paper. For example, on page 3, Query transmitter and receiver, "the output o_k = M_k(q_k) received from M_k is modified using receiver function as v_k = R_{k->n}(s^t, o_k). " There are multiple new variables in this paragraph, without specifying the dimension and meaning for each attribute, it's really hard to understand. On page 4, State update function, what is the meaning of variable "Epsilon" in the equation? From the supplementary, it seems Epsilon means the environment?
3. On the object counting task, the query transmitter needs to produce a query for a relationship module. The authors mentioned that this is softly calculated by softmax on the importance score. Since q_rel require one hot vector as input, how to sample the q_rel given the importance score and how backprob the gradient in this case?
4. The cider score of image captioning is 109 compared to the baseline 108. The explanation is the COCO dataset has a fixed set of 80 object categories and does not benefit from training the diverse data. Since the input visual feature is the same, the only difference is the proposed model has additional label embedding as input. My assumption is the visual feature already contains the label information for image captioning.
5. On relational detection task, is there a way to compare with the STOA method on some specific data split? This will leads to much more convincing results.
6. Similar as above question, on the object counting task, is there a way to compare with previous counting methods?
7. In Table 4, the accuracy of number on Zhang et.al is 49.39, which is higher than other methods, while on test-dev, the accuracy is 51.62, which is lower than others. Is the number right?


The expressive power of end-to-end automatic speech recognition (ASR) systems enables direct estimation of the character or word label sequence from a sequence of acoustic features. Direct optimization of the whole system is advantageous because it not only eliminates the internal linkage necessary for hybrid systems, but also extends the scope of potential application use cases by training the model for multiple objectives. Several multi-lingual ASR systems were recently proposed based on a monolithic neural network architecture without language-dependent modules, showing that modeling of multiple languages is well within the capabilities of an end-to-end framework. There has also been growing interest in multispeaker speech recognition, which enables generation of multiple label sequences from single-channel mixed speech. In particular, a multi-speaker end-to-end ASR system that can directly model one-to-many mappings without additional auxiliary clues was recently proposed. In this paper, we propose an all-in-one end-to-end multi-lingual multi-speaker ASR system that integrates the capabilities of these two systems. The proposed model is evaluated using mixtures of two speakers generated by using 10 languages, including mixed-language utterances.

This paper presents an end-to-end system that can recognize single-channel multiple-speaker speech with multiple languages.
Pros:
- The paper is well written.
- It shows the existing end-to-end multi-lingual ASR (Seki et al., 2018b) and end-to-end multi-speaker ASR (Seki et al., 2018a) techniques can be combined without any change to achieve reasonable performance.
- It demonstrates the challenge of single-channel multi-lingual multiple-speaker speech recognition, and compares the performance of the multiple-speaker system on the mixed speech and the single-speaker system on the isolated speech.
Cons:
- It lacks novelty: the proposed framework just simply combines the two existing techniques as mentioned above.
- The training and evaluation data are both artificially created by randomly concatenating utterances with different languages from different speakers with different context. I am not sure of how useful the evaluation is, since this situation is not realistic. Also, currently it cannot test the real code-switching since the utterances are not related and not from the same speaker.
- There are not enough analyses. E.g. it would be good to analyze what contributes to the gap between the single-speaker ASR system performance on the isolated speech and the multi-lingual multi-speaker ASR system on the mixed speech. How well does the proposed end-to-end framework perform compared to a two-step framework with speaker separation followed by multi-lingual single-speaker ASR?


Random Matrix Theory (RMT) is applied to analyze the weight matrices of Deep Neural Networks (DNNs), including both production quality, pre-trained models such as AlexNet and Inception, and smaller models trained from scratch, such as LeNet5 and a miniature-AlexNet. Empirical and theoretical results clearly indicate that the empirical spectral density (ESD) of DNN layer matrices displays signatures of traditionally-regularized statistical models, even in the absence of exogenously specifying traditional forms of regularization, such as Dropout or Weight Norm constraints. Building on recent results in RMT, most notably its extension to Universality classes of Heavy-Tailed matrices, we develop a theory to identify 5+1 Phases of Training, corresponding to increasing amounts of Implicit Self-Regularization. For smaller and/or older DNNs, this Implicit Self-Regularization is like traditional Tikhonov regularization, in that there is a "size scale" separating signal from noise. For state-of-the-art DNNs, however, we identify a novel form of Heavy-Tailed Self-Regularization, similar to the selforganization seen in the statistical physics of disordered systems. This implicit Self-Regularization can depend strongly on the many knobs of the training process. By exploiting the generalization gap phenomena, we demonstrate that we can cause a small model to exhibit all 5+1 phases of training simply by changing the batch size.

This manuscript studies the implicit regularization of neural networks from the perspective of random matrix theory. The authors provide both empirical and theoretical results that aim to show that the empirical spectral density of weights of DNN captures the implicit regularization phenomenon. However, the results are far from rigorous theory and it is not clear how recent results in MP theory yields the statements made in the paper.
Detailed comments:
1. The empirical studies seem interesting. It seems that two kinds of results are shown. The first one is that ESD fits perfectly for small models, and the second one is that deep models fit heavy-tailed random matrices class. It would be interesting to see more details about how these models are trained, as training greatly affects the value of the weights.
2. Theoretical results are not clearly stated. In section 2, the authors introduce the basics of MP theory. However, it is not clear how to derive the theory in this paper based on the MP theory. It seems that the main theory is the "5+1 phases of training". The definition of these 6 phases is not even explicitly given in this paper. Moreover, the theory of all these phases seems to depend on equation (3) , but there are no lemmas or propositions that gives a rigourious theoretical guarantee.


MINIMUM DIVERGENCE VS. MAXIMUM MARGIN: AN EMPIRICAL COMPARISON ON SEQ2SEQ MODELS
Sequence to sequence (seq2seq) models have become a popular framework for neural sequence prediction. While traditional seq2seq models are trained by Maximum Likelihood Estimation (MLE), much recent work has made various attempts to optimize evaluation scores directly to solve the mismatch between training and evaluation, since model predictions are usually evaluated by a task specific evaluation metric like BLEU or ROUGE scores instead of perplexity. This paper puts this existing work into two categories, a) minimum divergence, and b) maximum margin. We introduce a new training criterion based on the analysis of existing work, and empirically compare models in the two categories. Our experimental results show that our new training criterion can usually work better than existing methods, on both the tasks of machine translation and sentence summarization.

The authors have updated the paper and clarified some things, and now my impression of the paper has improved. It still feels a little incremental to me, but the potential application areas of these sorts of models are quite large and therefore incremental improvements are not insignificant. This paper suggests some natural follow-up work in exploring Hellinger distance and other variations for these models.
----- original review follows: ------
This paper discusses loss functions for sequence generation tasks that take into account cost functions that reflect the task-specific evaluation metric. They compare RAML and risk (MRT) formally and empirically, and also test a loss based on Hellinger distance. They compare these to some standard max-margin losses. MRT and the Hellinger distance loss perform best in NMT and summarization experiments.
Pros:
There are some interesting aspects of this paper:
- It is interesting to note that RAML and MRT are (similar to) different directions of KL divergences between the same two distributions. (Caveat: the entropy regularizer, which I discuss in "Cons" below.)
- The new Hellinger-distance-based loss seems promising.
- The empirical comparison among losses for standard NMT/summarization tasks is a potentially valuable contribution.
Cons:
A.
The focus/story of the paper need some work. It is unclear what the key contributions are. I think the Hellinger distance loss is potentially the most important contribution, but the authors don't spend much time on that.. it seems that they think the comparison of divergence and max-margin losses is more central. However, I think the authors' conclusion (that the divergence losses are better than the max-margin losses) is not the main story, because RAML is not much better than the max-margin losses. Also, I have some concerns about some of the details of the max-margin losses (listed and discussed below), so I'm not sure how reliable the empirical comparison is.
B.
As for the connections and comparison between RAML and MRT:
It does not seem that MRT corresponds to a divergence of the form given at the start of Sec. 4. There is also an entropy regularizer in Eq. (9). Sec. 4.3 states: "By comparing the above two methods, we find that both RAML and MRT are minimizing the KL divergence between the model output distribution and the exponentiated payoff distribution, but with different directions of D_KL." However, this statement ignores the entropy regularizer in Eq. (9).
Maybe I'm being dense, but I didn't understand where Equation (10) comes from. I understand the equations above it for RAML, but I don't understand the MRT case in Eq. (10). Can you provide more details?
I also don't understand the following sentence: "It turns out that the hyperparameter \tau in RAML and \alpha in MRT have the same effect." What does this mean mathematically? Also, does this equivalence also require ignoring the entropy regularizer? As formulated, L_{RAML} necessarily contains a \tau, but L_{MRT} does not necessarily contain an alpha. It is only when moving to the sample approximation does the alpha become introduced. (MRT does not require this sample approximation; Some older work on MRT developed dynamic programming algorithms to exactly compute the gradients for structured output spaces like sequences, so samples were not used in those cases.) So I think the paper needs to clarify what exactly is meant by the connection between tau and alpha, under what conditions there is a connection between the two, and what exactly is the nature of this connection. If more space is needed for this, many of the details in Sec 3 can be cut or moved to appendices because those are standard and not needed for what follows.
In the experimental results (Sec. 7), MRT outperforms RAML consistently. The authors discuss the impact of the directionality of the KL divergence, but what about the entropy regularizer? It would be interesting to compare MRT both with and without the entropy regularizer. Without the regularizer, MRT would actually correspond to the KL that the authors are describing in the discussion. As it currently stands, two things are changing between MRT and RAML and we don't know which is responsible for the sizable performance gains.
C.
There are several technical issues with the writing (and potentially with the claims/conclusions), most of which are potentially flexible with some corrections and more exposition by the authors:
Is L_{RAML} to be maximized or minimized? Looks like maximized, but clearly both L_{MLE} and L_{MRT} are supposed to be minimized, so the use of L for all of these seems confusing. If different conventions are to be used for each one, it should be explicitly mentioned in each case whether the term is to be maximized or minimized.
At the end of Sec. 4.1, q' is not defined. I can guess what it is, but it is not entirely clear from the context and should be defined.
In Equation 6, please use different notation for the y in the denominator (e.g., y') to avoid collision with the y in the numerator and that on the left-hand side.
The discussion of max-margin losses in Sec. 5 has some things that should be fixed.
1. In Sec. 5, it is unclear why \Delta is defined to be the difference of two r() functions. Why not just make it -r(y, y^*)? Are there some implied conditions on \Delta that are not stated explicitly? If \Delta is assumed to be nonnegative, that should be stated.
2. In Eq. (11), F appears to be a function of y and theta, but in the definition of F, it has no functional arguments. But then further down, F appears to be a function of y only. Please make these consistent.
3. Eq. (11) is not only hard for structured settings; it is also hard in the simplest settings (binary classification with 0-1 loss). This is the motivation for surrogate loss functions in empirical risk minimization for classification settings. The discussion in the paper makes it sound as if these challenges only arise in the structured prediction setting.
4. I'm confused by what the paper is attempting to communicate with Equations 12 and 13. In Eq. 12, y on the left-hand side is not bound to anything, so it is unclear what is being stated exactly. Is it for all y? For any y? In Eq. 13, the \Delta on the right-hand side is outside the max over y -- is that really what was intended? I thought the max (the slack-rescaled loss-augmented inference step) should take into account \Delta. Otherwise, it is just doing an argmax over the score function.
5. If the authors are directly optimizing the right-hand side of the inequality in Equation 12 (as would be suggested for the formula for the gradient), then there is no global minimum of the loss. It would go to negative infinity. Typically people use a "max(0, )" outside the loss so that the global minimum is 0.
Typos and minor issues follow:
Sec. 1:
"the SEARN" --> "SEARN"
"the DAGGER" --> "DAGGER"
Sec. 2:
"genrally" --> "generally"
"took evaluation metric into training" --> "incorporated the evaluation metric into training"
"consistant" --> "consistent"
Sec. 3:
Use \exp instead of exp.
"a detail explanation" --> "a detailed explanation"
Sec. 4.1:
"predition" --> "prediction"
Sec. 4.2:
"only single sample" --> "only a single sample"
Sec. 6.1:
"less significant than" --> "less significantly than"


We introduce advocacy learning, a novel supervised training scheme for classification problems. This training scheme applies to a framework consisting of two connected networks: 1) the Advocates, composed of one subnetwork per class, which take the input and provide a convincing class-conditional argument in the form of an attention map, and 2) a Judge, which predicts the inputs class label based on these arguments. Each Advocate aims to convince the Judge that the input example belongs to their corresponding class. In contrast to a standard network, in which all subnetworks are trained to jointly cooperate, we train the Advocates to competitively argue for their class, even when the input belongs to a different class. We also explore a variant, honest advocacy learning, where the Advocates are only trained on data corresponding to their class. Applied to several different classification tasks, we show that advocacy learning can lead to small improvements in classification accuracy over an identical supervised baseline. Through a series of follow-up experiments, we analyze when and how Advocates improve discriminative performance. Though it may seem counter-intuitive, a framework in which subnetworks are trained to competitively provide evidence in support of their class shows promise, performing as well as or better than standard approaches. This provides a foundation for further exploration into the effect of competition and class-conditional representations.

This paper presents a novel concept of supervised learning, advocacy learning. In this framework, supervised learning procedure is given by two subnetworks, advocates and judge. Advocates generate evidence in the form of attention for individual classes and judge decide the final class labels.
The main idea looks interesting, and the paper is clear enough to deliver the idea. However, this paper has the following major issues.
1. There is no formal justification of the idea. Although the idea looks interesting, there is no theoretical background and no clear intuition.
2. Experiment is weak and even inconsistent. Evaluation is performed on very small datasets only, where all baseline methods already show very high accuracy and accuracy gain given by the proposed method is very marginal. In particular, Table 1 and 2  have inconsistent results; advocacy network is better in Table 1 while worse in Table 2 compared to honest advocacy network. To make the idea more convincing, it is required to test it on much larger datasets, at least ImageNet scale, and more desirable to show results in other tasks such as object detection and image segmentation.
3. I am not sure if advocacy network has any separate supervision to enforce it to be learned in a class-conditional manner. Also, in honest advocacy network, each subnetwork can look at only a part of dataset (data corresponding to the class), and I wonder if there is any problem given by data deficiency issue.
Overall, the paper does not look ready for publication because the idea is clearly justified neither theoretically nor empirically.