Ranking is a central task in machine learning and information retrieval. In this task, it is especially important to present the user with a slate of items that is appealing as a whole. This in turn requires taking into account interactions between items, since intuitively, placing an item on the slate affects the decision of which other items should be placed alongside it. In this work, we propose a sequence-to-sequence model for ranking called seq2slate. At each step, the model predicts the next item to place on the slate given the items already selected. The recurrent nature of the model allows complex dependencies between items to be captured directly in a flexible and scalable way. We show how to learn the model end-to-end from weak supervision in the form of easily obtained click-through data. We further demonstrate the usefulness of our approach in experiments on standard ranking benchmarks as well as in a real-world recommendation system.

The authors consider the problem of re-ranking an initial ranker that doesn’t consider interactions between items (e.g., a point-wise ranker) with a pointer-network approach that considers these interactions when re-ordering the input ranking. Notably, this is performed during decoding as opposed to {pairwise, list-wise} learning to rank approaches that consider interactions during training, but emit an item-wise score during inference. Operationally in practice, this has to be trained from click-through data for which the authors consider both a RL approach (Reinforce) and supervised training (a sequence-level hinge loss function) and decoded either with a single-step greedy policy or a sampling procedure. Experiments are conducted on learning-to-rank benchmarks where interactions are introduced to test the validity of the method and on a real-world, large-scale recommendation engine — showing solid improvements in both cases.
From a high-level perspective, the methodological innovation (a pointer-network trained on sequence loss from logged data), setting (re-ranking a slate to consider interactions), and empirical analyses are largely ‘incremental’ — although I think non-trivial to put together and the paper itself is well-written and fairly convincing. In framing the paper this way, I would have expected some comparison to sub-modular methods on the ‘diverse-clicks’ generated data for completeness, although I would be surprised if the Seq2Slate method doesn’t perform better (but all the more reason to conduct). In addition to reporting how to resolve some of the details of applying this, the most interesting results may very well be the real-world experiments as the result improvements are fairly impressive (such that I intend to play with this myself). Thus, as the focus is on details and empirical results over methodological innovation, this paper reads a bit like an industry-track paper — but I find the results interesting overall and am marginally inclined to accept.
Evaluating the paper along the requested dimensions:
= Quality: The paper clearly states its motivation, proposes a model, discusses practical issues, and provides convincing experiments (given the constraints of proprietary data, etc.). I didn’t observe any technical flaws and everything was relatively self-contained and easy to read. I could think of a few more experiments regarding submodular-based models, possibly different settings of the ‘diverse-click’ data for a sensitivity analysis, and a more direct comparison to [Ai, et al., SIGIR18], but this isn’t required to make the results sufficiently convincing. (6/10)
= Clarity: The paper is very clearly written. (7/10)
= Originality: This is the weakest aspect of the paper from a methodological perspective. It is a fairly straightforward application of pointer-networks. Even the path forward is fairly straightforward as outlined in the conclusion. One additional pointer that is methodologically similar, but for ‘discrete choice’ as opposed to re-ranking is [Mottini & Acuna-Agost, Deep Choice Model Using Pointer Networks for Airline Itinerary Prediction; KDD17] (which honestly, is probably a better venue for this specific work). Non-trivial and complete, but not particularly innovative. (5/10)
= Significance: Methodologically, probably will influence some work regarding re-ranking methodologically. From a practical perspective, seems very promising. A few more experiments would make this case stronger, but has real-world data. (6/10)
=== Pros ===
+ extends a widely used model (pointer-networks) to the re-ranking setting
+ discusses practical issues in getting this to work at scale
+ shows that it works in a real-world setting
+ contextualization within existing research shows good understanding of related work
=== Cons ===
- is a fairly direct application of pointer-networks with the innovation being in the details (i.e., is more of an ‘industry’ paper)
- additional experiments around ‘diverse-clicks’ settings (to see how smooth the performance curve) and submodular comparisons may have been interesting
In summary, I think there is room for improvement (some outlined in the conclusion), but is an interesting finding with promise that I plan to try myself. Thus, I lean toward an accept.


This report introduces a training and recognition scheme, in which classification is realized via class-wise discerning. Trained with datasets whose labels are randomly shuffled except for one class of interest, a neural network learns class-wise parameter values, and remolds itself from a feature sorter into feature filters, each of which discerns objects belonging to one of the classes only. Classification of an input can be inferred from the maximum response of the filters. A multiple check with multiple versions of filters can diminish fluctuation and yields better performance. This scheme of discerning, maximum response and multiple check is a method of general viability to improve performance of feedforward networks, and the filter training itself is a promising feature abstraction procedure. In contrast to the direct sorting, the scheme mimics the classification process mediated by a series of one component picking.

The paper discusses a method to increase accuracy of deep-nets on multi-class classification tasks by what seems to be a reduction of multi-class to binary classification following the classical one-vs-all mechanism. I fail to see any novelty in the paper. The problem of reducing multi-class to binary classification has been studied thoroughly with many classical papers like:
1. Usual one-vs-all - this paper does the same thing as one vs all in my opinion even though this technique is known for a decade or so.
2. Weighted One-Against-All - http://hunch.net/~jl/projects/reductions/woa/woa.pdf
and more sophisticated techniques like:
3. Alina Beygelzimer, John Langford, Pradeep D. Ravikumar. Error-Correcting Tournaments. CoRR, abs/0902.3176, 2009.
4. Erin L. Allwein, Robert E. Schapire, Yoram Singer, Pack Kaelbling. Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers. Journal of Machine Learning Research, 113—141, 2000.
5. Thomas G. Dietterich, Ghulum Bakiri. Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research, 2:263—286, 1995.
In my opinion the methods in [1,2,3,5] above can be used with any binary learners and therefore deep-networks. This paper makes no effort in comparing with any of these well-known papers. Moreover the experiments do not show any gain in state of the art performances in the data-sets used, as experiments are done with toy-networks. Further some rules for selecting the class is discussed in Section 3. There are many known rules for generating probability scores in one-vs-all classification and the relations to these are not discussed.
Therefore, I fail to see any novelty in this paper (theoretical or empirical).


Bottleneck structures with identity (e.g., residual) connection are now emerging popular paradigms for designing deep convolutional neural networks (CNN), for processing large-scale features efficiently. In this paper, we focus on the information-preserving nature of identity connection and utilize this to enable a convolutional layer to have a new functionality of channel-selectivity, i.e., redistributing its computations to important channels. In particular, we propose Selective Convolutional Unit (SCU), a widely-applicable architectural unit that improves parameter efficiency of various modern CNNs with bottlenecks. During training, SCU gradually learns the channel-selectivity on-the-fly via the alternative usage of (a) pruning unimportant channels, and (b) rewiring the pruned parameters to important channels. The rewired parameters emphasize the target channel in a way that selectively enlarges the convolutional kernels corresponding to it. Our experimental results demonstrate that the SCU-based models without any postprocessing generally achieve both model compression and accuracy improvement compared to the baselines, consistently for all tested architectures.

This is an architecture design paper. It proposes a general structure called Selective Convolutional Unit that the authors claim to be useful for various CNN models. The SCU structure contains two major parts: CD and NC. CD for compressing/pruning channels and NC for multiplicative noise. The paper gives a measure, called expected channel damage score, on the change of the output for SCU. It also shows the effectiveness of SCU on CIFAR-10, CIFAR-100 and imagenet.
Some questions and concerns:
1. The paper spends too much space introducing the bottleneck structures and a whole lot of the details on the optimization of NC and CD are put in the appendix. I would suggest to reduce the section of introductory part and put a shorter version of appendix A and B to the main text so that the readers know more about the architecture and how it is optimized. In particular, the description on NC is confusing since without looking at the appendix it is not clear how the prior p(\theta) is used.
2. The experiment shows improvement on densenet and resnetXT, but the result is not the state-of-the-art. Wide-Resnet seems to get better accuracy on both CIFAR-10 and CIFAR-100 compared to the best accuracy reported in this paper. Also the number reported by the original densenet paper on imagenet seems to be better (densenet-264 has an error rate of 22.15/20.80)
3. In your CD design, channel assignment \pi is a discrete variable. How is it optimized in the training process?
4. The proof of proposition 1 does not look correct to me. The optimization procedure makes use of the data X to determine your NC variable \theta so \theta depends on X. In this way you cannot factorize the expectation in the equation below (20) in your appendix.


TOWARDS UNDERSTANDING REGULARIZATION IN BATCH NORMALIZATION
Batch Normalization (BN) improves both convergence and generalization in training neural networks. This work understands these phenomena theoretically. We analyze BN by using a basic block of neural networks, consisting of a kernel layer, a BN layer, and a nonlinear activation function. This basic network helps us understand the impacts of BN in three aspects. First, by viewing BN as an implicit regularizer, BN can be decomposed into population normalization (PN) and gamma decay as an explicit regularization. Second, learning dynamics of BN and the regularization show that training converged with large maximum and effective learning rate. Third, generalization of BN is explored by using statistical mechanics. Experiments demonstrate that BN in convolutional neural networks share the same traits of regularization as the above analyses.

This is a thought provoking paper that aims to understand the regularization effects of batch-normalization (BN) under a probabilistic interpretation. The authors connect BN to population normalization (PN) and a gamma-decay term that penalizes the scale of the weights. They analyze the generalization error of BN for a single-layer perceptron using ideas in statistical physics.
Detailed comments:
1. Theorem 1 uses the loss function of a single-layer perceptron in the proof. This is not mentioned in the main writeup. This theorem is not valid in general.
2. The main contribution of this paper is Theorem 1 which connects BN to population normalization and weight normalization. It shows that the regularization of BN can be split into two components that depend on the mini-batch mean and variances: the former penalizes the magnitude of activations while the latter penalizes their correlation.
3. Although the theoretical analysis is conducted under simplistic models, this paper corroborates a number of widely-known observations about BN in practice. It validates these predictions on standard experiments.
4. The scaling of BN regularization with batch-size can be easily seen from Teye et al., 2018, so I think the experiments that validate this prediction are not strictly necessary.
5. It is difficult to use these techniques for deep non-linear networks.
6. The predictions in Section 3.3 are very interesting: it is often seen that fully-connected layers (where BN helps significantly) need small learning rates to train without BN; with BN one can use larger learning rates.
7. The experimental section is very rough. In particular the experiments on CIFAR-10 and downsampled-ImageNet with CNNs seem to have very high errors and it is difficult to understand whether some of the predictions about generalization error apply here. Why not use a more recent architecture for CIFAR-10?
8. There is a very large number of grammatical and linguistic errors in the narrative.
9. The presentation of the paper is very dense, I would advise the authors to move certain parts to the appendix and remove the inlining of important equations to improve readability.


Single-image super-resolution (SISR) is a canonical problem with diverse applications. Leading methods like SRGAN (Ledig et al., 2017)  produce images that contain various artifacts, such as high-frequency noise, hallucinated colours and shape distortions, which adversely affect the realism of the result. In this paper, we propose an alternative approach based on an extension of the method of Implicit Maximum Likelihood Estimation (IMLE) (Li & Malik, 2018). We demonstrate greater effectiveness at noise reduction and preservation of the original colours and shapes, yielding more realistic super-resolved images.

Quality: The overall quality of this paper is good. It adopts a simple but novel idea on SISR and shows clear improvement against existing method (e.g., SRGAN).
Clarify: This paper is well written and easy to follow. It shows a clear motivation for adopting the implicit probabilistic model.
Originality: To the best of my knowledge, this paper is the first work to learn multi-modal probabilistic model for SISR.
Significance: While the results can be further improved (still look a bit blurred), this paper shows an interesting and important direction to learn better mappings for SISR.
Pros:
+ The writing is clear.
+ The proposed method is well motivated and easy to understand.
+ The experimental results include both objective and subjective evaluations.
Cons:
- The two-stage architecture is similar to the following generative models and SR methods. It’s suggested to discuss them as well.
[a] Denton, E. L., Chintala, S., & Fergus, R. “Deep generative image models using a￼ laplacian pyramid of adversarial networks”. NIPS, 2015.
[b] Karras, T., Aila, T., Laine, S., & Lehtinen, J. “Progressive growing of gans for improved quality, stability, and variation”. ICLR 2018.
[c] Lai, W. S., Huang, J. B., Ahuja, N., & Yang, M. H. “Deep laplacian pyramid networks for fast and accurate super-resolution.” CVPR 2017.
[d] Wang, Y., Perazzi, F., McWilliams, B., Sorkine-Hornung, A., Sorkine-Hornung, O., & Schroers, C. “A Fully Progressive Approach to Single-Image Super-Resolution.”. CVPR Workshops 2018.
- In the hierarchical sampling (section 2.4), it’s not clear how to generate the upper noise vector “conditioned on the lower noise vector”.
- The hierarchical sampling seems to improve the efficiency of training. I wonder does it affect the results of testing?
- In the implementation details (section 2.5), I don’t understand why you need to transfer the the feature activations from GPU to CPU? I think all the computation can be done on GPU for most common toolboxes. Projecting the activations to a lower dimension with a “random Gaussian matrix” sounds harmful to the results.
- How do you generate the low-resolution images? Are you using bicubic downsampling or other approaches? This detail should be clarified.
- While the evaluation with PSNR and SSIM is a reference to show the quality, many literatures already show that PSNR and SSIM are not correlated well with human perception. It is suggested to also evaluate with some perceptual metrics, e.g., LPIPS [e].
[e] Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. “The unreasonable effectiveness of deep features as a perceptual metric.” CVPR 2018.
- In Figure 7, how do you generate different results from the same input image for SRGAN? From my understanding, SRGAN doesn’t take any noise vector as input and cannot generate multi-modal results.
- I feel that the comparison with only SRGAN is not enough. There are some GAN-based SR methods [f][g]. It’s also suggested to compare with MSE-based state-of-the-art SR algorithms [h][i].
[f] Sajjadi, M. S., Schölkopf, B., & Hirsch, M. “Enhancenet: Single image super-resolution through automated texture synthesis.“ ICCV 2017.
[g] Wang, X., Yu, K., Dong, C., & Loy, C. C. “Recovering realistic texture in image super-resolution by deep spatial feature transform.” CVPR 2018.
[h] Lim, B., Son, S., Kim, H., Nah, S., & Lee, K. M. “Enhanced deep residual networks for single image super-resolution.” CVPR Workshops 2017.
[i] Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y. “Residual dense network for image super-resolution.” CVPR 2018.


We introduce geomstats, a Python package for Riemannian modelization and optimization over manifolds such as hyperspheres, hyperbolic spaces, spaces of SPD matrices or Lie groups of transformations. Our contribution is threefold. First, geomstats allows the flexible modeling of many a machine learning problem through an efficient and extensively unit-tested implementations of these manifolds, as well as the set of useful Riemannian metrics, exponential and logarithm maps that we provide. Moreover, the wide choice of loss functions and our implementation of the corresponding gradients allow fast and easy optimization over manifolds. Finally, geomstats is the only package to provide a unified framework for Riemannian geometry, as the operations implemented in geomstats are available with different computing backends (numpy, tensorflow and keras), as well as with a GPU-enabled mode-thus considerably facilitating the application of Riemannian geometry in machine learning. In this paper, we present geomstats through a review of the utility and advantages of manifolds in machine learning, using the concrete examples that they span to show the efficiency and practicality of their implementation using our package 1 .

The paper introduces the software package geomstats which provides simple use of Riemannian manifolds and metrics within machine learning models. Like theanogeometry, geomstats provides a backend for fast computation. Instead of theano, they interface tensorflow and numpy.
The core problem the author’s have to argue against is the existence of various other packages like pymanopt (which are mentioned in the paper) providing similar functionality.
The main advantage to pymanopt is stated to be lower computational cost. Unfortunately, this is not evaluated empirically. Pymanopt similarly provides the option to provide the cost function with tensorflow and uses numpy/scipy internally, therefore also making use of vectorization. A favorable empirical comparison would have been a compelling case for geomstats. While geomstats provides some more metrics than pymanopt, it lacks in other areas in comparison. Such metrics could be added relatively easily to pymanopt (or some of the other competing libraries).
Truly novel is the update for Keras which allows Riemannian gradient descent on parameters living on manifolds. Unfortunately, this is not directly shown and discussed further in the paper, but the reader is referred to the code base. While plenty of examples were provided in the supplementary material, I’d have preferred to see specific example(s) being shown and discussed in the paper. In the end, the main paper alone only gives an overview of what exists, but gives me no idea on how the package is used.
In parts, the paper reads more like an argument in favor of Riemannian modelization and optimization, instead of advocating for the specific package. While it is very important to demonstrate potential applications for the framework - an area in which this paper excelled - other, more important parts (mentioned above), were omitted because of it. Similarly, a lot of time is needlessly spent on defining well-known manifolds.
On the formal side, the formatting of the citations within the text don’t adhere to the official style guide which prescribes the use of authors’ last names and year.
Overall, the software package seems to provide nice functionality with integration into a currently popular machine learning framework, but it’s novelty compared to existing software packages is limited. The novel parts (performance improvements, integration with keras) are not sufficiently demonstrated in the paper.


Model-based reinforcement learning approaches have the promise of being sample efficient. Much of the progress in learning dynamics models in RL has been made by learning models via supervised learning. There is enough evidence that humans build a model of the environment, not only by observing the environment but also by interacting with the environment. Interaction with the environment allows humans to carry out experiments: taking actions that help uncover true causal relationships which can be used for building better dynamics models. Analogously, we would expect such interaction to be helpful for a learning agent while learning to model the environment dynamics. In this paper, we build upon this intuition, by using an auxiliary cost function to ensure consistency between what the agent observes (by acting in the real world) and what it imagines (by acting in the "learned" world). Our empirical analysis shows that the proposed approach helps to train powerful policies as well as better dynamics models.

-------------
Summary
-------------
The authors propose to train a policy while concurrently learning a dynamics model. In particular, the policy is updated using both the RL loss (rewards from the environment) and the "consistency constraint", which the authors introduce. This consistency constraint is a supervised learning signal, which compares trajectories in the environment with trajectories in the imagined world (produced with the dynamics model).
---------------------
Main Feedback
---------------------
I feel like there might be some interesting ideas in this work, and the results suggest that this approach performs well. However, I had a difficult time understanding how exactly the method works, and what its advantages are. These are my main questions:
1) At the beginning of Section 4 the authors write "The learning agent has two pathways for improving its behaviour: (...) (ii) the open loop path, where it imagines taking actions and hallucinates the state transitions that could happen". Do you actually do this? This is not mentioned in anywhere. And as far as I understand, the reward function is not learned - hence there will be no training signal in the open loop path. Does the reward signal always come from the true environment?
2) Is the dynamics model used for anything else than action-selection during training? Planning? If not, I don't really understand the results and why this works at all (k=20 being better than k=5, for example).
3) Is the dynamics model pre-trained in any way? I find it surprising that the model-free method and the proposed method perform similar at the beginning (Figure 3). If the agent chooses its actions based on the state that is predicted by the dynamics model, this should throw off the learning of the policy at the beginning (when the dynamics model hasn't learned anything sensible yet).
-----------------------
Other Questions
-----------------------
4) How exactly does training without the consistency constraint look? Is this the same as k=1?
5) Could the authors comment on the evaluation protocol in the experimental section? Are the results averages over multiple runs? If so, it would help to see confidence intervals to make a fair assessment of the results.
6) For the swimmer in Figure 2, the two lines (with consistency and without consistency) start at different initial returns, why is that so? If the same architecture and seed was used, shouldn't this be the same (or can you just not see it in the graph)?
---------
Clarity
---------
The title and introduction initially gave me a slightly wrong impression on what the paper is going to be about, and several things were not followed up on later in the paper.
Title:
8) "generative models" reminds of things like a VAE or GAN; however, I believe the authors mean "dynamics models" instead
9) "by interaction" is a bit vague as to what the contribution is (aren't policies and dynamic models in general trained by interacting with the environment?); the main idea of the paper is the consistency constraint
Abstract / Introduction:
10) The authors talk about humans carrying out "experiments via interaction" to help uncover "true causal relationships". This idea is not brought up again in the methods section, and I don't see evidence that with the proposed approach, the policy does targeted experiments to uncover causal relationships. It is not clear to me why this is the intuition that motivates the consistency constraint.
11) As the authors state in the introduction, the hope of model-based RL is better sample complexity. This is usually achieved by using the model in some way, for example by planning several steps ahead when choosing the current action. Could the authors comment on where they would place their proposed method - how does it address sample complexity?
12) In the introduction, the authors discuss the problem of compounding errors. These must be a problem in the proposed method as well, especially as k grows. Could the authors comment on that? How come that the performance is so good for k=20?
13) The authors write that in most model-based approaches, the dynamics model is "learned with supervised learning techniques, i.e., just by observing the data" and not via interaction. There's two things I don't understand: (1) in the existing model-based approaches the authors refer to, the policy also interacts with the world to get the data to do supervised learning - what exactly is the difference? (2) The auxiliary loss "which explicitly seeks to match the generative behaviour to the observed behaviour" is just a supervised learning loss as well, so how is this different?
For me, it would help the readability and understanding of the paper if some concepts were introduced more formally.
14) In Section 2, it would help me to see a formal definition of the MDP and what exactly is optimised. The authors write "optimise a reward signal" and "maximise its expected reward", however I believe it should be the expected cumulative reward (i.e., return).
15) The loss function for the dynamics model is not explicitly stated. From the text I assume that it is the mean squared error for the per-step loss, and a GAN loss for the trajectory-wise loss.
16) Could the authors explicitly state what the overall loss function is, and how the RL and supervised objective are combined? Is the dynamics model f trained only on the supervised loss, and the policy pi only on the RL loss?
17) In 2.3 the variable z_t is not formally introduced. What does it represent?
------------------------
Other Comments
------------------------
18) I find it problematic to use words such as "hallucination" and "imagination" when talking about learning algorithms. I would much prefer to see formal/factual language (like saying that the dynamics model is used to do make predictions / do planning, rather than that the agent is hallucinating).
-- edit (19.11.) ---
- updated score to 5
- corrected summary


Flow-based generative models are powerful exact likelihood models with efficient sampling and inference. Despite their computational efficiency, flow-based models generally have much worse density modeling performance compared to stateof-the-art autoregressive models. In this paper, we investigate and improve upon three limiting design choices employed by flow-based models in prior work: the use of uniform noise for dequantization, the use of inexpressive affine flows, and the use of purely convolutional conditioning networks in coupling layers. Based on our findings, we propose Flow++, a new flow-based model that is now the state-of-the-art non-autoregressive model for unconditional density estimation on standard image benchmarks. Our work has begun to close the significant performance gap that has so far existed between autoregressive models and flow-based models.

I think the ideas are of sufficient interest to the community to merit acceptance & discussion, but I still miss the high resolution samples we got with the Glow paper. Responses to my concerns somewhat addressed, though simpler alternatives to uniform dequant would be nice.
=====
Improvements are attained on two image datasets by (a) variational dequantization, (b) mixture CDF coupling layers, and (c) self-attention in conditioning net.
Quality: The work is fine, demonstrating familiarity with recent work in flows and improving upon it. The experiments are on CIFAR-10 and 32x32 ImageNet. Unclear if the evaluation numbers are on a test set or a 'validation' set. I will be assuming test set. The visualizations are fine, but not nearly as convincing as the Glow visualizations on CelebA.
Clarity: The presentation is clear enough, and the motivation seems reasonable, though the assertion that all AR models are slow seems a bit belied by the recent WaveRNN work, which gets a Wavenet like model running in realtime on a phone. On the other hand, I felt like the proposed fixes were all a bit scattered here & there. Each could stand as a research topic on its own, and one paper can't fit in much analysis of all three. For example, a RealNVP style model usually needs to shuffle or reverse the channels to attain decent performance, but there's no discussion of how/whether that is done here. Folks wanting to replicate this work would want a formula for the tractable log-abs-det-jacobian of the coupling layer, but all we have is "involves calculating the pdf of the logistic mixtures".
Originality: Self-attention is not new, though its uptake in the conditioning networks of flow models has been slow/nonexistent. I found the dequantization improvement more novel. The new proposal for a coupling layer seems like a clever way of introducing more parameters in a structured manner.
Significance: Bringing flow models closer to the performance of AR models is good progress.
Questions
I wonder whether some kind of spline or cubic interpolation might achieve similar improvement over the uniform dequantization. Perhaps uniform is not the best baseline?
The new coupling layer might just be viewed as a way of introducing many more parameters in a structured manner. Have you compared parameter counts?
Appendix B shows some portion of the code, but seems like a missed opportunity to fit this into a framework like tfp.bijectors. The code seems glued in somewhat slapdash. For example, the tf_go function looks like debugging/logging code (unwanted), and lacks any usage.
I think this work is promising and interesting to the probabilistic modeling community, but needs some cleanup and some more compelling presentation (non image data? Glow-style graphics?).


DARTS: DIFFERENTIABLE ARCHITECTURE SEARCH
This paper addresses the scalability challenge of architecture search by formulating the task in a differentiable manner. Unlike conventional approaches of applying evolution or reinforcement learning over a discrete and non-differentiable search space, our method is based on the continuous relaxation of the architecture representation, allowing efficient search of the architecture using gradient descent. Extensive experiments on CIFAR-10, ImageNet, Penn Treebank and WikiText-2 show that our algorithm excels in discovering high-performance convolutional architectures for image classification and recurrent architectures for language modeling, while being orders of magnitude faster than state-of-the-art non-differentiable techniques. Our implementation has been made publicly available to facilitate further research on efficient architecture search algorithms.

This paper proposes a novel way to formulate neural architecture search as a differentiable problem.
It uses the idea of weight sharing introduced in previous papers (convolutional neural fabrics, ENAS, and Bender et al's one shot model) and combines this with a relaxation of discrete choices between k operators into k continuous weights. Then, it uses methods based on hyperparameter gradient search methods to optimize in this space and in the end removes the relaxation by dropping weak connections and selecting the single choice of the k options with the highest weight. This leads to an efficient solution for architecture search. Overall, this is a very interesting paper that has already created quite a buzz due to the simplicity of the methods and the strong results. It is a huge plus that there is code with the paper! This will dramatically increase the paper's impact.
In my first read through, I thought this might be a candidate for an award paper, but the more time I spent with it the more issues I found. I still think the paper should be accepted, but I do have several points of criticism / questions I detail below, and to which I would appreciate a response.
Some criticisms / questions:
1. The last step of how to move from the one-shot model to a single model is in a sense the most interesting aspect of this work, but also the one that leaves the most questions open: Why does this work? Are there cases where we lose arbitrarily badly by rounding the solution to the closest discrete value or is the performance loss bounded? How would other ways of moving from the relaxation to a discrete choice work? I don't expect the paper to answer all of these questions, but it would be useful if the authors acknowledge that this is a critical part of the work that deserves further study. Any insights from other approaches the authors may have tried before the mechanism in Section 2.4 would also be useful.
2. The related work is missing several papers, namely the entire category of work on using network morphisms to speed up the optimization process, Bender et al's one shot model, and several early papers on neural architecture search (work on NAS did not only start in 2017 but goes back to work in the 1990s on neuroevolution that is very similar to the evolution approach by Real). This is a useful survey useful for further references: https://arxiv.org/abs/1808.05377
3. I find a few of the claims to be a bit too strong. In the introduction, the paper claims to outperform ENAS, but really the paper doesn't give a head-to-head comparison. In the experiments, ENAS is faster and gives slightly worse results. The authors state explicitly that their method is slower because they run it 4 times and pick the best result. One could obviously also do that with ENAS, and since ENAS is 8 times faster one could even run it 8 times! This is unfair and should be fixed. I don't really care even if it turns out that ENAS performs a bit better with the same budget, but comparisons should be fair and on even ground in order to help our science advance -- something that is far too often ignored in the ML literature in order to obtain a table with bold numbers in one's own row.
Likewise, why is ENAS missing in the Figure 3 plots for CIFAR, and why is its performance not plotted over time like that of DARTS?
4. The paper is not really forthcoming about clearly stating the time required to obtain the results:
- On CIFAR, there are 4 DARTS run of 1 day each
- Then, the result of each of these is evaluated for 100 epochs (which is only stated in the caption of Figure 3) to pick the best. Each of these validation runs takes 4 hours (which, again, one has to be inferred from the fact that random search can do 24 such evaluations in 4 GPU days), so this step takes another 16 GPU hours.
- Then, one needs to train the final network for 600 epochs; this is a larger network, so this should take another 2-3 GPU days.
So, overall, to obtain the result on CIFAR-10 requires about one GPU week. That's still cheap, but it's a different story than 1 day.
Likewise, DARTS is *not* able to obtain 55.7 perplexity on PTB in 6 hours with 4 GPUs; again, there is the selection step (probably another 4*6 hours?) and I think training the final model takes about 2 GPU days. These numbers should be stated prominently next to the stated "search times" to not mislead the reader.
5. One big question I have is where the hyperparameters come from, for both the training pipeline and the final evaluation pipeline (which actually differ a lot!).
For example, here are the hyperparameters for CIFAR, in this format: training pipeline value -> final evaluation pipeline value:
#cells: 8 -> 20
batch size: 64 -> 96
initial channels: 16 -> 36
#epochs: 50 -> 600
droppath: no -> yes (with probability 0.2)
auxiliary head: no -> yes (with weight 0.4)
BatchNorm: enabled (no learnable parameters) -> enabled
The situation is similar for PTB:
embedding size: 300 -> 850
hidden units per RNN layer: 300 -> 850
#epochs: 500 -> 8000
batch size: 256 (SGD) -> 64 (ASGD), sped up by starting with SGD
weight decay: 5e-7 -> 8e-7
BatchNorm: enabled (no learnable parameters) -> disabled
The fact that there are so many differences in the pipelines is disconcerting, since it looks like a lot of manual work is required to get these right. Now you need to tune hyperparameters for both the training and the final evaluation pipeline? If you have to tune them for the final evaluation pipeline, then you can't capitalize at all on the fact that DARTS is fast, since hyperparameter optimization on the full final evaluation pipeline will be order of magnitudes more expensive than running DARTS.
6. How was the final evaluation pipeline chosen? Before running DARTS the first time, or was it chosen to be tuned for architectures found by DARTS?
7. A question about how the best of 4 DARTS runs is selected, and how the best of the 24 random samples in random search is evaluated: is this based on 100 epochs using the *training* procedure or the *final evaluation* procedure? Seeing how different the hyperparameters are above, this should be stated.
8. A few questions to the authors related to the above: how did you choose the hyperparameters of DARTS? The DARTS learning rate for PTB is 10 times higher than for CIFAR-10, and the momentum also differs a lot (0.9 vs. 0.5). Did you ever consider different hyperparameters for DARTS? If so, how did you decide on the ones used? Is it sensitive to the choice of hyperparameters? In the author response period, could you please report the
(1) result of running DARTS on PTB using the same DARTS hyperparameters as used for CIFAR-10 (learning rate 3*e-4 and momentum (0.5,0.999)) and
(2) result of running DARTS on CIFAR-10 using the same DARTS hyperparameters as used for PTB (learning rate 3*e-3 and momentum (0.9,0.999))?
9. DARTS is being critizized in https://openreview.net/pdf?id=rylqooRqK7#page=10&zoom=180,-16,84
I am wondering whether the authors have a reply to this.
The algorithm for solving the relaxed problem is also not mathematically derived from the optimization problem to be solved (equations 3,4), but it is more a heuristic. A derivation, or at least a clearer motivation for the algorithm would be useful.
10. Further comments:
- Equation 1: This looks like a typo, shouldn't this be x(j) = \sum_{i<j} o(i,j) x(i) ? Even if the authors wanted to use the non-intuitive way of edges going from j to i, then o(i,j) should still be o(j,i).
- Just above Equation 5: "the the"
- Equation 5: I would have found it more intuitive had \alpha_{k-1} already just been a generic \alpha here.
- It would be nice if the authors gave the explicit equations for the extension with momentum in the appendix for completeness.
- The authors should include citations for techniques such as batch normalization, Adam, and cosine annealing.
Despite these issues (which I hope the authors will address in the author response and the final version), as stated above, I'm arguing for accepting the paper, due to the simplicity of the method combined with its very promising results and the direct availability of code.


AN EMPIRICAL STUDY OF BINARY NEURAL NETWORKS' OPTIMISATION
Neural networks with deterministic binary weights using the Straight-ThroughEstimator (STE) have been shown to achieve state-of-the-art results, but their training process is not well-founded. This is due to the discrepancy between the evaluated function in the forward path, and the weight updates in the backpropagation, updates which do not correspond to gradients of the forward path. Efficient convergence and accuracy of binary models often rely on careful finetuning and various ad-hoc techniques. In this work, we empirically identify and study the effectiveness of the various ad-hoc techniques commonly used in the literature, providing best-practices for efficient training of binary models. We show that adapting learning rates using second-moment methods is crucial for the successful use of the STE, and that other optimisers can easily get stuck in local minima. We also find that many of the commonly employed tricks are only effective towards the end of the training, with these methods making early stages of the training considerably slower. Our analysis disambiguates necessary from unnecessary ad-hoc techniques for the training of binary neural networks, paving the way for future development of solid theoretical foundations for these. Our newly-found insights further lead to new procedures which make training of existing binary neural networks notably faster.

The paper systematically studies the training of binary neural networks, where binary in this case refers to single bit weight elements in the network. In particular, different existing training methods are tested and compared for training both MLPs and CNNs.
The main findings of the paper are:
- Using methods such as AdaGrad, AdaDelta, RMSProp and ADAM yields better performance than simpler momentum-based methods such as vanilla momentum and Nesterov momentum, which in turn are much better than vanilla SGD
- When training binary models, it is common to clip weights and/or gradients for the proxy weights in the network. In the paper it is however shown that these methods hinder using a fast learning rate in the beginning of training, while the methods are required in later stages of training in order to achieve good results
- Pre-training the model with full-precision training works well in speeding up training
For a practitioner, the paper presents a very useful reference for what methods work well when training binary networks. Although there are some proposals and hypotheses for reasons behind the results, I see the paper as a review paper of existing methods for training binary networks, showing experiments where the methods are tested using the same benchmark and training procedure in order to give a fair comparison.
As a practical guide, the paper therefore has clear value. What is lacking compared to typical ICLR papers is rigorously presenting new findings. The authors present a hypothesis for why different batch sizes are needed in the beginning compared to the end of training, but I found neither the justification nor the results very convincing with respect to the hypothesis. The way I see it, the actual novel proposals that are made in the paper are two two-stage training methods: one in which the tricks of weight and gradient clipping are only used towards the end of training, and one where the first stage of training is done using a full precision model. It is however quite well known that some training schemes with different stages can lead to improved performance: for instance with ADAM, even if it is an adaptive method, lowering the learning rate towards the end of training is often beneficial. It might therefore be fair to compare the methods to other multi-stage training methods. In addition, I could not find the training curves or final performance figures of the method where clipping is only activated towards the end of training.
To put it all together, the paper is clearly useful for the community as it provides a useful summary of the performance of different methods for training binary neural networks. In addition, it presents two two-stage training schemes that seem to make training even faster. What the paper lacks is rigorous theoretical justifications and clearly novel ideas.
Small comments:
- How are the training lengths decided for the different methods? If I am not mistaken, in Figure 2, it seems like the SGD and momentum methods have not yet converged when training is halted. Is there a budget for wall clock time or is early stopping used or something similar? Considering the nature of the paper, I would see this kind of decisions as important to report.
- In the abstract, you might want to refer to binary weights somewhere. Based on the abstract it is easy to mix the binary networks in this paper with stochastic binary networks that can also be trained using the STE estimator
- Are the differences in the performances in Table 3 statistically significant?


DEEP CONVOLUTIONAL NETWORKS AS SHALLOW GAUSSIAN PROCESSES
We show that the output of a (residual) convolutional neural network (CNN) with an appropriate prior over the weights and biases is a Gaussian process (GP) in the limit of infinitely many convolutional filters, extending similar results for dense networks. For a CNN, the equivalent kernel can be computed exactly and, unlike "deep kernels", has very few parameters: only the hyperparameters of the original CNN. Further, we show that this kernel has two properties that allow it to be computed efficiently; the cost of evaluating the kernel for a pair of images is similar to a single forward pass through the original CNN with only one filter per layer. The kernel equivalent to a 32-layer ResNet obtains 0.84% classification error on MNIST, a new record for GPs with a comparable number of parameters.

This paper shows that deep convolutional networks (CNNs, without pooling) with a suitable prior over weights can be seen as shallow Gaussian processes (GPs) with a specific covariance function. It shows that this covariance function can be computed efficiently (when compared to previous attempts at resembling convolutional networks with GPs), with a cost that only depends linearly on the number of layers and the input dimensionality, i.e.~O(N^2 L D).
To show the equivalence between deep CNNs and shallow GPs, the paper uses similar ideas to those proposed by Matthews et al (2018a) and Lee et al (2017), i.e. using the multivariate central limit theorem in very large networks, where in the case of this paper the limit is taken as the number of channels at each layer goes to infinity. Therefore, from a theoretical perspective, these ideas have been proposed before. However, the paper presents a novel efficient way to compute the convolutional kernel, which I believe has merits on its own.
However, the model setting for classification (where deep CNNs have been successful) and consequent evaluation on the MNIST dataset is less than convincing. One of the main motivations for Bayesian CNNs and GPs (and the paper argue for this in the intro) is to be able to provide good uncertainty estimates. However, the classification problem is framed in a regression setting, where neither probabilistic estimates are evaluated or even provided. Indeed, only the error rate is given on Table 1. To me, this is certainly not enough for a Bayesian/GP method and it is a critical deficiency of the paper in its current form. While I understand having a non-Gaussian likelihood will complicate things and conflate the kernel contribution with the approximations, I believe it is necessary to provide and evaluate such probabilistic estimates and compare them to other GP approaches (even using other less than satisfying methods such as calibration/scaling). Along a similar vein, it is unclear what objective function was used for hyper-parameter learning but, given that the authors actually “sample hyper-parameters”, I am guessing a proper probabilistic objective such as the marginal likelihood is out of the question.
Other (perhaps minor) deficiencies is that the method is not scalable to large datasets (I am even surprised the authors managed to run this on full MNIST) and that no theoretical analysis is done (e.g. as in Mattews et al, 2018a).
Minor comments:
* In the intro, “Other methods such as Gaussian Processes”: GPs are not a method and I believe the authors really mean here Gaussian process regression.
* The prior variance over filters in Eq (3) divides over the number of channels.  Why does a Gaussian prior with infinite precision make sense here?
* The authors should report the state of the art of using GPs for MNIST classification using non-convolutional kernels).


Generative adversarial network (GAN) is one of the best known unsupervised learning techniques these days due to its superior ability to learn data distributions. In spite of its great success in applications, GAN is known to be notoriously hard to train. The tremendous amount of time it takes to run the training algorithm and its sensitivity to hyper-parameter tuning have been haunting researchers in this area. To resolve these issues, we need to first understand how GANs work. Herein, we take a step toward this direction by examining the dynamics of GANs. We relate a large class of GANs including the Wasserstein GANs to max-min optimization problems with the coupling term being linear over the discriminator. By developing new primal-dual optimization tools, we show that, with a proper stepsize choice, the widely used first-order iterative algorithm in training GANs would in fact converge to a stationary solution with a sublinear rate. The same framework also applies to multi-task learning and distributional robust learning problems. We verify our analysis on numerical examples with both synthetic and real data sets. We hope our analysis shed light on future studies on the theoretical properties of relevant machine learning problems.

This paper studies the convergence of a primal-dual algorithm on a certain min-max problem and experimentally shows that it works in GANs and multi-task learning.
This paper is clear and well-written. The convergence guarantee looks neat, and convergence to stationary points is a sensible thing on non convex-concave problems. I am not super familiar with the literature of saddle-point optimization and may not have a good sense about the significance of the theoretical result.
My main concern is that the assumptions in the theory are rather over-restrictive and it’s not clear what intuitions or new messages they bring in for the practice of GANs. The convergence theorem requires the maximization problem (over discriminators) to be strictly concave. On GANs, this assumption is not (near) satisfied beyond the simple case of the LQG setting (PSD quadratic discriminators). On the other hand, the experiments on GANs just seem to say that the algorithm works but not much more beyond that. There is a brief discussion about the improvement in time consumption but it doesn’t have a report a quantitative comparison in the wall time.
On multi-task learning, the proposed algorithm shows improvement over the baseline. However it is also unclear whether it is the *formulation* (12) that brings in the improvement, or it is the actual primal-dual *algorithm*. Perhaps it might be good to try gradient descent on (12) and see if it also works well.
In general, I would recommend the authors to have a more convincing demonstration of the strength of this algorithm over baseline methods on min-max problems, either theoretical or empirical.


Stochastic video prediction models take in a sequence of image frames, and generate a sequence of consecutive future image frames. These models typically generate future frames in an autoregressive fashion, which is slow and requires the input and output frames to be consecutive. We introduce a model that overcomes these drawbacks by generating a latent representation from an arbitrary set of frames that can then be used to simultaneously and efficiently sample temporally consistent frames at arbitrary time-points. For example, our model can "jump" and directly sample frames at the end of the video, without sampling intermediate frames. Synthetic video evaluations confirm substantial gains in speed and functionality without loss in fidelity. We also apply our framework to a 3D scene reconstruction dataset. Here, our model is conditioned on camera location and can sample consistent sets of images for what an occluded region of a 3D scene might look like, even if there are multiple possibilities for what that region might contain. Reconstructions and videos are available at https://bit.ly/2O4Pc4R.

This paper presents a method for predicting future frames of a video (or unseen views of a 3D scene) in a "jumpy" way (you can query arbitrary viewpoints or timesteps) and "consistent" way (when you sample different views, the scene will be consistent). They use a VAE that encodes the input video in a permutation invariant way, which is achieved by summing the per-frame latent vectors. Then, they sample a latent vector using a DRAW prior. This latent vector can then be used to render the video/scene from different times/viewpoints via an LSTM decoder. They test the model on several toy datasets: they compare to video prediction methods on a dataset of moving shapes, and 3D viewpoint prediction on a 3D MNIST "dice" dataset.
Pros:
- The idea of developing new methods for viewpoint and video synthesis that allow for "jumpy" and "consistent" predictions is an important problem.
- The paper is fairly well written.
- The design of the model is reasonable (it is a natural extension of VAE viewpoint/future prediction methods).
Cons:
- All of the experiments were done on toy datasets. These are also not well-established toy datasets, and seem tailored to debugging the model, so it is not particularly surprising that the method worked. Since the main contribution is not very novel from a technical perspective (it is more about designing a model from existing, well-established components), this is a significant limitation.
- The paper suggests with experiments that GQN generates predictions that are less consistent across samples, but it is not clear exactly which design decisions lead to this difference. Why is this model more jumpy and consistent than GQN?
- The paper claims that JUMP trains more reliably than several video prediction methods in Figure 5. Yet, in the 3D viewpoint synthesis task, they suggest that JUMP had trouble with convergence, i.e.: "We ran 7 runs for each model, and picked the best 6/7 runs for each model (1 run for JUMP failed to converge)." This is confusing for two reasons. First, why was this evaluation protocol chosen (i.e. running 7 times and picking the 6 best)? If it was a post-hoc decision to remove one training run, then this should be clarified, and the experiment should be redesigned and rerun. Second, is the implication that JUMP is more stable than video prediction methods, but not necessarily more stable than GQN for viewpoint prediction?
- The paper should consider citing older representation learning work that deals with synthesizing images from multiple viewpoints. For example:
M. Tatarchenko, A. Dosovitskiy, T. Brox. "Multi-view 3D Models from Single Images with a Convolutional Network". ECCV 2016.
- There is insufficient explanation of the BADJ baseline. What architectural changes are different?
- The decision to use DRAW, instead of a normal VAE prior, is unusual and not explained in much detail. Why does this improve the visual fidelity of the samples?
Overall:
The paper does not present enough evidence that this model is better at jumpy/consistent predictions than other approaches. It is evaluated only on toy datasets: if the technical approach were more novel (and if it was clearer where the performance gains are coming from) then this could be OK, but it seems to be a fairly straightforward extension of existing models.


We introduce a new normalization technique that exhibits the fast convergence properties of batch normalization using a transformation of layer weights instead of layer outputs. The proposed technique keeps the contribution of positive and negative weights to the layer output in equilibrium. We validate our method on a set of standard benchmarks including CIFAR-10/100, SVHN and ILSVRC 2012 ImageNet.

This manuscript introduces a new layer-wise transform, EquiNorm, to improve upon batch normalization. As with batch normalization and related techniques, the idea is to introduce a simple linear transform at each layer to reduce the dependency of the features to the data. Unlike batch normalization, the procedure does not modify the inputs to the layers but rather the layer weights. For this purpose, a scaling factor and a shift is computed on a mini batch, separating positive and negative weights to compute easily both running estimates of shift and of spread (here in the l1 sense). The method is compared to BatchNorm and GroupNorm on several classic computer vision datasets. Empirically, the method converges faster in the beginning of the optimization, in the sense that in the first few epochs the test accuracy is higher than for BatchNorm. However, this benefit decreases with more epochs and when the results are close to peak performance the difference between methods is a small. In addition, test accuracy can decrease at the end of the optimization, which the authors interpret as a sign of increased overfit, and tackle with clever data augmentation. The paper reads well.
The strengths of the paper are the elegance of the solution proposed, and the faster convergence. To me, the main drawback of the manuscript is that the results do not show a clear benefit at convergence of EquiNorm compared to BatchNorm. It seems that as the learning rate decreases there is no consistent difference: in none of the 4 datasets does EquiNorm give a gain larger than a quartile of the distribution of scores and in 2 out of 4 it performs less well.
My advice to improve this work would be to do more experiments and to show better that the lack of performance gain is due to overfit, and can be fixed with larger data or data augmentation.
The faster initial convergence could be a real benefit of this method. However, the time to useful convergence is the same than with BatchNorm. This is probably because the learning-rate decrease schedule is the same. It might be interesting to study adapted learning-rate decrease.
To argue for increased overfit, it would have been interesting to compare test error with train error, and to plot train loss alongside with figure 2.
While the fast convergence is a benefit, it would have been interesting to also compare EquiNorm with BatchNorm with learning scale schedules (ie outside "super convergence" settings). While costly, such an experiment would help separating the benefits of EquiNorm from choice of learning rate and give a standard baseline to compare to. Ideally, EquiNorm achieves this baseline with less compute time.
I would like to congratulate the authors for reporting multiple runs with different RNGs and the quantile of the distribution. This is absolutely good practice to judge the significance of improvements and is seldom done.
Why does figure 2 shows only test loss, while the other experimental figures show test accuracy and loss?


OF WIKIPEDIA: KNOWLEDGE-POWERED CONVERSATIONAL AGENTS
In open-domain dialogue intelligent agents should exhibit the use of knowledge, however there are few convincing demonstrations of this to date. The most popular sequence to sequence models typically "generate and hope" generic utterances that can be memorized in the weights of the model when mapping from input utterance(s) to output, rather than employing recalled knowledge as context. Use of knowledge has so far proved difficult, in part because of the lack of a supervised learning benchmark task which exhibits knowledgeable open dialogue with clear grounding. To that end we collect and release a large dataset with conversations directly grounded with knowledge retrieved from Wikipedia. We then design architectures capable of retrieving knowledge, reading and conditioning on it, and finally generating natural responses. Our best performing dialogue models are able to conduct knowledgeable discussions on open-domain topics as evaluated by automatic metrics and human evaluations, while our new benchmark allows for measuring further improvements in this important research direction.

This paper introduces a new dataset and method for chatbots. In contrast to previous work, this paper specifically probes how well a dialogue system can use external unstructured knowledge.
Quality:
Overall, this is a very high-quality paper. The dataset is developed well, the experimental setup is well thought-through and the authors perform many ablation studies to test different model variants. The main criticism I have would be that the human evaluation is rather simple (rating 1-5), I would have expected more fine-grained categories, especially ones that relate to how much knowledge the system uses (I appreciate the "Wiki F1" metric, but that is an automatic metric). As it is, the human evaluation shows that most of their contributions are not appreciated by human annotators. Further, the paper ends a bit abruptly, I would have expected a more in-depth discussion of next steps.
Clarity:
The description of the work is clear in most places. I particularly like the abstract and introduction, which set up the rest of the paper nicely. In some places, perhaps due to space restrictions, method descriptions are a bit too short.
Originality:
The paper is fairly original, especially the aspect about specifically using external knowledge. The authors could have been more clear on how the work differs from other work on non-goal directed dialogue work though (last paragraph of related work section).
Significance:
The dataset is really well-developed, hence I believe many working in the dialogue systems community will re-use the developed benchmark and build on this paper.
More detailed comments:
- Missing reference for goal-oriented dialogue datasets: Wen et al. 2017, A Network-based End-to-End Trainable Task-oriented Dialogue System, https://arxiv.org/abs/1604.04562
- How does the proposed dataset differ from the Reddit and Wikipedia datasets discussed in the last paragraph of the related work section? This should be explained.
- Page 3, paragraph "Conversational Flow": what is the maximum number of turns, if the minimum is 5?
- Page 3, paragraph "Knowledge Retrieval": how were the top 7 articles and first 10 sentences choices made? This seems arbitrary. Also, why wasn't the whole text used?
- Page 3, paragraph "Knowledge Selection and Response Generation": how do you deal with co-reference problems if you only ever select one sentence at a time? The same goes for the "Knowledge Attention" model described in Section 4.
- Page 3, paragraph "Knowledge Selection and Response Generation": how often do annotators choose "no sentence selected"? It would be interesting to see more such statistics about the dataset
- Section 4.2: did you run experiments for BPE encoding? Would be good to see as this is a bit of a non-standard choice.
- Section 4.2: it would be good to explain the Cer et al. 2018 method directly in the paper
- Section 4.2: is there a reference for knowledge dropout? Also, it would be good to show ablation results for this.
- Section 5.1: why did you choose to pre-train on the Reddit data? There should be some more in-depth description of the Reddit dataset to motivate this choice.
- Section 5.1: what is the setup you use for multi-task learning on SQuAD? Is it just a hard parameter sharing model, or?
- Section 5.3: as stated above, the human evaluation is a little bit underwhelming, both in terms of setup and results. I'd expect a more fine-grained way of assessing conversations by humans, and also an explanation of why the retrieval performer without knowledge was assessed as being on par with the retrieval transformer memnet.
- Section 5.3: I assume higher=better for the human scores? This should be made explicit.
- Section 5.3: Have others used the "F1 overlap score"? If so, cite.
- Section 5.3: I don't understand the argument that the human evaluation shows that humans prefer more natural responses. How does it show that?
- Section 5.3: The Wiki F1 score is kind of interesting because it shows to what degree the model uses knowledge. But the side-by-side comparison with the human scores shows that humans don't necessarily prefer chatbot models that use a lot of knowledge. I'd expect this to be discussed, and suggestions for future work to be made accordingly.
- Section 6: The paper ends a bit abruptly. It's be nice to suggest future areas of improvement.


GAN DISSECTION: VISUALIZING AND UNDERSTANDING GENERATIVE ADVERSARIAL NETWORKS
Generative Adversarial Networks (GANs) have recently achieved impressive results for many real-world applications, and many GAN variants have emerged with improvements in sample quality and training stability. However, they have not been well visualized or understood. How does a GAN represent our visual world internally? What causes the artifacts in GAN results? How do architectural choices affect GAN learning? Answering such questions could enable us to develop new insights and better models.
In this work, we present an analytic framework to visualize and understand GANs at the unit-, object-, and scene-level. We first identify a group of interpretable units that are closely related to object concepts using a segmentation-based network dissection method. Then, we quantify the causal effect of interpretable units by measuring the ability of interventions to control objects in the output. We examine the contextual relationship between these units and their surroundings by inserting the discovered object concepts into new images. We show several practical applications enabled by our framework, from comparing internal representations across different layers, models, and datasets, to improving GANs by locating and removing artifact-causing units, to interactively manipulating objects in a scene. We provide open source interpretation tools to help researchers and practitioners better understand their GAN models.

The paper proposes a method for visualizing and understanding GANs representation. This seems an important topic as several such methods were performed for networks trained in supervised learning, which relate
to the predicted outcome, but there is lack of methods for interpreting GANs which are learned in an unsupervised manner and it is generally unclear what is the representation learned by GANs.
The method is finding correlations between the appearance of objects and the activation of units in each layer of the learned network.
In addition, the paper presents a 'causal' measure, where a causal effect of a unit is measured by removing and adding this unit from/to the network and computing the average effect on object appearance.
The authors demonstrate how the methods are applied by improving the appearance of images, by modifying units which were detected as important for specific objects.
The authors also provide an interactive interface where users can manually examine and modify their trained GANs in order to add/remove objects and to remove artifacts.
The method proposed by the authors seem to be appropriate for convolutional neural networks, where 'units' in each layer may correspond to objects and can be searched for in particular locations of image.
It is not clear to me if and how one can apply the author's methods to other architecture, and to other application domains (besides images), or whether the method is limited to vision applications.
The authors do not explain specifically how do they choose the 'units' for which they seek interpretation when reporting their results. It is written that each layer is divided into two sets:
u  and u-bar, where we seek interpretation of u. But how large does u tend to be? how would one choose it? is it one filter out of all filters in a certain layer? when optimizing for sets of units together
(using the alpha probabilities and the optimization in eq. 6) what is d? is it performed for all units in a single layer? more details would be useful here.
The paper is overall clearly written, with lots of visual examples demonstrating the methods presented in it.
The paper presents a new methodological idea, which allows for nice practical contribution. There is no theoretical contribution or any deep analysis.
There is no reference in the paper to the supp. info. figures and therefore it is not clear if and how the supp. info. adds valuable information to the reader.
The authors use scores like SWD and FIT for performance, but give no explanations for what do these scores measure.
Minor:
Abstract: immprovements -> improvements
Page 6, middle: 'train on four LSUN' -> 'trained on four LSUN'
Page 7, bottom: Fig. 14a and 14b should be Fig. 8a and 8b


META-LEARNING PROBABILISTIC INFERENCE FOR PREDICTION
This paper introduces a new framework for data efficient and versatile learning. Specifically: 1) We develop ML-PIP, a general framework for Meta-Learning approximate Probabilistic Inference for Prediction. ML-PIP extends existing probabilistic interpretations of meta-learning to cover a broad class of methods. 2) We introduce VERSA, an instance of the framework employing a flexible and versatile amortization network that takes few-shot learning datasets as inputs, with arbitrary numbers of shots, and outputs a distribution over task-specific parameters in a single forward pass. VERSA substitutes optimization at test time with forward passes through inference networks, amortizing the cost of inference and relieving the need for second derivatives during training. 3) We evaluate VERSA on benchmark datasets where the method sets new state-of-the-art results, handles arbitrary numbers of shots, and for classification, arbitrary numbers of classes at train and test time. The power of the approach is then demonstrated through a challenging few-shot ShapeNet view reconstruction task.

Summary:
This work tackles few-shot (or meta) learning from a probabilistic inference viewpoint. Compared to previous work, it uses a simpler setup, performing task-specific inference only for single-layer head models, and employs an objective based on predictive distributions on train/test splits for each task (rather than an approximation to log marginal likelihood). Inference is done amortized by a network, whose input is the task training split. The same network is used for parameters of each class (only feeding training points of that class), which allows an arbitrary number of classes per task. At test time, inference just requires forward passes through this network, attractive compared to non-amortized approaches which need optimization or gradients here.
It provides a clean, decision-theoretic derivation, and clarifies relationships to previous work. The experimental results are encouraging: the method achieves a new best on 5-way, 5-shot miniImageNet, despite the simple setup. In general, explanations in the main text could be more complete (see questions). I'd recommend shortening Section 4, which is pretty obvious.
- Quality: Several interesting differences to prior work. Well-done experiments
- Clarity: Clean derivation, easy to understand. Some details could be spelled out better
- Originality: Several important novelties (predictive criterion, simple model setup, amortized inference network). Closely related to "neural processes" work, but this happened roughly at the same time
- Significance: The few-shot learning results are competitive, in particular given they use a simpler model setup than most previous work. I am not an expert on these kind of experiments, but I found the comparisons fair and rather extensive
Interesting about this work:
- Clean Bayesian decision-theoretic viewpoint. Key question is of course whether
   an inference network of this simple structure (no correlations, sum combination
   of datapoints, same network for each class) can deliver a good approximation to
   the true posterior.
- Different to previous work, task-specific inference is done only on the weights of
   single-layer head models (logistic regression models, with shared features).
   Highly encouraging that this is sufficient for state-of-the-art few-shot classification
   performance. The authors could be more clear about this point.
- Simple and efficient amortized inference model, which along with the neural
   network features, is learned on all data jointly
- Optimization criterion is based on predictive distributions on train/test splits, not
   on the log marginal likelihood. Has some odd consequences (question below),
   but clearly works better for few-shot classification
Experiments:
- 5.1: Convincing results, in particular given the simplicity of the model setup and
   the inference network. But some important points are not explained:
   - Which of the competitors (if any) use the same restricted model setup (inference
      only on the top-layer weights)? Clearly, MAML does not, right? Please state this
      explicitly.
   - For Versa, you use k_c training and 15 test points per task update during
      training. Do competitors without train/test split also get k_c + 15 points, or
      only k_c points? The former would be fair, the latter not so much.
- 5.2: This seems a challenging problem, and both your numbers and reconstructions
   look better than the competitor. I cannot say more, based on the very brief
   explanations provided here.
   The main paper does not really state what the model or the likelihood is. From
   F.4 in the Appendix, this model does not have the form of your classification
   models, but psi is input at the bottom of the network. Also, the final layer has
   sigmoid activation. What likelihood do you use?
   One observation: If you used the same "inference on final layer weights" setup
   here, and Gaussian likelihood, you could compute the posterior over psi in closed
   form, no amortization needed. Would this setup apply to your problem?
Further questions:
- Confused about the input to the inference network. Real Bayesian inference would
   just see features h_theta(x) as inputs, not the x's. Why not simply feed features in
   then?
   Please do improve the description of the inference network, this is a major
   novelty of this paper, and even the appendix is only understandable by reading
   other work as well. Be clear how it depends on theta (I think nothing is lost by
   feeding in the h_theta(x)).
- The learning criterion based on predictive distributions on train/test splits seem
   to work better than ELBO-like criteria, for few-shot classification.
   But there are some worrying aspects. The marginal likelihood has an Occam's
   razor argument to prevent overfitting. Why would your criterion prevent overfitting?
   And it is quite worrying that the prior p(psi | theta) drops out of the method
   entirely. Can you comment more on that?
Small:
- p(psi_t | tilde{x}_t, D_t, theta) should be p(psi_t | D_t, theta). Please avoid a more
   general notation early on, if you do not do it later on. This is confusing


THEORETICAL ANALYSIS OF AUTO RATE-TUNING BY BATCH NORMALIZATION
Batch Normalization (BN) has become a cornerstone of deep learning across diverse architectures, appearing to help optimization as well as generalization. While the idea makes intuitive sense, theoretical analysis of its effectiveness has been lacking. Here theoretical support is provided for one of its conjectured properties, namely, the ability to allow gradient descent to succeed with less tuning of learning rates. It is shown that even if we fix the learning rate of scale-invariant parameters (e.g., weights of each layer with BN) to a constant (say, 0.3), gradient descent still approaches a stationary point (i.e., a solution where gradient is zero) in the rate of T −1/2 in T iterations, asymptotically matching the best bound for gradient descent with well-tuned learning rates. A similar result with convergence rate T −1/4 is also shown for stochastic gradient descent.

* Description
The work is motivated by the empirical performance of Batch Normalization and in particular the observed better robustness of the choice of the learning rate.  Authors analyze theoretically the asymptotic convergence rate for objectives involving normalization, not necessarily BN, and show that for scale-invariant groups of parameters (appearing as a result of normalization) the initial learning rate may be set arbitrary while still asymptotic convergence is guaranteed with the same rate as the best known in the general case. Offline gradient descent and stochastic gradient descent cases are considered.
* Strengths
The work addresses better theoretical understanding of successful heuristics in deep learning, namely batch normalization and other normalizations. The technical results obtained are non-trivial and detailed proofs are presented. Also I did not verify the proofs the paper appears technically correct and technically clear. The result may be interpreted in the following form: if one chooses to use BN or other normalization, the paper gives a recommendation that only the learning rate of scale-variant parameters need to be set, which may have some practical advantages. Perhaps more important than the rate of convergence, is the guarantee that the method will not diverge (and will not get stuck in a non-local minimum).
* Criticism
This paper presents non-trivial theoretical results that are worth to be published but as I argue below its has a weak relevance to practice and the applicability of the obtained results is unclear.
-- Concerns regarding the clarity of presentation and interpretation of the results.

The properties of BN used as motivation for the study, are observed non-asymptotically with constant or empirically decreased learning rate schedules for a limited number of iterations. In contrast, the studied learning rates are asymptotic and there is a big discrepancy. SGD is observed to be significantly faster than batch gradient when far from convergence (experimental evidence), and this is with or without normalization. In practice, the training is stopped much before convergence, in the hope of finding solutions close to minimum with high probability. There is in fact no experimental evidence that the practical advantages of BN are relevant to the results proven. It makes a nice story that the theoretical properties justify the observations, but they may be as well completely unrelated.
As seen from the formal construction, the theoretical results apply equally well to all normalization methods. It occludes the clarity that BN is emphasized amongst them.
Considering theoretically, what advantages truly follow from the paper for optimizing a given function? Let’s consider the following cases.
1. For optimizing a general smooth function with all parameters forming a single scale-invariant vector. In this case, the paper proves that no careful selection of the learning rate is necessary. This result is beyond machine learning and unfortunately I cannot evaluate its merit. Is it known / not known in optimization?
2. The case of data-independent normalization (such as weight normalization).
Without normalization, we have to tune learning rate to achieve the optimal convergence. With normalization we still have to tune the learning rate (as scale-variant parameters remain or are reintroduced with each invariance to preserve the degrees of freedom), then we have to wait for the phase two of Lemma 3.2 so that the learning rate of scale-invariant parameters adapts, and from then on the optimal convergence rate can be guaranteed.
3. The case of Batch Normalization. Note that there is no direct correspondence between the loss of BN-normalized network (2) and the loss of the original network because of dependence of the normalization on the batches. In other words, there is no setting of parameters of the original network that would make its forward pass equivalent to that of BN network (2) for all batches. The theory tells the same as in case 2 above but with an additional price of optimizing a different function.
These points remain me puzzled regarding either practical or theoretical application of the result. It would be great if authors could elaborate.
-- Difference from Wu et al. 2018
This works is cited as a source of inspiration in several places in the paper. As the submission is a theoretical result with no immediate applicability, it would be very helpful if the authors could detail the technical improvements over this related work. Note, ICLR policy says that arxiv preprints earlier than one month before submission are considered a prior art. Could the authors elaborate more on possible practical/theoretical applications?

* Side Notes (not affecting the review recommendation)
I believe that the claim that “BN reduces covariate shift” (actively discussed in the intro) was an imprecise statement in the original work. Instead, BN should be able to quickly adapt to the covariate shift when it occurs. It achieves this by using the parameterization in which the mean and variance statistics of neurons (the quantities whose change is called the covariate shift) depend on variables that are local to the layer (gamma, beta in (1)) rather than on the cumulative effect of all of the preceding layers.
* Revision
I took into account the discussion and the newly added experiments and increased the score. The experiments verify the proven effect and make the paper more substantial. Some additional comments about experiments follow.
Training loss plots would be more clear in the log scale.
Comparison to "SGD BN removed" is not fair because the initialization is different (application of BN re-initializes weight scales and biases). The same initialization can be achieved by performing one training pass with BN with 0 learning rate and then removing it, see e.g. Gitman, I. and Ginsburg, B. (2017). Comparison of batch normalization and weight normalization algorithms for the large-scale image classification.
The use of Glorot uniform initializer is somewhat subtle. Since BN is used, Glorot initialization has no effect for a forward pass. However, it affects the gradient norm. Is there a rationale in this setting or it is just a more tricky method to fix the weight norm to some constant, e.g. ||w||=1?


A significant challenge for the practical application of reinforcement learning to real world problems is the need to specify an oracle reward function that correctly defines a task. Inverse reinforcement learning (IRL) seeks to avoid this challenge by instead inferring a reward function from expert behavior. While appealing, it can be impractically expensive to collect datasets of demonstrations that cover the variation common in the real world (e.g. opening any type of door). Thus in practice, IRL must commonly be performed with only a limited set of demonstrations where it can be exceedingly difficult to unambiguously recover a reward function. In this work, we exploit the insight that demonstrations from other tasks can be used to constrain the set of possible reward functions by learning a "prior" that is specifically optimized for the ability to infer expressive reward functions from limited numbers of demonstrations. We demonstrate that our method can efficiently recover rewards from images for novel tasks and provide intuition as to how our approach is analogous to learning a prior.

This paper aims to address the problem of lacking sufficient demonstrations in inverse reinforcement learning (IRL) problems. They propose to take a meta learning approach, in which a set of i.i.d. IRL tasks are provided to the learner and the learner aims to learn a strategy to quickly recover a good reward function for a new task that is assumed to be sampled from the same task distribution. Particularly, they adopt the gradient-based meta learning algorithm, MAML, and the maximal entropy (MaxEnt) IRL framework, and derive the required meta gradient expression for parameter update. The proposed algorithm is evaluated on a synthetic grid-world problem, SpriteWorld. The experimental results suggest the proposed algorithm can learn to mimic the optimal policy under the true reward function that is unknown to the learner.
Strengths:
1) The use of meta learning to improve sample efficiency of IRL is a good idea.
2) The combination of MAML and MaxEnt IRL is new to my knowledge.
3) Providing the gradient expression is useful, which is the main technical contribution of this paper. (But it needs to be corrected; see below.)
4) The paper is well motivated and clearly written "in a high level" (see below).
Weakness:
1) The derivation of (5) assumes the problem is tabular, and the State-Visitations-Policy procedure assumes the dynamics/transition of the MDP is known. These two assumption are rather strong and therefore should be made explicitly in the problem definition in Section 3.
2)  Equation (8) is WRONG. The direction of the derivation takes is correct, but the final expression is incorrect. This is mostly because of the careless use of notation in derivation on p 15 in the appendix (the last equation), in which the subscript i is missed for the second term. The correct expression of (8) should have a rightmost term in the form  (\partial_\theta r_\theta) D  (\partial_\theta r_\theta)^T, where D is a diagonal matrix that contains \partial_{r_i} (\E_{\tau} [ \mu_\tau])_i and i is in 1,...,|S||A|.
3) Comparison with imitation learning and missing details of the experiments.
a) The paper assumes the expert is produced by the MaxEnt model. In the experiments, it is unclear whether this is true or not, as the information about the demonstration and the true reward is not provided.
b) While the experimental results suggest the algorithm can recover the similar performance to the optimal policy of the true reward function, whether this observation can generalize outside the current synthetic environment is unclear to me. In imitation learning, it is known that the expert policy is often sub-optimal, and therefore the goal in imitation learning is mostly only to achieve expert-level performance. Given this, the way this paper evaluate the performance is misleading and improper to me, which leads to an overstatement of the benefits of the algorithm.
c) It would be interesting to compare the current approach with, e.g., the policy-based supervised learning approach to imitation learning (i.e. behavior cloning).
4) The rigorousness in technicality needs to be improved. While the paper is well structured, the writing at the mathematical level is careless, which leads to ambiguities and mistakes (though one might be able to work out the right formula after going through the details of the entire paper). Below I list a few points.
    a) The meta-training set {T_i; i=1,...,N} and the meta-test set {T_j; i=1,...,M} seems to overload the notation. I suppose this is unintentional but it may appear that the two sets share the first T_1,.., T_M tasks, e.g., when N>=M, instead of being disjoint.
    b) The set over which the summation is performed in (4) is unclear; alpha in (4) is not defined, though I guess it's a positive step size.
    c) On p4, "we can view this problem as aiming to learn a prior over the intentions of human demonstrators" is an overstatement to me. At best, this algorithm learns a prior over rewards for solving maximal entropy IRL, not intention. And the experiment results do not corroborate  the statement about "human" intention.
    d) On p4,  "since the space of relevant reward functions is much smaller than the space of all possible rewards deﬁnable on the raw observations" needs to be justified. This may not be true in general, e.g., learning the set of relevant functions may require a larger space than learning the reward functions.
    e) The authors call \mu_\tau the "state" visitation, but this is rather confusing, as it is the visiting frequency of state and action (which is only made clear late in the appendix).
    f) On p5, it writes "... taking a small number of gradient steps on a few demonstrations from given task leads" But the proposed algorithm actually only takes "one" gradient step in training.
    g) The convention of derivatives used in the appendix is the transpose of the one used in the main paper.
Minor points:
1) typo in (2)
2) p_\phi is not defined, L_{IRL} is not defined, though the definition of both can be guessed.
3) T^{tr} seems to be typo in (11)
4) A short derivation of (2) in the Appendix would be helpful.


BOOSTING ROBUSTNESS CERTIFICATION OF NEURAL NETWORKS
We present a novel approach for the certification of neural networks against adversarial perturbations which combines scalable overapproximation methods with precise (mixed integer) linear programming. This results in significantly better precision than state-of-the-art verifiers on challenging feedforward and convolutional neural networks with piecewise linear activation functions.

This paper proposed a mixed strategy to obtain better precision on robustness verifications of feed-forward neural networks with piecewise linear activation functions.
The topic of robustness verification is important. The paper is well-written and the overview example is nice and helpful.
The central idea of this paper is simple and the results can be expected: the authors combine several verification methods (the complete verifier MILP, the incomplete verifier LP and AI2) and thus achieve better precision compared with imcomplete verifiers while being more scalable than the complete verifiers. However, the verified networks are fairly small (1800 neurons) and it is not clear how good the performance is compared to other state-of-the-art complete/incomplete verifiers.
About experiments questions:
1. The experiments compare verified robustness with AI2 and show that RefineAI can verify more than AI2 at the expense of much more computation time (Figure 3). However, the problem here is how is RefineAI or AI2 compare with other complete and incomplete verifiers as described in  the second paragraph of introduction? The AI2 does not seem to have public available codes that readers can try out but for some complete and incomplete verifiers papers mentioned in the introductions,  I do find some public codes available:
* complete verifiers
1. Tjeng & Tedrake (2017): github.com/vtjeng/MIPVerify.jl
2. SMT Katz etal (2017): https://github.com/guykatzz/ReluplexCav2017
* incomplete verifiers
3. Weng etal (2018) : https://github.com/huanzhang12/CertifiedReLURobustness
4. Wong & Kolter (2018): http://github.com/locuslab/convex_adversarial
How does Refine AI proposed in this paper compare with the above four papers in terms of the verified robustness percentage on test set, the robustness bound (the epsilon in the paragraph Abstract Interpretation p.4) and the run time? The verified robustness percentage of Tjeng & Tedrake is reported but the robustness bound is not reported.  Also, can Refine AI scale to other datasets?
About other questions:
1. Can RefineAI handle only piece-wise linear activation functions? How about other activation functions, such as sigmoid? If so, what are the modifications to be made to handle other non-piece-wise linear activation functions?
2. In Sec 4, the Robustness properties paragraph. "The adversarial attack considered here is untargeted and therefore stronger than ...". The approaches in Weng etal (2018) and Tjeng & Tedrake (2017) seem to be able to handle the untargeted robustness as well?
3. In Sec 4, the Effect of neural selection heuristic paragraph. "Although the number of images verified change by only 3 %... produces tighter output bounds...". How tight the output bounds improved by the neuron selection heuristics?