In this paper, we propose a novel adversarial learning framework, namely Delib-GAN, for generating high-quality sentences without supervision. Our framework consists of a coarse-to-fine generator, which contains a first-pass decoder and a second-pass decoder, and a multiple instance discriminator. And we propose two training mechanisms DelibGAN-I and DelibGAN-II. The discriminator is used to fine-tune the second-pass decoder in DelibGAN-I and further evaluate the importance of each word and tune the first-pass decoder in DelibGAN-II. We compare our models with several typical and state-of-the-art unsupervised generic text generation models on three datasets (a synthetic dataset, a descriptive text dataset and a sentimental text dataset). Both qualitative and quantitative experimental results show that our models produce more realistic samples, and DelibGAN-II performs best.

Summary:
The authors propose a novel adversarial learning framework consisting of a coarse-to-fine generator and a multiple instance discriminator to generate high-quality sentences without supervision, along with two training mechanisms to make the generator produce higher-quality sentences.
The coarse-to-fine generator is implemented as a two-stage decoder where the first stage decoder is initialized with a noise vector z and produces an initial sequence by sampling from its distribution at each step. The second stage decoder is able to attend to first stage decoder hidden states when generating its sequence and is initialized with the last hidden state of the first state decoder
Two training frameworks are defined to help the coarse-to-fine generator learn to produce coherent text. In both frameworks, a multi-instance learner is used as a discriminator and the overall score for the sequence is an average of word-level scores.
In DelibGAN-I, the first stage decoder is just trained to minimize the negative log likelihood of producing the sampled sequence. I’m not sure I understand how G1 learns to generate coherent text by just maximizing the probability of its samples.
In DelibGAN-2, the word-level scores from passing the first stage decoder’s output through the discriminator is used to score the sequence. This makes more sense to me w.r.t to why the first stage generator could learn to produce coherent text.
Review:
The writing could be clearer. Spacing is consistently an issue in w.r.t. to citations, referencing equations an figures, and using punctuations. Equations are difficult to understand, as well. See Equation 5. I’m confused by how the discriminator score of x_t is computed in equation 5. It seems to be the score of the multi-instance learner when applied to the concatenation of the previous generated tokens and the most recently generated token. This isn’t really a roll-out policy, however, since only a single step is taken in the future. It’s just scoring the next token according to the discriminator. In this case, I’m not sure what the summation over j is supposed to represent. It seems to index different histories when scoring word x_t. The authors should clarify exactly what the scoring procedure is for sequences generated by G2 and make sure that clarification matches what is written in the equation.
Because deliberation has previous been explored in Xia et al., 2017, the novelty of this work rests on the application of deliberation to GAN training of generative text models, as well as the development of the multi-instance discriminator for assigning word-level scores to the first-stage decoder's actions. However, it’s difficult to know how much of the improvement in performance is due to the modeling improvement because the evaluations are missing key details.
First, no information about the model’s hyperparameter setting were provided. Naturally, having two generators would double the number of parameters of the total model. Was an ablation run that looked at doubling the parameters of a single-generator? How powerful are the models being used in this new setup. With powerful models such as the OpenAI GPT being available for generative modeling, how powerful are the base units in this framework?
Second, I don’t have much intuition for how difficult the tasks being performed are. Because the authors aren’t evaluating on particularly difficult language datasets, they should provide statistics about the datasets w.r.t to vocabulary size, average length, etc.
Consequently, a lot of information that is necessary for evaluating the strengths and weaknesses of this paper are missing from the write-up.
Questions:
Equation 2 is slightly confusing. Is the representation of the word at that time step not conditioned on previous and future hidden states?
Is the multi-instance discriminator used in DelibGAN-II or is G2 only scored using the sentence level discriminator score?
Small:
-Wang & Wan, 2018a are definitely not the first work to discover that using word-level losses to train models for sentence-level evaluations is suboptimal
-Please fix issues with spacing. There are issues when citing papers, or referencing equations, or using punctuation.
-Equation 3 is missing a closing parenthesis on the softmax


We develop new approximation and statistical learning theories of convolutional neural networks (CNNs) via the ResNet-type structure where the channel size, filter size, and width are fixed. It is shown that a ResNet-type CNN is a universal approximator and its expression ability is no worse than fully-connected neural networks (FNNs) with a block-sparse structure even if the size of each layer in the CNN is fixed. Our result is general in the sense that we can automatically translate any approximation rate achieved by block-sparse FNNs into that by CNNs. Thanks to the general theory, it is shown that learning on CNNs satisfies optimality in approximation and estimation of several important function classes. As applications, we consider two types of function classes to be estimated: the Barron class and Hölder class. We prove the clipped empirical risk minimization (ERM) estimator can achieve the same rate as FNNs even the channel size, filter size, and width of CNNs are constant with respect to the sample size. This is minimax optimal (up to logarithmic factors) for the Hölder class. Our proof is based on sophisticated evaluations of the covering number of CNNs and the nontrivial parameter rescaling technique to control the Lipschitz constant of CNNs to be constructed.

The paper studies approximation and estimation properties of CNNs with residual blocks in the context
of non-parametric regression, by constructing equivalent fully-connected architectures (with a block-sparse structure),
and leveraging previous approximation results for such functions.
Explicit risk bounds are obtained for regression functions in Barron and Holder classes.
The main contribution of the paper is Theorem 1, which shows that a class of ResNet-type CNNs
contains a class of "block-sparse" fully-connected networks, with appropriate constraints on various size quantities.
This result allows the authors to obtain a general risk bound for the ResNet CNN that minimizes empirical risk
(Theorem 2, which mostly follows Schmidt-Hieber (2017)),
as well as adaptations of the bound for the Barron and Holder classes, by relying on existing approximation results.
The construction of Theorem 1 is interesting, and shows that ResNet CNNs can be quite powerful function approximators,
even with a filter size that is arbitrarily fixed.
However, the obtained CNN approximating architectures look quite unrealistic compared to most practical use-cases of CNNs,
since they specifically try to reproduce a fully-connected architecture, leading to residual blocks of depth ~= D/K,
which is very deep compared to usual CNNs/ResNets (considering, e.g. K=3 and D in the hundreds for images).
In particular, CNNs are typically used when there is some relevant inductive bias such as equivariance
to translations (and invariance with pooling operations) to take advantage of,
so removing this inductive bias by approximating fully-connected architectures seems a bit twisted.
The approach of reducing the function class to be approximated would seem more relevant here,
as in the cited papers Petersen & Voigtlaender (2018) and Yarotsky (2018), and perhaps the results of
the present paper can be useful in such a scenario as well.
Separately, the presentation of the paper could be significantly improved,
for instance by introducing relevant notions more clearly in the introduction and related work sections,
and by providing more insight and discussion of the obtained results in the main paper.
More specific comments:
- Section 1, p.2: define M? define D? M seems to be used for different things in different paragraphs
- Section 2: Explain what is "s" in the Barron class, or at least point to the relevant definition in the paper
- Section 3.1:
  * 'estimation error' is usually called '(expected) risk' in the statistical literature (also in the introduction). estimation error would have to do with relating R and R^hat
  * why is the estimator "regularized"?
- Definition 2: shouldn't it be D_m^(0) = D instead of 1?
- Theorem 1: What is L? Also, it would be helpful to sketch the construction in the main paper given that this is the main result.
- Section 4.2: M_1 is the Lipschitz constant of what function?
- Section 5.1: "M = 1" this is confusing, maybe use a different letter for the ridge expansion? The discussion on 'relative scale' could be made clearer.
- Section 5.2, 'if we carefully look at their proofs': more details on this should be provided.


Beam search is the most popular inference algorithm for decoding neural sequence models. Unlike greedy search, beam search allows for a non-greedy local decisions that can potentially lead to a sequence with a higher overall probability. However, previous work found that the performance of beam search tends to degrade with large beam widths. In this work, we perform an empirical study of the behavior of the beam search algorithm across three sequence synthesis tasks. We find that increasing the beam width leads to sequences that are disproportionately based on early and highly non-greedy decisions. These sequences typically include a very low probability token that is followed by a sequence of tokens with higher (conditional) probability leading to an overall higher probability sequence. However, as beam width increases, such sequences are more likely to have a lower evaluation score. Based on our empirical analysis we propose to constrain the beam search from taking highly non-greedy decisions early in the search. We evaluate two methods to constrain the search and show that constrained beam search effectively eliminates the problem of beam search degradation and in some cases even leads to higher evaluation scores. Our results generalize and improve upon previous observations on copies and training set predictions.

This paper addresses issues with the beam search decoding algorithm that is commonly applied to recurrent models during inference. In particular, the paper investigates why using larger beam widths, resulting in output sequences with higher log-probabilities, often leads to worse performance on evaluation metrics of interest such as BLEU. The paper argues that this effect is related to ‘search discrepancies’ (deviations from greedy choices early in decoding), and proposes a constrained decoding mechanism as a heuristic fix.
Strengths:
- The reduction in performance from using larger beam widths has been often reported and needs more investigation.
- The paper views beam search decoding through the lens of heuristic and combinatorial search, and suggests an interesting connection with methods such as limited discrepancy search (Harvey and Ginsberg 1995) that seek to eliminate early ‘wrong turns’.
- In most areas the paper is clear and well-written, although it may help to be more careful about explaining and / or defining terms such as ‘highly non-greedy’, ‘search discrepancies’ in the introduction.
Weaknesses and suggestions for improvement:
- Understanding: The paper does not offer much in the way of a deeper understanding of search discrepancies. For example, are search discrepancies caused by exposure bias or label bias, i.e. an artifact of local normalization at each time step during training, as suggested in the conclusion? Or are they actually a linguistic phenomenon (noting that English, French and German have common roots)? As there are neural network methods that attempt to do approximate global normalization (e.g. https://www.aclweb.org/anthology/P16-1231), there may be ways to investigate this question by looking at whether search discrepancies are reduced in these models (although I haven’t looked deeply into this).
- Evaluation: In the empirical evaluation, the results seem quite marginal. Taking the best performing beam size for the proposed method, and comparing the score to the best performing beam size for the baseline, the scores appear to be within around 1% for each task. Although the proposed method allows larger beam widths to be used without degradation during decoding, of course this is not actually beneficial unless the larger beam can improve the score. In the end, the evidence that search discrepancies are the cause of the problems with large beam widths, and therefore the best way to mitigate these problems, is not that strong.
- Evaluation metrics and need for human evals: The limitations of automatic linguistic evaluations such as BLEU are well known. For image captioning, the SPICE (ECCV 2016 https://arxiv.org/abs/1607.08822) and CIDEr (CVPR 2015 https://arxiv.org/abs/1411.5726) metrics show much greater correlation with human judgements of caption quality, and should be reported in preference (or in addition) to BLEU. More generally, it is quite possible that the proposed fix based on constraining discrepancies could improve the generated output in the eyes of humans, even if this is not strongly reflected in automatic evaluation metrics. Therefore, it would be interesting to see human evaluations for the generated outputs in each task.
- Rare words: The authors reference Koehn and Knowles’ (2017) six challenges for NMT, which includes beam search decoding. One of the other six challenges is low-frequency words. However, the impact of the proposed constrained decoding approach on the generation of rare words is not explored. It seems reasonable that limiting search discrepancies might also further limit the generation of rare words. Therefore, I would like to suggest that an analysis of the diversity of the generated outputs for each approach be included in the evaluation.
- Constrained beam search: There is a bunch of prior work on constrained beam search. For example, an algorithm called constrained beam search was introduced at EMNLP 2017 (http://aclweb.org/anthology/D17-1098). This is a general algorithm for decoding RNNs with constraints defined by a finite state acceptor. Other works have also been proposed that are variations on this idea, e.g. http://aclweb.org/anthology/P17-1141, http://aclweb.org/anthology/N18-1119). It might be helpful to identify these in the related work section to help limit confusion when talking about this ‘constrained beam search’ algorithm.
Minor issues:
- Section 3. The image captioning splits used by Xu et al. 2015 were actually first proposed by Karpathy & Li, ‘Deep visual-semantic alignments for generating image descriptions’, CVPR 2015, and should be cited as such. (Some papers actually refer to them as the ‘Karpathy splits’.)
- In Table 4 it is somewhat difficult to interpret the comparison between the baseline results and the constrained beam search methods, because the best results appear in different columns. Bolding the highest score in every row would be helpful.
Summary:
In summary, improving beam search is an important direction, and to the best of my knowledge the idea of looking at beam search through the lens of search discrepancies is novel. Having said, I don't feel that this paper in it's current form contributes very much to our understanding of RNN decoding, since it is not clear if search discrepancies are actually a problem. Limiting search discrepancies during decoding has minimal impact on BLEU scores, and it seems possible that search discrepancies could just be an aspect of linguistic structure. I rate this paper marginally below acceptance, although I would encourage the authors to keep working in this direction and have tried to provide some suggestions for improvement.


LEARNING TO NAVIGATE THE WEB
Learning in environments with large state and action spaces, and sparse rewards, can hinder a Reinforcement Learning (RL) agent's learning through trial-anderror. For instance, following natural language instructions on the Web (such as booking a flight ticket) leads to RL settings where input vocabulary and number of actionable elements on a page can grow very large. Even though recent approaches improve the success rate on relatively simple environments with the help of human demonstrations to guide the exploration, they still fail in environments where the set of possible instructions can reach millions. We approach the aforementioned problems from a different perspective and propose guided RL approaches that can generate unbounded amount of experience for an agent to learn from. Instead of learning from a complicated instruction with a large vocabulary, we decompose it into multiple sub-instructions and schedule a curriculum in which an agent is tasked with a gradually increasing subset of these relatively easier sub-instructions. In addition, when the expert demonstrations are not available, we propose a novel meta-learning framework that generates new instruction following tasks and trains the agent more effectively. We train DQN, deep reinforcement learning agent, with Q-value function approximated with a novel QWeb neural network architecture on these smaller, synthetic instructions. We evaluate the ability of our agent to generalize to new instructions on World of Bits benchmark, on forms with up to 100 elements, supporting 14 million possible instructions. The QWeb agent outperforms the baseline without using any human demonstration achieving 100% success rate on several difficult environments.

UPDATE:
Thank you to the authors for a comprehensive response.  I have increased my score based on these changes.  I apologize for the misunderstanding about ArXiV papers and indeed the authors are correct on that point.  Thank you as well for reporting the learning speeds.  As you mentioned, they confirm our intuitions and complete the picture of the algorithm’s behavior.  The addition of pseudo-code does make the paper and algorithm easier to follow.  Thank you for adding it.  The rewritten section 5 is indeed much easier to follow and makes the coordination between the agents clear.  Seeing that the instructor is a fixed policy resolves the game theoretic issue form the original review.
Summary:
The paper proposes a deep reinforcement learning approach to filling out web forms, called QWeb.  In addition to both deep and shallow embeddings of the states, the authors evaluate various methods for improving the learning system, including reward shaping, introducing subgoals, and even a meta-learning algorithm that is used as an instructor.  These variations are tested in several environments and basic QWeb is shown to outperform the baselines and many of the adaptations perform even better than that in more complex domains.
Review:
Overall, the problem the paper considers is important and their results seem significant.  The authors have derived a novel architecture and are the first to tackle the problem of filling in web forms at this scale with an autonomous learning agent rather than one that is taught mostly by demonstration.
The related work section is very well written with topical references to recent results and solid differentiations to the new algorithm.  However, I see many references in the paper are not from peer reviewed conferences or journals.  Unless absolutely necessary, such papers should not be cited because they have not been properly peer reviewed.  If the papers cited have actually been in a conference or journal, please add the correct attribution.
The experiments seem well conducted.  I liked that each new addition to the algorithm was tested incrementally in Figure 7 to give a realistic view of the gains introduced by each change.  I also thought the earlier comparisons to the baselines were well done and I liked that they were done against modern cutting-edge LfD demonstrations.  The only thing I would have liked to seen beyond these results are actual learning curves showing, after X iterations, what percentage of the tasks could be completed.  I suspect that in many domains the baseline LfD techniques are learning much faster since learning from teachers tends to be more targeted and sample efficient.  Learning curves would show us whether or not this is the case.
The weakest part of the paper was the description of the instructor network and the Meta-training in general.  This portion seemed ill-described and largely speculative, despite the promising results in Figure 7.  In particular, Section 5 is very unclear on how exactly the Meta-Learning works.  Pseudocode is definitely needed in this portion well beyond the quick descriptions in Figure 4 and 5, which I could not understand, despite multiple readings.  I suggest eliminating those figures and providing concrete pseudo—code describing the meta learning and also addressing the following open questions in the text:
•	Why is a rule based randomized policy good to learn from?  How is this different from learning from demonstration in the baselines?
•	How is a “fine grained signal” generated?  What does that mean?  Is it a reward?
•	In Section 5.1, are there two RL agents, an instructor and a learner with different reward functions?  If so, isn’t this becoming game theoretic and is this likely to converge in most scenarios?
•	What does Q_D^I actually represent?  Why is maximizing these values a good thing?
There are a few grammatical mistakes in the paper including:
Abstract – simpler environments -> simple environments
Abstract- with gradually increasing -> with a gradually increasing
Page 2 – generate unbounded -> generate an unbounded
Page 7 – correct value -> correct values
Page 9 – episode length -> episode lengths


Learning synthesizers and generating music in the raw audio domain is a challenging task. We investigate the learned representations of convolutional autoregressive generative models. Consequently, we show that mappings between musical notes and the harmonic style (instrument timbre) can be learned based on the raw audio music recording and the musical score (in binary piano roll format). Our proposed architecture, SynthNet uses minimal training data (9 minutes), is substantially better in quality and converges 6 times faster than the baselines. The quality of the generated waveforms (generation accuracy) is sufficiently high that they are almost identical to the ground truth. Therefore, we are able to directly measure generation error during training, based on the RMSE of the Constant-Q transform. Mean opinion scores are also provided. We validate our work using 7 distinct harmonic styles and also provide visualizations and links to all generated audio.

This paper proposes a neural model for synthesizing instrument sounds, using an architecture based on the WaveNet and DeepVoice models. The model generates raw waveforms conditioned on a piano roll representation of aligned MIDI input.
My biggest gripe with this work is that the model is trained entirely on a synthetic dataset generated from a sample-based synthesizer using a sound font. I feel that this defeats the purpose, as it will never work better than just sampling the original sound library. One potential argument in favour would be to save storage space, but the sound font used for the work is only ~140 MB, which is not prohibitive these days (indeed, many neural models require a comparable amount of storage).
It would be much more interesting to train the model on real instrument recordings, because then it could capture all the nuances of the instruments that sample-based synthesizers cannot replicate. As it stands, all the model has to do is reproduce a fixed (and fairly small) set of audio samples. This is arguably a much simpler task, which could also explain why reducing the model size (SynthNet's depthwise convolutions have many fewer parameters than the regular convolutions used in WaveNet and DeepVoice) works so well here.
That said, I think the proposed architectural modifications for raw audio models could be interesting and should be tested for other, more challenging tasks. The proposed RMSE-CQT error measure is potentially quite valuable for music generation research, and its correlation with MOS scores is promising (but this should also be tested on more realistic audio).
The fact that the models were trained to convergence on only 9 minutes of data per instrument is also impressive, despite the limitations of the dataset. The use of dithering to reduce perceptual noise is also interesting and some comparison experiments there would have been interesting, especially to corroborate the claim that it is critical for the learning process.
I think the paper slightly overstates its contributions in terms of providing insight into the representations that are learned in generative convolutional models. The Gram matrix projections showing that the activations of different layers diverge for different input types as we advance through the model is not particularly surprising, and similar plots could probably be made for almost any residual model.
Overall, I feel the work has some fundamental flaws, mostly stemming from the dataset that was used.
Miscellany:
- In the abstract: "is substantially better in quality", compared to what?
- In the introduction, it is mentioned that words in a speech signal cannot overlap, but notes in a musical signal can. I would argue that these are not comparable abstractions though, words themselves are composed of a sequence of phonemes, which are probably a better point of comparison (and phonemes, while they don't tend to overlap, can affect neighbouring phonemes in various ways). That said, I appreciate that this is probably quite subjective.
- Overall, the formulation of paragraph 2 of the introduction is a bit unusual, I think the same things are said in a much better way in Section 3.
- "Conditioning Deep Generative Raw Audio Models for Structured Automatic Music" by Manzelli et al. (2018) also proposes a MIDI-conditional neural audio generation model, trained on real instrument recordings from the MusicNet dataset. I think this is a very relevant reference.
- In the contributions of the paper, it is stated that "the generated audio is practically identical to ground truth as can be seen in Figure 4" but the CQTs in this figure are visibly different.
- I don't think it is fair to directly compare this setup to Engel et al. (2017) and Mor et al. (2018) as is done in the last paragraph of Section 2, as these are simply different tasks (mapping from audio to audio as opposed to generating audio).
- At the start of Section 3.1 it would be good to explicitly mention whether 8-bit mu-law audio is used, to explain why the waveform is 256-valued.
- Why is the conditioning causal? It does not need to be, as the piano roll is fully available in advance of the audio generation. I guess one argument in favour would be to enable real-time generation, but it would still be good to compare causal and non-causal conditioning.
- Since the piano roll representation is binary, does that mean MIDI velocity is not captured in the conditioning signal? It would probably be useful for the model to provide this information, so it can capture the differences in timbre between different velocities.
- The use of a MIDI prediction loss to regularise the conditioning part of the model is interesting, but I would have liked to see a comparison experiment (with/without).
- In Section 4.3, specify the unit, i.e. "Delta < 1 second".
- For the task of recreating synthetic audio samples, the WaveNet models seem to be quite large. As far as I can tell the size hyperparameters were chosen based on the literature, but the inherited parameters were originally optimised for different tasks.
- In Section 4.3 under "global conditioning", the benchmark is said to be between DeepVoice L26 and SynthNet L24, but Table 4 lists DeepVoice L26 and SynthNet L26, which version was actually used?


Most deep neural networks use simple, fixed activation functions, such as sigmoids or rectified linear units, regardless of domain or network structure. We introduce differential equation networks, an improvement to modern neural networks in which each neuron learns the particular nonlinear activation function that it requires. We show that enabling each neuron with the ability to learn its own activation function results in a more compact network capable of achieving comperable, if not superior performance when compared to much larger networks. We also showcase the capability of a differential equation neuron to learn behaviors, such as oscillation, currently only obtainable by a large group of neurons. The ability of differential equation networks to essentially compress a large neural network, without loss of overall performance makes them suitable for on-device applications, where predictions must be computed locally. Our experimental evaluation of realworld and toy datasets show that differential equation networks outperform fixed activatoin networks in several areas.

This paper proposes the use of structured activations based on ordinary differential equations, as an activation in neural network architectures.
There are validations that the approach discovers different activations, and comparisons to a variety of other architectures with fixed activations. In general, I would like to see additional contextualization of your work with other approaches which learn activation functions. How does your approach differ from max-out and Agostinelli? References to Network in Network, Lin et al and even Hypernetworks, Ha et al would also be helpful. Some of these papers are cited, but only the comparison to Ramachadran et. al. is directly discussed and the methodological difference there is not true for the case of Maxout.
The paper indicates that the ODE should be solved before beginning to use the network. Should this process be performed once per network design? Once per dataset? What happens if instead of solving f1 and f2, these are set randomly? How is this actually solved in the case of the MNIST convnet?
The experiments were useful in demonstrating the proposed method. However, some discussion and comparison to other learned activation functions would be helpful (for instance, one could perform similar experiments as in Maxout). Performance on larger datasets, as seen in Swish, would make the results more compelling.
The MNIST experiments shown are also pretty far from standard baselines. See for example the benchmark performance in Maxout, which also references an architecture from Jarrett et. al., 2009 which is quite similar to the baseline architecture, but ~.5% error. It isn't necessary to get the absolute best performance with a new activation, just show that the proposed doesn't actively hurt and enables new interpretation or direction. But as it stands, it isn't possible to tell from the experiments if the proposed method has serious limitations, because the baselines on MNIST are below where they should be.
In general, a larger test suite that compares on more standard datasets is necessary to really prove out this idea, or see if there are problems with cases on larger datasets. CIFAR10 at a minimum would be a key addition as well as other datasets (besides the diabetes dataset shown) where there can be direct comparison to existing work. Currently, only MNIST is filling that role. Many of the cited / compared work (PReLU, Swish, LReLU, SELU) has a broad suite of benchmarks, all on datasets with existing numbers tuned by the authors of respective past papers.
What are the downsides of this method? Tradeoffs in memory and training time should be discussed in detail if application to low power hardware is a real application area, perhaps along with an inclusion of the time / effort required for solving the ODEs in Maple. A difference of 2x+ in parameter count may not be a difference if the computational time is much worse. Can you consider a case where "normal" architectures don't fit in memory, but one based on ODE activations will? The paper directly discusses mobile and low footprint deployments, but without discussion of the computational overhead and complexity it is speculative, especially when there are also numerous methods for compressing or distilling very large architectures to much smaller sizes, as well as small models which directly achieve high performance. A few relevant methods are linked below.
MobileNets - https://arxiv.org/abs/1704.04861
ENet - https://arxiv.org/abs/1606.02147
In the Network Compression section, the paper fails to discuss a number of successful foundational and modern network compression techniques that would improve the argument, including:
Optimal brain damage  - “removing unimportant weights can actually improve performance” - https://papers.nips.cc/paper/250-optimal-brain-damage
Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding -  pruned state-of-the-art CNN models with no loss of accuracy - https://arxiv.org/abs/1510.00149
Bayesian Compression for Deep Learning - https://arxiv.org/abs/1705.08665
Practical Variational Inference for Neural Networks - http://papers.nips.cc/paper/4329-practical-variational-inference-for-neural-networks
There are repeated claims of first use of ODE in neural networks, which is frankly false. Though the specific use proposed here may be new, neural networks and ODEs have been used together many times. Clarifying what particular usage of ODE inside this setting is novel would be much better than a broad claim such as "While the presented model, algorithms and results in this paper are the first application of ODEs in neural networks...". Much of this work has been about controlling or solving ODEs, but particularly the setting of Meade Jr. et. al. strongly resembles a "neuron" in this architecture, so a discussion of the relevant differences would be useful. In addition Neural Ordinary Differential Equations allows the end-to-end training of ODEs in larger models, which also closely resembles the use of ODEs here.
Artificial Neural Networks for Solving Ordinary and Partial Differential Equations - https://ieeexplore.ieee.org/document/712178/
Solution of nonlinear ordinary differential equations by feedforward neural networks - https://www.sciencedirect.com/science/article/pii/089571779400160X
Neural Ordinary Differential Equations - https://arxiv.org/abs/1806.07366
Overall, a stronger focus on empirical results on comparable datasets would be beneficial, especially larger tasks. If larger tasks are not possible, a description of what it may take to "scale up" would be useful. The written focus on novelty detracts from the presentation, and a discussion of neural ODE methods (whether acting as activations, or solvers) would serve as good background material. If compute / performance in low footprints or mobile hardware is a focus, it should be described and tested. If lower parameter count is a perceived benefit, a more direct exploration and discussion of parameter count settings for this architecture and baselines would also be useful. Particularly, hyperparameters become very important in small architectures, so "Dropout probability,
batch size, epochs and learning rate were consistent across all networks" is not a positive (presuming the authors have likely tuned toward their own architecture). Baselines should be given equal treatment and tuning in order to compare "best-on-best" performance.
The description of universal approximation, visualization of the adaptivity of the method, and background are all very nice. My concerns come primarily to relation to prior and relevant work, strength of relevant experimentation, and claims of application and novelty / "first past the post".
——
Minor Nitpicks:
Page 1: In the sentence - “researchers have introduced highly effective network structures such as convolutional neural networks”, it seems inconsistent to cite a foundational paper for CNNs and not RNNs.
Page 2: It seems like there is a word missing here - “The size of a neural network is delineated its number of hidden neurons and their interconnections, which together determine the network’s complexity”
There seems to be a missing word in “3.3 DIFEN IS UNIVERSAL APPROXIMATOR” .
Numerous spelling errors should be corrected -
3.1 differentiatial
4.1 challnging
Figure 1 - fucntion


UNIVERSAL STAGEWISE LEARNING FOR NON- CONVEX PROBLEMS WITH CONVERGENCE ON AV- ERAGED SOLUTIONS
Although stochastic gradient descent (SGD) method and its variants (e.g., stochastic momentum methods, ADAGRAD) are algorithms of choice for solving nonconvex problems (especially deep learning), big gaps still remain between the theory and the practice with many questions unresolved. For example, there is still a lack of theories of convergence for SGD and its variants that use stagewise step size and return an averaged solution in practice. In addition, theoretical insights of why adaptive step size of ADAGRAD could improve non-adaptive step size of SGD is still missing for non-convex optimization. This paper aims to address these questions and fill the gap between theory and practice. We propose a universal stagewise optimization framework for a broad family of non-smooth nonconvex problems with the following key features: (i) at each stage any suitable stochastic convex optimization algorithms (e.g., SGD or ADAGRAD) that return an averaged solution can be employed for minimizing a regularized convex problem; (ii) the step size is decreased in a stagewise manner; (iii) an averaged solution is returned as the final solution. Our theoretical results of stagewise ADAGRAD exhibit its adaptive convergence, therefore shed insights on its faster convergence than stagewise SGD for problems with slowly growing cumulative stochastic gradients. To the best of our knowledge, this is the first work for addressing the unresolved issues of existing theories mentioned earlier. Besides theoretical contributions, our empirical studies show that our stagewise SGD and ADAGRAD improve the generalization performance of existing variants/implementations of SGD and ADAGRAD.

Summary:
The paper presents an analysis and numerical evaluation of stagewise SGD, ADAGRAD and Stochastic momentum methods for solving stochastic non-smooth non-convex optimization problems.
Comments:
I find the ideas presented in this paper very interesting. The convergence analysis seems correct and the paper is reasonably well written, and tackles an important problem.
The analysis holds for μ-weekly convex functions. This assumption is really important for the development of the algorithm and the proposed analysis. I like the fact that the authors provide two examples showing that popular objective functions in machine learning satisfy this assumption.
The numerical evaluation is adequate showing the effectiveness  of the proposed stagewise algorithms.  However i have the follow suggestions/minor comments:
1) It will be nice to have also some plots showing the performance of the proposed method on the ImageNet dataset.
2) Another possible nice experiment will be a comparison of the four stagewise methods (SGD,ADAGRAD,SHB,SNAG) on the same dataset. Which one behaves better?
Minor Comments:
1) The captions of the figures can be more informative (mention also the division by column). First column is SGD, Second column Adagrad, etc.
2) Typos:
Section 1, last bullet point, second line: "stagwise"
Section 5, second paragraph , first line :"their their"
page 8, 3 line from the bottom:  "seems, indicate"
2) Missing reference.
In the area of stochastic gradient methods with momentum many papers have been proposed recently for the case of convex optimization that worth to be mentioned:
Gadat, Sébastien, Fabien Panloup, and Sofiane Saadane. "Stochastic heavy ball." Electronic Journal of Statistics 12.1 (2018): 461-529.
Loizou, Nicolas, and Peter Richtárik. "Momentum and stochastic momentum for stochastic gradient, Newton, proximal point and subspace descent methods." arXiv preprint arXiv:1712.09677 (2017).
Lan, Guanghui, and Yi Zhou. "An optimal randomized incremental gradient method." Mathematical programming (2017): 1-49.
Overall, I suggest to accept this paper.


Quantum computers promise significant advantages over classical computers for a number of different applications. We show that the complete loss function landscape of a neural network can be represented as the quantum state output by a quantum computer. We demonstrate this explicitly for a binary neural network and, further, show how a quantum computer can train the network by manipulating this state using a well-known algorithm known as quantum amplitude amplification. We further show that with minor adaptation, this method can also represent the meta-loss landscape of a number of neural network architectures simultaneously. We search this meta-loss landscape with the same method to simultaneously train and design a binary neural network.

Neural Network Cost Landscapes as Quantum States
The authors describe a method where a deep learning framework can be quantised, this is done by considering the two state form of a Bloch sphere/qubit and mapping binary neural network. onto the quantum object creating a quantum binary neural network
I have to say I liked the paper, it is indeed novel and I haven’t seen this anywhere else. More then that, it addresses the quantum aspects of deep learning which has only recently started getting so much attention rather the regular machine learning algorithms. And such I think it’s a good fit for ICLR.
With that I have a few concerns/ nitpicking issues I would like the authors to address if possible
1. While authors show basically a discreet network (much like Soudry’s work) there has also been recently a show of continues variable networks (https://arxiv.org/pdf/1806.06871.pdf), how does your scaling compare to that ? Could one think of this is the continuum limit (albeit you showed a *very* small system) of your model ?
2. As a general style remark there is a lot of introduction on quantum information and I wondering if authors could just reference classical text such as Isaac Chuang, Michael Nielsen or for the CS flavour Classical and Quantum Computation  ? I would have liked to see all 3.1,3.2 maybe in an appendix and a lot more details on the experiments and setup for example
3. At the end of section 3.2 authors mention the lack of correspondence principle in some quantum systems, I would be happy for a refinement in the aspect of quantum computing that is true and also in general quantum mechanics but there is a huge body of work and an entire field dedicated to just that, settling the difference in correspondence principle and it’s meaning (for example Chaos and the semiclassical limit of quantum mechanics (is the moon there when somebody looks?) by Michael Berry. While this is defiantly far from a deal breaker I would be happy for a bit more clarification on subtle difference. My guess is that authors are referring to the fact that an essential part of quantum computing is the lack of correspondence that can also be taken advantage of for quantum parallelism/ quantum speed ups ?
Same for the last paragraph of 3.1.
4. In the method section  all of this procedure is described for a fault tolerant machine, can you say something about coupling to error correction codes for near term quantum devices ?
5. On the same note, there is a feeling that non linearities are swept under the rug, do you have a hunch how can one use non linear activation functions ? (General question, you don’t really have to answer and can take the fifth )
6. In the problem statement (4.2)  x is in fact the binary representation of the qubit state ?
7. In your method Fig. 2, should there be some sort of measurements ?
8. Can you elaborate what exactly goes into the U gates and how are they constructed in Fig.2 ? Or is it some  oracle model that can do all the actions
9. You report a quadratic speedup, is there some relationship to random walk  is this in fact a case of a RW search over the parameter space ?


Extractive summarization methods operate by ranking and selecting the sentences which best encapsulate the theme of a given document. They do not fare well in domains like fictional narratives where there is no central theme and core information is not encapsulated by a small set of sentences. For the purpose of reducing the size of the document while conveying the idea expressed by each sentence, we need more sentence specific methods. Telegraphic summarization, which selects short segments across several sentences, is better suited for such domains. Telegraphic summarization captures the plot better by retaining shorter versions of each sentence while not really concerning itself with grammatically linking these segments. In this paper, we propose an unsupervised deep learning network (NUTS) to generate telegraphic summaries. We use multiple encoderdecoder networks and learn to drop portions of the text that are inferable from the chosen segments. The model is agnostic to both sentence length and style. We demonstrate that the summaries produced by our model show significant quantitative and qualitative improvement over those produced by existing methods and baselines.

The authors consider the problem of telegraphic sentence compression: they train a system in an unsupervised fashion to predict which words can be dropped from a sentence without drastic loss of information. To that end, they propose a new auto-encoding type architecture which uses the extracted words as latent code, and, most importantly, a linkage loss which relates a word's perplexity given the summary of its left context to its likelihood of being retained. The model itself is sober and well motivated, and the linkage loss is, to the best of my knowledge, original. The authors show that their method outperforms some simple baselines in terms of ROUGE and compression on a small human-annotated test set.
The paper is generally well written, although the initial presentation of the model could be made a little clearer (it is not obvious from the text that the Decoder takes the text as input -- Figure 2 helps, but comes a couple pages later). However, the authors fail to appropriately justify the choice of their hyper-parameters (e.g. "The optimum value of r for our experiments was found to be 0.65", "the best value of b was found to be 5", "The weights λ1, λ2, λ3, and λ4 have been set to 3, 2, 50 and 3 respectively for our experiments" -> how is "best" measured on the validation set, which does not have gold references?). The choice of the specific sparsity constraint (one could as well imagine using a simpe L1 regularization for the Binarization loss) and of \Chi_i (why not simply use the likelihood?) could also be better motivated.
The model also relies on a hand-crafted rules (Section 3.3) whose effect needs to be made more evident. What weights are used in practice? How were they chosen ("We observed that..." needs to be further developed)? The authors claim that "the quantitative scores are not affected significantly", but that is presumably only the ROUGE score, what about annotator's preferences?
Most importantly, however, the task of telegraphic sentence compression, whose usefulness is not a priori obvious, is barely motivated.  The author refer to "Malireddy et al. (2018)" for a justification, but it is important to note that the latter provides a telegraphic summary of a whole document, with a compression factor of 0.37. The claim is that the concatenation of the telegraphic sentence compression can act as a summary of a whole document, but given the fact that compression for individual sentences is closer to 0.69, this is yet to be demonstrated. And even if that were true, it is unclear whether the cognitive load of reading a sequence of telegraphic sentences would be that much lower than that of reading the original text.
This paper presents some interesting ideas and is well written, but the content is not quite sufficient for publication. In addition to the clarifications and justifications requested above, the authors are encouraged to apply there methods to full lengths documents, which would make for a more substantial contribution.


Deep Neural Networks (DNNs) thrive in recent years in which Batch Normalization (BN) plays an indispensable role. However, it has been observed that BN is costly due to the reduction operations. In this paper, we propose alleviating the BN's cost by using only a small fraction of data for mean & variance estimation at each iteration. The key challenge to reach this goal is how to achieve a satisfactory balance between normalization effectiveness and execution efficiency. We identify that the effectiveness expects less data correlation while the efficiency expects regular execution pattern. To this end, we propose two categories of approach: sampling or creating few uncorrelated data for statistics' estimation with certain strategy constraints. The former includes "Batch Sampling (BS)" that randomly selects few samples from each batch and "Feature Sampling (FS)" that randomly selects a small patch from each feature map of all samples, and the latter is "Virtual Dataset Normalization (VDN)" that generates few synthetic random samples. Accordingly, multi-way strategies are designed to reduce the data correlation for accurate estimation and optimize the execution pattern for running acceleration in the meantime. All the proposed methods are comprehensively evaluated on various DNN models, where an overall training speedup by up to 21.7% on modern GPUs can be practically achieved without the support of any specialized libraries, and the loss of model accuracy and convergence rate are negligible. Furthermore, our methods demonstrate powerful performance when solving the well-known "micro-batch normalization" problem in the case of tiny batch size.

This paper proposes a new technique that can reduce the computational complexity of batch normalization. Several sampling methods called NS, BS, and FS are proposed, and additionally, VDN is proposed to generate random virtual samples.  Experiment results follow to support the authors' goal.
pros)
(+) The paper is clearly written and easy to follow.
(+) The way of reducing the computational cost looks good.
(+) The method can be easily adapted to BN or other batch-based methods.
cons)
(-) Any motivations or insights into NS, BS, and FS are not provided. Furthermore, the proposed sampling strategy looks heuristic without any studies.
(-) For VDN, how to generate virtual samples is not clearly stated. I think the way of generating samples is critical to the performance of VDN but hard to find the exact way to do that.
(-) How to determine the sampling ratio for each normalization method is not provided, and it would be better if the authors can show some studies about sampling ratio versus the speed gain.
(-) It is hard to choose which normalization among FS and BS is better as looking at Table 2 and 3 only. So how about the speedup using BS+VDN?
comments)
- It is something strange why the authors used shallower ResNet on ImageNet and deeper ones on CIFAR datasets, maybe it was due to the training time, but the authors should clarify it.
- What is the goal of the correlation analysis section? Especially, Figure 7 looks similar among BS, FS, VDN, and NS. Furthermore, the authors could include BN into the comparison.
- This kind of paper should incoporate different ablation studies as much as the authors can, but it seems to be lacking.
The paper has an interesting idea about sampling some features to speed up the batch normalization. However, it looks quite obvious and needs more experimental grounds such as ablation studies to support the idea.


RECALL TRACES: BACKTRACKING MODELS FOR EFFICIENT REINFORCEMENT LEARNING
In many environments only a tiny subset of all states yield high reward. In these cases, few of the interactions with the environment provide a relevant learning signal. Hence, we may want to preferentially train on those high-reward states and the probable trajectories leading to them. To this end, we advocate for the use of a backtracking model that predicts the preceding states that terminate at a given high-reward state. We can train a model which, starting from a high value state (or one that is estimated to have high value), predicts and samples which (state, action)-tuples may have led to that high value state. These traces of (state, action) pairs, which we refer to as Recall Traces, sampled from this backtracking model starting from a high value state, are informative as they terminate in good states, and hence we can use these traces to improve a policy. We provide a variational interpretation for this idea and a practical algorithm in which the backtracking model samples from an approximate posterior distribution over trajectories which lead to large rewards. Our method improves the sample efficiency of both on-and off-policy RL algorithms across several environments and tasks.

Revision:
The authors have thoroughly addressed my review and I have consequently updated my rating accordingly.
Summary:
Model-free reinforcement learning is inefficient at exploration if rewards are
sparse / low probability.
The paper proposes a variational model for online learning to backtrack
state / action traces that lead to high reward states based on best previous
samples.
The backtracking models' generated recall traces are then used to augment policy
training by imitation learning, i.e. by optimizing policy to take actions that
are taken from the current states in generated recall traces.
Overall, the methodology seems akin to an adaptive importance sampling
approach for reinforcement learning.
Evaluation:
The paper gives a clear (at least mathematically) presentation of the core idea
but it some details about modeling choices seem to be missing.
The experimental evaluation seems preliminary and it is not fully evident when
and how the proposed method will be practically relevant (and not relevant).
My knowledgable of the previous literature is not sufficient to validate the
claimed novelty of the approach.
Details:
The paper is well written and easy to follow in general.
I'm not familiar enough with reinforcment learning benchmarks to judge the
quality of the experiments compared to the literature as a whole.
Although there are quite a few experiments they seem rather preliminary.
It is not clear whether enough work was done to understand the effect of the
many different hyperparameters that the proposed method surely must have.
The authors claim to show empirically that their method can improve sample
efficiency.
This is not necessarily a strong claim as such and could be achieved on
relatively simple tests.
In the discussion the authors claim their results indicate that their approach
is able to accelearte learning on a variety of tasks, also not a strong claim.
The paper could be improved by adding a more clear explanation of the exact way
by which the method helps with exploration and how it affects finding sparse
rewards (based on e.g. Figure 1).
It seems that since only knowledge of seen trajectories can be used to generate
paths to high reward states it only works for generating new trajectories
through previously visited states.
Questions that could be clarified:
- It is not entirely obvious to me what parametric models are used for the
backtracking distributions.
- Does this method not also potentially hinder exploration by making the agent
learn to go after the same high rewards / Does the direction of the variational
problem guarantee coverage of the support of the R > L distribution by samples?
- What would be the effect of a hyperparameter that balances learning the recall
traces and learning the true environment?
- Are there also reinforcement learning tasks where the proposed methods'
improvement is marginal and the extra modeling effort is not justified (e.g.
due to increase complexity).
Page 1: iwth (Typo)
Page 2: r(s_t) -> r(s_t, a_t)
Page 6: Prioritize d (Typo)


Most classification and segmentation datasets assume a closed-world scenario in which predictions are expressed as distribution over a predetermined set of visual classes. However, such assumption implies unavoidable and often unnoticeable failures in presence of out-of-distribution (OOD) input. These failures are bound to happen in most real-life applications since current visual ontologies are far from being comprehensive. We propose to address this issue by discriminative detection of OOD pixels in input data. Different from recent approaches, we avoid to bring any decisions by only observing the training dataset of the primary model trained to solve the desired computer vision task. Instead, we train a dedicated OOD model which discriminates the primary training set from a much larger "background" dataset which approximates the variety of the visual world. We perform our experiments on high resolution natural images in a dense prediction setup. We use several road driving datasets as our training distribution, while we approximate the background distribution with the ILSVRC dataset. We evaluate our approach on WildDash test, which is currently the only public test dataset that includes out-of-distribution images. The obtained results show that the proposed approach succeeds to identify out-of-distribution pixels while outperforming previous work by a wide margin.

Summary:
This paper addresses the problem of out-of-distribution detection for helping the segmentation process. Therefore, the detection is performed on a pixel basis. The application of the approach is to datasets used for autonomous driving, where semantic segmentation of the view of the road is a typical application. Since in a road view there will be pixels that are projections of objects that are likely not in the set of classes known by the semantic segmentation algorithm, it makes sense to flag them as being out of distribution (OOD), or not known, or to assign to them a low confidence level. The proposed approach is trivial: train a binary classifier that distinguishes image patches from a known set of classes from image patches coming from an unknown (background class). The classifier output applied at every pixel will give the confidence value. While there are different dataset options to represent the known classes, the background class is represented by images from ILSVRC. The results show that for the segmentation application the approach works better than using an adaptation of more elaborate out-of-distribution methods.
Quality and clarity:
The paper is well organized and is described very clearly and provides an ok set of results, despite the simplicity of the approach.
Originality and significance:
Unfortunately, I do not see any relevant technical novelty, and this is a major issue. Perhaps the only significant conclusion about this paper is that before designing a new OOD detector, if representing the set of “unknown” classes with ILVRC is reasonable, then it makes sense to simply train a binary classifier and see how it works.
Besides the novelty, I disagree with the way the paper has been positioned and motivated. It brings into play epistemic and aleatoric uncertainty concepts to justify (the simplicity of) the approach, and it overlooks a large body of machine learning (novelty detection, one-class classification, …). This is also a major issue.
Additional comments:
One of the biggest motivations for this work is that other approaches do not distinguish between epistemic and aleatoric uncertainty and this is why they do not work. This is regarded as a distinctive advantage of the proposed approach. It is claimed that the proposed formulation is insensitive to any aleatoric uncertainty. On the other hand, the paper is written in a way that ignores a large body of literature that goes under the name of “novelty detection”, “anomaly detection”, “one-class classification”, and related names. So, I am wondering how the approaches just mentioned compare with the proposed method, when epistemic and aleatoric uncertainty become part of the discussion. Isn’t every novelty detector insensitive to aleatoric uncertainty as well? Could the Authors clarify what they claim with that statement, while considering a broader view?
The paper should relate to the literature mentioned above. In particular, I would point the Authors to a couple of recent works that seem to precisely contradict the premises of the proposed approach, which are given at the beginning of section 3:
- Adversarially Learned One-Class Classifier for Novelty Detection, CVPR 2018
- Generative Probabilistic Novelty Detection with Adversarial Autoencoders, arXiv, July 2018.
Again, related to novelty detection, it looks like the proposed approach still requires tuning one or more thresholds. Therefore, it would not be that different from tuning the threshold of a novelty detector, or a one-class classifier. It would have strengthened the paper if the approach was compared also to a novelty detector.
It is not clear if the fully convolutional OOD detector is working on a patch or on the entire image. If it is a patch, of what size?
Page 4, define the “ID” acronym.


ANALYSING MATHEMATICAL REASONING ABILITIES OF NEURAL MODELS
Mathematical reasoning-a core ability within human intelligence-presents some unique challenges as a domain: we do not come to understand and solve mathematical problems primarily on the back of experience and evidence, but on the basis of inferring, learning, and exploiting laws, axioms, and symbol manipulation rules. In this paper, we present a new challenge for the evaluation (and eventually the design) of neural architectures and similar system, developing a task suite of mathematics problems involving sequential questions and answers in a free-form textual input/output format. The structured nature of the mathematics domain, covering arithmetic, algebra, probability and calculus, enables the construction of training and test splits designed to clearly illuminate the capabilities and failure-modes of different architectures, as well as evaluate their ability to compose and relate knowledge and learned processes. Having described the data generation process and its potential future expansions, we conduct a comprehensive analysis of models from two broad classes of the most powerful sequence-to-sequence architectures and find notable differences in their ability to resolve mathematical problems and generalize their knowledge.

Summary: This paper is about models for solving basic math problems. The main contribution is a synthetically generated dataset that includes a variety of types and difficulties of math problems; it is both larger and more varied than previous datasets of this type. The dataset is then used to evaluate a number of recurrent models (LSTM, LSTM+attention, transformer); these are very powerful models for general sequence-sequence tasks, but they are not explicitly tailored to math problems. The results are then analyzed and insights are derived explaining where neural models seemingly cope well with math tasks, and where they fall down.
Strengths: I am happy to see the proposal of a very large dataset with a lot of different axes for measuring and examining the performance of models. There are challenging desiderata involved in building the training+tests sets, and the authors have an interesting and involved methodology to accomplish these. The paper is very clearly written. I'm not aware of a comparable work, so the novelty here seems good.
Weaknesses: The dataset created here is entirely synthetic, and the paper only includes one single small real-world case; it seems like it would be easy to generate a larger and more varied real world dataset as well (possibly from the large literature of extant solved problems in workbooks). It would have been useful to compare the general models here with some specific math problem-focused ones as well. Some details weren't clear to me. More in the comments below.
Verdict: I thought this was generally an interesting paper that has some very nice benefits, but also has some weaknesses that could be resolved. I view it as borderline, but I'm willing to change my mind based on the discussion.

Comments:
- One area that could stand to be improved is prior work. I'd like to see more of a discussion of *prior data sets* rather than papers proposing models for problems. Since this is the core contribution, this should also be the main comparison. For example, EMLNP 2017 paper "Deep Neural Solver for Math Word Problems" mentions a size 60K problem dataset. A more extensive discussion will help convince the readers that the proposed dataset is indeed the largest and most diverse.
- The authors note that previous datasets are often specific to one type of problem (i.e., single variable equation solving). Why not then combine multiple types of extant problem sets?
- The authors divide dataset construction into crowdsourcing and synthetic. This seems incomplete to me: there are tens of thousands (probably more) of exercises and problems available in workbooks for elementary, middle, and high school students. These are solved, and only require very limited validation. They are also categorized by difficulty and area. Presumably the cost here would be to physically scan some of these workbooks, but this seems like a very limited investment. Why not build datasets based on workbooks, problem solving books, etc?
- How do are the difficulty levels synthetically determined?
- When generating the questions, the authors "first sample the answer". What's the distribution you use on the answer? This seems like it dramatically affects the resulting questions, so I'm curious how it's selected.
- The general methodology of generating questions and ensuring that no question is too rare or too frequent and the test set is sufficiently different---these are important questions and I commend the authors for providing a strong methodology.
- I didn't understand the motivation for testing only very general-purpose models (this is described in Section 3). This is certainly a scientific decision, i.e., the authors are determining which models to use in order to determine the possible insights they will derive. But it's not clear to me why testing more sophisticated models that are tailored for math questions would *not* be useful. In fact, assuming that such methods outperform general-purpose models, we could investigate why and where this is the case (in fact the proposed dataset is very useful for this). On the other hand, if these specialized approaches largely fail to outperform general-purpose models, we would have the opposite insights---that these models' benefits are dataset-specific and thus limited.
- Really would be good to do real-world tests in a more extensive way. A 40-question exam for 16 year olds is probably far too challenging for the current state of general recurrent models. Can you add some additional grades here, and more questions?
- For the number of thinkings steps, how does it scale up as you increase it from 0 to 16? Is there a clear relationships here?
- The 1+1+...+1 example is pretty intriguing, and could be a nice "default" question!
- Minor typo: in the abstract: "test spits" should be "test splits"


We study reinforcement learning algorithms with nonlinear function approximation in the online setting. By formulating both the problems of value function estimation and policy learning as bilevel optimization problems, we propose online Q-learning and actor-critic algorithms for these two problems respectively. Our algorithms are gradient-based methods and thus are computationally efficient. Moreover, by approximating the iterates using differential equations, we establish convergence guarantees for the proposed algorithms. Thorough numerical experiments are conducted to back up our theory.

The paper casts the problems of value learning and policy optimization, which can be problematic the non-linear setting, into the bilevel optimization framework. It proposes two novel algorithms with convergence guarantees. Although other works with similar guarantees exist, these algorithms are very appealing for their simplicity. A limited empirical evaluation is provided for the value-based method in Acrobot and Mountain Car and in the Atari games Pong and Breakout for the proposed bilevel Actor Critic.
There are a few missing references to similar, recent work, including Dai et al’s saddle-point algorithm (https://arxiv.org/pdf/1712.10285.pdf). Also, the claim that this is “the first attempt to study the convergence of online reinforcement learning algorithms with nonlinear function approximation” can’t be true (even replacing ‘attempt’ by ‘successfully’, there is e.g. Maei et al.’s nonlinear GTD paper, see below).
Although certainly interesting, the claims relating bilevel optimization and the target network are not completely right. E.g. Equation 3.6 as given is a hard constraint on omega. More explicitly, there are no guarantees that either network is the minimizer of the RHS quantity in 3.6.
The two-timescale algorithm is closer in spirit to the use of a target network, but in DQN and variants the target network is periodically reset, as opposed to what the presented theory would suggest. A different breed of “soft target” networks, which is more closely related to bilevel optimization has been used to stabilize training in DDPG (https://arxiv.org/abs/1509.02971).
There was some confusion for me on the first pass that you define two algorithms called ‘online Q-learning’ and ‘actor-critic’. Neither algorithm is actually that, and they should be renamed accordingly (perhaps ‘bilevel Q-Learning’ and ‘bilevel actor-critic’?). In particular, standard Q-Learning is online; and the actor-critic method does not minimize the Bellman residual (i.e. I believe the RHS of 3.8 is novel within policy-gradient methods).
Once we’re operating on a bounded space with continuous operators, Theorem 4.2 is not altogether surprising – a case of Brouwer’s fixed point theorem, short of the result that theta* = omega*, which is explained in the few lines below the theorem. While I do think Theorem 4.2 is important, it would be good to contrast it to existing results from the GTD family of approaches. Also, requiring that |Q_theta(s,a)| <= Qmax is a significant issue -- effectively this test fails for most commonly used value-based algorithms.
The empirical evaluation lacks any comparison to baselines and serves for little more than as a sanity check of the developed theory. This is probably the biggest weakness of the paper, and is unfortunate given the claim of relevance to e.g. deep RL.
Questions
Throughout, the assumption of the data being sampled on-policy is made without a clear argument as to why. Would the relaxation of this assumption affect the convergence results?
Can the authors provide an intuitive explanation if/why bilevel optimization is necessary?
Can you contrast your work with Maei et al., “Convergent Temporal-Difference Learning with Arbitrary Smooth Function Approximation”?
Suggestions
The discussion surrounding the target network should be improved. In particular, claiming that the DQN target network can be viewed “as the parameter of the upper level optimization subproblem” is a stretch from what is actually shown.
The paper was sometimes hard to follow, in part because the claims are not crisply made. I strongly encourage the authors to more clearly relate their results to existing work, and ensure that the names they use match common usage.
I would have liked to know more about bilevel optimization, what it aims to solve, and the tools used to do it. Instead all I found was very standard two time-scale methods, which was a little disappointing – I don’t think these have been found to work particularly well in practice. This is particularly relevant in the context of e.g. the target network question.
A proper empirical comparison to existing algorithms would significantly improve the quality and relevancy of this work. There are tons of open-source baselines out there, in particular good state of the art implementations. Modifying a standard implementation to optimize its target network along the lines of bilevel optimization should be relatively easy.
Revision:
I thank the authors for their detailed feedback, but still think the work isn't quite ready for publication. After reading the other reviews, I will decrease my score from 6 to 5. Some sticking points/suggestions:
- Some of my concerns remain unanswered. E.g. the actor-critic method 3.8 is driven by the Bellman residual, which is not the same as e.g. the MSPBE used with linear function approximation. There is no harm in proposing variations on existing algorithms, and I'm not sure why the authors are reluctant to do. Also, Brouwer's fixed point theorem, unlike Banach's, does not require a contractive mapping.
- The paper over-claims in a number of places. I highly recommend that the authors make their results more concrete by demonstrating the implications of their method on e.g. linear function approximation. This will also help contrast with Dai et al., etc.


DEFENSIVE QUANTIZATION: WHEN EFFICIENCY MEETS ROBUSTNESS
Neural network quantization is becoming an industry standard to efficiently deploy deep learning models on hardware platforms, such as CPU, GPU, TPU, and FPGAs. However, we observe that the conventional quantization approaches are vulnerable to adversarial attacks. This paper aims to raise people's awareness about the security of the quantized models, and we designed a novel quantization methodology to jointly optimize the efficiency and robustness of deep learning models. We first conduct an empirical study to show that vanilla quantization suffers more from adversarial attacks. We observe that the inferior robustness comes from the error amplification effect, where the quantization operation further enlarges the distance caused by amplified noise. Then we propose a novel Defensive Quantization (DQ) method by controlling the Lipschitz constant of the network during quantization, such that the magnitude of the adversarial noise remains non-expansive during inference. Extensive experiments on CIFAR-10 and SVHN datasets demonstrate that our new quantization method can defend neural networks against adversarial examples, and even achieves superior robustness than their fullprecision counterparts, while maintaining the same hardware efficiency as vanilla quantization approaches. As a by-product, DQ can also improve the accuracy of quantized models without adversarial attack.

Summary of paper
This paper presents an approach  for quantising neural networks such that the resulting quantised model is robust to adversarial and random perturbations.
The core idea of the paper is to enforce the Lipschitz constant of each linear layer of the network approximately close to 1. Since the Lipschitz constant of the neural network is bounded by the product of the
Lipschitz constant of its linear layer (assuming Lipschitz 1 activation functions) the Lipschitz constant of the trained neural network is bounded by 1. This results in a model which is robust to adversarial and random noise ad all directions in the model space are non-expansive. Algorithmically, controlling the Lipschitz constant is achieved by using the orthogonal regulariser presented in the paper Cisse et.al which has the same motivation for this work but for standard neural network training but not quantising. The authors presents thorough experimental study showing why standard quantisation schemes are prone to adversarial noise and demonstrate clearly how this approach improves robustness of quantised network and sometimes even improve over the accuracy of original model.
Review:
The paper is well written with clear motivation and very easy to follow.
The core idea of using orthogonal regulariser for improving the robustness of neural network models have been presented in Cisse et.al and the authors re-use it for improving the robustness of quantised models. The main contribution of this work is in identifying that the standard quantised models are very vulnerable to adversarial noise which is illustrated through experiments and then empirically showing that the regulariser presented in Cisse et. al improves the robustness of quantised models with rigorous experiments. The paper add value to the research community through thorough experimental study as well as in industry since quantised models are widely used and the presented model is simple and easy to use.
Some suggestions and ideas:
1. It will be great if the authors could add a simple analytical explanation why the quantised networks are not robust.
2. The manifold of Orthogonal matrices does not include all 1 - Lipschitz matrices and also the Orthogonal set is not convex. I think a better strategy for this problem is to regularise the spectral norm to be 1.  Regularising the spectral norm is computationally cheaper than Orthogonal regulariser when combined with SGD using power iterations.  Moreover the regulariser part of the model becomes nice and convex.
3. Another strategy to control the Lipschitz constant of the network is to directly penalise the norm of the Jacobian as explained in Improved Training of Wasserstein GANs (Gulrajani et. al).


We introduce a novel embedding of directed graphs derived from the singular value decomposition (SVD) of the normalized adjacency matrix. Specifically, we show that, after proper normalization of the singular vectors, the distances between vectors in the embedding space are proportional to the mean commute times between the corresponding nodes by a forward-backward random walk in the graph, which follows the edges alternately in forward and backward directions. In particular, two nodes having many common successors in the graph tend to be represented by close vectors in the embedding space. More formally, we prove that our representation of the graph is equivalent to the spectral embedding of some co-citation graph, where nodes are linked with respect to their common set of successors in the original graph. The interest of our approach is that it does not require to build this co-citation graph, which is typically much denser than the original graph. Experiments on real datasets show the efficiency of the approach.

# Summary of the paper
This paper proposes an embedding of directed graphs based on the SVD of a normalized adjacency matrix. This embedding is shown to be equivalent to the spectral embedding of a co-citation graph, which is more complex to calculate. Interestingly, the proposed approach does not require the *explicit* representation of this graph. Moreover, the paper also shows that distances of the embedded vectors are proportional to mean commute times of a forward--backward random walk in the original graph. A suite of experiments is run on graphs from KONECT.
# Review
This is a well-written paper, which I enjoyed reading. The extension of embeddings to the case of directed graphs is significant and warrants a detailed exploration.
The principal issues I see with this paper are as follows:
- The originality or scope of the contribution is not clear
- The experimental section is uncompelling
- Several relevant works appear to have been ignored
Overall, I like the way the paper treats the subject. In particular, I appreciate the fact that proofs are explained well; additionally, code is provided, which will increase reproducibility. This is uncommon and praiseworthy!
As for the originality of the paper, I find it hard to judge the scope of the contribution. The paper is extremely well written and employs a very pedagogical treatment of the subject, which I appreciate. Yet, it is hard for me to judge the utility and novelty of the proposed method in light of Section 8, where the paper shows that a spectral embedding of the undirected variant of the graph leads to essentially the *same* eigenvectors (up to renormalization and permutations). To prove that a new method is more effective, this point should be emphasized more:
1. In a sense, I would see the results from Section 8 as the equal to what 'Laplacian Eigenmaps' (LE) yields. This needs to be stressed, and analysed in an experimental section.
2. I understand that the order of the singular vectors is different, so embeddings that use only parts of them will be different. However, a convincing experiment should assess the differences. For example, in which regime for $d$ (number of used vectors) will the new method be surpassed by the old one? Is there such a regime? Ideally, this will be answered in the form of an asymptotic theorem; it could also be a larger experiment, though (to simulate the conditions in practice).
3. I understand that the new approach has a lower run-time, because the SVD is more efficient than eigendecomposition. However, what about a simple baseline algorithm that uses SVD for the *undirected* graph of the input data? This should be simple to accomplish, and would be a way to ascertain the benefit of using edge directions.
  To my understanding, LE should be this embedding, but from the table, I can see that its runtime is a lot worse than the novel method. What causes this? The fact that eigendecomposition is used instead of SVD?  If so, an additional SVD-based approach should be implemented.
This brings me to the experimental section. Here, the paper demonstrates the superiority of the new embedding based on evaluating modularity of a set of different clusterings of larger graphs, obtained using $k$-means. I have several concerns about this:
1. Modularity has problems with larger networks because only a small part of the network will be used in its configuration.
2. Since the embeddings cannot be easily compared due to missing ground truth information, other metrics should be employed. Here are a few, which are often used  by the community. See 'Is there a best quality metric for graph clusters?' by Almeida et al. for more details and a description of their shortcomings:
    - Silhouette coefficient
    - Coverage
    - Conductance
  Different ones should be evaluated here in order to show the behaviour of the new embedding. Do the embeddings differ if the modularities are similar?
3. How do the results change for different values of $d$? I find it hard to disentangle such a discussion from instabilities in $k$-means, but to my understanding of the method, tuning $d$ means that more or less information is used from the singular vectors.
   This could also be quantified in a proof (about asymptotic behaviour) but an experiment would be equally fine.
Concerning the bibliography, or the treatment of prior works, there are some issues:
- There appear to be some missing references of earlier works that used SVD or variants in order to cluster graphs or embed them:
  - Drineas et al.: 'Clustering Large Graphs via the Singular Value Decomposition'
  - Malliaros and Vazirgiannis: 'Clustering and Community Detection in Directed Networks: A Survey'
- Likewise, the use of pseudo-inverse Laplacians has a lot more papers attached to it (these are only a few that are relevant):
  - Ho and Dooren: 'On the pseudo-inverse of the Laplacian of a bipartite graph'
  - Gutman and Xia: 'Generalized inverse of the Laplacian matrix and some applications'
# Suggestions for improvement
- In some sense, this work can be seen as an extension of Laplacian eigenmaps to the directed case. The paper needs to be more clear about these extensions with respect to prior work. In Section 2, it is claimed that 'our main contribution is a proper normalization'. This strikes me as a rather small contribution in light of the experimental section, as outlined above.
- I am also hesitant to speak about a better interpretability of the mean commute time. I agree that it is nice to know that the distance permits such an interpretation in terms of random walks, but what is the impact of knowing the MCT? It is not only used in the embedding insofar as one obtains a vector representation.
- Section 5 is then the standard way of defining random walks based on a Laplacian matrix, and the correspondence to the pseudo-inverse of the Laplacian is shown. This is mathematically interesting, but appears to me to be in line with previous research.
- The section about co-citation graphs should make it more clear that 'successors' are to be taken in terms of the original graph and the directionality of edges. Since this is a standard definition in the domain of network analysis I would suggest citing a textbook here.
- In Section 6, the paper could give more details about random walk concepts such as 'stochastic', 'stationary distribution' etc., as it would make the paper more accessible (I am familiar with these concepts but since the writing of the paper is of high quality in the other sections, I am convinced this would improve its impact, and attract more readers).
Typos & grammar issues:
- 'in terms of random walk' --> 'in terms of random walks'
- 'equivalent to build' --> 'equivalent to building'
- 'with corresponding unitary matrix' --> 'with a corresponding unitary
  matrix'
- 'square Euclidean distance' --> 'squared Euclidean distance'
- 'equivalent to consider' --> 'equivalent to considering'
- 'irreductible' --> 'irreducible'
- 'and provide generally' --> 'and provides generally'
- 'in low dimension' --> 'in a lower dimension'
Furthermore, the bibliography should employ consistent capitalization and journal names for articles.


A significant roadblock in multilingual neural language modeling is the lack of labeled non-English data. One potential method for overcoming this issue is learning cross-lingual text representations that can be used to transfer the performance from training on English tasks to non-English tasks, despite little to no task-specific non-English data. In this paper, we explore a natural setup for learning crosslingual sentence representations: the dual-encoder. We provide a comprehensive evaluation of our cross-lingual representations on a number of monolingual, crosslingual, and zero-shot/few-shot learning tasks, and also give an analysis of different learned cross-lingual embedding spaces.

This paper proposes a novel cross-lingual multi-tasking framework based on a dual-encoder model that can learn cross-lingual sentence representations which are useful in monolingual tasks and cross-lingual tasks for both languages involved in the training, as observed on the experiments for three language pairs. The main idea of the approach is to model all tasks as input-response ranking tasks and introduce cross-lingual representation tying through the translation ranking task, introduced by Guo et al. (2018). All components of the framework are quite standard and deja-vu, but I like the paper in general, and the results seem quite encouraging. I have several comments on how to further strengthen the paper and improve the presentation of the main findings.
The proposed framework does not offer any substantial modeling contribution (i.e., all major components are based on SOTA models), but the framework is still quite interesting as a mixture of these SOTA components. I believe that some additional experiments would make the main contributions clearer and would also provide additional insights into the main properties of the proposed framework: 1) cross-linguality and 2) multi-tasking.
*Most of all, I am surprised not to see any ablation studies. For instance, what happens if we remove one of the two monolingual tasks in each language? How does that reduced model compare to the full model? Which monolingual task is more beneficial for the final performance in downstream tasks? Can we think of adding another monolingual task to boost performance further? I think that this sort of experiment would be more beneficial for the paper than a pretty long analysis from Section 5 (this analysis is still valid, but should be shortened substantially). Evaluating only multi-tasking without any cross-lingual training would also be very beneficial to recognise the extent of improvement achieved by adding cross-linguality to the model.
*How much does the proposed architecture depend on the choice of the encoding model for the function g? Have the authors experimented with other (recent and (near-)SOTA) encoding models? I would like to see a comparative analysis of this 'hyper-parameter'.
*I would like to see more experiments on more distant language pairs. This would make the paper even more interesting imho. I am also curious whether there would be a drop in performance reported conditioned on the distance/proximity between two languages in a language pair.
*I would like to see a more detailed description of the two best performing STS systems (ECNU and BIT). In what respect are these systems state-of-the-art feature engineered and mixed? I am not sure what this means without providing any additional context to the claim and description.
*How does the monolingual English STS model trained with the cross-lingual multi-task framework compare to the work of Conneau et al. (EMNLP 2017) which also used SNLI as the task on which to learn universal sentence representations. This would be a good experiment imho as it would show how much we gain from cross-lingual training and multi-tasking.
Minor:
*Page 3: Could you add a short footnote discussing how hard-negatives for the translation ranking task are selected? How do you compute similarity here?
*Do you expect performance to improve further by training MultiNLI instead of SNLI (or combining the two datasets)?
*"All hyperparameters are tuned based on preliminary experiments on a development set." -> What is used as the development set? More details needed.
*"Finally, as an additional training heuristic, we multiply the gradients to the word and character embeddings by a factor of 100." -> How is the value for the embedding gradient multiplier determined? Is there an automatic procedure to fine-tune this hyper-parameter or has this been done in a completely empirical way?
*Table 1: please define the task abbreviations before showing them in the table. It is not clear what each task is by relying only on the abbreviation.
*This dataset was not available at the time of the submission, but for the revision it would make sense to also evaluate on the new XNLI dataset of Conneau et al. (EMNLP 2018) for multilingual NLI experiments.
(After the first revision) I have raised the score after the very detailed author response (thanks for that!), but this is also conditioned on the authors making the actual revisions promised in their response. I am still quite interested to check how well the method works in a setup with distant language pairs.


The gradient of a deep neural network (DNN) w.r.t. the input provides information that can be used to explain the output prediction in terms of the input features and has been widely studied to assist in interpreting DNNs. In a linear model (i.e., g(x) = wx + b), the gradient corresponds solely to the weights w. Such a model can reasonably locally linearly approximate a smooth nonlinear DNN, and hence the weights of this local model are the gradient. The other part, however, of a local linear model, i.e., the bias b, is usually overlooked in attribution methods since it is not part of the gradient. In this paper, we observe that since the bias in a DNN also has a non-negligible contribution to the correctness of predictions, it can also play a significant role in understanding DNN behaviors. In particular, we study how to attribute a DNN's bias to its input features. We propose a backpropagation-type algorithm "bias back-propagation (BBp)" that starts at the output layer and iteratively attributes the bias of each layer to its input nodes as well as combining the resulting bias term of the previous layer. This process stops at the input layer, where summing up the attributions over all the input features exactly recovers b. Together with the backpropagation of the gradient generating w, we can fully recover the locally linear model g(x) = wx + b. Hence, the attribution of the DNN outputs to its inputs is decomposed into two parts, the gradient w and the bias attribution, providing separate and complementary explanations. We study several possible attribution methods applied to the bias of each layer in BBp . In experiments, we show that BBp can generate complementary and highly interpretable explanations of DNNs in addition to gradient-based attributions.

Summary:
The paper's main contribution is to attribute the bias term seen at the output to each of the input dimension appropriately.  The other claim is that together with gradient information , this could enhance existing explainability methods.

The paper considers DNNs with piecewise linear activation functions. Then the final DNN output is a piece linear function. So for any given point x, the point lies in one of the linear pieces and there fore, can be written as a linear model. The gradient term can be computed using back propagation methods (although back propagating keeping the weights fixed and keeping the input as the variable). However there are no know existing works that attribute the bias of the linear piece at the final layer to the input dimensions. This paper provides a method to do it such that when you add the vector contribution of all the dimension in the input - it results in the bias vector at the output layer.
The basic idea is to distribute dimension wise bias attribution  at layer \ell to layer \ell-1 by using N_{\ell} x N_{\ell-1} attribution matrix where N_{\ell} is the dimension of layer \ell. This is done using two methods - one using exponential weights and the other using some sort of variance measure from a fixed average bias.
The authors then show using examples from STL-10 and Imagenet datasets, how the gradients and biases attributed to the inputs compare for explanation purposes.
Strengths:
The notion of attributing final layer biases to input layer is novel. Its important given that it carries important information regarding the final classification output.
Weaknesses:
a) This paper lacks quite a bit on comparison with existing work. For example, LRP (layer wise relevance propagation) has been referred to by the authors. However, there is no comparison with LRP heat maps. This website - http://www.explain-ai.org/ - documents state of the LRP methods with code, videos, papers (some of which have been cited by the authors). I think the authors could produce heat map produced by LRP for all these different examples in page 8. For instance, pls look at Image A in Fig .2 in https://arxiv.org/pdf/1708.08296.pdf. There is 'cup' and a 'volcano' in the same picture. The paper compares gradient based heatmaps and LRP based ones. LRP based ones are very crisp (for this image of course). LRPcode is readily available from a well maintained project page - http://www.explain-ai.org/. Will the heat maps of grad/bias look crisper than LRP ? Other papers and more examples of LRP can be found at http://www.explain-ai.org/
LRP takes the final probability weight at the output layer and assigns recursively to other neurons in the penultimate very similar to what the current paper does for bias. However, the attribution mechanism (the weights) are different. A couple of variations are explored in this survey (https://arxiv.org/pdf/1708.08296.pdf).
b)  This point is related to the first- The authors say "Therefore, the interpretation of the DNN’s behavior on the input data should be exclusive to the information embedded in the linear model." - I disagree a little bit here. It is true that behavior of the DNN on the input is exclusive to the linear piece. However, interpretation of the behavior/ explanation of it is another matter. For example, I quote two methods in the literature that have been used for explanation of an input sample - but does not use the linear piece or the gradient information.
     1)  Pls look at - https://arxiv.org/pdf/1703.02647.pdf - (Figure 3 + Section A.8 in the appendix). The paper used streaming submodularity algorithms to actually assemble a part of the image with everything else "blacked out" to determine which sparse parts of the image are responsible for the final output. They have exhaustive comparisons with LIME too. These methods actually rely on behavior of DNN far away from the actual x to explain the behavior at x. "Zeroing out irrelevant" parts is one of the ways of explaining adopted by LIME and these approaches. Ideally the authors should compare with these too.

 2) In this paper - https://arxiv.org/abs/1802.07623 - authors provide pertinent negatives - what should "not be" there in the image so that the label does not change - In fact for MNIST data, this produces very interesting additional explanations that is not produced by methods (including LRP and LIME) that rely on things in the image. Again the explanation is not at all related to the linear piece.
In general, linear pieces are so close by , the nearby movement and possibly change of gradients can also provide useful explanation of the behavior.
Overall - at least the authors must discuss the above references + also survey works that highlights "relevant parts of the image" like LIME and streaming sub modularity etc. Further an actual comparison to LRP (code is easily available) is crucial to evaluate the efficacy of the proposed methods given that the authors of LRP have compared with gradient based methods. Comparison with LIME would also be interesting and desirable.