Recurrent Neural Networks architectures excel at processing sequences by modelling dependencies over different timescales. The recently introduced Recurrent Weighted Average (RWA) unit captures long term dependencies far better than an LSTM on several challenging tasks. The RWA achieves this by applying attention to each input and computing a weighted average over the full history of its computations. Unfortunately, the RWA cannot change the attention it has assigned to previous timesteps, and so struggles with carrying out consecutive tasks or tasks with changing requirements. We present the Recurrent Discounted Attention (RDA) unit that builds on the RWA by additionally allowing the discounting of the past. We empirically compare our model to RWA, LSTM and GRU units on several challenging tasks. On tasks with a single output the RWA, RDA and GRU units learn much quicker than the LSTM and with better performance. On the multiple sequence copy task our RDA unit learns the task three times as quickly as the LSTM or GRU units while the RWA fails to learn at all. On the Wikipedia character prediction task the LSTM performs best but it followed closely by our RDA unit. Overall our RDA unit performs well and is sample efficient on a large variety of sequence tasks.

The authors present RDA, the Recurrent Discounted Attention unit, that improves upon RWA, the earlier introduced Recurrent Weighted Average unit, by adding a discount factor. While the RWA was an interesting idea with bad results (far worse than the standard GRU or LSTM with standard attention except for hand-picked tasks), the RDA brings it more on-par with the standard methods.
On the positive side, the paper is clearly written and adding discount to RWA, while a small change, is original. On the negative side, in almost all tasks the RDA is on par or worse than the standard GRU - except for MultiCopy where it trains faster, but not to better results and it looks like the difference is between few and very-few training steps anyway. The most interesting result is language modeling on Hutter Prize Wikipedia, where RDA very significantly improves upon RWA - but again, only matches a standard GRU or LSTM. So the results are not strongly convincing, and the paper lacks any mention of newer work on attention. This year strong improvements over state-of-the-art have been achieved using attention for translation ("Attention is All You Need") and image classification (e.g., Non-local Neural Networks, but also others in ImageNet competition). To make the evaluation convincing enough for acceptance, RDA should be combined with those models and evaluated more competitively on multiple widely-studied tasks.


Large-scale publicly available datasets play a fundamental role in training deep learning models. However, large-scale datasets are difficult to collect in problems that involve processing of sensitive information. Collaborative learning techniques provide a privacy-preserving solution in such cases, by enabling training over a number of private datasets that are not shared by their owners. Existing collaborative learning techniques, combined with differential privacy, are shown to be resilient against a passive adversary which tries to infer the training data only from the model parameters. However, recently, it has been shown that the existing collaborative learning techniques are vulnerable to an active adversary that runs a GAN attack during the learning phase. In this work, we propose a novel keybased collaborative learning technique that is resilient against such GAN attacks. For this purpose, we present a collaborative learning formulation in which class scores are protected by class-specific keys, and therefore, prevents a GAN attack. We also show that very high dimensional class-specific keys can be utilized to improve robustness against attacks, without increasing the model complexity. Our experimental results on two popular datasets, MNIST and AT&T Olivetti Faces, demonstrate the effectiveness of the proposed technique against the GAN attack. To the best of our knowledge, the proposed approach is the first collaborative learning formulation that effectively tackles an active adversary, and, unlike model corruption or differential privacy formulations, our approach does not inherently feature a trade-off between model accuracy and data privacy.

In this paper, the authors proposed a counter measure to protect collaborative training of DNN against the GAN attack in (Hitaj et al. 2017). The motivation of the paper is clear and so is the literature review. But for me the algorithm is not clearly defined and it is difficult to evaluate how the proposed procedure works. I am not saying that this is not the solution. I am just saying that the paper is not clear enough to say that it is (or it is not). From, my perspective this will make the paper a clear reject.
I think the authors should explain a few things more clearly in order to make the paper foolproof. The first one seems to me the most clear problem with the approach proposed in the paper:
1 $\psi(c)$ defines the mapping from each class to a high dimensional vector that allows protection against the GAN attack. $\psi(c)$ is suppose to be private for each class (or user if each class belong only to one user). This is the key aspect in the paper. But if more than one user have the same class they will need to share this key. Furthermore, at test time, these keys need to be known by everyone, because the output of the neural network needs to be correlated against all keys to see which is the true label. Of course the keys can only be released after the training is completed. But the adversary can also claim to have examples from the class it is trying to attack and hence the legitimate user that generated the key will have to give the attacker the key from the training phase. For example, let assume the legitimate user only has ones from MNIST and declares that it only has one class. The attacker says it has two classes the same one that the legitimate user and some other label. In this case the legitimate user needs to share $\psi(c)$ with the attacker. Of course this sounds “fishy” and might be a way of finding who the attacker is, but there might be many cases in which it makes sense that two or more users shares the same labels and in a big system might be complicated to decide who has access to which key.
2 I do not understand the definition of $\phi(x)$. Is this embedding fixed for each user? Is this embedding the DNN? In Eq. 4 I would assume that $\phi(x)$ is the DNN and that it should be $\phi_\theta(x)$, because otherwise the equation does not make sense. But this is not clearly explained in the paper and Eq 4 makes no sense at all. In a way the solution to the maximization in Eq 4 is Theta=\infty. Also the term $\phi(x)$ is not mentioned in the paper after page 5. My take is that the authors want to maximize the inner product, but then the regularizer should go the other way around.
3 In the paper in page 5 we can read: “Here, we emphasize the first reason why it is important to use l2-normalized class keys and embedding outputs: in this manner, the resulting classification score is by definition restricted to the range [-1; +1],” If I understand correctly the authors are dividing the inner product by ||$\psi(c)|| ||$\phi(x)||. I can see that we can easily divide by ||$\psi(c)||, but I cannot see how we can do dive by ||$\phi(x)||, if this term depends on \theta. If this term does not depend on \theta, then Eq 4 does not make sense.
To summarize, I have the impression that there are many elements in the paper that does not makes sense in the way that they are explained and that the authors need to tell the paper in a way that can be easily understood and replicated. I recommend the authors to run the paper by someone in their circle that could help them rewrite the paper in a way that is more accessible.


Workshop track -ICLR 2018 TREE-TO-TREE NEURAL NETWORKS FOR PROGRAM TRANSLATION
Program translation is an important tool to migrate legacy code in one language into an ecosystem built in a different language. In this work, we are the first to consider employing deep neural networks toward tackling this problem. We observe that program translation is a modular procedure, in which a sub-tree of the source tree is translated into the corresponding target sub-tree at each step. To capture this intuition, we design a tree-to-tree neural network as an encoderdecoder architecture to translate a source tree into a target one. Meanwhile, we develop an attention mechanism for the tree-to-tree model, so that when the decoder expands one non-terminal in the target tree, the attention mechanism locates the corresponding sub-tree in the source tree to guide the expansion of the decoder. We evaluate the program translation capability of our tree-to-tree model against several state-of-the-art approaches. Compared against other neural translation models, we observe that our approach is consistently better than the baselines with a margin of up to 15 points. Further, our approach can improve the previous state-of-the-art program translation approaches by a margin of 20 points on the translation of real-world projects.

This paper aims to translate source code from one programming language to another using
a neural network architecture that maps trees to trees. The encoder uses an upward pass of
a Tree LSTM to compute embeddings for each subtree of the input, and then the decoder
constructs a tree top-down. As nodes are created in the decoder, a hidden state is passed
from parents to children via an LSTM (one for left children, one for right children), and
an attention mechanism allows nodes in the decoder to attend to subtrees in the encoder.
Experimentally, the model is applied to two synthetic datasets, where programs in the
source domain are sampled from a PCFG and then translated to the target domain with a
hand-coded translator. The model is then trained on these pairs. Results show that the
nproposed approach outperforms sequence representations or serialized tree representations
of inputs and outputs.
Pros:
- Nice model which seems to perform well.
- Reasonably clear explanation.
A couple questions about the model:
- the encoder uses only bottom-up information to determine embeddings of subtrees. I wonder
if top-down information would create embeddings with more useful information for the attention
in the decoder to pick up on.
- I would be interested to know more details about how the hand-coded translator works. Does
it work in a context-free, bottom-up fashion? That is, recursively translate two children nodes
and then compute the translation of the parent as a function of the parent node and
translations of the two children? If so, I wonder what is missing from the proposed model
that makes it unable to perfectly solve the first task?
Cons:
- Only evaluated on synthetic programs, and PCFGs are known to generate unrealistic programs,
so we can only draw limited conclusions from the results.
- The paper overstates its novelty and doesn't properly deal with related work (see below)
The paper overstates its novelty and has done a poor job researching related work.
Statements like "We are the first to consider employing neural network approaches
towards tackling the problem [of translating between programming languages]" are
obviously not true (surely many people have *considered* it), and they're particularly
grating when the treatment of related  work is poor, as it is in this paper. For example,
there are several papers that frame the code migration problem as one of statistical
machine translation (see Sec 4.4 of [1] for a review and citations), but this paper
makes no reference to them. Further, [2] uses distributed representations for the purpose
of code migration, which I would call a "neural network approach," so there's not any
sense that I can see in which this statement is true. The paper further says, "To the best
of our knowledge, this is the first tree-to-tree neural network architecture in the
literature." This is worded better, but it's definitely not the first tree-to-tree
neural network. See, e.g., [3, 4, 5], one of which is cited, so I'm confused about
this claim.
In total, the model seems clean and somewhat novel, but it has only been tested on
unrealistic synthetic data, the framing with respect to related work is poor, and the
contributions are overstated.
[1] https://arxiv.org/abs/1709.06182
[2]  Trong Duc Nguyen, Anh Tuan Nguyen, and Tien N Nguyen. 2016b. Mapping API elements for code migration with
vector representations. In Proceedings of the International Conference on Software Engineering (ICSE).
[3] Socher, Richard, et al. "Semi-supervised recursive autoencoders for predicting sentiment distributions." Proceedings of the conference on empirical methods in natural language processing. Association for Computational Linguistics, 2011.
[4] https://arxiv.org/abs/1703.01925
[5] Parisotto, Emilio, et al. "Neuro-symbolic program synthesis." arXiv preprint arXiv:1611.01855 (2016).


In recent years deep reinforcement learning has been shown to be adept at solving sequential decision processes with high-dimensional state spaces such as in the Atari games. Many reinforcement learning problems, however, involve highdimensional discrete action spaces as well as high-dimensional state spaces. In this paper, we develop a novel policy gradient methodology for the case of large multidimensional discrete action spaces. We propose two approaches for creating parameterized policies: LSTM parameterization and a Modified MDP (MMDP) giving rise to Feed-Forward Network (FFN) parameterization. Both of these approaches provide expressive models to which backpropagation can be applied for training. We then consider entropy bonus, which is typically added to the reward function to enhance exploration. In the case of high-dimensional action spaces, calculating the entropy and the gradient of the entropy requires enumerating all the actions in the action space and running forward and backpropagation for each action, which may be computationally infeasible. We develop several novel unbiased estimators for the entropy bonus and its gradient. Finally, we test our algorithms on two environments: a multi-hunter multi-rabbit grid game and a multi-agent multi-arm bandit problem.

In this paper, the authors suggest introducing dependencies between actions in RL settings with multi-dimensional action spaces by way of two mechanisms (using an RNN and making partial action specification as part of the state); they then introduce entropy pseudo-rewards whose maximization corresponding to joint entropy maximization.
In general, the multidimensional action methods either seem incremental or non novel to me. The combined use of the chain rule and RNNs (LSTM or not) to induce correlations in multi-dimensional outputs is well know (sequence-to-sequence networks, pixelRNN, etc.) and the extension to RL presents no difficulties, if it is not already known. Note very related work in https://arxiv.org/pdf/1607.07086.pdf and https://www.media.mit.edu/projects/improving-rnn-sequence-generation-with-rl/overview/ .
As for the MMDP technique, I believe it is folklore (it can for instance be found as a problem in a problem set - http://stellar.mit.edu/S/course/2/sp04/2.997/courseMaterial/topics/topic2/readings/problemset4/problemset4.pdf). Note that both approaches could be combined; the first idea is essentially a policy method, the second, a value method. The second method could be used to provide stronger, partial action-conditional baselines (or even critics) to the first method.
The entropy derivation are more interesting - and the smoothed entropy technique is as far as I know, novel. The experiments are well done, though on simple toy environments.
Minor:
- In section 3.2, one should in principle tweak the discount factor of the modified MDP to recover behavior identical to the original one with large action space. This should be noted (alternatively, the discount between non-environment transitions should be set to 1).
- From the description at the end of 3.2, and figure 1.b, it seems actions fed to the MMDP feed-forward network are not one-
hot; I thought this was pretty surprising as it would almost certainly affect performance? Note also that the collection of feed-forward network which collectively output the joint vector can be thought of as an RNN with non-learned state transition.
- Since the function optimized can be written as an expectation of reward+pseudo-reward, the proof of theorem 4 can be simplified by using generic score-function optimization arguments (see Stochastic Computation Graphs, Schulman et al).


VITERBI-BASED PRUNING FOR SPARSE MATRIX WITH FIXED AND HIGH INDEX COMPRESSION RATIO
Weight pruning has proven to be an effective method of reducing the model size and computation cost without sacrificing its model accuracy. Conventional sparse matrix formats, however, involve irregular index structures with large storage requirement and a sequential reconstruction process, resulting in inefficient use of highly parallel computing resources. Hence, pruning is usually restricted to inference with a batch size of one, for which an efficient parallel matrix-vector multiplication method exists. In this paper, a new class of sparse matrix representation is proposed utilizing the Viterbi algorithm that has a high, and more importantly, fixed index compression ratio regardless of the pruning rate. In this approach, numerous sparse matrix candidates are first generated by the Viterbi encoder, and the candidate that aims to minimize the model accuracy degradation is then selected by the Viterbi algorithm. The model pruning process based on the proposed Viterbi encoder and Viterbi algorithm is highly parallelizable, and can be implemented efficiently in hardware to achieve low-energy and a high-performance index decoding process. Compared with the existing magnitude-based pruning methods, the index data storage requirement can be further compressed by 85.2% in MNIST and 83.9% in AlexNet while achieving a similar pruning rate. Even compared with the relative index compression technique, our method can still reduce the index storage requirement by 52.7% in MNIST and 35.5% in AlexNet.

The paper proposes VCM, a novel way to store sparse matrices that is based on the Viterbi Decompressor. Only a subset of sparse matrices can be represented in the VCM format, however, unlike CSR format, it allows for faster parallel decoding and requires much less index space. The authors also propose a novel method of pruning of neural network that constructs an (sub)optimal (w.r.t. a weight magnitude based loss) Viterbi-compressed matrix given the weights of a pretrained DNN.
VCM is an interesting analog to the conventional CSR format that may be more computationally efficient given particular software and/or hardware implementations of the Viterbi Decompressor. However, the empirical study of possible acceleration remains as an open question.
However, I have a major concern regarding the efficiency of the pruning procedure. Authors report practically the same level of sparsity, as the pruning procedure from the Deep Compression paper. Both the proposed Viterbi-based pruning, and Deep Compression pruning belong to the previous era of pruning methods. They separate the pruning procedure and the training procedure, so that the model is not trained end-to-end. However, during the last two years a lot of new adaptive pruning methods have been developed, e.g. Dynamic Network Surgery, Soft Weight Sharing, and Sparse Variational DropOut. All of them in some sense incorporate the pruning procedure into the training procedure and achieve a much higher level of sparsity (e.g. DC achieves ~13x compression of LeNet5, and SVDO achieves ~280x compression of the same network). Therefore the reported 35-50% compression of the index storage is not very significant.
It is not clear whether it is possible to take a very sparse matrix and transform it into the VCM format without a high accuracy degradation. It is also not clear whether the VCM format would be efficient for storage of extremely sparse matrices, as they would likely be more sensitive to the mismatch of the original sparsity mask, and the best possible VCM sparsity mask. Therefore I’m concerned whether it would be possible to achieve a close-to-SotA level of compression using this method, and it is not yet clear whether this method can be used for practical acceleration or not.
The paper presents an interesting idea that potentially has useful applications, however the experiments are not convincing enough.


Deep neural networks trained on large supervised datasets have led to impressive results in recent years. However, since well-annotated datasets can be prohibitively expensive and time-consuming to collect, recent work has explored the use of larger but noisy datasets that can be more easily obtained. In this paper, we investigate the behavior of deep neural networks on training sets with massively noisy labels. We show on multiple datasets such as MINST, CIFAR-10 and ImageNet that successful learning is possible even with an essentially arbitrary amount of noise. For example, on MNIST we find that accuracy of above 90 percent is still attainable even when the dataset has been diluted with 100 noisy examples for each clean example. Such behavior holds across multiple patterns of label noise, even when noisy labels are biased towards confusing classes. Further, we show how the required dataset size for successful training increases with higher label noise. Finally, we present simple actionable techniques for improving learning in the regime of high label noise.

The paper makes a bold claim, that deep neural networks are robust to arbitrary level of noise. It also implies that this would be true for any type of noise, and support this later claim using experiments on CIFAR and MNIST with three noise types: (1) uniform label noise (2) non-uniform but image-independent label noise, which is named "structured noise", and (3) Samples from out-of-dataset classes. The experiments show robustness to these types of noise.
Review:
The claim made by the paper is overly general, and in my own experience incorrect when considering real-world-noise. This is supported by the literature on "data cleaning" (partially by the authors), a procedure which is widely acknowledged as critical for good object recognition.  While it is true that some image-independent label noise can be alleviated in some datasets, incorrect labels in real world datasets can substantially harm classification accuracy.
It would be interesting to understand the source of the difference between the results in this paper and the more common results (where label noise damages recognition quality). The paper did not get a chance to test these differences, and I can only raise a few hypotheses. First, real-world noise depends on the image and classes in a more structured way. For instance, raters may confuse one bird species from a similar one, when the bird is photographed from a particular angle. This could be tested experimentally, for example by adding incorrect labels for close species using the CUB data for fine-grained bird species recognition.  Another possible reason is that classes in MNIST and CIFAR10 are already very distinctive, so are more robust to noise. Once again, it would be interesting for the paper to study why they achieve robustness to noise while the effect does not hold in general.
Without such an analysis, I feel the paper should not be accepted to ICLR because the way it states its claim may mislead readers.
Other specific comments:
-- Section 3.4 the experimental setup, should clearly state details of the optimization, architecture and hyper parameter search. For example, for Conv4, how many channels at each layer? how was the net initialized? which hyper parameters were tuned and with which values? were hyper parameters tuned on a separate validation set? How was the train/val/test split done, etc. These details are useful for judging technical correctness.
-- Section 4, importance of large datasets. The recent paper by Chen et al (2017) would be relevant here.
-- Figure 8 failed to show for me.
-- Figure 9,10, need to specify which noise model was used.


This paper introduces the concept of continuous convolution to neural networks and deep learning applications in general. Rather than directly using discretized information, input data is first projected into a high-dimensional Reproducing Kernel Hilbert Space (RKHS), where it can be modeled as a continuous function using a series of kernel bases. We then proceed to derive a closed-form solution to the continuous convolution operation between two arbitrary functions operating in different RKHS. Within this framework, convolutional filters also take the form of continuous functions, and the training procedure involves learning the RKHS to which each of these filters is projected, alongside their weight parameters. This results in much more expressive filters, that do not require spatial discretization and benefit from properties such as adaptive support and non-stationarity. Experiments on image classification are performed, using classical datasets, with results indicating that the proposed continuous convolutional neural network is able to achieve competitive accuracy rates with far fewer parameters and a faster convergence rate.

The paper introduces the notion of continuous convolutional neural networks.
The main idea of the paper is to project examples into an RK Hilbert space
and performs convolution and filtering into that space. Interestingly, the
filters defined in the Hilbert space  have parameters that are learnable.
While the idea may be novel and interesting, its motivation is not clear for
me. Is it for space? for speed? for expressivity of hypothesis spaces?
Most data that are available for learning are in discrete forms and hopefully,
they have been digitalized according to Shannon theory. This means that they bring
all necessary information for rebuilding their continuous counterpart. Hence, it is
not clear why projecting them back into continuous functions is of interest.
Another point that is not clear or at least misleading is the so-called Hilbert Maps.
As far as I understand, Equation (4) is not an embedding into an Hilbert space but
is more a proximity space representation [1]. Hence, the learning framework of the
authors can be casted more as a learning with similarity function than learning
into a RKHS [2]. A proper embedding would have mapped $x$ into a function
belonging to $\mH$. In addition, it seems that all computations are done
into a \ell^2 space instead of in the RKHS (equations 5 and 11).
Learning good similarity functions is also not novel [3] and Equations
(6) and (7) corresponds to learning these similarity functions.
As far as I remember, there exists also some paper from the nineties that
learn the parameters of RBF networks but unfortunately I have not been able to
google some of them.
Part 3 is the most interesting part of the paper, however it would have been
great if the authors provide other kernel functions with closed-form convolution
formula that may be relevant for learning.
The proposed methodology is evaluated on some standard benchmarks in vision. While
results are pretty good, it is not clear how the various cluster sets have been obtained
and what are their influence on the performances (if they are randomly initialized, it
would be great to see standard deviation of performances with respect to initializations).
I would also be great to have intuitions on why a single continuous filter works betters
than 20 discrete ones (if this behaviour is consistent accross initialization).
On the overall, while the idea may be of interested, the paper lacks in motivations
in connecting to relevant previous works and in providing insights on why it works.
However, performance results seem to be competitive and that's the reader may
be eager for insights.
minor comments
---------------
* the paper employs vocabulary that is not common in ML. eg. I am not sure what
occupancy values, or inducing points are.
* Supposingly that the authors properly consider computation in RKHS, then \Sigma_i
should be definite positive right? how update in (7) is guaranteed to be DP?
This constraints may not be necessary if instead they used proximity space representation.

[1] https://alex.smola.org/papers/1999/GraHerSchSmo99.pdf
[2] https://www.cs.cmu.edu/~avrim/Papers/similarity-bbs.pdf
[3] A. Bellet, A. Habrard and M. Sebban. Similarity Learning for Provably Accurate Sparse Linear Classification.


DECISION-BASED ADVERSARIAL ATTACKS: RELIABLE ATTACKS AGAINST BLACK-BOX MACHINE LEARNING MODELS
Many machine learning algorithms are vulnerable to almost imperceptible perturbations of their inputs. So far it was unclear how much risk adversarial perturbations carry for the safety of real-world machine learning applications because most methods used to generate such perturbations rely either on detailed model information (gradient-based attacks) or on confidence scores such as class probabilities (score-based attacks), neither of which are available in most real-world scenarios. In many such cases one currently needs to retreat to transfer-based attacks which rely on cumbersome substitute models, need access to the training data and can be defended against. Here we emphasise the importance of attacks which solely rely on the final model decision. Such decision-based attacks are (1) applicable to real-world black-box models such as autonomous cars, (2) need less knowledge and are easier to apply than transfer-based attacks and (3) are more robust to simple defences than gradient-or score-based attacks. Previous attacks in this category were limited to simple models or simple datasets. Here we introduce the Boundary Attack, a decision-based attack that starts from a large adversarial perturbation and then seeks to reduce the perturbation while staying adversarial. The attack is conceptually simple, requires close to no hyperparameter tuning, does not rely on substitute models and is competitive with the best gradient-based attacks in standard computer vision tasks like ImageNet. We apply the attack on two black-box algorithms from Clarifai.com. The Boundary Attack in particular and the class of decision-based attacks in general open new avenues to study the robustness of machine learning models and raise new questions regarding the safety of deployed machine learning systems. An implementation of the attack is available as part of Foolbox (https://github.com/bethgelab/foolbox).

This is a nice paper proposing a simple but effective heuristic for generating adversarial examples from class labels with no gradient information or class probabilities. Highly relevant prior work was overlooked and there is no theoretical analysis, but I think this paper still makes a valuable contribution worth sharing with a broader audience.
What this paper does well:
- Suggests a type of attack that hasn't been applied to image classifiers
- Proposes a simple heuristic method for performing this attack
- Evaluates the attack on both benchmark neural networks and a commercial system
Problems and limitations:
1. No theoretical analysis. Under what conditions does the boundary attack succeed or fail? What geometry of the classification boundaries is necessary? How likely are those conditions to hold? Can we measure how well they hold on particular networks?
Since there is no theoretically analysis, the evidence for effectiveness is entirely empirical. That weakens the paper and suggests an important area of future work, but I think the empirical evidence is sufficient to show that there's something interesting going. Not a fatal flaw.
2. Poor framing. The paper frames the problem in terms of "machine learning models" in general (beginning with the first line of the abstract), but it only investigates image classification. There's no particular reason to believe that all machine learning algorithms will behave like convolutional neural network image classifiers. Thus, there's an implicit claim of generality that is not supported.
This is a presentation issue that is easily fixed. I suggest changing the title to reflect this, or at least revising the abstract and introduction to make the scope clearer.
A minor presentation quibble/suggestion: "adversarial" is used in this paper to refer to any class that differs from the true class of the instance to be disguised. But an image of a dalmation that's labeled as a dalmation isn't adversarial -- it's just a different image that's labeled correctly. The adversarial process is about constructing something that will be mislabeled, exploiting some kind of weakness that doesn't show up on a natural distribution of inputs. I suggest rewording some of the mentions of adversarial.
3. Ignorance of prior work. Finding deceptive inputs using only the classifier output has been done by Lowd and Meek (KDD 2005) for linear classifiers and Nelson et al. (AISTATS 2010, JMLR 2012) for convex-inducing classifiers. Both works include theoretical bounds on the number of queries required for near-optimal adversarial examples. Biggio et al. (ECML 2013) further propose training a surrogate classifier on similar training data, using the predictions of the target classifier to relabel the training data. In this way, decision information from the target model is used to help train a more similar surrogate, and then attacks can be transferred from the surrogate to the target.
Thus, "decision-based attacks" are not new, although the algorithm and experiments in this paper are.
Overall, I think this paper makes a worthwhile contribution, but needs to revise the claims to match what's done in the paper and what's been done before.


Variational Autoencoder plays an important role in disentangled representation learning. However, it is found facing posterior collapse problem and learning multiple variants in one factor. What would be learned by variational autoencoder (VAE)  and what influence the disentanglement of VAE? This paper tries to preliminarily address VAE's intrinsic dimension, real factor, disentanglement and indicator issues theoretically in the idealistic situation and implementation issue practically through noise modeling perspective in the realistic case. On intrinsic dimension issue, due to information conservation, the idealistic VAE learns and only learns intrinsic factor dimension. Besides, suggested by mutual information separation property, the constraint induced by Gaussian prior to the VAE objective encourages the information sparsity in dimension. On disentanglement issue, subsequently, inspired by information conservation theorem the clarification on disentanglement in this paper is made. On real factor issue, due to factor equivalence, the idealistic VAE possibly learns any factor set in the equivalence class. On indicator issue, the behavior of current disentanglement metric is discussed, and several performance indicators regarding the disentanglement and generating influence are subsequently raised to evaluate the performance of VAE model and to supervise the used factors. On implementation issue, the experiments under noise modeling and constraints empirically testify the theoretical analysis and also show their own characteristic in pursuing disentanglement.

This paper proposes to modify how noise factors are treated when developing VAE models.  For example, the original VAE work from (Kingma and Welling, 2013) applies a deep network to learn a diagonal approximation to the covariance on the decoder side.  Subsequent follow-up papers have often simplified this covariance to sigma^2*I, where sigma^2 is assumed to be known or manually tuned.  In contrast, this submission suggests either treating sigma^2 as a trainable parameter, or else introducing a more flexible zero-mean mixture-of-Gaussians (MoG) model for the decoder noise.  These modeling adaptations are then analyzed using various performance indicators and empirical studies.
The primary issues I have with this work are threefold:  (i) The paper is not suitably organized/condensed for an ICLR submission, (ii) the presentation quality is quite low, to the extent that clarity and proper understanding are jeopardized, and (iii) the novelty is limited.  Consequently my overall impression is that this work is not yet ready for acceptance to ICLR.
First, regarding the organization, this submission is 19 pages long (*excluding* references and appendices), despite the clear suggestion in the call for papers to limit the length to 8 pages: "There is no strict limit on paper length. However, we strongly recommend keeping the paper at 8 pages, plus 1 page for the references and as many pages as needed in an appendix section (all in a single pdf). The appropriateness of using additional pages over the recommended length will be judged by reviewers."  In the present submission, the first 8+ pages contain minimal new material, just various background topics and modified VAE update rules to account for learning noise parameters via basic EM algorithm techniques.  There is almost no novelty here.  In my mind, this type of well-known content is in no way appropriate justification for such a long paper submission, and it is unreasonable to expect reviewers to wade through it all during a short review cycle.
Secondly, the presentation quality is simply too low for acceptance at a top-tier international conference (e.g., it is full of strange sentences like "Such amelioration facilitates the VAE capable of always reducing the artificial intervention due to more proper guiding of noise learning."  While I am sympathetic to the difficulties of technical writing, and realize that at times sufficiently good ideas can transcend local grammatical hiccups, my feeling is that, at least for now, another serious pass of editing is seriously needed.  This is especially true given that it can be challenging to digest so many pages of text if the presentation is not relatively smooth.
Third and finally, I do not feel that there is sufficient novelty to overcome the issues already raised above.  Simply adapting the VAE decoder noise factors via either a trainable noise parameter or an MoG model represents an incremental contribution as similar techniques are exceedingly common.  Of course, the paper also invents some new evaluation metrics and then applies them on benchmark datasets, but this content only appears much later in the paper (well after the soft 8 page limit) and I admittedly did not read it all carefully.  But on a superficial level, I do not believe these contributions are sufficient to salvage the paper (although I remain open to hearing arguments to the contrary).


Published as a conference paper at ICLR 2018 WAVELET POOLING FOR CONVOLUTIONAL NEURAL NETWORKS
Convolutional Neural Networks continuously advance the progress of 2D and 3D image and object classification. The steadfast usage of this algorithm requires constant evaluation and upgrading of foundational concepts to maintain progress. Network regularization techniques typically focus on convolutional layer operations, while leaving pooling layer operations without suitable options. We introduce Wavelet Pooling as another alternative to traditional neighborhood pooling. This method decomposes features into a second level decomposition, and discards the first-level subbands to reduce feature dimensions. This method addresses the overfitting problem encountered by max pooling, while reducing features in a more structurally compact manner than pooling via neighborhood regions. Experimental results on four benchmark classification datasets demonstrate our proposed method outperforms or performs comparatively with methods like max, mean, mixed, and stochastic pooling.

The paper proposes "wavelet pooling" as an alternative for traditional subsampling methods, e.g. max/average/global pooling, etc., within convolutional neural networks.
Experiments on the MNIST, CIFAR-10, SHVN and KDEF datasets, shows the proposed wavelet-based method has
competitive performance with existing methods while still being able to address the overfitting behavior of max pooling.
Strong points
- The method is sound and well motivated.
- The proposes method achieves competitive performance.
Weak points
- No information about added computational costs is given.
- Experiments are conducted in relatively low-scale datasets.
Overall the method is well presented and properly motivated. The paper as a good flow and is easy to follow. The authors effectively demonstrate with few toy examples the weaknesses of traditional methods, i.e max pooling and average pooling. Moreover, their extended evaluation on several datasets show the performance of the proposed method in different scenarios.
My main concerns with the manuscript are the following.
Compared to traditional methods, the proposed methods seems to require higher computation costs. In a deep neural network setting where operations are conducted a large number of times, this is a of importance. However, no indication is given on what are the added computation costs of the proposed method and how that compares to existing methods. A comparison on that regard would strengthen the paper.
In many of the experiments, the manuscript stresses the overfitting behavior of max pooling. This makes me wonder whether this is caused by the fact that experiments are conducted or relatively smaller datasets. While the currently tested datasets are a good indication of the performance of the proposed method, an evaluation on a large scale scenario, e.g. ILSVRC'12, could solidify the message sent by this manuscript. Moreover, it would increase the relevance of this work in the computer vision community.
Finally, related to the presentation, I would recommend presenting the plots, i.e. Fig. 8,10,12,14, for the training and validation image subsets in two separate plots. Currently, results for training and validation sets are mixed in the same plot, and due to the clutter it is not possible to see the trends clearly.
Similarly, I would recommend referring to the Tables added in the paper when discussing the performance of the proposed method w.r.t. traditional alternatives.
I encourage the authors to address my concerns in their rebuttal


Published as a conference paper at ICLR 2018 GRADIENT ESTIMATORS FOR IMPLICIT MODELS
Implicit models, which allow for the generation of samples but not for point-wise evaluation of probabilities, are omnipresent in real-world problems tackled by machine learning and a hot topic of current research. Some examples include data simulators that are widely used in engineering and scientific research, generative adversarial networks (GANs) for image synthesis, and hot-off-the-press approximate inference techniques relying on implicit distributions. The majority of existing approaches to learning implicit models rely on approximating the intractable distribution or optimisation objective for gradient-based optimisation, which is liable to produce inaccurate updates and thus poor models. This paper alleviates the need for such approximations by proposing the Stein gradient estimator, which directly estimates the score function of the implicitly defined distribution. The efficacy of the proposed estimator is empirically demonstrated by examples that include gradient-free MCMC, meta-learning for approximate inference and entropy regularised GANs that provide improved sample diversity.

Post rebuttal phase (see below for original comments)
================================================================================
I thank the authors for revising the manuscript. The methods makes sense now, and I think its quite interesting. While I do have some concerns (e.g. choice of eta, batching may not produce a consistent gradient estimator etc.), I  think the paper should be accepted. I have revised my score accordingly.
That said, the presentation (esp in Section 2) needs to be improved. The main problem is that many symbols have been used without being defined. e.g. phi, q_phi, \pi,  and a few more. While the authors might assume that this is obvious, it can be tricky to a reader - esp. someone like me who is not familiar with GANs. In addition, the derivation of the estimator in Section 3 was also sloppy. There are neater ways to derive this using RKHS theory without doing this on a d' dimensional space.
Revised summary: The authors present a method for estimating the gradient of some training objective for generative models used to sample data, such as GANs. The idea is that this can be used in a training procedure. The idea is based off the Stein's identity, for which the authors propose a kernelized solution. The key insight comes from rewriting the variational lower bound so that we are left with having to compute the gradients w.r.t a random variable and then applying Stein's identity. The authors present applications in Bayesian NNs and GANs.
Summary
================================================================
The authors present a method for estimating the gradient of some training objective
for generative models used to sample data, such as GANs. The idea is that this can be
used in a training procedure. The idea is based off the Stein's identity, for which the
authors propose a kernelized solution. The authors present applications in Bayesian NNs
and GANs.

Detailed Reviews
================================================================
My main concern is what I raised via a comment, for which I have not received a response
as yet. It seems that you want the gradients w.r.t the parameters phi in (3). But the
line immediately after claims that you need the gradients w.r.t the domain of a random
variable z and the subsequent sections focus on the gradients of the log density with
respect to the domain. I am not quite following the connection here.
Also, it doesn't help that many of the symbols on page 2 which elucidates the set up
have not been defined. What are the quantities phi, q, q_phi, epsilon, and pi?
Presentation
- Bottom row in Figure 1 needs to be labeled. I eventually figured that the colors
  correspond to the figures above, but a reader is easily confused.
- As someone who is not familiar with BNNs, I found the description in Section 4.2
  inadequate.

Some practical concerns:
- The fact that we need to construct a kernel matrix is concerning. Have you tried
  batch verstions of these estimator which update the gradients with a few data points?
- How is the parameter \eta chosen in practice? Can you comment on the values that you
  used and how it compared to the eigenvalues of the kernel matrix?
Minor
- What is the purpose behind sections 3.1 and 3.2? They don't seem pertinent to the rest
  of the exposition. Same goes for section 3.5? I don't see the authors using the
  gradient estimators for out-of-sample points?
I am giving an indifferent score mostly because I did not follow most of the details.


Representation learning is one of the foundations of Deep Learning and allowed important improvements on several Machine Learning tasks, such as Neural Machine Translation, Question Answering and Speech Recognition. Recent works have proposed new methods for learning representations for nodes and edges in graphs. Several of these methods are based on the SkipGram algorithm, and they usually process a large number of multi-hop neighbors in order to produce the context from which node representations are learned. In this paper, we propose an effective and also efficient method for generating node embeddings in graphs that employs a restricted number of permutations over the immediate neighborhood of a node as context to generate its representation, thus ego-centric representations. We present a thorough evaluation showing that our method outperforms state-of-the-art methods in six different datasets related to the problems of link prediction and node classification, being one to three orders of magnitude faster than baselines when generating node embeddings for very large graphs.

The paper includes the terms first-order proximity ("the concept that connected nodes in a graph should have similar properties") and second-order proximity ("the concept that nodes with similar neighborhoods should have common characteristics"). These are called homophily in social network analysis. It is also known as assortativity in network science literature. The paper states on Page 4: "A trade-off between first and second order proximity can be achieved by changing the parameter k, which simultaneously controls both the sizes of sentences generated and the size of the wind used in the SkipGram algorithm." It is not readily clear why this statement should hold. Also the paper does not include a discussion on how the amount of homophily in the graph affects the results. There are various ways of measuring the level of homophily in a graph. There is simple local consistency, which is % of edges connecting nodes that have the same characteristics at each endpoint. Neville & Jensen's JMLR 2007 paper describes relational auto-correlation, which is Pearson contingency coefficient on the characteristics of endpoints of edges. Park & Barabasi's PNAS 2007 paper describes dyadicity and heterophilicity, which measures connections of nodes with the same characteristics compared to a random model and the connections of nodes with different characteristics compared to a random model.
k ("which simultaneously controls both the sizes of sentences generated and the size of the wind used in the SkipGram algorithm") is a free-parameter in the proposed algorithm. The paper needs an in-depth discussion of the role of k in the results. Currently, no discussion is provided on k except that it was set to 5 for the experiments.  From a network science perspective, it makes sense to have k vary per node.
It is also not clear why d = 128 was chosen as the size of the embedding.
From the description of the experimental setup for link prediction, it is not clear if a stratified sample of the entries of the adjacency matrix (i.e., both 0 and 1 entries) where selected.
For the node classification experiments, information on class distribution and homophily levels would be helpful.
In Section 5.1, the paper states: "For highly connected graphs, larger numbers of permutations should be chosen (n in [10, 1000]) to better represent distributions, while for sparser graphs, smaller values can be used (n in [1, 10])." How high is highly connected graphs? How spare is a sparser graph? In general, the paper lacks an in-depth analysis of when the approach works and when it does not.  I recommend running experiments on synthetic graphs (such as Barabasi-Albert, Watts-Strogatz, Forest Fire, Kronecker, and/or BTER graphs), systematically changing various characteristics of the graph, and reporting the results.
The faster runtime is interesting but not surprising given the ego-centric nature of the approach.


Workshop track -ICLR 2018 LEARNING EFFICIENT TENSOR REPRESENTATIONS WITH RING STRUCTURE NETWORKS
Tensor train (TT) decomposition is a powerful representation for high-order tensors, which has been successfully applied to various machine learning tasks in recent years. In this paper, we propose a more generalized tensor decomposition with ring structure network by employing circular multilinear products over a sequence of lower-order core tensors, which is termed as TR representation. Several learning algorithms including blockwise ALS with adaptive tensor ranks and SGD with high scalability are presented. Furthermore, the mathematical properties are investigated, which enables us to perform basic algebra operations in a computationally efficiently way by using TR representations. Experimental results on synthetic signals and real-world datasets demonstrate the effectiveness of TR model and the learning algorithms. In particular, we show that the structure information and high-order correlations within a 2D image can be captured efficiently by employing an appropriate tensorization and TR decomposition.

The paper addresses the problem of tensor decomposition which is relevant and interesting. The paper proposes Tensor Ring (TR) decomposition which improves over and bases on the Tensor Train (TT) decomposition method. TT decomposes a tensor in to a sequences of latent tensors where the first and last tensors are a 2D matrices.
The proposed TR method generalizes TT in that the first and last tensors are also 3rd-order tensors instead of 2nd-order. I think such generalization is interesting but the innovation seems to be very limited.
The paper develops three different kinds of solvers for TR decomposition, i.e., SVD, ALS and SGD. All of these are well known methods.
Finally, the paper provides experimental results on synthetic data (3 oscillated functions) and image data (few sampled images). I think the paper could be greatly improved by providing more experiments and ablations to validate the benefits of the proposed methods.
Please refer to below for more comments and questions.
-- The rating has been updated.
Pros:
1. The topic is interesting.
2. The generalization over TT makes sense.
Cons:
1. The writing of the paper could be improved and more clear: the conclusions on inner product and F-norm can be integrated into "Theorem 5". And those "theorems" in section 4 are just some properties from previous definitions; they are not theorems.
2. The property of TR decomposition is that the tensors can be shifted (circular invariance). This is an interesting property and it seems to be the major strength of TR over TT. I think the paper could be significantly improved by providing more applications of this property in both theory and experiments.
3. As the number of latent tensors increase, the ALS method becomes much worse approximation of the original optimization. Any insights or results on the optimization performance vs. the number of latent tensors?
4. Also, the paper mentions Eq. 5 (ALS) is optimized by solving d subproblems alternatively. I think this only contains a single round of optimization. Should ALS be applied repeated (each round solves d problems) until convergence?
5. What is the memory consumption for different solvers?
6. SGD also needs to update at least d times for all d latent tensors. Why is the complexity O(r^3) independent of the parameter d?
7. The ALS is so slow (if looking at the results in section 5.1), which becomes not practical. The experimental part could be improved by providing more results and description about a guidance on how to choose from different solvers.
8. What does "iteration" mean in experimental results such as table 2? Different algorithms have different cost for "each iteration" so comparing that seems not fair. The results could make more sense by providing total time consumptions and time cost per iteration. also applies to table 4.
9. Why is the \epsion in table 3 not consistent? Why not choose \epsion = 9e-4 and \epsilon=2e-15 for tensorization?
10. Also, table 3 could be greatly improved by providing more ablations such as results for (n=16, d=8), (n=4, d=4), etc. That could help readers to better understand the effect of TR.
11. Section 5.3 could be improved by providing a curve (compression vs. error) instead of just providing a table of sampled operating points.
12. The paper mentions the application of image representation but only experiment on 32x32 images. How does the proposed method handle large images? Otherwise, it does not seem to be a practical application.
13. Figure 5: Are the RSE measures computed over the whole CIFAR-10 dataset or the displayed images?
Minor:
- Typo: Page 4 Line 7 "Note that this algorithm use the similar strategy": use -> uses


We introduce a model that learns to convert simple hand drawings into graphics programs written in a subset of L A T E X. The model combines techniques from deep learning and program synthesis. We learn a convolutional neural network that proposes plausible drawing primitives that explain an image. These drawing primitives are like a trace of the set of primitive commands issued by a graphics program. We learn a model that uses program synthesis techniques to recover a graphics program from that trace. These programs have constructs like variable bindings, iterative loops, or simple kinds of conditionals. With a graphics program in hand, we can correct errors made by the deep network and extrapolate drawings. Taken together these results are a step towards agents that induce useful, humanreadable programs from perceptual input.

Summary of paper:
This paper tackles the problem of inferring graphics programs from hand-drawn images by splitting it into two separate tasks:
(1) inferring trace sets (functions to use in the program) and
(2) program synthesis, using the results from (1).
The usefulness of this split is referred to as the trace hypothesis.
(1) is done by training a neural network on data [input = rendered image; output = trace sets] which is generated synthetically. During test time, a trace set is generated using a population-based method which samples and assigns weights to the guesses made by the neural network based on a similarity metric. Generalization to hand-drawn images is ensured by by learning the similarity metric.
(2) is done by feeding the trace set into a program synthesis tool of Solar Lezama. Since this is too slow, the authors design a search policy which proposes a restriction on the program search space, making it faster. The final loss for (2) in equation 3 takes into consideration the time taken to synthesize images in a search space.
---
Quality: The experiments are thorough and it seems to work. The potential limitation is generalization to non-synthetic data.
Clarity: The high level idea is clear however some of the details are not clear.
Originality: This work is one of the first that tackles the problem described.
Significance: There are many ad-hoc choices made in the paper, making it hard to extract an underlying insight that makes things work. Is it the trace hypothesis? Or is it just that trying enough things made this work?
---
Some questions/comments:
- Regarding the trace set inference, the loss function during training and the subsequent use of SMC during test time is pretty unconventional. The use of the likelihood P_{\theta}[T | I] as a proposal, as the paper also acknowledges, is also unconventional. One way to look at this which could make it less unconventional is to pose the training phase as learning the proposal distribution in an amortized way (instead of maximizing likelihood) as, for example, in [1, 2].
- In section 2.1., the paper talks about learning the surrogate likelihood function L_{learned} in order to work well for actual hand drawings. This presumably stems from the problem of mismatch between the distribution of the synthetic data used for training and the actual hand drawings. But then L_{learned} is also learned from synthetic data. What makes this translate to non-synthetic data? Does this translate to non-synthetic data?
- What does "Intersection over Union" in Figure 8 mean?
- The details for 3.1 are not clear. In particular, what does t(\sigma | T) in equation 3 refer to? Time to synthesize all images in \sigma? Why is the concept of Bias-optimality important?
- It seems from Table 4 that by design, the learned policy for the program search space already limits the search space to programs with maximum depth of the abstract syntax tree of 3. What is the usual depth of an AST when using Sketch?
---
Minor Comments:
- In page 4, section 2.1: "But pixel-wise distance fares poorly... match the model's renders." and "Pixel-wise distance metrics are sensitive... search space over traces." seem to be saying the same thing
- End of page 5: \citep Polozov & Gulwani (2015)
- Page 6: \citep Solar Lezama (2008)
---
References
[1] Paige, B., & Wood, F. (2016). Inference Networks for Sequential Monte Carlo in Graphical Models. In Proceedings of the 33rd International Conference on Machine Learning, JMLR W&CP 48: 3040-3049.
[2] Le, T. A., Baydin, A. G., & Wood, F. (2017). Inference Compilation and Universal Probabilistic Programming. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (Vol. 54, pp. 1338–1348). Fort Lauderdale, FL, USA: PMLR.


This paper we present a defogger, a model that learns to predict future hidden information from partial observations. We formulate this model in the context of forward modeling and leverage spatial and sequential constraints and correlations via convolutional neural networks and long short-term memory networks, respectively. We evaluate our approach on a large dataset of human games of StarCraft: Brood War, a real-time strategy video game. Our models consistently beat strong rule-based baselines and qualitatively produce sensible future game states.

The authors introduce the task of "defogging", by which they mean attempting to infer the contents of areas in the game StarCraft hidden by "the fog of war".
The authors train a neural network to solve the defogging task, define several evaluation metrics, and argue that the neural network beats several naive baseline models.
On the positive side, the task is a nice example of reasoning about a complex hidden state space, which is an important problem moving forwards in deep learning.
On the negative side, from what I can tell, the authors don't seem to have introduced any fundamentally new architectural choices in their neural network, so the contribution seems fairly specific to mastering StarCraft, but at the same time, the authors don't evaluate how much their defogger actually contributes to being able to win StarCraft games.  All of their evaluation is based on the accuracy of defogging.
Granted, being able to infer hidden states is of course an important problem, but the authors appear to mainly have applied existing techniques to a benchmark that has minimal practical significance outside of being able to win StarCraft competitions, meaning that, at least as the paper is currently framed, the critical evaluation metric would be showing that a defogger helps to win games.
Two ways I could image the contribution being improved are either highlighting and generalizing novel insights gleaned from the process of building the neural network that could help people build "defoggers" for other domains (and spelling out more explicitly what domains the authors expect their insights to generalize to), or doubling down on the StarCraft application specifically and showing that the defogger helps to win games.  A minimal version of the second modification would be having a bot that has access to a defogger play against a bot that does not have access to one.

All that said, as a paper on an application of deep learning, the paper appears to be solid, and if the area chairs are looking for that sort of contribution, then the work seems acceptable.
Minor points:
- Is there a benefit to having a model that jointly predicts unit presence and count, rather than having two separate models (e.g., one that feeds into the next)?  Could predicting presence or absence separately be a way to encourage sparsity, since absence of a unit is already representable as a count of zero?  The choice to have one model seems especially peculiar given the authors say they couldn't get one set of weights that works for both their classification and regression tasks
- Notation: I believe the space U is never described in the main text. What components precisely does an element of U have?
- The authors say they use gameplay from no later than 11 minutes in the game to avoid the difficulties of increasing variance. How long is a typical game?  Is this a substantial fraction of the time of the games studied?  If it is not, then perhaps the defogger would not help so much at winning.
- The F1 performance increases are somewhat small. The L1 performance gains are bigger, but the authors only compare L1 on true positives. This means they might have very bad error on false positives. (The authors state they are favoring the baseline in this comparison, but it would be nice to have those numbers.)
- I don't understand when the authors say the deep model has better memory than baselines (which includes a perfect memory baseline)


Search engine users nowadays heavily depend on query completion and correction to shape their queries. Typically, the completion is done by database lookup which does not understand the context and cannot generalize to prefixes not in the database. In the paper, we propose to use unsupervised deep language models to complete and correct the queries given an arbitrary prefix. We address two main challenges that renders this method practical for large-scale deployment: 1) we propose a method for integrating error correction into the language model completion via an edit-distance potential and a variant of beam search that can exploit such a potential function; and 2) we show how to efficiently perform CPUbased computation to complete the queries, with error correction, in real time (generating top 10 completions within 16 ms). Experiments show that the method substantially increases hit rate over standard approaches, and is capable of handling tail queries.

This paper presents methods for query completion that includes prefix correction, and some engineering details to meet particular latency requirements on a CPU.  Regarding the latter methods: what is described in the paper sounds like competent engineering details that those performing such a task for launch in a real service would figure out how to accomplish, and the specific reported details may or may not represent the 'right' way to go about this versus other choices that might be made.  The final threshold for 'successful' speedups feels somewhat arbitrary -- why 16ms in particular?  In any case, these methods are useful to document, but derive their value mainly from the fact that they allow the use of the completion/correction methods that are the primary contribution of the paper.
While the idea of integrating the spelling error probability into the search for completions is a sound one, the specific details of the model being pursued feel very ad hoc, which diminishes the ultimate impact of these results.  Specifically, estimating the log probability to be proportional to the number of edits in the Levenshtein distance is really not the right thing to do at all.  Under such an approach, the unedited string receives probability one, which doesn't leave much additional probability mass for the other candidates -- not to mention that the number of possible misspellings would require some aggressive normalization.  Even under the assumption that a normalized edit probability is not particularly critical (an issue that was not raised at all in the paper, let alone assessed), the fact is that the assumptions of independent errors and a single substitution cost are grossly invalid in natural language.  For example, the probability p_1 of 'pkoe' versus p_2 of 'zoze' as likely versions of 'poke' (as, say, the prefix of pokemon, as in your example) should be such that p_1 >>> p_2, not equal as they are in your model.  Probabilistic models of string distance have been common since Ristad and Yianlios in the late 90s, and there are proper probabilistic models that would work with your same dynamic programming algorithm, as well as improved models with some modest state splitting.  And even with very simple assumptions some unsupervised training could be used to yield at least a properly normalized model.  It may very well end up that your very simple model does as well as a well estimated model, but that is something to establish in your paper, not assume.  That such shortcomings are not noted in the paper is troublesome, particularly for a conference like ICLR that is focused on learned models, which this is not.  As the primary contribution of the paper is this method for combining correction with completion, this shortcoming in the paper is pretty serious.
Some other comments:
Your presentation of completion cost versus edit cost separation in section 3.3 is not particularly clear, partly since the methods are discussed prior to this point as extension of (possibly corrected) prefixes.  In fact, it seems that your completion model also includes extension of words with end point prior to the end of the prefix -- which doesn't match your prior notation, or, frankly, the way in which the experimental results are described.
The notation that you use is a bit sloppy and not everything is introduced in a clear way.  For example, the s_0:m notation is introduced before indicating that s_i would be the symbol in the i_th position (which you use in section 3.3).  Also, you claim that s_0 is the empty string, but isn't it more correct to model this symbol as the beginning of string symbol?  If not, what is the difference between s_0:m and s_1:m?  If s_0 is start of string, the s_0:m is of length m+1 not length m.
You spend too much time on common, well-known information, such as the LSTM equations.  (you don't need them, but also why number if you never refer to them later?)  Also the dynamic programming for Levenshtein is foundational, not required to present that algorithm in detail, unless there is something specific that you need to point out there (which your section 3.3 modification really doesn't require to make that point).
Is there a specific use scenario for the prefix splitting, other than for the evaluation of unseen prefixes?  This doesn't strike me as the most effective way to try to assess the seen/unseen distinction, since, as I understand the procedure, you will end up with very common prefixes alongside less common prefixes in your validation set, which doesn't really correspond to true 'unseen' scenarios.  I think another way of teasing apart such results would be recommended.
You never explicitly mention what your training loss is in section 5.1.
Overall, while this is an interesting and important problem, and the engineering details are interesting and reasonably well-motivated, the main contribution of the paper is based on a pretty flawed approach to modeling correction probability, which would limit the ultimate applicability of the methods.


Recently several different deep learning architectures have been proposed that take a string of characters as the raw input signal and automatically derive features for text classification. Few studies are available that compare the effectiveness of these approaches for character based text classification with each other. In this paper we perform such an empirical comparison for the important cybersecurity problem of DGA detection: classifying domain names as either benign vs. produced by malware (i.e., by a Domain Generation Algorithm). Training and evaluating on a dataset with 2M domain names shows that there is surprisingly little difference between various convolutional neural network (CNN) and recurrent neural network (RNN) based architectures in terms of accuracy, prompting a preference for the simpler architectures, since they are faster to train and less prone to overfitting.

SUMMARY
This paper addresses the cybersecurity problem of domain generation algorithm (DGA)  detection. A class of malware uses algorithms to automatically generate artificial domain names for various purposes, e.g. to generate large numbers of rendezvous points. DGA detection concerns the (automatic) distinction of actual and artificially generated domain names. In this paper, a basic problem formulation and general solution approach is investigated, namely that of treating the detection as a text classification task and to let domain names arrive to the classifier as strings of characters. A set of five deep learning architectures (both CNNs and RNNs) are compared empirical on the text classification task. A domain name data set with two million instances is used for the experiments. The main conclusion is that the different architectures are almost equally accurate and that this prompts a preference of simpler architectures over more complex architectures, since training time and the likelihood for overfitting can potentially be reduced.
COMMENTS
The introduction is well-written, clear, and concise. It describes the studied real-world problem and clarifies the relevance and challenge involved in solving the problem. The introduction provides a clear overview of deep learning architectures that have already been proposed for solving the problem as well as some architectures that could potentially be used. One suggestion for the introduction is that the authors take some of the description of the domain problem and put it into a separate background section to reduce the text the reader has to consume before arriving at the research problem and proposed solution.
The methods section (Section 2) provides a clear description of each of the five architectures along with brief code listings and details about whether any changes or parameter choices were made for the experiment. In the beginning of the section, it is not clarified why, if a 75 character string is encoded as a 128 byte ASCII sequence, the content has to be stored in a 75 x 128 matrix instead of a vector of size 128. This is clarified later but should perhaps be discussed earlier to allow readers from outside the subarea to grasp the approach.
Section 3 describes the experiment settings, the results, and discusses the learned representations and the possible implications of using either the deep architectures or the “baseline” Random Forest classifier. Perhaps, the authors could elaborate a little bit more on why Random Forests were trained on a completely different set of features than the deep architectures? The data is stated to be randomly divided into training (80%), validation (10%), and testing (10%). How many times is this procedure repeated? (That is, how many experimental runs were averaged or was the experiment run once?).
In summary, this is an interesting and well-written paper on a timely topic. The main conclusion is intuitive. Perhaps the conclusion is even regarded as obvious by some but, in my opinion, the result is important since it was obtained from new, rather extensive experiments on a large data set and through the comparison of several existing (earlier proposed) architectures. Since the main conclusion is that simple models should be prioritised over complex ones (due to that their accuracy is very similar), it would have been interesting to get some brief comments on a simplicity comparison of the candidates at the conclusion.
MINOR COMMENTS
Abstract: “Little studies” -> “Few studies”
Table 1: “approach” -> “approaches”
Figure 1: Use the same y-axis scale for all subplots (if possible) to simplify comparison. Also, try to move Figure 1 so that it appears closer to its inline reference in the text.
Section 3: “based their on popularity” -> “based on their popularity”


Published as a conference paper at ICLR 2018 COMPRESSING WORD EMBEDDINGS VIA DEEP COMPOSITIONAL CODE LEARNING
Natural language processing (NLP) models often require a massive number of parameters for word embeddings, resulting in a large storage or memory footprint. Deploying neural NLP models to mobile devices requires compressing the word embeddings without any significant sacrifices in performance. For this purpose, we propose to construct the embeddings with few basis vectors. For each word, the composition of basis vectors is determined by a hash code. To maximize the compression rate, we adopt the multi-codebook quantization approach instead of binary coding scheme. Each code is composed of multiple discrete numbers, such as (3, 2, 1, 8), where the value of each component is limited to a fixed range. We propose to directly learn the discrete codes in an end-to-end neural network by applying the Gumbel-softmax trick. Experiments show the compression rate achieves 98% in a sentiment analysis task and 94% ∼ 99% in machine translation tasks without performance loss. In both tasks, the proposed method can improve the model performance by slightly lowering the compression rate. Compared to other approaches such as character-level segmentation, the proposed method is language-independent and does not require modifications to the network architecture.

This paper proposed a new method to compress the space complexity of word embedding vectors by introducing summation composition over a limited number of basis vectors, and representing each embedding as a list of the basis indices. The proposed method can reduce more than 90% memory consumption while keeping original model accuracy in both the sentiment analysis task and the machine translation tasks.
Overall, the paper is well-written. The motivation is clear, the idea and approaches look suitable and the results clearly follow the motivation.
I think it is better to clarify in the paper that the proposed method can reduce only the complexity of the input embedding layer. For example, the model does not guarantee to be able to convert resulting "indices" to actual words (i.e., there are multiple words that have completely same indices, such as rows 4 and 6 in Table 5), and also there is no trivial method to restore the original indices from the composite vector. As a result, the model couldn't be used also as the proxy of the word prediction (softmax) layer, which is another but usually more critical bottleneck of the machine translation task.
For reader's comprehension, it would like to add results about whole memory consumption of each model as well.
Also, although this paper is focused on only the input embeddings, authors should refer some recent papers that tackle to reduce the complexity of the softmax layer. There are also many studies, and citing similar approaches may help readers to comprehend overall region of these studies.
Furthermore, I would like to see two additional analysis. First, if we trained the proposed model with starting from "zero" (e.g., randomly settling each index value), what results are obtained? Second, What kind of information is distributed in each trained basis vector? Are there any common/different things between bases trained by different tasks?


Understanding how people represent categories is a core problem in cognitive science, with the flexibility of human learning remaining a gold standard to which modern artificial intelligence and machine learning aspire. Decades of psychological research have yielded a variety of formal theories of categories, yet validating these theories with naturalistic stimuli remains a challenge. The problem is that human category representations cannot be directly observed and running informative experiments with naturalistic stimuli such as images requires having a workable representation of these stimuli. Deep neural networks have recently been successful in a range of computer vision tasks and provide a way to represent the features of images. In this paper, we introduce a method for estimating the structure of human categories that draws on ideas from both cognitive science and machine learning, blending human-based algorithms with state-of-theart deep representation learners. We provide qualitative and quantitative results as a proof of concept for the feasibility of the method. Samples drawn from human distributions rival the quality of current state-of-the-art generative models and outperform alternative methods for estimating the structure of human categories.

This paper presents a method based on GANs for visualizing how humans represent visual categories. Authors perform experiments on two datasets: Asian Faces Dataset and ImageNet Large Scale Recognition Challenge dataset.
Positive aspects:
+ The idea of using GANs for this goal is smart and interesting
+ The results seem interesting too
Weaknesses:
- Some aspects of the paper are not clear and presentation needs improvement.
- I miss a clearer results comparison with previous methods, like Vondrick et al. 2015.
Specific comments and questions:
-  Figure 1 is not clear. Authors should clarify how they use the inference network and what the two arrows from this inference network represent.
- Figure 2 is also not clear. Just the FLD projections of the MCMCP chains are difficult to interpret. The legend of the figure is too tiny. The right part of the figure should be better described in the text or in the caption, I don't understand well what this illustrates.
- Regarding to the human experiments with AMT: how do the authors deal with noise on the workers performance? Is any qualification task used? What are the instructions given to the workers?
- In section 4.2. the authors state "We also simultaneously learn a corresponding inference network, .... granular human biases captured". This seems interesting but I didn't find any result on that in the paper. Can you give more details or refer to where in the paper it is discussed/tested?
- Figure 4 shows "most interpretable mixture components".  How this "most interpretable" were selected?
- In second paragraph Section 4.3, it should be Table 1 instead of Figure 1.
- It would be interesting to see a discussion on why MCMCP Density is better for group 1 and MCMCP Mean is better for group 2. To see the confusion matrixes could be useful.
I like this paper. The addressed problem is challenging and the proposed idea seems interesting.  However, the aspects mentioned make me think the paper needs some improvements to be published.


The ability to deploy neural networks in real-world, safety-critical systems is severely limited by the presence of adversarial examples: slightly perturbed inputs that are misclassified by the network. In recent years, several techniques have been proposed for training networks that are robust to such examples; and each time stronger attacks have been devised, demonstrating the shortcomings of existing defenses. This highlights a key difficulty in designing an effective defense: the inability to assess a network's robustness against future attacks. We propose to address this difficulty through formal verification techniques. We construct ground truths: adversarial examples with a provably-minimal distance from a given input point. We demonstrate how ground truths can serve to assess the effectiveness of attack techniques, by comparing the adversarial examples produced by those attacks to the ground truths; and also of defense techniques, by computing the distance to the ground truths before and after the defense is applied, and measuring the improvement. We use this technique to assess recently suggested attack and defense techniques.

Summary: The paper proposes a method to compute adversarial examples with minimum distance to the original inputs, and to use the method to do two things: Show how well heuristic methods do in finding "optimal/minimal" adversarial examples (how close the come to the minimal change that flips the label) and to assess how a method that is designed to make the model more robust to adversarial examples actually works.
Pros:
I like the idea and the proposed applications. It is certainly highly relevant, both in terms of assessing models for critical use cases as well as a tool to better understand the phenomenon.
Some of the suggested insights in the analysis of defense techniques are interesting.
Cons:
The is not much technical novelty. The method boils down to applying Reluplex (Katz et al. 2017b) in a binary search (although I acknowledge the extension to L1 as distance metric).
The practical application of the method is very limited since the search is very slow and is only feasible at all for relatively small models. State-of-the-art practical models that achieve accuracy rates that make them interesting for deployment in potentially safety critical applications are out of reach for this analysis. The network analysed here does not reach the state-of-the-art on MNIST from almost two decades ago. The analysis also has to be done for each sample. The long runtime does not permit to analyse large amounts of input samples, which makes the analysis in terms of the increase in robustness rather weak. The statement can only be made for the very limited set of tested samples.
It is also unclear whether it is possible to include distance metrics that capture more sophisticated attacks that fool network even under various transformations of the input.
The paper does not consider the more recent and highly relevant Moosavi-Dezfooli et al. “Universal Adversarial Perturbations” CVPR 2017.
The distance metrics that are considered are only L_inf and L1, whereas it would be interesting to see more relevant “perceptual losses” such as those used in style transfer and domain adaptation with GANs.
Minor details:
* I would consider calling them “minimal adversarial samples” instead of “ground-truth”.
* I don’t know if the notation in the Equation in the paragraph describing Carlini & Wagner comes from the original paper, but the inner max would be easier to read as \max_{i \neq t} \{Z(x’)_i \}
* Page 3 “Neural network verification”: I dont agree with the statement that neural networks commonly are trained on “a small set of inputs”.
* Algorithm 1 is essentially only a description of binary search, which should not be necessary.
* What is the timeout for the computation, mentioned in Sec 4?
* Page 7, second paragraph: I wouldn’t say the observation is in line with Carlini & Wagner, because they take a random step, not necessarily one in the direction of the optimum? That’s also the conclusion two paragraphs below, no?
* I don’t fully agree with the conclusion that the defense of Madry does not overfit to the specific method of creating adversarial examples. Those were not created with the CW attack, but are related because CW was used to initialize the search.


Neural Style Transfer  has become a popular technique for generating images of distinct artistic styles using convolutional neural networks. This recent success in image style transfer has raised the question of whether similar methods can be leveraged to alter the "style" of musical audio. In this work, we attempt long time-scale high-quality audio transfer and texture synthesis in the time-domain that captures harmonic, rhythmic, and timbral elements related to musical style, using examples that may have different lengths and musical keys. We demonstrate the ability to use randomly initialized convolutional neural networks to transfer these aspects of musical style from one piece onto another using 3 different representations of audio: the log-magnitude of the Short Time Fourier Transform (STFT), the Mel spectrogram, and the Constant-Q Transform spectrogram. We propose using these representations as a way of generating and modifying perceptually significant characteristics of musical audio content. We demonstrate each representation's shortcomings and advantages over others by carefully designing neural network structures that complement the nature of musical audio. Finally, we show that the most compelling "style" transfer examples make use of an ensemble of these representations to help capture the varying desired characteristics of audio signals.

This paper studies style transfer for musical audio, and largely proposes some additions to the framework proposed by Ulyanov and Lebedev.  The changes are designed to improve the long-term temporal structure and harmonic matching of the stylized audio.  They carry out a few experiments to demonstrate how their proposed approach improves upon the baseline model.
Overall, I don't think this paper provides sufficiently novel or justified contributions compared to the baseline approach of Ulyanov and Lebedev.  It largely studies what happens when a different spectrogram representation is used on the input, and when somewhat different network architectures are used.  These changes are interesting, but don't provide a lot of additional information which I believe would be interesting to the ICLR community.  They seem better suited for an (audio) signal processing venue, or a more informal venue.  In addition, the results are not terribly compelling.  If the proposed changes (which are heuristically, not theoretically motivated) resulted in huge improvements to sound quality, I might be convinced.  More concretely, the results are still very far away from being able to be used in a commercial application (in contrast with image style transfer, whose impressive results were immediately commercially applied).  One reason I think the results remain bad is that the audio signal is still fundamentally represented as a phase-invariant representation.  Even if you backpropagate through the time-frequency transformation, the transformation itself discards phase, and so various signals (with different phase characteristics) will appear the same after transformation.  I believe this contributes the most to the fact that the resulting audio sounds very artifact-ridden and unrealistic.  If the paper had been able to overcome this limitation, I might be more convinced, but as-is I don't think it warrants acceptance at ICLR.
Specific comments:
- The description of Ulyanov & Lebedev's algorithm in 3.1 is confusingly structured.  For example, the sentence "The L2
distance between the generated audio and the content audio’s feature maps..." is basically a concatenation of roughly 6 thoughts which should be separated into different sentences.  The location of the equations (1), (2) do not correspond to where they are introduced in the text.  In addition, I don't understand how S and C are generated.  It is written that S and C are the "log-magnitude feature maps for style and content".  But the "feature maps" X are themselves a log-magnitude time frequency representation (x) convolved with the filterbank.  So how are S and C "log-magnitude feature maps"?  Surely you aren't computing the log of the output of the filterbank?  More equations would be helpful here.  Finally, it would be helpful to provide an equation both for G and W instead of just saying that W is analogously defined.
- I don't see any reason to believe that a mel-scaled spectrogram would better capture longer time scales or rhythmic information.  Mel-scaling your spectrogram just changes the frequency axis to a mel scale, which makes it somewhat closer to our perception; it does not modify the way time is represented in any way.  In fact, in musical machine learning tasks usually swapping between CQT and mel-scaled spectrograms (with a comparable number of frequency bins) has little effect, so I don't see any compelling reason to use one for "rhythm".  You need to provide strong empirical or theoretical evidence to back up your claim that this is a principled approach.  Instead, I would expect that your change of convolutional structure (to the dilated convolutions, etc) for the "mel spectrogram" branch of your network would account more heavily for stronger modeling of longer timescales.
- You refer to "WaveNet auto-encoders" and cite van den Oord et al.  The original wavenet paper did not propose an auto-encoder structure; Engel et al. did.
- "neither representation is capable of representing spatial patterns along the frequency axis" What do you mean by this?  Mel or linear-frequency (or CQT) spectrograms exhibit very strong patterns along their frequency axis.
- The method for automatically setting the scales of the different loss terms seems interesting, but I can't find anywhere a description of how you apply each of the beta terms.  Are they analogous to the alpha and beta parameters in equation (4)?  If so, it appears that gamma is shared across each beta term; this would mean that changing the value of gamma simply changed the scale of all loss terms at once, which would have no effect on optimization.
- "This is entirely possible though the ensemble representation" typo, through -> through
- That instance normalization causes noisy audio is an interesting empirical result, but I'm interested in a principled explanation of why this would happen.
- "the property of having 0 mean and unit variance" - you use this to describe the SeLU nonlinearity.  That's not a property of the nonlinearity, it's a property of the activations of different layers when using the nonlinearity (given correct initialization).
- How are the "Inter-Onset Interval Length Distributions" computed?  How are you getting the onsets, etc?
- " the maximum cross-correlation value between the time-domain audio waveforms are not significantly affected by the length of this field" - there are many ways effective copying could happen without the time-domain cross-correlation being large.


Under review as a conference paper at ICLR 2018 INTERPRETABLE AND PEDAGOGICAL EXAMPLES
Teachers intentionally pick the most informative examples to show their students. However, if the teacher and student are neural networks, the examples that the teacher network learns to give, although effective at teaching the student, are typically uninterpretable. We show that training the student and teacher iteratively, rather than jointly, can produce interpretable teaching strategies. We evaluate interpretability by (1) measuring the similarity of the teacher's emergent strategies to intuitive strategies in each domain and (2) conducting human experiments to evaluate how effective the teacher's strategies are at teaching humans. We show that the teacher network learns to select or generate interpretable, pedagogical examples to teach rule-based, probabilistic, boolean, and hierarchical concepts.

The authors define a novel method for creating a pair of models, a student and a teacher model, that are co-trained in a manner such that the teacher provides useful examples to the student to communicate a concept that is interpretable to people. They do this by adapting a technique from computational cognitive science called rational pedagogy. Rather than jointly optimize the student and teacher (as done previously), they have form a coupled relation between the student and teacher where each is providing a best response to the other. The authors demonstrate that their method provides interpretable samples for teaching in commonly used psychological domains and conduct human experiments to argue it can be used to teach people in a better manner than random teaching.
Understanding how to make complex models interpretable is an extremely important problem in ML for a number of reasons (e.g., AI ethics, explainable AI). The approach proposed by the authors is an excellent first step in this direction, and they provide a convincing argument for why a previous approach (joint optimization) did not work. It is an interesting approach that builds on computational cognitive science research and the authors provide strong evidence their method creates interpretable examples. They second part of their article, where they test the examples created by their models using behavioral experiments was less convincing. This is because they used the wrong statistical tests for analyzing the studies and it is unclear whether their results would stand with proper tests (I hope they will! – it seems clear that random samples will be harder to learn from eventually, but I also hoped there was a stronger baseline.).
For analysis, the authors use t-tests directly on KL-divergence and accuracy scores; however, this is inappropriate (see Jaeger, 2008; Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language, 59(4), 434-446.). This is especially applicable to the accuracy score results and the authors should reanalyze their data following the paper referenced above. With respect to KL-divergence, a G-test can be used (see https://en.wikipedia.org/wiki/G-test#Relation_to_Kullback.E2.80.93Leibler_divergence). I suspect the results will still be meaningful, but the appropriate analysis is essential to be able to interpret the human results.
Also, a related article: One article testing rational pedagogy in more ML contexts and using it to train ML models that is
Ho, M. K., Littman, M., MacGlashan, J., Cushman, F., & Austerweil, J. L. (NIPS 2016). Showing versus Doing. Teaching by Demonstration.
For future work, it would be nice to show that the technique works for finding interpretable examples in more complex deep learning networks, which motivated the current push for explainable AI in the first place.