Building chatbots that can accomplish goals such as booking a flight ticket is an unsolved problem in natural language understanding. Much progress has been made to build conversation models using techniques such as sequence2sequence modeling. One challenge in applying such techniques to building goal-oriented conversation models is that maximum likelihood-based models are not optimized toward accomplishing goals. Recently, many methods have been proposed to address this issue by optimizing a reward that contains task status or outcome. However, adding the reward optimization on the fly usually provides little guidance for language construction and the conversation model soon becomes decoupled from the language model. In this paper, we propose a new setting in goal-oriented dialogue system to tighten the gap between these two aspects by enforcing model level information isolation on individual models between two agents. Language construction now becomes an important part in reward optimization since it is the only way information can be exchanged. We experimented our models using self-play and results showed that our method not only beat the baseline sequence2sequence model in rewards but can also generate human-readable meaningful conversations of comparable quality.

I like the idea of coupling the language and the conversation model. This is in line with the latest trends of constructing end-to-end NN models that deal with the conversation in a holistic manner. The idea of enforcing information isolation is brilliant. Creating hidden information and allowing the two-party model to learn through self-play is a very interesting approach and the results seem promising.
Having said that, I feel important references are missing and specific statements of the paper, like that "Their success is however limited to conversations with very few turns and without goals" can be argued. There are papers that are goal oriented and have many turns. I will just provide one example, to avoid being overwhelming, although more can be found in the literature. That would be the paper of T.-H. Wen, D. Vandyke, N. Mrksic, M. Gasic, L. Rojas-Barahona, P.-H. Su, S. Ultes and S. Young (2017). "A Network-based End-to-End Trainable Task-oriented Dialogue System." EACL 2017, Valencia, Spain. In fact in this paper even more dialogue modules are coupled. So, the "fresh challenge" of the paper can be argued.
It is not clear to me how you did the supervised part of the training. To my experience, although supervised learning can be used, reinforcement learning seems to be the most popular choice. Also, I had to read most of the paper to understand that the system is based on a simulator. Additionally, it is not clear how you got the ground-truth for the training. How are the action and the dialogue generated by the simulator guaranteed to follow the optimal policy?

I also disagree with the statement that "based on those... to estimate rewards". If ruled-based systems were sufficient, there would not be a need for statistical dialogue managers. However, the latter is a very active research area.
Figure 1 is missing information (for my likings), like not defined symbols. In addition, it's not self-contained. Also, I would prefer a longer, descriptive and informative label to make the figure as self-explained as possible. I believe it would add to the clarity of the paper.
Also, fundamental information, according to my opinion is missing. For example, what are the restrictions R and how is the database K formed? What is the size of the database? How many actions do you define? Some of them are defined in the action state decoder, but it is not clear if it is all of them.
GRU -> abbreviation not defined
I would really appreciate a figure to better explain the subsection "Encoding External Knowledge". In the current form I am struggling to understand what the authors mean.
How is the embedding matrix E created?
Have you tried different unit sizes d? Have you tried different unit sizes for the customer and the service?
"we use 2 transformation matrixes" -> if you could please provide more details
How is equation 2 related to figure 1?
Typo: "name of he person"
"During the supervised learning... and the action states". I am not sure I get what you mean. May you make this statement more clearly by adding an equation for example?
What happens if you use random rather than supervised learning weight initialisation?
Equation 7: What does T stand for?
I cannot find Table 1, 2 and 5 referred to in-text. Moreover I am not sure about quite some items. For example, what is number db? What is the inference set?
500k of data is quite some. A figure on convergence would be nice.
Setting generator: You mention the percentage of book and flight not found. What about the rest of the cases?
Typo: “table 3 and table 3”
The set of the final states of the dialogue is not the same as the ones presented at Fig. 2.
Sub section reward generation is poorly described. After all, reward seems to play a very important role for the proposed system. Statements like  “things such as” (instead the exhaustive list of rules for example) or “the error against the optimal distance” with no note what should be considered the optimal distance make the paper clarity decreased and the results not possible to be reproduced. Personally I would prefer to see some equations or a flow chart. By the way, have you tried and alternative reward function?
Table 4 is not easy for me to understand. For example, what do you mean when you say eval reward?
Implementation details. I fail to understand how the supervised learning is used (as said already). Also you make a note for the value network, but not for the policy network.
There are some minor issues with the references such as pomdp or lstm not being capitalised
In general, I believe that the paper has a great potential and is a noticeable work. However,
the paper could be better organised. Personally, I struggled with the clarity of some text portions.
For me, the main drawback of the paper is that it was't tested with human users. The actual success of the system when evaluated by humans can be surprisingly different from the one that comes from simulation.


While much of the work in the design of convolutional networks over the last five years has revolved around the empirical investigation of the importance of depth, filter sizes, and number of feature channels, recent studies have shown that branching, i.e., splitting the computation along parallel but distinct threads and then aggregating their outputs, represents a new promising dimension for significant improvements in performance. To combat the complexity of design choices in multi-branch architectures, prior work has adopted simple strategies, such as a fixed branching factor, the same input being fed to all parallel branches, and an additive combination of the outputs produced by all branches at aggregation points. In this work we remove these predefined choices and propose an algorithm to learn the connections between branches in the network. Instead of being chosen a priori by the human designer, the multi-branch connectivity is learned simultaneously with the weights of the network by optimizing a single loss function defined with respect to the end task. We demonstrate our approach on the problem of multi-class image classification using four different datasets where it yields consistently higher accuracy compared to the state-of-the-art "ResNeXt" multi-branch network given the same learning capacity.
While much of the work in the design of convolutional networks over the last 1 five years has revolved around the empirical investigation of the importance of 2 depth, filter sizes, and number of feature channels, recent studies have shown that 3 branching, i.e., splitting the computation along parallel but distinct threads and then 4 aggregating their outputs, represents a new promising dimension for significant 5 improvements in performance. To combat the complexity of design choices in 6 multi-branch architectures, prior work has adopted simple strategies, such as a 7 fixed branching factor, the same input being fed to all parallel branches, and an 8 additive combination of the outputs produced by all branches at aggregation points.

The authors extend the ResNeXt architecture. They substitute the simple add operation with a selection operation for each input in the residual module. The selection of the inputs happens through gate weights, which are sampled at train time. At test time, the gates with the highest values are kept on, while the other ones are shut. The authors fix the number of the allowed gates to K out of C possible inputs (C is the multi-branch factor in the ResNeXt modules). They show results on CIFAR-100 and ImageNet (as well as mini ImageNet). They ablate the choice of K, the binary nature of the gate weights.
Pros:
(+) The paper is well written and the method is well explained
(+) The authors ablate and experiment on large scale datasets
Cons:
(-) The proposed method is a simple extension of ResNeXt
(-) The gains are reasonable, yet not SOTA, and come at a price of more complex training protocols (see below)
(-) Generalization to other tasks not shown
The authors do a great job walking us through the formulation and intutition of their proposed approach. They describe their training procedure and their sampling approach for the gate weights. However, the training protocol gets complicated with the introduction of gate weights. In order to train the gate weights along with the network parameters, the authors need to train the parameters jointly followed by the training of only the network parameters while keeping the gates frozen. This makes training of such networks cumbersome.
In addition, the authors report a loss in performance when the gates are not discretized to {0,1}. This means that a liner combination with the real-valued learned gate parameters is suboptimal. Could this be a result of suboptimal, possibly compromised training?
While the CIFAR-100 results look promising, the ImageNet-1k results are less impressive. The gains from introducing gate weights in the input of the residual modules vanish when increasing the network size.
Last, the impact of ResNeXt/ResNet lies in their ability to generalize to other tasks. Have the authors experimented with other tasks, e.g. object detection, to verify that their approach leads to better performance in a more diverse set of problems?


A NEW METHOD OF REGION EMBEDDING FOR TEXT CLASSIFICATION
To represent a text as a bag of properly identified "phrases" and use the representation for processing the text is proved to be useful. The key question here is how to identify the phrases and represent them. The traditional method of utilizing n-grams can be regarded as an approximation of the approach. Such a method can suffer from data sparsity, however, particularly when the length of n-gram is large. In this paper, we propose a new method of learning and utilizing task-specific distributed representations of n-grams, referred to as "region embeddings". Without loss of generality we address text classification. We specifically propose two models for region embeddings. In our models, the representation of a word has two parts, the embedding of the word itself, and a weighting matrix to interact with the local context, referred to as local context unit. The region embeddings are learned and used in the classification task, as parameters of the neural network classifier. Experimental results show that our proposed method outperforms existing methods in text classification on several benchmark datasets. The results also indicate that our method can indeed capture the salient phrasal expressions in the texts.

The authors present a model for text classification. The parameters of the model are an embedding for each word and a local context unit. The local context unit can be seen as a filter for a convolutional layer, but which filter is used at location i depends on the word at location i (i.e. there is one filter per vocabulary word). After the filter is applied to the embeddings and after max pooling, the word-context region embeddings are summed and fed into a neural network for the classification task. The embeddings, the context units and the neural net parameters are trained jointly on a supervised text classification task. The authors also offer an alternative model, which changes the role of the embedding an the context unit, and results in context-word region embeddings. Here the embedding of word i is combined with the elements of the context units of words in the context. To get the region embeddings both model (word-context and context-word) combine attributes of the words (embeddings) with how their attributes should be emphasized or deemphasized based on nearby words (local context units and max pooling) while taking into account the relative position of the words in the context (columns of the context units).
The method beats existing methods for text classification including d-LSTMs , BoWs, and ngram TFIDFs on held out classification accuracy. the choice of baselines is convincing. What is the performance of the proposed method if the embeddings are initialized to pretrained word embeddings and a) trained for the classification task together with randomly initialized context units b) frozen to pretrained embeddings and only the context units are trained for the classification task?
The introduction was fine. Until page 3 the authors refer to the context units a couple of times without giving some simple explanation of what it could be. A simple explanation in the introduction would improve the writing.
The related work section only makes sense *after* there is at least a minimal explanation of what the local context units do. A simple explanation of the method, for example in the introduction, would then make the connections to CNNs more clear. Also, in the related work, the authors could include more citations (e.g. the d-LSTM and the CNN based methods from Table 2) and explain the qualitative differences between their method and existing ones.
The authors should consider adding equation numbers. The equation on the bottom of page 3 is fine, but the expressions in 3.2 and 3.3 are weird. A more concise explanation of the context-word region embeddings and the word-context region embeddings would be to instead give the equation for r_{i,c}.
The included baselines are extensive and the proposed method outperforms existing methods on most datasets. In section 4.5 the authors analyze region and embedding size, which are good analyses to include in the paper. Figure 2 and 3 could be next to each other to save space.
I found the idea of multi region sizes interesting, but no description is given on how exactly they are combined. Since it works so well, maybe it could be promoted into the method section? Also, for each data set, which region size worked best?
Qualitative analysis: It would have been nice to see some analysis of whether the learned embeddings capture semantic similarities, both at the embedding level and at the region level. It would also be interesting to investigate the columns of the context units, with different columns somehow capturing the importance of relative position. Are there some words for which all columns are similar meaning that their position is less relevant in how they affect nearby words? And then for other words with variation along the columns of the context units, do their context units modulate the embedding more when they are closer or further away?
Pros:
 + simple model
 + strong quantitative results
Cons:
 - notation (i.e. precise definition of r_{i,c})
 - qualitative analysis could be extended
 - writing could be improved


Anomaly detection discovers regular patterns in unlabeled data and identifies the non-conforming data points, which in some cases are the result of malicious attacks by adversaries. Learners such as One-Class Support Vector Machines (OCSVMs) have been successfully used in anomaly detection, yet their performance may degrade significantly in the presence of sophisticated adversaries, who target the algorithm itself by compromising the integrity of the training data. With the rise in the use of machine learning in mission critical day-to-day activities where errors may have significant consequences, it is imperative that machine learning systems are made secure. To address this, we propose a defense mechanism that is based on a contraction of the data, and we test its effectiveness using OCSVMs. The proposed approach introduces a layer of uncertainty on top of the OCSVM learner, making it infeasible for the adversary to guess the specific configuration of the learner. We theoretically analyze the effects of adversarial perturbations on the separating margin of OCSVMs and provide empirical evidence on several benchmark datasets, which show that by carefully contracting the data in low dimensional spaces, we can successfully identify adversarial samples that would not have been identifiable in the original dimensional space. The numerical results show that the proposed method improves OCSVMs performance substantially (2-7%).

Although the problem addressed in the paper seems interesting, but there lacks of evidence to support some of the arguments that the authors make. And the paper does not contribute novelty to representation learning, therefore, it is not a good fit for the conference. Detailed critiques are as following:
1. The idea proposed by the authors seems too quite simple. It is just performing random projections for 1000 times and choose the set of projection parameters that results in the highest compactness as the dimensionality reduction model parameter before one-class SVM.
2. It says in the experiments part that the authors have used 3 different S_{attack} values, but they only present results for S_{attack} = 0.5. It would be nicer if they include results for all S_{attack} values that they have used in their experiments, which would also give the reader insights on how the anomaly detection performance degrades when the S_attack value change.
3. The paper claims that the nonlinear random projection is a defence against adversary due to the randomness, but there is no results in the paper proving that other non-random projections are susceptible to adversary that is designed to target that projection mechanism and nonlinear random projection is able to get away with that. And PCA as a non-random projection would a nice baseline to compare against.
4. The paper seems to misuse the term “False positive rate” as the y label of figure 3(d/e/f). The definition of false positive rate is FP/(FP+TN), so if the FPR=1 it means that all negative samples are labeled as positive. So it is surprising to see FPR=1 in Figure 3(d) when feature dimension=784 while the f1 score is still high in Figure 3(a). From what I understand, the paper means to present the percentage of adversarial examples that are misclassified instead of all the anomaly examples that get misclassified. The paper should come up with a better term for that evaluation.
5. The conclusion, that robustness of the learned model increases wrt the integrity attacks increases when the projection dimension becomes lower, cannot be drawn from Figure 3(d). Need more experiment on more dimensionality to prove that.
6. In the appendix B results part, sometimes the word ’S_attack’ is typed wrong. And the values in  “distorted/distorted” columns in Table 5 do not match up with the ones in Figure 3(c).


In this paper, we propose the generation of accented speech using generative adversarial networks (GANs). Through this work we make two main contributions a) The ability to condition latent representations while generating realistic speech samples b) The ability to efficiently generate long speech samples by using a novel latent variable transformation module that is trained using policy gradients. Previous methods are limited in being able to generate only relatively short samples or are not very efficient at generating long samples. The generated speech samples are validated through a number of various evaluation measures viz, a Wasserstein-GAN critic loss and through subjective scores on user evaluations against a competitive speech synthesis baseline. The evaluations demonstrate that the model generates realistic long speech samples conditioned on accent efficiently.

This paper presents a method for generating speech audio in a particular accent. The proposed approach relies on a generative adversarial network (GAN), combined with a policy approach for joining together generated speech segments. The latter is used to deal with the problem of generating very long sequences (which is generally difficult with GANs).
The problem of generating accented speech is very relevant since accent plays a large role in human communication and speech technology. Unfortunately, this paper is hard to follow. Some of the approach details are unclear and the research is not motivated well. The evaluation does not completely support the claims of the paper, e.g., there is no human judgment of whether the generated audio actually matches the desired accent.
Detailed comments, suggestions, and questions:
- It would be very useful to situate the research within work from the speech community. Why is accented modelling important? How is this done at the moment in speech synthesis systems? The paper gives some references, but without context. The paper from Ikeno and Hansen below might be useful.
- Accents are also a big problem in speech recognition (references below). Could your approach give accent-invariant representations for recognition?
- Figure 1: Add $x$, $y$, and the other variables you mention in Section 3 to the figure.
- What is $o$ in eq. (1)?
- Could you add a citation for eq. (2)? This would also help justifying that "it has a smoother curve and hence allows for more meaningful gradients".
- With respect to the critic $C_\nu$, I can see that it might be helpful to add structure to the hidden representation. In the evaluation, could you show the effect of having/not having this critic (sorry if I missed it)? The statement about "more efficient layers" is not clear.
- Section 3.4: If I understand correctly, this is a nice idea for ensuring that generated segments are combined sensibly. It would be helpful defining with "segments" refer to, and stepping through the audio generation process.
- Section 4.1: "using which we can" - typo.
- Section 5.1: "Figure 1 shows how the Wasserstein distance ..." I think you refer to the figure with Table 1?
- Figure 4: Add (a), (b) and (c) to the relevant parts in the figure.
References that might be useful:
- Ikeno, Ayako, and John HL Hansen. "The effect of listener accent background on accent perception and comprehension." EURASIP Journal on Audio, Speech, and Music Processing 2007, no. 3 (2007): 4.
- Van Compernolle, Dirk. "Recognizing speech of goats, wolves, sheep and… non-natives." Speech Communication 35, no. 1 (2001): 71-79.
- Benzeghiba, Mohamed, Renato De Mori, Olivier Deroo, Stephane Dupont, Teodora Erbes, Denis Jouvet, Luciano Fissore et al. "Automatic speech recognition and speech variability: A review." Speech communication 49, no. 10 (2007): 763-786.
- Wester, Mirjam, Cassia Valentini-Botinhao, and Gustav Eje Henter. "Are We Using Enough Listeners? No!—An Empirically-Supported Critique of Interspeech 2014 TTS Evaluations." In Sixteenth Annual Conference of the International Speech Communication Association. 2015.
The paper tries to address an important problem, and there are good ideas in the approach (I suspect Sections 3.3 and 3.4 are sensible). Unfortunately, the work is not presented or evaluated well, and I therefore give a week reject.


POLAR TRANSFORMER NETWORKS
Convolutional neural networks (CNNs) are inherently equivariant to translation. Efforts to embed other forms of equivariance have concentrated solely on rotation. We expand the notion of equivariance in CNNs through the Polar Transformer Network (PTN). PTN combines ideas from the Spatial Transformer Network (STN) and canonical coordinate representations. The result is a network invariant to translation and equivariant to both rotation and scale. PTN is trained end-to-end and composed of three distinct stages: a polar origin predictor, the newly introduced polar transformer module and a classifier. PTN achieves stateof-the-art on rotated MNIST and the newly introduced SIM2MNIST dataset, an MNIST variation obtained by adding clutter and perturbing digits with translation, rotation and scaling. The ideas of PTN are extensible to 3D which we demonstrate through the Cylindrical Transformer Network.

This paper presents a new convolutional network architecture that is invariant to global translations and equivariant to rotations and scaling. The method is combination of a spatial transformer module that predicts a focal point, around which a log-polar transform is performed. The resulting log-polar image is analyzed by a conventional CNN.
I find the basic idea quite compelling. Although this is not mentioned in the article, the proposed approach is quite similar to human vision in that people choose where to focus their eyes, and have an approximately log-polar sampling grid in the retina. Furthermore, dealing well with variations in scale is a long-standing and difficult problem in computer vision, and using a log-spaced sampling grid seems like a sensible approach to deal with it.
One fundamental limitation of the proposed approach is that although it is invariant to global translations, it does not have the built-in equivariance to local translations that a ConvNet has. Although we do not have data on this, I would guess that for more complex datasets like imagenet / ms coco, where a lot of variation can be reasonably well modelled by diffeomorphisms, this will result in degraded performance.
The use of the heatmap centroid as the prediction for the focal point is potentially problematic as well. It would not work if the heatmap is multimodal, e.g. when there are multiple instances in the same image or when there is a lot of clutter.
There is a minor conceptual confusion on page 4, where it is written that "Group-convolution requires integrability over a group and identification of the appropriate measure dg. We ignore this detail as implementation requires application of the sum instead of integral."
When approximating an integral by a sum, one should generally use quadrature weights that depend on the measure, so the measure cannot be ignored. Fortunately, in the chosen parameterization, the Haar measure is equal to the standard Lebesque measure, and so when using equally-spaced sampling points in this parameterization, the quadrature weights should be one. (Please double-check this - I'm only expressing my mathematical intuition but have not actually proven this).
It does not make sense to say that "The above convolution requires computation of the orbit which is feasible with respect to the finite rotation group, but not for general rotation-dilations", and then proceed to do exactly that (in canonical coordinates). Since the rotation-dilation group is 2D, just like the 2D translation group used in ConvNets, this is entirely feasible. The use of canonical coordinates is certainly a sensible choice (for the reason given above), but it does not make an infeasible computation feasible.
The authors may want to consider citing
- Warped Convolutions: Efficient Invariance to Spatial Transformations, Henriques & Vedaldi.
This paper also uses a log-polar transform, but lacks the focal point prediction / STN.
Likewise, although the paper makes a good effort to rewiev the literature on equivariance / steerability, it missed several recent works in this area:
- Steerable CNNs, Cohen & Welling
- Dynamic Steerable Blocks in Deep Residual Networks, Jacobsen et al.
- Learning Steerable Filters for Rotation Equivariant CNNs, Weiler et al.
The last paper reports 0.71% error on MNIST-rot, which is slightly better than the PTN-CNN-B++ reported on in this paper.
The experimental results presented in this paper are quite good, but both MNIST and ModelNet40 seem like simple / toyish datasets. For reasons outlined above, I am not convinced that this approach in its current form would work very well on more complicated problems. If the authors can show that it does (either in its current form or after improving it, e.g. with multiple saccades, or other improvements) I would recommend this paper for publication.
Minor issues & typos
- Section 3.1, psi_gh = psi_g psi_h. I suppose you use psi for L and L', but this is not very clear.
- L_h f = f(h^{-1}), p. 4
- "coordiantes", p. 5


In recent years Convolutional Neural Networks (CNN) have been used extensively for Superresolution (SR). In this paper, we use inverse problem and sparse representation solutions to form a mathematical basis for CNN operations. We show how a single neuron is able to provide the optimum solution for inverse problem, given a low resolution image dictionary as an operator. Introducing a new concept called Representation Dictionary Duality, we show that CNN elements (filters) are trained to be representation vectors and then, during reconstruction, used as dictionaries. In the light of theoretical work, we propose a new algorithm which uses two networks with different structures that are separately trained with low and high coherency image patches and show that it performs faster compared to the state-of-the-art algorithms while not sacrificing from performance.

The method proposes a new architecture for solving image super-resolution task. They provide an analysis that connects aims to establish a connection between how CNNs for solving super resolution and solving sparse regularized inverse problems.
The writing of the paper needs improvement. I was not able to understand the proposed connection, as notation is inconsistent and it is difficult to figure out what the authors are stating. I am willing to reconsider my evaluation if the authors provide clarifications.
The paper does not refer to recent advances in the problem, which are (as far as I know), the state of the art in the problem in terms of quality of the solutions. This references should be added and the authors should put their work into context.
1) Arguably, the state of the art in super resolution are techniques that go beyond L2 fitting. Specifically, methods using perceptual losses such as:
Johnson, J. et al "Perceptual losses for real-time style transfer and super-resolution." European Conference on Computer Vision. Springer International Publishing, 2016.
Ledig, Christian, et al. "Photo-realistic single image super-resolution using a generative adversarial network." arXiv preprint arXiv:1609.04802 (2016).
PSNR is known to not be directly related to image quality, as it favors blurred solutions. This should be discussed.
2) The overall notation of the paper should be improved. For instance, in (1), g represents the observation (the LR image), whereas later in the text, g is the HR image.
3) The description of Section 2.1 is quite confusing in my view. In equation (1), y is the signal to be recovered and K is just the downsampling plus blurring. So assuming an L1 regularization in this equation assumes that the signal itself is sparse. Equation (2) changes notation referring y as f.
4) Equation (2) seems wrong. The term multiplying K^T is not the norm (should be parenthesis).
5) The first statement of Section 2.2. seems wrong. DL methods do state the super resolution problem as an inverse problem. Instead of using a pre-defined basis function they learn an over-complete dictionary from the data, assuming that natural images can be sparsely represented. Also, this section does not explain how DL is used for super resolution. The cited work by Yang et al learns a two coupled dictionaries (one for LR and HL), such that for a given patch, the same sparse coefficients can reconstruct both HR and LR patches. The authors just state the sparse coding problem.
6) Equation (10) should not contain the \leq \epsilon.
7) In the second paragraph of Section 3, the authors mention that the LR image has to be larger than the HR image to prevent border effects. This makes sense. However, with the size of the network (20 layers), the change in size seems to be quite large. Could you please provide the sizes? When measuring PSNR, is this taken into account?
8) It would be very helpful to include an image explaining the procedure described in the second paragraph of Section 3.
9) I find the description in Section 3 quite confusing. The authors relate the training of a single filter (or neuron) to equation (7), but they define D, that is not used in all of Section 2.1. And K does not show in any of the analysis given in the last paragraph of page 4. However, D and K seem two different things (it is not just one for the other), see bellow.
10) I cannot understand the derivation that the authors do in the last paragraph of page 4 (and beginning of page 5). What is phi_l here? K in equation (7) seems to match to D here, but D here is a collection of patches and in (7) is a blurring and downsampling operator. I cannot review this section. I will wait for the author's response clarifications.
11) The authors describe a change in roles between the representations and atoms in the training and testing phase respectively. I do not understand this. If I understand correctly, the final algorithm, the authors train a CNN mapping LR to HR images. The network is used in the same way at training and testing.
12) It would be useful to provide more details about the training of the network. Please describe the training set used by Kim et al. Are the two networks trained independently? One could think of fine-tuning them jointly (including the aggregation).
13) The authors show the advantage of separating networks on a single image, Barbara. It would be good to quantify this better (maybe in terms of PSNR?). This observation might be true only because the training loss, say than the works cited above. Please comment on this.
14) In figures 3 and 4, the learned filters are those on the top (above the yellow arrow). It is not obvious to me that the reflect the predominant structure in the data. (maybe due to the low resolution).
15) This work is related to (though clearly different)  that of LISTA (Learned ISTA) type of networks, proposed in:
Gregor, K., & LeCun, Y. (2010). Learning fast approximations of sparse coding. In Proceedings of the 27th International Conference on Machine Learning (ICML)
Which connect the network architecture with the optimization algorithm used for solving the sparse coding problem. Follow up works have used these ideas for solving inverse problems as well.


There have been multiple attempts with variational auto-encoders (VAE) to learn powerful global representations of complex data using a combination of latent stochastic variables and an autoregressive model over the dimensions of the data. However, for the most challenging natural image tasks the purely autoregressive model with stochastic variables still outperform the combined stochasticautoregressive models. In this paper, we present simple additions to the VAE framework that generalize to natural images by embedding spatial information in the stochastic layers. We significantly improve the state-of-the-art results on MNIST, OMNIGLOT, CIFAR10 and ImageNet when the feature map parameterization of the stochastic variables are combined with the autoregressive PixelCNN approach. Interestingly, we also observe close to state-of-the-art results without the autoregressive part. This opens the possibility for high quality image generation with only one forward-pass.

The paper combines several recent advances on generative modelling including a ladder variational posterior and a PixelCNN decoder together with the proposed convolutional stochastic layers to boost the NLL results of the current VAEs. The numbers in the tables are good but I have several comments on the motivation, originality and experiments.
Most parts of the paper provide a detailed review of the literature. However, the resulting model is quite like a combination of the existing advances and the main contribution of the paper, i.e. the convolution stochastic layer, is not well discussed. Why should we introduce the convolution stochastic layers? Could the layers encode the spatial information better than a deterministic convolutional layer with the same architecture? What's the exact challenge of training VAEs addressed by the convolution stochastic layer? Please strengthen the motivation and originality of the paper.
Though the results are good, I still wonder what is the exact contribution of the convolutional stochastic layers to the NLL results?  Can the authors provide some results without the ladder variational posterior and the PixelCNN decoder on both the gray-scaled and the natural images?
According to the experimental setting in the Section 3 (Page 5 Paragraph 2), "In case of gray-scaled images the stochastic latent layers are dense with sizes 64, 32, 16, 8, 4 (equivalent to Sønderby et al. (2016)) and for the natural images they are spatial (cf. Table 1). There was no significant difference when using feature maps (as compared to dense layers) for modelling gray-scaled images." there is no stochastic convolutional layer.  Then is there anything new in FAME on the gray images? Furthermore, how could FAME advance the previous state-of-the-art? It seems because of other factors instead of the stochastic convolutional layer.
The results on the natural images are not complete. Please present the generation results on the ImageNet dataset and the reconstruction results on both the CIFAR10 and ImageNet datasets. The quality of the samples on the CIFAR10 dataset seems not competitive to the baseline papers listed in the table. Though the visual quality does not necessarily agree with the NLL results but such large gap is still strange. Besides, why FAME can obtain both good NLL and generation results on the MNIST and OMNIGLOT datasets when there is no stochastic convolutional layer? Meanwhile, why FAME cannot obtain good generation results on the CIFAR10 dataset? Is it because there is a lot randomness in the stochastic convolutional layer? It is better to provide further analysis and it is not safe to say that the stochastic convolutional layer helps learn better latent representations based on only the NNL results.
Minor things:
Please rewrite the sentence "When performing reconstructions during training ... while also using the stochastic latent variables z = z 1 , ..., z L." in the caption of Figure 1.


Designing a metric manually for unsupervised sequence generation tasks, such as text generation, is essentially difficult. In a such situation, learning a metric of a sequence from data is one possible solution. The previous study, SeqGAN, proposed the framework for unsupervised sequence generation, in which a metric is learned from data, and a generator is optimized with regard to the learned metric with policy gradient, inspired by generative adversarial nets (GANs) and reinforcement learning. In this paper, we make two proposals to learn better metric than SeqGAN's: partial reward function and expert-based reward function training. The partial reward function is a reward function for a partial sequence of a certain length. SeqGAN employs a reward function for completed sequence only. By combining long-scale and short-scale partial reward functions, we expect a learned metric to be able to evaluate a partial correctness as well as a coherence of a sequence, as a whole. In expert-based reward function training, a reward function is trained to discriminate between an expert (or true) sequence and a fake sequence that is produced by editing an expert sequence. Expert-based reward function training is not a kind of GAN frameworks. This makes the optimization of the generator easier. We examine the effect of the partial reward function and expert-based reward function training on synthetic data and real text data, and show improvements over SeqGAN and the model trained with MLE. Specifically, whereas SeqGAN gains 0.42 improvement of NLL over MLE on synthetic data, our best model gains 3.02 improvement, and whereas SeqGAN gains 0.029 improvement of BLEU over MLE, our best model gains 0.250 improvement.

This paper considers the problem of improving sequence generation by learning better metrics. Specifically, it focuses on addressing the exposure bias problem, where traditional methods such as SeqGAN uses GAN framework and reinforcement learning. Different from these work, this paper does not use GAN framework. Instead, it proposed an expert-based reward function training, which trains the reward function (the discriminator) from data that are generated by randomly modifying parts of the expert trajectories. Furthermore, it also introduces partial reward function that measures the quality of the subsequences of different lengths in the generated data. This is similar to the idea of hierarchical RL, which divide the problem into potential subtasks, which could alleviate the difficulty of reinforcement learning from sparse rewards. The idea of the paper is novel. However, there are a few points to be clarified.
In Section 3.2 and in (4) and (5), the authors explains how the action value Q_{D_i} is modeled and estimated for the partial reward function D_i of length L_{D_i}. But the authors do not explain how the rewards (or action value functions) of different lengths are aggregated together to update the model using policy gradient. Is it a simple sum of all of them?
It is not clear why the future subsequences that do not contain y_{t+1} are ignored for estimating the action value function Q in (4) and (5). The authors stated that it is for reducing the computation complexity. But it is not clear why specifically dropping the sequences that do not contain y_{t+1}. Please clarify more on this point.


Published as a conference paper at ICLR 2018 LATENT CONSTRAINTS: LEARNING TO GENERATE CONDITIONALLY FROM UNCONDITIONAL GENERATIVE MODELS
Deep generative neural networks have proven effective at both conditional and unconditional modeling of complex data distributions. Conditional generation enables interactive control, but creating new controls often requires expensive retraining. In this paper, we develop a method to condition generation without retraining the model. By post-hoc learning latent constraints, value functions that identify regions in latent space that generate outputs with desired attributes, we can conditionally sample from these regions with gradient-based optimization or amortized actor functions. Combining attribute constraints with a universal "realism" constraint, which enforces similarity to the data distribution, we generate realistic conditional images from an unconditional variational autoencoder. Further, using gradient-based optimization, we demonstrate identity-preserving transformations that make the minimal adjustment in latent space to modify the attributes of an image. Finally, with discrete sequences of musical notes, we demonstrate zero-shot conditional generation, learning latent constraints in the absence of labeled data or a differentiable reward function.

UPDATE: I think the authors' rebuttal and updated draft address my points sufficiently well for me to update my score and align myself with the other reviewers.
-----
ORIGINAL REVIEW: The paper proposes a method for learning post-hoc to condition a decoder-based generative model which was trained unconditionally. Starting from a VAE trained with an emphasis on good reconstructions (and at the expense of sample quality, via a small hard-coded standard deviation on the conditional p(x | z)), the authors propose to train two "critic" networks on the latent representation:
1. The "realism" critic receives either a sample z ~ q(z) (which is implicitly defined as the marginal of q(z | x) over all empirical samples) or a sample z ~ p(z) and must tell them apart.
2. The "attribute" critic receives either a (latent code, attribute) pair from the dataset or a synthetic (latent code, attribute) pair (obtained by passing both the attribute and a prior sample z ~ p(z) through a generator) and must tell them apart.
The goal is to find a latent code which satisfies both the realism and the attribute-exhibiting criteria, subject to a regularization penalty that encourages it to stay close to its starting point.
It seems to me that the proposed realism constraint hinges exclusively on the ability to implictly capture the marginal distribution q(z) via a trained discriminator. Because of that, any autoencoder could be used in conjunction with the realism constraint to obtain good-looking samples, including the identity encoder-decoder pair (in which case the problem reduces to generative adversarial training). I fail to see why this observation is VAE-specific. The authors do mention that the VAE semantics allow to provide some weak form of regularization on q(z) during training, but the way in which the choice of decoder standard deviation alters the shape of q(z) is not explained, and there is no justification for choosing one standard deviation value in particular.
With that in mind, the fact that the generator mapping prior samples to "realistic" latent codes works is expected: if the VAE is trained in a way that encourages it to focus almost exclusively on reconstruction, then its prior p(z) and its marginal q(z) have almost nothing to do with each other, and it is more convenient to view the proposed method as a two-step procedure in which an autoencoder is first trained, and an appropriate prior on latent codes is then learned. In other words, the generator represents the true prior by definition.
The paper is also rather sparse in terms of comparison with existing work. Table 1 does compare with Perarnau et al., but as the caption mentions, the two methods are not directly comparable due to differences in attribute labels.
Some additional comments:
- BiGAN [1] should be cited as concurrent work when citing (Dumoulin et al., 2016).
- [2] and [3] should be cited as concurrent work when citing (Ulyanov et al., 2016).
Overall, the relative lack of novelty and comparison with previous work make me hesitant to recommend the acceptance of this paper.
References:
[1] Donahue, J., Krähenbühl, P., and Darrell, T. (2017). Adversarial feature learning. In Proceedings of the International Conference on Learning Representations.
[2] Li, C., and Wand, M. (2016). Precomputed real-time texture synthesis with markovian generative adversarial networks. In European Conference on Computer Vision.
[3] Johnson, J., Alahi, A., and Fei-Fei, L. (2016). Perceptual losses for real-time style transfer and super-resolution. In European Conference on Computer Vision.


Measuring visual (dis)similarity between two or more instances within a data distribution is a fundamental task in many applications, especially in image retrieval. Theoretically, non-metric distances are able to generate a more complex and accurate similarity model than metric distances, provided that the non-linear data distribution is precisely captured by the similarity model. In this work, we analyze a simple approach for deep learning networks to be used as an approximation of non-metric similarity functions and we study how these models generalize across different image retrieval datasets.

The authors of this work propose learning a similarity measure for visual similarity and obtain, by doing that, an improvement in the very well-known datasets of Oxford and Paris for image retrieval. The work takes high-level image representations generated with an existing architecture (R-MAC), and train on top a neural network of two fully connected layers.
The training of such network is performed in three stages: firstly approximating the cosine similarity with a large amount of random feature vectors, secondly using image pairs from the same class, and finally using the hard examples.
PROS
P1. Results indicate the benefit of this approach in terms of similarity estimation and, overall, the paper present results that extend the state of the art in well-known datasets.
P2. The authors make a very nice effort in motivation the paper, relating it with the state of the art and funding their proposal on studies regarding human visual perception. The whole text is very well written and clear to follow.
CONS
C1. As already observed by the authors, training a similarity function without considering images from the target dataset is actually harmful. In this sense, the simple cosine similarity does not present this drawback in terms of lack of generalization. This observation is not new, but relevant in the field of image retrieval, where in many applications the object of interest for a query is actually not present in the training dataset.
C2. The main drawback of this approach is in terms of computation. Feed-forwarding the two samples through the trained neural network is far more expensive that computing the simple cosine similarity, which is computed very quickly with a GPU as a matrix multiplication. The authors already point at this in Section 4.3.
C3. I am somehow surprised that the authors did not explore also training the network that would extract the high-level representations, that is, a complete end-to-end approach. While I would expect to have the weights frozen in the first phase of training to miimic the cosine similarity, why not freeing the rest of layers when dealing with pairs of images ?
C4. There are a couple of recent papers that include results of the state of the art which are closer and sometimes better than the ones presented in this work. I do not think they reduce at all the contribution of this work, but they should be cited and maybe included in the tables:
A. Gordo, J. Almazan, J. Revaud, and D. Larlus. End-to-end learning of deep visual representations for image retrieval.
International Journal of Computer Vision, 124(2):237–254, 2017.
Albert Jimenez, Jose M. Alvarez, and Xavier Giro-i-Nieto. “Class-Weighted Convolutional Features for Visual Instance Search.” In Proceedings of the 28th British Machine Vision Conference (BMVC). 2017.


Neural sequence generation is commonly approached by using maximumlikelihood (ML) estimation or reinforcement learning (RL). However, it is known that they have their own shortcomings; ML presents training/testing discrepancy, whereas RL suffers from sample inefficiency. We point out that it is difficult to resolve all of the shortcomings simultaneously because of a tradeoff between ML and RL. In order to counteract these problems, we propose an objective function for sequence generation using α-divergence, which leads to an ML-RL integrated method that exploits better parts of ML and RL. We demonstrate that the proposed objective function generalizes ML and RL objective functions because it includes both as its special cases (ML corresponds to α → 0 and RL to α → 1). We provide a proposition stating that the difference between the RL objective function and the proposed one monotonically decreases with increasing α. Experimental results on machine translation tasks show that minimizing the proposed objective function achieves better sequence generation performance than ML-based methods.

This paper considers a dichitomy between ML and RL based methods for sequence generation. It is argued that the ML approach has some "discrepancy" between the optimization objective and the learning objective, and the RL approach suffers from bad sample complexity. An alpha-divergence formulation is considered to combine both methods.
Unfortunately, I do not understand main points made in this paper and am thus not able to give an accurate evaluation of the technical content of this paper. I therefore have no option but to vote for reject of this paper, based on my educated guess.
Below are the points that I'm particularly confused about:
1. For the ML formulation, the paper made several particularly confusing remarks. Some of them are blatantly wrong to me. For example,
1.1 The q(.|.) distribution in Eq. (1) *cannot* really be the true distribution, because the true distribution is unknown and therefore cannot be used to construct estimators. From the context, I guess the authors mean "empirical training distribution"?
1.2 I understand that the ML objective is different from what the users really care about (e.g., blue score), but this does not seem a "discrepancy" to me. The ML estimator simply finds a parameter that is the most consistent to the observed sequences; and if it fails to perform well in some other evaluation criterion such as blue score, it simply means the model is inadequate to describe the data given, or the model class is so large that the give number of samples is insufficient, and as a result one should change his/her modeling to make it more apt to describe the data at hand. In summary, I'm not convinced that the fact that ML optimizes a different objective than the blue score is a problem with the ML estimator.
In addition, I don't see at all why this discrepancy is a discrepancy between training and testing data. As long as both of them are identically distributed, then no discrepancy exists.
1.3 In point (ii) under the maximum likelihood section, I don't understand it at all and I think both sentences are wrong. First, the model is *not* trained on the true distribution which is unknown. The model is trained on an empirical distribution whose points are sampled from the true distribution. I also don't understand why it is evaluated using p_theta; if I understand correctly, the model is evaluated on a held-out test data, which is also generated from the underlying true distribution.
2. For the RL approach, I think it is very unclear as a formulation of an estimator. For example, in Eq. (2), what is r and what is y*? It is mentioned that r is a "reward" function, but I don't know what it means and the authors should perhaps explain further. I just don't see how one obtains an estimated parameter theta from the formulation in Eq. (2), using training examples.


DECISION BOUNDARY ANALYSIS OF ADVERSARIAL EXAMPLES
Deep neural networks (DNNs) are vulnerable to adversarial examples, which are carefully crafted instances aiming to cause prediction errors for DNNs. Recent research on adversarial examples has examined local neighborhoods in the input space of DNN models. However, previous work has limited what regions to consider, focusing either on low-dimensional subspaces or small balls. In this paper, we argue that information from larger neighborhoods, such as from more directions and from greater distances, will better characterize the relationship between adversarial examples and the DNN models. First, we introduce an attack, OPT-MARGIN, which generates adversarial examples robust to small perturbations. These examples successfully evade a defense that only considers a small ball around an input instance. Second, we analyze a larger neighborhood around input instances by looking at properties of surrounding decision boundaries, namely the distances to the boundaries and the adjacent classes. We find that the boundaries around these adversarial examples do not resemble the boundaries around benign examples. Finally, we show that, under scrutiny of the surrounding decision boundaries, our OPTMARGIN examples do not convincingly mimic benign examples. Although our experiments are limited to a few specific attacks, we hope these findings will motivate new, more evasive attacks and ultimately, effective defenses.

Summary of paper:
The authors present a novel attack for generating adversarial examples, deemed OptMargin, in which the authors attack an ensemble of classifiers created by classifying at random L2 small perturbations. They compare this optimization method with two baselines in MNIST and CIFAR, and provide an analysis of the decision boundaries by their adversarial examples, the baselines and non-altered examples.
Review summary:
I think this paper is interesting. The novelty of the attack is a bit dim, since it seems it's just the straightforward attack against the region cls defense. The authors fail to include the most standard baseline attack, namely FSGM. The authors also miss the most standard defense, training with adversarial examples. As well, the considered attacks are in L2 norm, and the distortion is measured in L2, while the defenses measure distortion in L_\infty (see detailed comments for the significance of this if considering white-box defenses). The provided analysis is insightful, though the authors mostly fail to explain how this analysis could provide further work with means to create new defenses or attacks.
If the authors add FSGM to the batch of experiments (especially section 4.1) and address some of the objections I will consider updating my score.
A more detailed review follows.
Detailed comments:
- I think the novelty of the attack is not very strong. The authors essentially develop an attack targeted to the region cls defense. Designing an attack for a specific defense is very well established in the literature, and the fact that the attack fools this specific defense is not surprising.
- I think the authors should make a claim on whether their proposed attack works only for defenses that are agnostic to the attack (such as PGD or region based), or for defenses that know this is a likely attack (see the following comment as well). If the authors want to make the second claim, training the network with adversarial examples coming from OptMargin is missing.
- The attacks are all based in L2, in the sense that the look for they measure perturbation in an L2 sense (as the paper evaluation does), while the defenses are all L_\infty based (since the region classifier method samples from a hypercube, and PGD uses an L_\infty perturbation limit). This is very problematic if the authors want to make claims about their attack being effective under defenses that know OptMargin is a possible attack.
- The simplest most standard baseline of all (FSGM) is missing. This is important to compare properly with previous work.
- The fact that the attack OptMargin is based in L2 perturbations makes it very susceptible to a defense that backprops through the attack. This and / or the defense of training to adversarial examples is an important experiment to assessing the limitations of the attack.
- I think the authors rush to conclude that "a small ball around a given input distance can be misleading". Wether balls are in L2 or L_\infty, or another norm makes a big difference in defense and attacks, given that they are only equivalent to a multiplicative factor of sqrt(d) where d is the dimension of the space, and we are dealing with very high dimensional problems. I find the analysis made by the authors to be very simplistic.
- The analysis of section 4.1 is interesting, it was insightful and to the best of my knowledge novel. Again I would ask the authors to make these plots for FSGM. Since FSGM is known to be robust to small random perturbations, I would be surprised that for a majority of random directions, the adversarial examples are brought back to the original class.
- I think a bit more analysis is needed in section 4.2. Do the authors think that this distinguishability can lead to a defense that uses these statistics? If so, how?
- I think the analysis of section 5 is fairly trivial. Distinguishability in high dimensions is an easy problem (as any GAN experiment confirms, see for example Arjovsky & Bottou, ICLR 2017), so it's not surprising or particularly insightful that one can train a classifier to easily recognize the boundaries.
- Will the authors release code to reproduce all their experiments and methods?
Minor comments:
- The justification of why OptStrong is missing from Table2 (last three sentences of 3.3) should be summarized in the caption of table 2 (even just pointing to the text), otherwise a first reader will mistake this for the omission of a baseline.
- I think it's important to state in table 1 what is the amount of distortion noticeable by a human.
=========================================
After the rebuttal I've updated my score, due to the addition of FSGM added as a baseline and a few clarifications. I now understand more the claims of the paper, and their experiments towards them. I still think the novelty, significance of the claims and protocol are still perhaps borderline for publication (though I'm leaning towards acceptance), but I don't have a really high amount of experience in the field of adversarial examples in order to make my review with high confidence.


Published as a conference paper at ICLR 2018 COMBINING SYMBOLIC EXPRESSIONS AND BLACK- BOX FUNCTION EVALUATIONS IN NEURAL PROGRAMS
Neural programming involves training neural networks to learn programs, mathematics, or logic from data. Previous works have failed to achieve good generalization performance, especially on problems and programs with high complexity or on large domains. This is because they mostly rely either on black-box function evaluations that do not capture the structure of the program, or on detailed execution traces that are expensive to obtain, and hence the training data has poor coverage of the domain under consideration. We present a novel framework that utilizes black-box function evaluations, in conjunction with symbolic expressions that define relationships between the given functions. We employ tree LSTMs to incorporate the structure of the symbolic expression trees. We use tree encoding for numbers present in function evaluation data, based on their decimal representation. We present an evaluation benchmark for this task to demonstrate our proposed model combines symbolic reasoning and function evaluation in a fruitful manner, obtaining high accuracies in our experiments. Our framework generalizes significantly better to expressions of higher depth and is able to fill partial equations with valid completions.

Summary
This paper presents a dataset of mathematical equations and applies TreeLSTMs to two tasks: verifying and completing mathematical equations. For these tasks, TreeLSTMs outperform TreeNNs and RNNs. In my opinion, the main contribution of this paper is this potentially useful dataset, as well as an interesting way of representing fixed-precision floats. However, the application of TreeNNs and TreeLSTMs is rather straight-forward, so in my (subjective) view there are only a few insights salvageable for the ICLR community and compared to Allamanis et al. (2017) this paper is a rather incremental extension.
Strengths
The authors present a new datasets for mathematical identities. The method for generating additional correct identities could be useful for future research in this area.
I find the representation of fixed-precision floats presented in this paper intriguing. I believe this contribution should be emphasized more as it allows the model to generalize to unseen numbers and I am wondering whether the authors see some wider application of this representation for neural programming models.
I liked the categorization of the related work.
Weaknesses
p2: It is mentioned that the framework is the first to combine symbolic expressions with black-box function evaluations, but I would argue that Neural Programmer-Interpreters (NPI; Reed & De Freitas) are already doing that (see Fig 1 in that paper where the execution trace is a symbolic expression and some expressions "Act(LEFT)" are black-box function applications directly changing the image).
The differences to Allamanis et al. (2017) are not worked out well. For instance, the authors use the TreeNN model from that paper as a baseline but the EqNet model is not mentioned at all. The obvious question is whether EqNets can be applied to the two tasks (verifying and completing mathematical equations) and if so why this has not been done.
The contribution regarding black box function application is unclear to me. On page 6, it is unclear to me what "handles […] function evaluation expressions". As far as I understand, the TreeLSTM learns to the return value of function evaluation expressions in order to predict equality of equations, but this should be clarified.
I find the connection of the proposed model and task to "neural programming" weak. For instance, as far as I understand there is no support for stateful programs. Furthermore, it would be interesting to hear how this work can be applied to existing programming languages such as Haskell. What are the limitations of the architecture? Could it learn to identify equality of two lists in Haskell?
p6: The paragraph on baseline models is rather uninformative. TreeLSTMs have been shown to outperform Tree NN's in various prior work. The statement that "LSTM cell […] helps the model to have a better understanding of the underlying functions in the domain" is vague. LSTM cells compared to fully-connected layers in Tree NNs ameliorate vanishing and exploding gradients along paths in the tree. Furthermore, I would like to see a qualitative analysis of the reasoning capabilities that are mentioned here. Did you observe any systematic differences in the ~4% of equations where the TreeLSTM fails to generalize (Table 3; first column).
Minor Comments
Abstract: "Our framework generalizes significantly better" I think it would be good to already mention in comparison to what this statement is.
p1: "aim to solve tasks such as learn mathematical" -> "aim to solve tasks such as learning mathematical"
p2: You could add a citation for Theano, Tensorflow and Mxnet.
p2: Could you elaborate how equation completion is used in Mathematical Q&A?
p3: Could you expand on "mathematical equation verification and completion […] has broader applicability" by maybe giving some concrete examples.
p3 Eq. 5: What precision do you consider? Two digits?
p3: "division because that they can" -> "division because they can"
p4 Fig. 1: Is there a reason 1 is represented as 10^0 here? Do you need the distinction between 1 (the integer) and 1.0 (the float)?
p5: "we include set of changes" -> "we include the set of changes"
p5: In my view there is enough space to move appendix A to section 2. In addition, it would be great to see more examples of generated identities at this stage (including negative ones).
p5: "We generate all possible equations (with high probability)" – what is probabilistic about this?
p5: I don't understand why function evaluation results in identities of depth 2 and 3. Is it both or one of them?
p6: The modules "symbol" and "number" are not shown in the figure. I assume they refer to projections using Wsymb and Wnum?
p6: "tree structures neural networks" -> "tree structured neural networks"
p6: A reference for the ADAM optimizer should be added.
p6: Which method was used for optimizing these hyperparameters? If a grid search was used, what intervals were used?
p7: "the superiority of Tree LSTM to Tree NN shows that is important to incorporate cells that have memory" is not a novel insight.
p8: When you mention "you give this set of equations to the models look at the top k predictions" I assume you ranked the substituted equations by the probability that the respective model assigns to it?
p8: Do you have an intuition why prediction function evaluations for "cos" seem to plateau certain points? Furthermore, it would be interesting to see what effect the choice of non-linearity on the output of the TreeLSTM has on how accurately it can learn to evaluate functions. For instance, one could replace the tanh with cos and might expect that the model has now an easy time to learn to evaluate cos(x).
p8 Fig 4b; p9: Relating to the question regarding plateaus in the function evaluation: "in Figure 4b […] the top prediction (0.28) is the correct value for tan with precision 2, but even other predictions are quite close" – they are all the same and this bad, right?
p9: "of the state-of-the-art neural reasoning systems" is very broad and in my opinion misleading too. First, there are other reasoning tasks (machine reading/Q&A, Visual Q&A, knowledge base inference etc.) too and it is not obvious how ideas from this paper translate to these domains. Second, for other tasks TreeLSTMs are likely not state-of-the-art (see for example models on the SQuAD leaderboard: https://rajpurkar.github.io/SQuAD-explorer/) .
p9: "exploring recent neural models that explicitly use memory cells" – I think what you mean is models with addressable differentiable memory.
# Update after the rebuttal
Thank you for the in-depth response and clarifications. I am increasing my score by one point. I have looked at the revised paper and I strongly suggest that you add the clarifications and in particular comments regarding comparison to related work (NPI, EqNet etc) to the paper. Regarding Fig. 4b, I am still not sure why all scores are the same (0.9977) -- I assume this is not the desired behavior?


Published as a conference paper at ICLR 2018 LEARNING A NEURAL RESPONSE METRIC FOR RETINAL PROSTHESIS
Retinal prostheses for treating incurable blindness are designed to electrically stimulate surviving retinal neurons, causing them to send artificial visual signals to the brain. However, electrical stimulation generally cannot precisely reproduce typical patterns of neural activity in the retina. Therefore, an electrical stimulus must be selected so as to produce a neural response as close as possible to the desired response. This requires a technique for computing the distance between a desired response and an achievable response that is meaningful in terms of the visual signal being conveyed. We propose a method to learn a metric on neural responses directly from recorded light responses of a population of retinal ganglion cells (RGCs) in the primate retina. The learned metric produces a measure of similarity of RGC population responses that accurately reflects the similarity of visual inputs. Using data from electrical stimulation experiments, we demonstrate that the learned metric could produce improvements in the performance of a retinal prosthesis.

* Summary of paper: The paper addresses the problem of optimizing metrics in the context of retinal prosthetics: Their goal is to learn a metric which assumes spike-patterns generated by the same stimulus to be more similar to each other than spike-patterns generated by different stimuli. They compare a conventional, quadratic metric to a neural-network based representation and a simple Hamming metric, and show that the neural-network based on achieves higher performance, but that the quadratic metric does not substantially beat the simple Hamming baseline. They subsequently evaluate the metric (unfortunately, only the quadratic metric) in two interesting applications involving electrical stimulation, with the goal of selecting stimulations which elicit spike-patterns which are maximally similar to spike-patterns evoked by particular stimuli.
* Quality: Overall, the paper is of high quality. What puzzled me, however is the fact that, in the applications using electrical stimulation in the paper (i.e. the applications targeted to retinal prosthetics, Secs 3.3 and 3.4), the authors do not actually used the well-performing neural-network based metric, but rather the quadratic metric, which is no better than the baseline Hamming metric?  It would be valuable for them to comment on what additional challenges would arise by using the neural network instead, and whether they think they could be surmonted.
* Clarity: The paper is overall clear, but specific aspects could be improved: First, it took me a while to understand (and is not entirely clear to me) what the goal of the paper is, in particular outside the setting studied by the authors (in which there is a small number of stimuli to be distinguished). Second, while the paper does not claim to provide a new metric-learning approach, it would benefit from more clearly explaining if and how their approach relates to previous approaches to metric learning.  Third, the paper, in my view, overstating some of the implications. As an example, Figure 5 is titled 'Learned quadratic response metric gives better perception than using a Hamming metric.': there is no psychophysical evaluation of perception in the paper, and even the (probably hand-picked?) examples in the figure do not look amazing.
* Originality: To the best of my knowledge, this is the first paper addressing the question of learning similarity metrics in the context of retinal prosthetics. Therefore, this specific paper and approach is certainly novel and original. From a machine-learning perspective, however, this seems like pretty standard metric learning with neural networks, and no attempt is made to either distinguish or relate their approach to prior work in this field (e.g. Chopra et al 2005, Schroff et al 2015 or Oh Song et al 2016.)
In addition, there is a host of metrics and kernels which have been proposed for measuring similarity between spike trains (Victor-Purpura) -- while they might not have been developed in the context of prosthetics, they might still be relevant to this tasks, and it would have been useful to see a comparison of how well they do relative to a Hamming metric. The paper states this as a goal ("This measure should expand upon...), but then never does that- why not?
* Significance: The general question the authors are approaching (how to improve retinal prosthetics) is,  an extremely important one both from a scientific and societal perspective. How important is the specific advance presented in this paper? The authors learn a metric for quantifying similarity between neural responses, and show that it performs better than a Hamming metric. It would be useful for the paper to comment on how they think that metric to be useful for retinal prosthetics. In a real prosthetic device, one will not be able learn a metric, as the metric learning her requires access to multiple trials of visual stimulation data, neuron-by-neuron. Clearly, any progress on the way to retinal prosthetics is important and this approach might contribute that. However, the current presentation of the manuscripts gives a somewhat misleading presentation of what has been achieved, and a more nuanced presentation would be important and appropriate.
Overall, this is a nice paper which could be of interest to  ICLR. Its strengths are that i) they identified a novel, interesting and potentially impactful problem that has not been worked on in machine learning before, ii) they provide a solution to it based on metric learning, and show that it performs better than a non-learned metrics. Its limitations are that i) no novel machine-learning methodology is used (and relationship to prior work in machine learning is not clearly described) ii) comparisons with previously proposed similarity measures of spike trains are lacking, iii) the authors do not actually use their learned, network based metric, but the metric which performs no better than the baseline in their main results, and  iv) it is not well explained how this improved metric could actually be used in the context of retinal prosthetics.
Minor comments:
  - p.2 The authors write that the element-wise product is denoted by $A \bullet B = \Tr(A^{\intercal}) B$
    This seems to be  incorrect, as the r.h.s. corresponds to a scalar.
  - p.3 What exactly is meant by “mining”?
  - p.4 It would be useful to give an example of what is meant by “similarity learning”.
  - p.4 “Please the Appendix” -> “Please see the Appendix”
  - p.5 (Fig. 3) The abbreviation “AUC” is not defined.
  - p.5 (Fig. 3B) The figure giving 'recall' should have a line indicating perfect performance, for comparison.
  - Sec 3.3: How was the decoder obtained ?
  - p.6 (Fig. 4) Would be useful to state that column below 0 is the target. Or just replace “0” by “target”.
  - p.6 (3rd paragraph) The sentence “Figure 4A bottom left shows the spatial profile of the linear decoding 20ms prior to the target response.” is unclear. It took me a very long time to realize that "bottom left" meant "column 0, 'decoded stimulus'" row. It's also unclear why the authors chose to look at 20ms prior to the target response.
  - p.6 The text says RMS distance, but the Fig. 4B caption says MSE— is this correct?


The quality of the features used in visual recognition is of fundamental importance for the overall system. For a long time, low-level hand-designed feature algorithms as SIFT and HOG have obtained the best results on image recognition. Visual features have recently been extracted from trained convolutional neural networks. Despite the high-quality results, one of the main drawbacks of this approach, when compared with hand-designed features, is the training time required during the learning process. In this paper, we propose a simple and fast way to train supervised convolutional models to feature extraction while still maintaining its high-quality. This methodology is evaluated on different datasets and compared with state-of-the-art approaches.

This paper deals with early stopping but the contributions are limited. This work would fit better a workshop as a preliminary result, furthermore it is too short. Following a short review section per section.
Intro: The name SFC is misleading as the method consists in stopping early the training with an optimized learning schedule scheme. Furthermore, the work is not compared to the appropriate baselines.
Proposal: The first motivation is not clear. The training time of the feature extractor has never been a problem for transfer learning tasks for example: once it is trained, you can reuse the architecture in a wide range of tasks. Besides, the training time of a CNN on CIFAR10 or even ImageNet is now quite small(for reasonable architectures), which allows fast benchmarking.
The second motivation, w.r.t. IB seems interesting but this should be empirically motivated(e.g. figures) in the subsection 2.1, and this is not done.
The section 3 is quite long and could be compressed to improve the relevance of this experimental section. All the accuracies(unsup dict, unsup, etc) on CIFAR10/CIFAR100 are reported from the paper (Oyallon & Mallat, 2015), ignoring 2-3 years of research that leads to new numerical results. Furthermore, this supervised technique is only compared to unsupervised or predefined methods, which is is not fair and the training time of the Scattering Transform is not reported, for example.
Finally, extracting features is mainly useful on ImageNet (for realistic images) and this is not reported here.
I believe re-thinking new learning rate schedules is interesting, however I recommend the rejection of this paper.


Recurrent neural networks like long short-term memory (LSTM) are important architectures for sequential prediction tasks. LSTMs (and RNNs in general) model sequences along the forward time direction. Bidirectional LSTMs (Bi-LSTMs), which model sequences along both forward and backward directions, generally perform better at such tasks because they capture a richer representation of the data. In the training of Bi-LSTMs, the forward and backward paths are learned independently. We propose a variant of the Bi-LSTM architecture, which we call Variational Bi-LSTM, that creates a dependence between the two paths (during training, but which may be omitted during inference). Our model acts as a regularizer and encourages the two networks to inform each other in making their respective predictions using distinct information. We perform ablation studies to better understand the different components of our model and evaluate the method on various benchmarks, showing state-of-the-art performance.

*Quality*
The paper is easy to parse, with clear diagrams and derivations at the start. The problem context is clearly stated, as is the proposed model.
The improvements in terms of average log-likelihood are clear. The model does improve over state-of-the-art in some cases, but not all.
Based on the presented findings, it is difficult to determine the quality of the learned models overall, since they are only evaluated in terms of average log likelihood. It is also difficult to determine whether the improvements are due to the model change, or some difference in how the models themselves were trained (particularly in the case of Z-Forcing, a closely related technique). I would like to see more exploration of this point, as the section titled “ablation studies” is short and does not sufficiently address the issue of what component of the model is contributing to the observed improvements in average log-likelihood.
Hence, I have assigned a score of "4" for the following reasons: the quality of the generated models is unclear; the paper does not clearly distinguish itself from the closely-related Z-Forcing concept (published at NIPS 2017); and the reasons for the improvements shown in average log-likelihood are not explored sufficiently, that is, the ablation studies don't eliminate key parts of the model that could provide this information.
More information on this decision is given in the remainder.
*Clarity*
A lack of generated samples in the Experimental Results section makes it difficult to evaluate the performance of the models; log-likelihood alone can be an inadequate measure of performance without some care in how it is calculated and interpreted (refer, e.g., to Theis et al. 2016, “A Note on the Evaluation of Generative Models”).
There are some typos and organizational issues. For example, VAEs are reintroduced in the Related Works section, only to provide an explanation for an unrelated optimization challenge with the use of RNNs as encoders and decoders.
I also find the motivations for the proposed model itself a little unclear. It seems unnatural to introduce a side-channel-cum-regularizer between a sequence moving forward in time and the same sequence moving backwards, through a variational distribution. In the introduction, improved regularization for LSTM models is cited as a primary motivation for introducing and learning two approximate distributions for latent variables between the forward and backward paths of a bi-LSTM. Is there a serious need for new regularization in such models? The need for this particular regularization choice is not particularly clear based on this explanation, nor are the improvements state-of-the-art in all cases. This weakens a possible theoretical contribution of the paper.
*Originality*
The proposed modification appears to amount to a regularizer for bi-LSTMs which bears close similarity to Z-Forcing (cited in the paper). I recommend a more careful comparison between the two methods. Without such a comparison, they are a little hard to distinguish, and the originality of this paper is hard to evaluate. Both appear to employ the same core idea of regularizing an LSTM using a learned variational distributions. The differences *seem* to be in the small details, and these details appear to provide better performance in terms of average log-likelihood on all tasks compared to Z-Forcing--but, crucially, not compared to other models in all cases.


Labeled text classification datasets are typically only available in a few select languages. In order to train a model for e.g news categorization in a language L t without a suitable text classification dataset there are two options. The first option is to create a new labeled dataset by hand, and the second option is to transfer label information from an existing labeled dataset in a source language L s to the target language L t . In this paper we propose a method for sharing label information across languages by means of a language independent text encoder. The encoder will give almost identical representations to multilingual versions of the same text. This means that labeled data in one language can be used to train a classifier that works for the rest of the languages. The encoder is trained independently of any concrete classification task and can therefore subsequently be used for any classification task. We show that it is possible to obtain good performance even in the case where only a comparable corpus of texts is available.

This paper addresses the problem of learning a cross-language text categorizer with no labelled information in the target language. The suggested solution relies on learning cross-lingual embeddings, and training a classifier using labelled data in the source language only.
The idea of using cross-lingual or multilingual representations to seamlessly handle documents across languages is not terribly novel as it has been use in multilignual categorization or semantic similarity for some time. This contribution however proposes a clean separation of the multiligual encoder and classifier, as well as a good (but long) section on related prior art.
One concern is that the modelling section stays fairly high level and is hardly sufficient, for example to re-implement the models. Many design decisions (e.g. #layers, #units) are not justified. They likely result from preliminary experiments, in that case it should be said.
The main concern is that the experiments could be greatly improved. Given the extensive related work section, it is odd that no alternate model is compared to. The details on the experiments are also scarce. For example, are all accuracy results computed on the same 8k test set? If so this should be clearly stated. Why are models tested on small subsets of the available data? You have 493k Italian documents, yet the largest model uses 158k... It is unclear where many such decisions come from -- e.g. Fig 4b misses results for 1000 and 1250 dimensions and Fig 4b has nothing between 68k and 137k, precisely where a crossover happens.
In short, it feels like the paper would greatly improve from a clearer modeling description and more careful experimental design.
Misc:
- Clarify early on what "samples" are in your categorization context.
- Given the data set, why use a single-label multiclass setup, rather than multilabel?
- Table 1 caption claims an average of 2.3 articles per topic, yet for 200 topics you have 500k to 1.5M articles?
- Clarify the use of the first 200 words in each article vs. snippets
- Put overall caption in Figs 2-4 on top of (a), (b), otherwise references like Fig 4b are unclear.


EMERGENCE OF LINGUISTIC COMMUNICATION FROM REFERENTIAL GAMES WITH SYMBOLIC AND PIXEL INPUT
The ability of algorithms to evolve or learn (compositional) communication protocols has traditionally been studied in the language evolution literature through the use of emergent communication tasks. Here we scale up this research by using contemporary deep learning methods and by training reinforcement-learning neural network agents on referential communication games. We extend previous work, in which agents were trained in symbolic environments, by developing agents which are able to learn from raw pixel data, a more challenging and realistic input representation. We find that the degree of structure found in the input data affects the nature of the emerged protocols, and thereby corroborate the hypothesis that structured compositional language is most likely to emerge when agents perceive the world as being structured.

--------------
Summary:
--------------
This paper presents a series of experiments on language emergence through referential games between two agents. They ground these experiments in both fully-specified symbolic worlds and through raw, entangled, visual observations of simple synthetic scenes. They provide rich analysis of the emergent languages the agents produce under different experimental conditions. This analysis (especially on raw pixel images) make up the primary contribution of this work.
--------------
Evaluation:
--------------
Overall I think the paper makes some interesting contributions with respect to the line of recent 'language emergence' papers. The authors provide novel analysis of the learned languages and perceptual system across a number of environmental settings, coming to the (perhaps uncontroversial) finding that varying the environment and restrictions on language result in variations in the learned communication protocols.
In the context of existing literature, the novelty of this work is somewhat limited -- consisting primarily of the extension of multi-agent reference games to raw-pixel inputs. While this is a non-trivial extension, other works have demonstrated language learning in similar referring-expression contexts (essentially modeling only the listener model [Hermann et.al 2017]).
I have a number of requests for clarification in the weaknesses section which I think would improve my understanding of this work and result in a stronger submission if included by the authors.
--------------
Strengths:
--------------
- Clear writing and document structure.
- Extensive experimental setting tweaks which ablate the information and regularity available to the agents. The discussion of the resulting languages is appropriate and provides some interesting insights.
- A number of novel analyses are presented to evaluate the learned languages and perceptual systems.
--------------
Weaknesses:
--------------
- How stable are the reported trends / languages across multiple runs within the same experimental setting? The variance of REINFORCE policy gradients (especially without a baseline) plus the general stochasticity of SGD on randomly initialized networks leads me to believe that multiple training runs of these agents might result is significantly different codes / performance. I am interested in hearing the author's experiences in this regard and if multiple runs present similar quantitative and qualitative results. I admit that expecting identical codes is unrealistic, but the form of the codes (i.e. primarily encoding position) might be consistent even if the individual mappings are not).
- I don't recall seeing descriptions of the inference-time procedure used to evaluate training / test accuracy. I will assume argmax decoding for both speaker and listener. Please clarify or let me know if I missed something.
- There is ambiguity in how the "protocol size" metric is computed. In Table 1, it is defined as 'the effective number of unique message used'. This comes back to my question about decoding I suppose, but does this count the 'inference-time' messages or those produced during training?
Furthermore, Table 2 redefines "protocol size" as the percentage of novel message. I assume this is an editing error given the values presented and take these columns as counts. It also seems "protocol size" is replaced with the term "lexicon" from 4.1 onward.
- I'm surprised by how well the agents generalize in the raw pixel data experiments. In fact, it seems that across all games the test accuracy remains very close to the train accuracy.
Given the dataset is created by taking all combinations of color / shape and then sampling 100 location / floor color variations, it is unlikely that a shape / color combo has not been seen in training. Such that the only novel variations are likely location and floor color. However, taking Game A as an example, the probe classifiers are relatively poor at these attributes -- indicating the speaker's representation is not capturing these attributes well. Then how do the agents effectively differentiate so well between 20 images leveraging primarily color and shape?
I think some additional analysis of this setting might shed some light on this issue. One thought is to compute upper-bounds based on ground truth attributes. Consider a model which knows shape perfectly, but cannot predict other attributes beyond chance. To compute the performance of such a model, you could take the candidate set, remove any instances not matching the ground truth shape, and then pick randomly from the remaining instances. Something similar could be repeated for all attributes independently as well as their combinations -- obviously culminating in 100% accuracy given all 4. It could be that by dataset construction, object location and shape are sufficient to achieve high accuracy because the odds of seeing the same shape at the same location (but different color) is very low.
Given these are operations on annotations and don't require time-consuming model training, I hope to see this analysis in the rebuttal to put the results into appropriate context.
- What is random chance for the position and floor color probe classifiers? I don't think it is mentioned how many locations / floor colors are used in generation.
- Relatively minor complaint: Both agents are trained via the REINFORCE policy gradient update rule; however, the listener agent makes a fairly standard classification decision and could be trained with a standard cross-entropy loss. That is to say, the listener policy need not make intermediate discrete policy decisions. This decision to withhold available supervision is not discussed in the paper (as far as I noticed), could the authors speak to this point?
--------------
Curiosities:
--------------
- I got the impression from the results (specifically the lack of discussion about message length) that in these experiments agents always issued full length messages even though they did not need to do so. If true, could the authors give some intuition as to why? If untrue, what sort of distribution of lengths do you observe?
- There is no long term planning involved in this problem, so why use reinforcement learning over some sort of differentiable sampler? With some re-parameterization (i.e. Gumbel-Softmax), this model could be end-to-end differentiable.
--------------
Minor errors:
--------------
[2.2 paragraph 1] LSTM citation should not be in inline form.
[3 paragraph 1] 'Note that these representations do care some' -> carry
[3.3.1 last paragraph] 'still able comprehend' --> to
-------
Edit
-------
Updating rating from 6 to 7.


Workshop track -ICLR 2018 ANYTIME NEURAL NETWORK: A VERSATILE TRADE- OFF BETWEEN COMPUTATION AND ACCURACY
We present an approach for anytime predictions in deep neural networks (DNNs). For each test sample, an anytime predictor produces a coarse result quickly, and then continues to refine it until the test-time computational budget is depleted. Such predictors can address the growing computational problem of DNNs by automatically adjusting to varying test-time budgets. In this work, we study a general augmentation to feed-forward networks to form anytime neural networks (ANNs) via auxiliary predictions and losses. Specifically, we point out a blindspot in recent studies in such ANNs: the importance of high final accuracy. In fact, we show on multiple recognition data-sets and architectures that by having near-optimal final predictions in small anytime models, we can effectively double the speed of large ones to reach corresponding accuracy level. We achieve such speed-up with simple weighting of anytime losses that oscillate during training. We also assemble a sequence of exponentially deepening ANNs, to achieve both theoretically and practically near-optimal anytime results at any budget, at the cost of a constant fraction of additional consumed budget.

1. Paper Summary
This paper adds a separate network at every layer of a residual network that performs classification. They minimize the loss of every classifier using two proposed weighting schemes. They also ensemble this model.
2. High level paper
The organization of this paper is a bit confusing. Two weighing schemes are introduced in Section 3.1, then the ensemble model is described in Section 3.2, then the weighing schemes are justified in Section 4.1.
Overall this method is essentially an cascade where each cascade classifier is a residual block. Every input is passed through as many stages as possible until the budget is reached. While this model is likely quite useful in industrial settings, I don't think the model itself is wholly original.
The authors have done extensive experiments evaluating their method in different settings. I would have liked to see a comparison with at least one other anytime method. I think it is slightly unfair to say that you are comparing with Xie & Tu, 2015 and Huang et al., 2017 just because they use the CONSTANT weighing schemes.
3. High level technical
I have a few concerns:
- Why does AANN+LINEAR nearly match the accuracy of EANN+SIEVE near 3e9 FLOPS in Figure 4b but EANN+LINEAR does not in Figure 4a? Shouldn't EANN+LINEAR be strictly better than AANN+LINEAR?
- Why do the authors choose these specific weighing schemes? Section 4.1 is devoted to explaining this but it is still unclear to me. They talk about there being correlation between the predictors near the end of the model so they don't want to distribute weight near the final predictors but this general observation doesn't obviously lead to these weighing schemes, they still seem a bit adhoc.
A few other comments:
- Figure 3b seems to contain strictly less information than Figure 4a, I would remove Figure 3b and draw lines showing the speedup you get for one or two accuracy levels.
Questions:
- Section 3.1: "Such an ideal θ* does not exist in general and often does not exist in practice." Why is this the case?
- Section 3.1: " In particular, spreading weights evenly as in (Lee et al., 2015) keeps all i away from their possible respective minimum" Why is this true?
- Section 3.1: "Since we will evaluate near depth b3L/4e, and it
is the center of L/2 low-weight layers, we increase it weight by 1/8." I am completely lost here, why do you do this?
4. Review summary
Ultimately because the model itself resembles previous cascade models, the selected weighings have little justification, and there isn't a comparison with another anytime method, I think this paper isn't yet ready for acceptance at ICLR.