Published as a conference paper at ICLR 2018 SPHERICAL CNNS
Convolutional Neural Networks (CNNs) have become the method of choice for learning problems involving 2D planar images. However, a number of problems of recent interest have created a demand for models that can analyze spherical images. Examples include omnidirectional vision for drones, robots, and autonomous cars, molecular regression problems, and global weather and climate modelling. A naive application of convolutional networks to a planar projection of the spherical signal is destined to fail, because the space-varying distortions introduced by such a projection will make translational weight sharing ineffective. In this paper we introduce the building blocks for constructing spherical CNNs. We propose a definition for the spherical cross-correlation that is both expressive and rotation-equivariant. The spherical correlation satisfies a generalized Fourier theorem, which allows us to compute it efficiently using a generalized (non-commutative) Fast Fourier Transform (FFT) algorithm. We demonstrate the computational efficiency, numerical accuracy, and effectiveness of spherical CNNs applied to 3D model recognition and atomization energy regression.

Summary:
The paper proposes a framework for constructing spherical convolutional networks (ConvNets) based on a novel synthesis of several existing concepts.  The goal is to detect patterns in spherical signals irrespective of how they are rotated on the sphere.  The key is to make the convolutional architecture rotation equivariant.
Pros:
+ novel/original proposal justified both theoretically and empirically
+ well written, easy to follow
+ limited evaluation on a classification and regression task is suggestive of the proposed approach's potential
+ efficient implementation
Cons:
- related work, in particular the first paragraph, should compare and contrast with the closest extant work rather than merely list them
- evaluation is limited; granted this is the nature of the target domain
Presentation:
While the paper is generally written well, the paper appears to conflate the definition of the convolutional and correlation operators?  This point should be clarified in a revised manuscript.
In Section 5 (Experiments), there are several references to S^2CNN.  This naming of the proposed approach should be made clear earlier in the manuscript.  As an aside, this appears a little confusing since convolution is performed first on S^2 and then SO(3).
Evaluation:
What are the timings of the forward/backward pass and space considerations for the Spherical ConvNets presented in the evaluation section?  Please provide specific numbers for the various tasks presented.
How many layers (parameters) are used in the baselines in Table 2?  If indeed there are much less parameters used in the proposed approach, this would strengthen the argument for the approach.  On the other hand, was there an attempt to add additional layers to the proposed approach for the shape recognition experiment in Sec. 5.3 to improve performance?
Minor Points:
- some references are missing their source, e.g., Maslen 1998 and Kostolec, Rockmore, 2007, and Ravanbakhsh, et al. 2016.
- some sources for the references are presented inconsistency, e.g., Cohen and Welling, 2017 and Dieleman, et al. 2017
- some references include the first name of the authors, others use the initial
- in references to et al. or not, appears inconsistent
- Eqns 4, 5, 6, and 8 require punctuation
- Section 4 line 2, period missing before "Since the FFT"
- "coulomb matrix" --> "Coulomb matrix"
- Figure 5, caption: "The red dot correcpond to" --> "The red dot corresponds to"
Final remarks:
Based on the novelty of the approach, and the sufficient evaluation, I recommend the paper be accepted.


Published as a conference paper at ICLR 2018 LEARNING TO SHARE: SIMULTANEOUS PARAMETER TYING AND SPARSIFICATION IN DEEP LEARNING
Deep neural networks (DNNs) may contain millions, even billions, of parameters/weights, making storage and computation very expensive and motivating a large body of work aimed at reducing their complexity by using, e.g., sparsityinducing regularization. Parameter sharing/tying is another well-known approach for controlling the complexity of DNNs by forcing certain sets of weights to share a common value. Some forms of weight sharing are hard-wired to express certain invariances; a notable example is the shift-invariance of convolutional layers. However, other groups of weights may be tied together during the learning process to further reduce the network complexity. In this paper, we adopt a recently proposed regularizer, GrOWL (group ordered weighted 1 ), which encourages sparsity and, simultaneously, learns which groups of parameters should share a common value. GrOWL has been proven effective in linear regression, being able to identify and cope with strongly correlated covariates. Unlike standard sparsity-inducing regularizers (e.g., 1 a.k.a. Lasso), GrOWL not only eliminates unimportant neurons by setting all their weights to zero, but also explicitly identifies strongly correlated neurons by tying the corresponding weights to a common value. This ability of GrOWL motivates the following two-stage procedure: (i) use GrOWL regularization during training to simultaneously identify significant neurons and groups of parameters that should be tied together; (ii) retrain the network, enforcing the structure that was unveiled in the previous phase, i.e., keeping only the significant neurons and enforcing the learned tying structure. We evaluate this approach on several benchmark datasets, showing that it can dramatically compress the network with slight or even no loss on generalization accuracy.

SUMMARY
The paper proposes to apply GrOWL regularization to the tensors of parameters between each pair of layers. The groups are composed of all coefficients associated to inputs coming from the same neuron in the previous layer. The proposed algorithm is a simple proximal gradient algorithm using the proximal operator of the GrOWL norm. Given that the GrOWL norm tend to empirically reinforce a natural clustering of the vectors of coefficients which occurs in some layers, the paper proposes to cluster the corresponding parameter vectors, to replace them with their centroid and to retrain with the constrain that some vectors are now equal. Experiments show that some sparsity is obtained by the model and that together with the clustering and high compression of the model is obtained which maintaining or improving over a good level of generalization accuracy. In comparison, plain group Lasso yields compressed versions that are too sparse, and tend to degrade performance. The method is also competitive with weight decay with much better compression.
REVIEW
Given the well known issue that the Lasso tends to select arbitrarily and in a non stable way variables
that are correlated *but* given that the well known elastic-net (and conceptually simpler than GrOWL) was proposed to address that issue already more than 10 years ago, it would seem relevant to compare the proposed method with the group elastic-net.
The proposed algorithm is a simple proximal gradient algorithm, but since the objective is non-convex it would be relevant to provide references for convergence guarantees of the algorithm.
How should the step size eta be chosen? I don't see that this is discussed in the paper.
In the clustering algorithm how is the threshold value chosen?
Is it chosen by cross validation?
Is the performance better with clustering or without?
Is the same threshold chosen for GrOWL and the Lasso?
It would be useful to know which values of p, Lambda_1 and Lambda_2 are selected in the experiments?
For Figures 5,7,8,9 given that the matrices do not have particular structures that need to be visualized but that the important thing to compare is the distribution of correlation between pairs, these figures that are hard to read and compare would be advantageously replaced by histograms of the values of the correlations between pairs (of different variables). Indeed, right now one must rely on comparison of shades of colors in the thin lines that display correlation and it is really difficult to appreciate how much of correlation of what level are present in each Figure. Histograms would extract exactly the relevant information...
A brief description of affinity propagation, if only in the appendix, would be relevant.
Why this method as opposed to more classical agglomerative clustering?
A brief reminder of what the principle of weight decay is would also be relevant for the paper to more self contained.
The proposed experiments are compelling, except for the fact that it would be nice to have a comparison with the group elastic-net.
I liked figure 6.d and would vote for inclusion in the main paper.
TYPOS etc
3rd last line of sec. 3.2 can fail at selecting -> fail to select
In eq. (5) theta^t should be theta^{(t)}
In section 4.1 you that the network has a single fully connected layer of hidden units -> what you mean is that the network has a single hidden layer, which is furthermore fully connected.
You cite several times Sergey (2015) in section 4.2. It seems you have exchanged first name and last name plus the corresponding reference is quite strange.
Appendix B line 5 ", while." -> incomplete sentence.


ON THE IMPORTANCE OF SINGLE DIRECTIONS FOR GENERALIZATION
Despite their ability to memorize large datasets, deep neural networks often achieve good generalization performance. However, the differences between the learned solutions of networks which generalize and those which do not remain unclear. Additionally, the tuning properties of single directions (defined as the activation of a single unit or some linear combination of units in response to some input) have been highlighted, but their importance has not been evaluated. Here, we connect these lines of inquiry to demonstrate that a network's reliance on single directions is a good predictor of its generalization performance, across networks trained on datasets with different fractions of corrupted labels, across ensembles of networks trained on datasets with unmodified labels, across different hyperparameters, and over the course of training. While dropout only regularizes this quantity up to a point, batch normalization implicitly discourages single direction reliance, in part by decreasing the class selectivity of individual units. Finally, we find that class selectivity is a poor predictor of task importance, suggesting not only that networks which generalize well minimize their dependence on individual units by reducing their selectivity, but also that individually selective units may not be necessary for strong network performance.

article summary:
The authors use ablation analyses to evaluate the reliance on single coordinate-aligned directions in activation space (i.e. the activation of single units or feature maps) as a function of memorization. They find that the performance of networks that memorize more are also more affected by ablations. This result holds even for identical networks trained on identical data. The dynamics of this reliance on single directions suggest that it could be used as a criterion for early stopping. The authors discuss this observation in relation to dropout and batch normalization. Although dropout is an effective regularizer to prevent memorization of random labels, it does not prevent over-reliance on single directions. Batch normalization does appear to reduce the reliance on single directions, providing an alternative explanation for the effectiveness of batch normalization. Networks trained without batch normalization also demonstrated a significantly higher amount of class selectivity in individual units compared to networks trained without batch normalization. Highly selective units were found to be no more important than units that were not selective to a particular class. These results suggest that highly selective units may actually be harmful to network performance.
* Quality: The paper presents thorough and careful empirical analyses to support their claims.
* Clarity: The paper is very clear and well-organized. Sufficient detail is provided to reproduce the results.
* Originality: This work is one of many recent papers trying to understand generalization in deep networks. Their description of the activation space of networks that generalize compared to those that memorize is novel. The authors throughly relate their findings to related work on generalization, regularization, and pruning. However, the authors may wish to relate their findings to recent reports in neuroscience observing similar phenomena (see below).
* Significance: The paper provides valuable insight that helps to relate existing theories about generalization in deep networks. The insights of this paper will have a large impact on regularization, early stopping, generalization, and methods used to explain neural networks.
Pros:
* Observations are replicated for several network architectures and datasets.
* Observations are very clearly contextualized with respect to several active areas of deep learning research.
Cons:
* The class selectivity measure does not capture all class-related information that a unit may pass on.
Comments:
* Regarding the class selectivity of single units, there is a growing body of literature in neurophysiology and neuroimaging describing similar observations where the interpretation has been that a primary role of any neural pathway is to “denoise” or cancel out the “distractor” rather than just amplifying the “signal” of interest.
    * Untuned But Not Irrelevant: The Role of Untuned Neurons In Sensory Information Coding, https://www.biorxiv.org/content/early/2017/09/21/134379
    * Correlated variability modifies working memory fidelity in primate prefrontal neuronal ensembles https://www.ncbi.nlm.nih.gov/pubmed/28275096
    * On the interpretation of weight vectors of linear models in multivariate neuroimaging http://www.sciencedirect.com/science/article/pii/S1053811913010914
        * see also LEARNING HOW TO EXPLAIN NEURAL NETWORKS https://openreview.net/forum?id=Hkn7CBaTW
* Regarding the intuition in section 3.1, "The minimal description length of the model should be larger for the memorizing network than for the structure- finding network. As a result, the memorizing network should use more of its capacity than the structure-finding network, and by extension, more single directions”. Does reliance on single directions not also imply a local encoding scheme? We know that for a fixed number of units, a distributed representation will be able to encode a larger number of unique items than a local one. Therefore if this behaviour was the result of needing to use up more of the capacity of the network, wouldn’t you expect to observe more distributed representations?
Minor issues:
* In the first sentence of section 2.3, you say you analyzed three models and then you only list two. It seems you forgot to include ResNet trained on ImageNet.


In high dimensions, the performance of nearest neighbor algorithms depends crucially on structure in the data. While traditional nearest neighbor datasets consisted mostly of hand-crafted feature vectors, an increasing number of datasets comes from representations learned with neural networks. We study the interaction between nearest neighbor algorithms and neural networks in more detail. We find that the network architecture can significantly influence the efficacy of nearest neighbor algorithms even when the classification accuracy is unchanged. Based on our experiments, we propose a number of training modifications that lead to significantly better datasets for nearest neighbor algorithms. Our modifications lead to learned representations that can accelerate nearest neighbor queries by 5×.

This paper investigates learning representations for the problem of nearest neighbor (NN) search by exploring various deep learning architectural choices. The crux of the paper is the connection between NN and the angles between the closest neighbors -- the higher this angle, more data points need to be explored for finding the nearest one, and thus more computational expense. Thus, the paper proposes to learn a network that tries to reduce the angles between the inputs and the corresponding class vectors in a supervised framework using softmax cross-entropy loss. Three architectural choices are investigated, (i) controlling the norm of output layers of the CNN (using batch norm essentially), (ii) removing relu so that the outputs are well-distributed in both positive and negative orthants, and (iii) normalizing the class vectors. Experiments are given on multiMNIST and Sports 1M and show improvements.
Pros:
1) The paper explores different architectural choices for the deep network to some depth and show extensive results.
2) The results do demonstrate clearly the advantage of the various choices and is useful
3) The theoretical connections between data angles and query times are quite interesting,
Cons:
1) Unclear Problem Statement.
I find the problem statement a bit vague. Standard NN search finds a data point in the database closest to a query under some distance metric. While, the current paper uses the cosine similarity as the distance, the deep framework is trained on class vectors using cross-entropy loss. I do not think class labels are usually assumed to be given in the standard definition of NN, and it is not clear to me how the proposed setup can accommodate NN without class labels.  Thus as such, I see this paper is perhaps proposing a classification problem and not an NN problem per se.
2) Lacks Focus
The paper lacks a good organization in my opinion. Things that are perhaps technically important are moved to the Appendix. For example, I find the theoretical part of the paper (e.g., Theorem 1) quite elegant and perhaps the main innovation in this paper. However, that is moved completely to the Appendix. So it cannot be really considered a contribution. It is also not clear if those theoretical results are novel.
2) Disconnect/Unclear Assumptions
There seems to be some disconnect between LSH and deep learning architectures explored in Sections 2 and 3 respectively. Are the assumptions used in the theoretical results for LSH also assumed in the deep networks? For example, as far as I know, the standard LSH works assumes the projection hyperplanes are randomly chosen and the theoretical results are based on such assumptions. It is not clear how a softmax output of a CNN, which is trained in a supervised way, follow such assumptions. It would be important if the paper could clarify such assumptions to make sure the sections are congruent.
3) No Related Work
There have been several efforts for adapting deep frameworks into KNN. The paper ignores all such works. Thus, it is not clear how significant is the proposed contribution. There are also not comparisons what-so-ever to competitive prior works.
4) Novelty
The main contribution of this paper is basically a set of experiments looking into architectural choices. However, the results of this study do not provide any surprises. It appears that batch normalization is essential for good performances, while using RELU is not so when one wants to use all directions for effective data encoding. Thus, as such, the novelty or the contributions of this paper are minor.
Overall, while I find there are some interesting theoretical bits in this paper, it lacks focus, the experiments do not offer any surprises, and there are no comparisons with prior literature. Thus, I do not think this paper is ready to be accepted in its present form.


The recent success of deep neural networks stems from their ability to generalize well on real data; however, Zhang et al. (Zhang et al., 2016)  have observed that neural networks can easily overfit random labels. This observation demonstrates that with the existing theory, we cannot adequately explain why gradient methods can find generalizable solutions for neural networks. In this work, we use a Fourierbased approach to study the generalization properties of gradient-based methods over 2-layer neural networks with sinusoidal activation functions. We prove that if the underlying distribution of data has nice spectral properties such as bandlimitedness, then the gradient descent method will converge to generalizable local minima. We also establish a Fourier-based generalization bound for bandlimited spaces, which generalizes to other activation functions. Our generalization bound motivates a grouped version of path norms for measuring the complexity of 2-layer neural networks with ReLU activation functions. We demonstrate numerically that regularization of this group path norm results in neural network solutions that can fit true labels without losing test accuracy while not overfitting random labels.

This work proposes to study the generalization of learning neural networks via the Fourier-based method. It first gives a Fourier-based generalization bound, showing that Rademacher complexity of functions with small bandwidth and Fourier l_1 norm will be small. This leads to generalization for 2-layer networks with appropriate bounded size. For 2-layer networks with sine activation functions, assuming that the data distribution has nice spectral property (ie bounded bandwidth), it shows that the local minimum of the population risk (if with isolated component condition) will have small size, and also shows that the gradient of the empirical risk is close to that of the population risk. Empirical results show that the size of the networks learned on random labels are larger than those learned on true labels, and shows that a regularizer implied by their Fourier-based generalization bound can effectively reduce the generalization gap on random labels.
The idea of applying the Fourier-based method to generalization is interesting. However, the theoretical results are not very satisfactory.
-- How do the bounds here compared to those obtained by directly applying Rademacher complexity to the neural network functions?
-- How to interpret the isolated components condition in Theorem 4? Basically, it means that B(P_X) should be a small constant. What type of distributions of X will be a good example?
-- It is not easy to put together the conclusions in Section 6.1 and 6.2. Suppose SGD leads to a local minimum of the empirical loss. One can claim that this is an approximate local minimum (ie, small gradient) by Corollary 3. But to apply Theorem 4, one will need a version of Theorem 4 for approximate local minima. Also, one needs to argue that the local minimum obtained by SGD will satisfy the isolated component condition. The argument in Section 8.6 is not convincing, ie, there is potentially a large approximation error in (41) and one cannot claim that Lemma 1 and Theorem 4 are still valid without the isolated component condition.


Published as a conference paper at ICLR 2018 MEMORY AUGMENTED CONTROL NETWORKS
Planning problems in partially observable environments cannot be solved directly with convolutional networks and require some form of memory. But, even memory networks with sophisticated addressing schemes are unable to learn intelligent reasoning satisfactorily due to the complexity of simultaneously learning to access memory and plan. To mitigate these challenges we propose the Memory Augmented Control Network (MACN). The network splits planning into a hierarchical process. At a lower level, it learns to plan in a locally observed space. At a higher level, it uses a collection of policies computed on locally observed spaces to learn an optimal plan in the global environment it is operating in. The performance of the network is evaluated on path planning tasks in environments in the presence of simple and complex obstacles and in addition, is tested for its ability to generalize to new environments not seen in the training set.

The paper presents a method for navigating in an unknown and partially observed environment is presented. The proposed approach splits planning into two levels: 1) local planning based on the observed space and 2) a global planner which receives the local plan, observation features, and access to an addressable memory to decide on which action to select and what to write into memory.
The contribution of this work is the use of value iteration networks (VINs) for local planning on a locally observed map that is fed into a learned global controller that references history and a differential neural computer (DNC), local policy, and observation features select an action and update the memory. The core concept of learned local planner providing additional cues for a global, memory-based planner is a clever idea and the thorough analysis clearly demonstrates the benefit of the approach.
The proposed method is tested against three problems: a gridworld, a graph search, and a robot environment. In each case the proposed method is more performant than the baseline methods.  The ablation study of using LSTM instead of the DNC and the direct comparison of CNN + LSTM support the authors’ hypothesis about the benefits of the two components of their method. While the author’s compare to DRL methods with limited horizon (length 4), there is no comparison to memory-based RL techniques. Furthermore, a comparison of related memory-based visual navigation techniques on domains for which they are applicable should be considered as such an analysis would illuminate the relative performance over the overlapping portions problem domains  For example, analysis of the metric map approaches on the grid world or of MACN on their tested environments.
Prior work in visual navigation in partially observed and unknown environments have used addressable memory (e.g., Oh et al.) and used VINs (e.g., Gupta et al.) to plan as noted. In discussing these methods, the authors state that these works are not comparable as they operate strictly on discretized 2d spaces. However, it appears to the reviewer that several of these methods can be adapted to higher dimensions and be applicable at least a subclass (for the euclidean/metric map approaches) or the full class of the problems (for Oh et al.), which appears to be capable to solve non-euclidean tasks like the graph search problem. If this assessment is correct, the authors should differentiate between these approaches more thoroughly and consider empirical comparisons. The authors should further consider contrasting their approach with “Neural SLAM” by Zhang et al.
A limitation of the presented method is requirement that the observation “reveals the labeling of nearby states.” This assumption holds in each of the examples presented: the neighborhood map in the gridworld and graph examples and the lidar sensor in the robot navigation example. It would be informative for the authors to highlight this limitation and/or identify how to adapt the proposed method under weaker assumptions such as a sensor that doesn’t provide direct metric or connectivity information such as a RGB camera.
Many details of the paper are missing and should be included to clarify the approach and ensure reproducible results. The reviewer suggests providing both more details in the main section of the paper and providing the precise architecture including hyperparameters in the supplementary materials section.


We study the precise mechanisms which allow autoencoders to encode and decode a simple geometric shape, the disk. In this carefully controlled setting, we are able to describe the specific form of the optimal solution to the minimisation problem of the training step. We show that the autoencoder indeed approximates this solution during training. Secondly, we identify a clear failure in the generalisation capacity of the autoencoder, namely its inability to interpolate data. Finally, we explore several regularisation schemes to resolve the generalisation problem. Given the great attention that has been recently given to the generative capacity of neural networks, we believe that studying in depth simple geometric cases sheds some light on the generation process and can provide a minimal requirement experimental setup for more complex architectures.

The paper considers a toy problem: the space of images of discs of variable radius - a one dimensional manifold.
An autoencoder based on convolutional layers with ReLU is experimented with, with a 1D embedding.
It is shown that
1) if the bias is not included, the resulting function is homogeneous (meaning f(ax)=af(x)), and so it fails because the 1D representation should be the radius, and the relationship from radius to image is more complex than a homogeneous function.
- if we include the bias and L2 regularise only the encoder weights, it works better in terms of interpolation for a limited data sample.
The thing is that 1) is trivial (the composition of homogeneous functions is homogeneous... so their proof is overly messy btw). Then, they continue by further analysing (see proposition 2) the solution for this case. Such analysis does not seem to shed much light on anything relevant, given that we know the autoencoder fails in this case due to the trivial proposition 1.
Another point: since the homogeneous function problem will not arise for other non-linearities (such as the sigmoid), the focus on the bias as the culprit seems arbitrary.
Then, the story about interpolation and regularisation is kind of orthogonal, and then is solved by an arbitrary regularisation scheme. The lesson learned from this case is basically the second last paragraph of section 3.2. In other words, it just works.
Since it's a toy problem anyway, the insights seem somewhat trivial.
On the plus side, such a toy problem seems like it might lead somewhere interesting. I'd like to see a similar setup but with a suite of toy problems. e.g. vary the aspect ratio of an oval (rather than a disc), vary the position, intensity, etc etc.


Action segmentation as a milestone towards building automatic systems to understand untrimmed videos has received considerable attention in the recent years. It is typically being modeled as a sequence labeling problem but contains intrinsic and sufficient differences than text parsing or speech processing. In this paper, we introduce a novel hybrid temporal convolutional and recurrent network (TricorNet), which has an encoder-decoder architecture: the encoder consists of a hierarchy of temporal convolutional kernels that capture the local motion changes of different actions; the decoder is a hierarchy of recurrent neural networks that are able to learn and memorize long-term action dependencies after the encoding stage. Our model is simple but extremely effective in terms of video sequence labeling. The experimental results on three public action segmentation datasets have shown that the proposed model achieves superior performance over the state of the art.

I will be upfront: I have already reviewed this paper when it was submitted to NIPS 2017, so this review is based heavily on the NIPS submission.
I am quite concerned that this paper has been resubmitted as it is, word by word, character by character. The authors could have benefited from the feedback they obtained from the reviewers of their last submissions to improved their paper, but nothing has been done. Even very easy remarks, like bolding errors (see below) have been kept in the paper.
The proposed paper describes a method for video action segmentation, a task where the video must be temporally densely labeled by assigned an action (sub) class to each frame. The method proceeds by extracting frame level features using convolutional networks and then passing a temporal encoder-decoder in 1D over the video, using fully supervised training.
On the positive side, the method has been tested on 3 different datasets, outperforming the baselines (recent methods from 2016) on 2 of them.
My biggest concern with the paper is novelty. A significant part of the paper is based on reference [Lea et al. 2017], the differences being quite incremental. The frame-level features are the same as in [Lea et al. 2017], and the basic encoder-decoder strategy is also taken from [Lea et al. 2017]. The encoder is also the same. Even details are reproduced, as the choice of normalized Relu activations.
The main difference seems to me that the decoder is not convolutional, but a recurrent network.
The encoder-decoder architecture seems to be surprisingly shallow, with only K=2 layers at each side.
The paper is well written and can be easily understood. However, a quite large amount of space is wasted on obvious and known content, as for example the basic equation for a convolutional layer (equation (1)) and the following half page of text and equations of LSTM and Bi-directional LSTM networks. This is very well known and the space can be used for more details on the paper's contributions.
While the paper is generally well written, there are a couple of exceptions in the form of ambiguous sentences, for example the lines before section 3.
There is a bolding error in table 2, where the proposed method is not state of the art (as indicated) w.r.t. to the accuracy metric.
To sum it up, the positive aspect of nicely executed experiments is contrasted by low novelty of the method.  To be honest, I am not totally sure whether the contribution of the paper should be considered as a new method or as architectural optimizations of an existing one. This is corroborated by the experimental results on the first two datasets (tables 2 and 3): on 50 salads, where ref. [Lea et al. 2017]. seems currently to obtain state of the art performance, the improvement obtained by the proposed method allows it to get state of the art performance. On GTEA, where [Lea et al. 2017] does not currently deliver state of the art performance, the proposed method performs (slightly) better than [Lea et al. 2017] but does not obtain state of the art performance.
On the third dataset, JIGSAWS, reference [Lea et al. 2017]. has not been tested, which is peculiar given the closeness.


Suppose a deep classification model is trained with samples that need to be kept private for privacy or confidentiality reasons. In this setting, can an adversary obtain the private samples if the classification model is given to the adversary? We call this reverse engineering against the classification model the Classifier-toGenerator (C2G) Attack. This situation arises when the classification model is embedded into mobile devices for offline prediction (e.g., object recognition for the automatic driving car and face recognition for mobile phone authentication). For C2G attack, we introduce a novel GAN, PreImageGAN. In PreImageGAN, the generator is designed to estimate the the sample distribution conditioned by the preimage of classification model f , P (X|f (X) = y), where X is the random variable on the sample space and y is the probability vector representing the target label arbitrary specified by the adversary. In experiments, we demonstrate PreImageGAN works successfully with hand-written character recognition and face recognition. In character recognition, we show that, given a recognition model of hand-written digits, PreImageGAN allows the adversary to extract alphabet letter images without knowing that the model is built for alphabet letter images. In face recognition, we show that, when an adversary obtains a face recognition model for a set of individuals, PreImageGAN allows the adversary to extract face images of specific individuals contained in the set, even when the adversary has no knowledge of the face of the individuals.

The paper proposes the use of a GAN to learn the distribution of image classes from an existing classifier, that is a nice and straightforward idea. From the point of view of forensic analysis of a classifier, it supposes a more principled strategy than a brute force attack based on the classification of a database and some conditional density estimation of some intermediate image features. Unfortunately, the experiments are inconclusive.
Quality: The key question of the proposed scheme is the role of the auxiliary dataset. In the EMNIST experiment, the results for the “exact same” and “partly same” situations are good, but it seems that for the “mutually exclusive” situation the generated samples look like letters, not numbers, and raises the question on the interpolation ability of the generator. In the FaceScrub experiment is even more difficult to interpret the results, basically because we do not even know the full list of person identities. It seems that generated images contain only parts of the auxiliary images related to the most discriminative features of the given classifier. Does this imply that the GAN models a biased probability distribution of the image class? What is the result when the auxiliary dataset comes from a different kind of images? Due to the difficulty of evaluating GAN results, more experiments are needed to determine the quality and significance of this work.
Clarity: The paper is well structured and written, but Sections 1-4 could be significantly shorter to leave more space to additional and more conclusive experiments. Some typos on Appendix A should be corrected.
Originality: the paper is based on a very smart and interesting idea and a straightforward use of GANs.
Significance: If additional simulations confirm the author’s claims, this work can represent a significant contribution to the forensic analysis of discriminative classifiers.


Published as a conference paper at ICLR 2018 NEURAL MAP: STRUCTURED MEMORY FOR DEEP RE- INFORCEMENT LEARNING
A critical component to enabling intelligent reasoning in partially observable environments is memory. Despite this importance, Deep Reinforcement Learning (DRL) agents have so far used relatively simple memory architectures, with the main methods to overcome partial observability being either a temporal convolution over the past k frames or an LSTM layer. More recent work (Oh et al., 2016) has went beyond these architectures by using memory networks which can allow more sophisticated addressing schemes over the past k frames. But even these architectures are unsatisfactory due to the reason that they are limited to only remembering information from the last k frames. In this paper, we develop a memory system with an adaptable write operator that is customized to the sorts of 3D environments that DRL agents typically interact with. This architecture, called the Neural Map, uses a spatially structured 2D memory image to learn to store arbitrary information about the environment over long time lags. We demonstrate empirically that the Neural Map surpasses previous DRL memories on a set of challenging 2D and 3D maze environments and show that it is capable of generalizing to environments that were not seen during training.

This paper presents a fully differentiable neural architecture for mapping and path planning for navigation in previously unseen environments, assuming near perfect* relative localization provided by velocity. The model is more general than the cognitive maps (Gupta et al, 2017) and builds on the NTM/DNC or related architectures (Graves et al, 2014, 2016, Rae et al, 2017) thanks to the 2D spatial structure of the associative memory. Basically, it consists of a 2D-indexed grid of features (the map) M_t that can be summarized at each time point into read vector r_t, and used for extracting a context c_t for the current agent state s_t, compute (thanks to an LSTM/GRU) an updated write vector w_{t+1}^{x,y} at the current position and update the map using that write vector. The position {x,y} is a binned representation of discrete or continuous coordinates. The absolute coordinate map can be replaced by a relative ego-centric map that is shifted (just like in Gupta et al, 2017) as the agent moves.
The experiments are exhaustive and include remembering the goal location with or without cues (similarly to Mirowski et al, 2017, not cited) in simple mazes of size 4x4 up to 8x8 in the 3D Doom environment. The most important aspect is the capability to build a feature map of previously unseen environments.
This paper, showing excellent and important work, has already been published on arXiv 9 months ago and widely cited. It has been improved since, through different sets of experiments and apparently a clearer presentation, but the ideas are the same. I wonder how it is possible that the paper has not been accepted at ICML or NIPS (assuming that it was actually submitted there). What are the motivations of the reviewers who rejected the paper - are they trying to slow down competing research, or are they ignorant, and is the peer review system broken? I quite like the formulation of the NIPS ratings: "if this paper does not get accepted, I am considering boycotting the conference".
* The noise model experiment in Appendix D is commendable, but the noise model is somewhat unrealistic (very small variance, zero mean Gaussian) and assumes only drift in x and y, not along the orientation. While this makes sense in grid world environments or rectilinear mazes, it does not correspond to realistic robotic navigation scenarios with wheel skid, missing measurements, etc... Perhaps showing examples of trajectories with drift added would help convince the reader (there is no space restriction in the appendix).


Published as a conference paper at ICLR 2018 SPATIALLY TRANSFORMED ADVERSARIAL EXAMPLES
Recent studies show that widely used deep neural networks (DNNs) are vulnerable to carefully crafted adversarial examples. Many advanced algorithms have been proposed to generate adversarial examples by leveraging the L p distance for penalizing perturbations. Researchers have explored different defense methods to defend against such adversarial attacks. While the effectiveness of L p distance as a metric of perceptual quality remains an active research area, in this paper we will instead focus on a different type of perturbation, namely spatial transformation, as opposed to manipulating the pixel values directly as in prior works. Perturbations generated through spatial transformation could result in large L p distance measures, but our extensive experiments show that such spatially transformed adversarial examples are perceptually realistic and more difficult to defend against with existing defense systems. This potentially provides a new direction in adversarial example generation and the design of corresponding defenses. We visualize the spatial transformation based perturbation for different examples and show that our technique can produce realistic adversarial examples with smooth image deformation. Finally, we visualize the attention of deep networks with different types of adversarial examples to better understand how these examples are interpreted.

This paper creates adversarial images by imposing a flow field on an image such that the new spatially transformed image fools the classifier. They minimize a total variation loss in addition to the adversarial loss to create perceptually plausible adversarial images, this is claimed to be better than the normal L2 loss functions.
Experiments were done on MNIST, CIFAR-10, and ImageNet, which is very useful to see that the attack works with high dimensional images. However, some numbers on ImageNet would be helpful as the high resolution of it make it potentially different than the low-resolution MNIST and CIFAR.
It is a bit concerning to see some parts of Fig. 2. Some of Fig. 2 (especially (b)) became so dotted that it no longer seems an adversarial that a human eye cannot detect. And model B in the appendix looks pretty much like a normal model. It might needs some experiments, either human studies, or to test it against an adversarial detector, to ensure that the resulting adversarials are still indeed adversarials to the human eye. Another good thing to run would be to try the 3x3 average pooling restoration mechanism in the following paper:
Xin Li, Fuxin Li. Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics . ICCV 2017.
to see whether this new type of adversarial example can still be restored by a 3x3 average pooling the image (I suspect that this is harder to restore by such a simple method than the previous FGSM or OPT-type, but we need some numbers).
I also don't think FGSM and OPT are this bad in Fig. 4. Are the authors sure that if more regularization are used these 2 methods no longer fool the corresponding classifiers?
I like the experiment showing the attention heat maps for different attacks. This experiment shows that the spatial transforming attack (stAdv) changes the attention of the classifier for each target class, and is robust to adversarially trained Inception v3 unlike other attacks like FGSM and CW.
I would likely upgrade to a 7 if those concerns are addressed.
After rebuttal: I am happy with the additional experiments and would like to upgrade to an accept.


Published as a conference paper at ICLR 2018 LEARNING WASSERSTEIN EMBEDDINGS
The Wasserstein distance received a lot of attention recently in the community of machine learning, especially for its principled way of comparing distributions. It has found numerous applications in several hard problems, such as domain adaptation, dimensionality reduction or generative models. However, its use is still limited by a heavy computational cost. Our goal is to alleviate this problem by providing an approximation mechanism that allows to break its inherent complexity. It relies on the search of an embedding where the Euclidean distance mimics the Wasserstein distance. We show that such an embedding can be found with a siamese architecture associated with a decoder network that allows to move from the embedding space back to the original input space. Once this embedding has been found, computing optimization problems in the Wasserstein space (e.g. barycenters, principal directions or even archetypes) can be conducted extremely fast. Numerical experiments supporting this idea are conducted on image datasets, and show the wide potential benefits of our method.

This paper proposes approximating the Wasserstein distance between normalized greyscale images based on a learnable approximately isometric embedding of images into Euclidean space. The paper is well written with clear and generally thorough prose. It presents a novel, straightforward and practical solution to efficiently computing Wasserstein distances and performing related image manipulations.
Major comments:
It sounds like the same image may be present in the training set and eval set. This is methodologically suspect, since the embedding may well work better for images seen during training. This affects all experimental results.
I was pleased to see a comparison between using exact and approximate Wasserstein distances for image manipulation in Figure 5, since that's a crucial aspect of whether the method is useful in practice. However the exact computation (OT LP) appears to be quite poor. Please explain why the approximation is better than the exact Wasserstein difference for interpolation. Relatedly, please summarize the argument in Cuturi and Peyre that is cited ("as already explained in").
Minor comments:
In section 3.1 and 4.1, "histogram" is used to mean normalized-to-sum-to-1 images, which is not the conventional meaning.
It would help to pick one of "Wasserstein Deep Learning" and "Deep Wasserstein Embedding" and use it and the acronym consistently throughout.
"Disposing of a decoder network" in section 3.1 should be "using a decoder network"?
In section 4.1, the architectural details could be clarified. What size are the input images? What type of padding for the convolutions? Was there any reason behind the chosen architecture? In particular the use of a dense layers followed by convolutional layers seems peculiar.
It would be helpful to say explicitly what "quadratic ground metric" means (i.e. W_2, I presume) in section 4.2 and elsewhere.
It would be helpful to give a sense of scale for the numbers in Table 1, e.g. give the 95th percentile Wasserstein distance. Perhaps use the L2 distance passed through a 1D-to-1D learned warping as a baseline.
Mention that OT stands for optimal transport in section 4.3.
Suggest mentioning "there is no reason for a Wasserstein barycenter to be a realistic sample" in the main text when first discussing barycenters.


MIXED PRECISION TRAINING OF CONVOLUTIONAL NEURAL NETWORKS USING INTEGER OPERATIONS
The state-of-the-art (SOTA) for mixed precision training is dominated by variants of low precision floating point operations, and in particular FP16 accumulating into FP32 . On the other hand, while a lot of research has also happened in the domain of low and mixed-precision Integer training, these works either present results for non-SOTA networks (for instance only AlexNet for ImageNet-1K), or relatively small datasets (like CIFAR-10). In this work, we train state-of-the-art visual understanding neural networks on ImageNet-1K dataset, with Integer operations on General Purpose (GP) hardware. In particular, we focus on Integer Fused-Multiply-and-Accumulate (FMA) operations which take two pairs of INT16 operands and accumulate results into an INT32 output.We propose a shared exponent representation of tensors, and develop a Dynamic Fixed Point (DFP) scheme suitable for common neural network operations. The nuances of developing an efficient integer convolution kernel is examined, including methods to handle overflow of the INT32 accumulator. We implement CNN training for ResNet-50, GoogLeNet-v1, VGG-16 and AlexNet; and these networks achieve or exceed SOTA accuracy within the same number of iterations as their FP32 counterparts without any change in hyper-parameters and with a 1.8X improvement in end-to-end training throughput. To the best of our knowledge these results represent the first INT16 training results on GP hardware for ImageNet-1K dataset using SOTA CNNs and achieve highest reported accuracy using half precision representation.

This paper describes an implementation of reduced precision deep learning using a 16 bit integer representation. This field has recently seen a lot of publications proposing various methods to reduce the precision of weights and activations. These schemes have generally achieved close-to-SOTA accuracy for small networks on datasets such as MNIST and CIFAR-10. However, for larger networks (ResNET, Vgg, etc) on large dataset such as ImageNET, a significant accuracy drop are reported. In this work, the authors show that a careful implementation of mixed-precision dynamic fixed point computation can achieve SOTA on 4 large networks on the ImageNET-1K datasets. Using a INT16 (as opposed to FP16) has the advantage of enabling the use of new SIMD mul-acc instructions such as QVNNI16.
The reported accuracy numbers show convincingly that INT16 weights and activations can be used without loss of accuracy in large CNNs. However, I was hoping to see a direct comparison between FP16 and INT16.
The paper is written clearly and the English is fine.


We propose a "Learning to Select" problem that selects the best among the flexible size candidates. This makes decisions based not only on the properties of the candidate, but also on the environment in which they belong to. For example, job dispatching in the manufacturing factory is a typical "Learning to Select" problem. We propose Variable-Length CNN which combines the classification power using hidden features from CNN and the idea of flexible input from Learning to Rank algorithms. This not only can handles flexible candidates using Dynamic Computation Graph, but also is computationally efficient because it only builds a network with the necessary sizes to fit the situation. We applied the algorithm to the job dispatching problem which uses the dispatching log data obtained from the virtual fine-tuned factory. Our proposed algorithm shows considerably better performance than other comparable algorithms.

The paper proposed a new framework called `Learning to select’, in which a best candidate needs to be identified in the decision making process such as job dispatching. A CNN architecture is designed, called `Variable-Length CNN’, to solve this problem.
My major concern is on the definition of the proposed concept of `learning-to-select’. Essentially, I’ve not seen its key difference from the classification problem. While `even in the case of completely identical candidates, the label can be 1 in some situations, and in some other situations the label can be 0’, why not including such `situations’ into your feature vector (i.e., x)? Once you do it, the gap between learning to select and classification will vanish. If this is not doable, you should better make more discussions, especially on what the so-called `situations’ are.  Furthermore, the application scope of the proposed framework is not very well discussed. If it is restricted to job dispatching scenarios, why do we need a new concept “learning to select”?
The proposed model looks quite straightforward. Standard CNN is able to capture the variable length input as is done in many NLP tasks. Dynamic computational graph is not new either. In this sense, the technical novelty of this work is somehow limited.
The experiments are weak in that the data are simulated and the baselines are not strong. I’ve not gained enough insights on why the proposed model could outperform the alternative approaches. More discussions and case studies are sorely needed.


In this work we present BinaryFlex, a neural network architecture that learns weighting coefficients of predefined orthogonal binary basis instead of the conventional approach of learning directly the convolutional filters. We have demonstrated the feasibility of our approach for complex computer vision datasets such as ImageNet. Our architecture trained on ImageNet is able to achieve top-5 accuracy of 65.7% while being around 2x smaller than binary networks capable of achieving similar accuracy levels. By using deterministic basis, that can be generated on-the-fly very efficiently, our architecture offers a great deal of flexibility in memory footprint when deploying in constrained microcontroller devices.

This paper proposes using a set of orthogonal basis and their combination to represent convolutional kernels. To learn the set of basis,  the paper uses an existing algorithm (OSVF)
-- Related Work
Related work suggests there is redundancy in the number of parameters (According to Denil et al) but the training can be done by learning a subset directly without drop in accuracy. I am not really sure this is strictly correct as many approaches (including Denil et al) suggest the additional parameters are needed to help the optimization process (therefore hard to learn directly a small model).
As in the clarity point below, please be consistent. Acronyms are not properly defined.
-- Method / Clarity
It is nice to read section 3.1 but at the same time probably redundant as it does not add any value (at least the first two paragraphs including Eq. 1). Reading the text, it is not clear to me why there is a lower number of parameters to be updated. To the best of my understanding so far in the explanation, the number of parameters is potentially the same but represented using a single bit. Rephrasing this section would probably improve readability.
Runtime is potentially reduced but not clear in current hardware.
Section 3.2 is nice as short overview but happens to take more than the actual proposal (so I get lost).
Figures 2 and 3. I am surprissed the FlexModule (a building block of BinaryFlex) is not mentioned in the binaryflex architecture and then, sparse blocks are not defined anywhere. Would be nice to be consistent here. Also note that filter banks among other details are not defined.
Now, w and b in eq 2 are meant to be binary, is that correct? The text defines them as real valued so this is confusing.
- From the explanations in the text, it is not clear to me how the basis and the weights are learned (except using backprop). How do we actually generate the filter bank, is this from scratch? or after some pretraining / preloaded model? What is the difference between BinaryFlex models and how do I generate them when replicating these results? It is correct to assume f_k is a pretrained kernel that is going to be approximated?

-- more on clarity
I would also appreciate rephrasing some parts of the paper. For instance, the paragraph under section 4.1 is confusing. There is no consistency with namings / acronyms and seems not to be in the right order. Note that the paragraph starts talking about ImageNet and then suggests different schedules for different datasets. The naming for state-of-the-art methods is not consistent.
Also note that acronyms are later used (such as BWN) but not defined here. This should be easy to improve.
I guess Figurre 4 needs clarification. What are the axis? Why square and cercles? Same for Figure 5.
Overall, text needs reviewing. There are typos all over the text. I think ImageNet is not a task but classification using ImageNet.
-- Results
I find it hard to follow the results. Section 4.1.1 suggests accuracy is comparable when constraints are relaxed and then only 7% drop in accuracy for a 4.5x model reduction. I have not been able to match these numbers with those in table 2. How do I get to see 7% lower accuracy for BinaryFlex-1.6?
Results suggest a model under 2MB is convenient for using in ARM, is this actually a fact (is it tested in an ARM?) or just guessing? This is also a point made in the introduction and I would expect at least an example of running time there (showing the benefit compared to competitors). It is also interesting the fact that in the text, the ARM is said to have 512KB while in the experiments there is no model achieving that lower-bound.
I would like to see an experiment on ImageNet where the proposed BinaryFlex leads to a model of approximately 7.5MB and see what the preformance is for that model (so comparable in size with the state-of-the-art).
I missed details for the exact implementation for the other datsets (as said in the paper). There are modifications that are obscure and the benefits in model size (at least compared to a baseline) are not mentioned. Why?


Most existing GANs architectures that generate images use transposed convolution or resize-convolution as their upsampling algorithm from lower to higher resolution feature maps in the generator. We argue that this kind of fixed operation is problematic for GANs to model objects that have very different visual appearances. We propose a novel adaptive convolution method that learns the upsampling algorithm based on the local context at each location to address this problem. We modify a baseline GANs architecture by replacing normal convolutions with adaptive convolutions in the generator. Experiments on CIFAR-10 dataset show that our modified models improve the baseline model by a large margin. Furthermore, our models achieve state-of-the-art performance on CIFAR-10 and STL-10 datasets in the unsupervised setting.

The paper operates under the hypothesis that the rigidity of the convolution operator is responsible in part for the poor performance of GANs on diverse visual datasets. The authors propose to replace convolutions in the generator with an Adaptive Convolution Block, which learns to generate the convolution weights and biases of the upsampling operation adaptively for each pixel location. State-of-the-art Inception scores are presented for the CIFAR-10 and STL-10 datasets.
I think the idea of leveraging adaptive convolutions in decoder-based models is compelling, especially given its success in video frame interpolation, which makes me wonder why the authors chose to restrict themselves to GANs. Wouldn't the arguments used to justify replacing regular convolutions in the generator with adaptive convolution blocks apply equally well to any other decoder-based generative model, like a VAE, for instance?
I find the paper lacking on the evaluation front. The evaluation of GANs is still very much an open research problem, which means that making a compelling case for the effectiveness of a proposed method requires nuance and contextualization. The authors claim a state-of-the-art Inception score but fail to explain what argument this claim supports. This is important, because the Inception score is not a universal measure of GAN performance: it provides a specific view on the ability of a generator to cover human-defined modes in the data distribution, but it does not inform on intra-class mode coverage and is blind to things like the generator collapsing on one or a few template samples per class.
I am also surprised that the relationship with HyperNetworks [1] is not outlined, given that both papers leverage the idea of factoring network parameters through a second neural network.
Some additional comments:
- Figure 1 should be placed much earlier in the paper, preferably above Section 3. In its current state, the paper provides a lot of mathematical notation to digest without any visual support.
- "[...] a transposed convolution is equivalent to a convolution [...]": This is inaccurate. A convolution's backward pass is a transposed convolution and vice versa, but they are not equivalent (especially when non-unit strides are involved).
- "The difficulties of training GANs is well known": There is a grammatical error in this sentence.
- "If [the discriminator] is too strong, log(1 - D(G(z))) will be close to zero and there would be almost no gradient [...]": This is only true for the minimax GAN objective, which is almost never used in practice. The non-saturating GAN objective does not exhibit this issue, as [2] re-iterated recently.
- "Several works have been done [...]": There is a grammatical error here.
- The WGAN-GP citation is wrong (Danihelka et al. rather than Gulrajani et al.).
Overall, the paper's lack of sufficient convincing empirical support prevents me from recommending its acceptance.
References:
[1] Ha, D., Dai, A., and Le, Q. V. (2016). HyperNetworks. arXiv:1609.09106.
[2] Fedus, W., Rosca, M., Lakshminarayanan, B., Dai, A. M., Mohamed, S., and Goodfellow, I. (2017). Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step. arXiv:1710.08446.


State-action value functions (i.e., Q-values) are ubiquitous in reinforcement learning (RL), giving rise to popular algorithms such as SARSA and Q-learning. We propose a new notion of action value defined by a Gaussian smoothed version of the expected Q-value. We show that such smoothed Q-values still satisfy a Bellman equation, making them learnable from experience sampled from an environment. Moreover, the gradients of expected reward with respect to the mean and covariance of a parameterized Gaussian policy can be recovered from the gradient and Hessian of the smoothed Q-value function. Based on these relationships we develop new algorithms for training a Gaussian policy directly from a learned smoothed Q-value approximator. Our approach is amenable to proximal optimization techniques by augmenting the objective with a penalty on KLdivergence from a previous policy. We find that the ability to learn both a mean and covariance during training allows this approach to achieve much better results on standard continuous control benchmarks.

This paper explores the idea of using policy gradients to learn a stochastic policy on complex control problems.  The central idea is to frame learning in terms of a new kind of Q-value that attempts to smooth out Q-values by framing them in terms of expectations over Gaussian policies.
To be honest, I didn't really "get" this paper.
* As far I understand, all of the original work policy gradients involved stochastic policies.  Many are/were Gaussian.
* All Q-value estimators are designed to marginalize out the randomness in these stochastic policies.
* As far as I can tell, this is equivalent to a slightly different formulation, where the agent emits a deterministic action (\mu,\Sigma) and the environment samples an action from that distribution.  In other words, it seems that if we just draw the box a bit differently, the environment soaks up the nondeterminism, instead of needing to define a new type of Q-value.
Ultimately, I couldn't discern /why/ this was a significant advance for RL, or even a meaningful new perspective on classic ideas.
I thought the little 2-mode MOG was a nice example of the premise of the model.
While I may or may not have understood the core technical contribution, I think the experiments can be critiqued: they didn't really seem to work out.  Figures 2&3 are unconvincing - the differences do not appear to be statistically significant.  Also, I was disappointed to see that the authors only compared to DDPG; they could have at least compared to TRPO, which they mention.  They dismiss it by saying that it takes 10 times as long, but gets a better answer - to which I respond, "Very well, run your algorithm 10x longer and see where you end up!"  I think we need to see a more compelling demonstration of why this is a useful idea before it's ready to be published.
The idea of penalizing a policy based on KL-divergence from a reference policy was explored at length by Bert Kappen's work on KL-MDPs.  Perhaps you should cite that?


WHAI: WEIBULL HYBRID AUTOENCODING INFERENCE FOR DEEP TOPIC MODELING
To train an inference network jointly with a deep generative topic model, making it both scalable to big corpora and fast in out-of-sample prediction, we develop Weibull hybrid autoencoding inference (WHAI) for deep latent Dirichlet allocation, which infers posterior samples via a hybrid of stochastic-gradient MCMC and autoencoding variational Bayes. The generative network of WHAI has a hierarchy of gamma distributions, while the inference network of WHAI is a Weibull upward-downward variational autoencoder, which integrates a deterministicupward deep neural network, and a stochastic-downward deep generative model based on a hierarchy of Weibull distributions. The Weibull distribution can be used to well approximate a gamma distribution with an analytic Kullback-Leibler divergence, and has a simple reparameterization via the uniform noise, which help efficiently compute the gradients of the evidence lower bound with respect to the parameters of the inference network. The effectiveness and efficiency of WHAI are illustrated with experiments on big corpora.

The authors propose a hybrid Bayesian inference approach for deep topic models that integrates stochastic gradient MCMC for global parameters and Weibull-based multilayer variational autoencoders (VAEs) for local parameters. The decoding arm of the VAE consists of deep latent Dirichlet allocation, and an upward-downward structure for the encoder. Gamma distributions are approximated as Weibull distributions since the Kullback-Leibler divergence is known and samples can be efficiently drawn from a transformation of samples from a uniform distribution.
The results in Table 1 are concerning for several reasons, i) the proposed approach underperfroms DLDA-Gibbs and DLDA-TLASGR. ii) The authors point to the scalability of the mini-batch-based algorithms, however, although more expensive, DLDA-Gibbs, is not prohibitive given results for Wikipedia are provided. iii) The proposed approach is certainly faster at test time, however, it is not clear to me in which settings such speed (compared to Gibbs) would be needed, provided the unsupervised nature of the task at hand. iv) It is not clear to me why there is no test-time difference between WAI and WHAI, considering that in the latter, global parameters are sampled via stochastic-gradient MCMC. One possible explanation being that during test time, the approach does not use samples from W but rather a summary of them, say posterior means, in which case, it defeats the purpose of sampling from global parameters, which may explain why WAI and WHAI perform about the same in the 3 datasets considered.
- \Phi is in a subset of R_+, in fact, columns of \Phi are in the P_0-dimensional simplex.
- \Phi should have K_1 columns not K.
- The first paragraph in Page 5 is very confusing because h is introduced before explicitly connecting it to k and \lambda. Also, if k = \lambda, why introduce different notations?


To construct representations for natural language sequences, information from two main sources needs to be captured: (i) semantic meaning of individual words, and (ii) their compositionality. These two types of information are usually represented in the form of word embeddings and compositional functions, respectively. For the latter, Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs) have been considered. There has not been a rigorous evaluation regarding the relative importance of each component to different text-representation-based tasks; i.e., how important is the modeling capacity of word embeddings alone, relative to the added value of a compositional function? In this paper, we conduct an extensive comparative study between Simple Word Embeddings-based Models (SWEMs), with no compositional parameters, relative to employing word embeddings within RNN/CNN-based models. Surprisingly, SWEMs exhibit comparable or even superior performance in the majority of cases considered. Moreover, in a new SWEM setup, we propose to employ a max-pooling operation over the learned word-embedding matrix of a given sentence. This approach is demonstrated to extract complementary features relative to the averaging operation standard to SWEMs, while endowing our model with better interpretability. To further validate our observations, we examine the information utilized by different models to make predictions, revealing interesting properties of word embeddings.

This paper empirically investigates the differences realized by using compositional functions over word embeddings as compared to directly operating the word embeddings. That is, the authors seek to explore the advantages afforded by RNN/CNN based models that induce intermediate semantic representations of texts, as opposed to simpler (parameter-free) approaches to composing these, like addition.
In sum, I think this is exploration is interesting, and suggests that we should perhaps experiment more regularly with simple aggregation methods like SWEM. On the other hand, the differences across the models is relatively modest, and the data resists clear conclusions, so I'm not sure that the work will be very impactful. In my view, then, this work does constitute a contribution, albeit a modest one. I do think the general notion of attempting to simplify models until performance begins to degrade is a fruitful path to explore, as models continue to increase in complexity despite compelling evidence that this is always needed.
Strengths
---
+ This paper does highlight a gap in existing work, as far as I am aware: namely, I am not sure that there are generally known trade-offs associated with different compositional models over token embeddings for NLP. However, it is not clear that we should expect there to be a consistent result to this question across all NLP tasks.
+ The results are marginally surprising, insofar as I would have expected the CNN/RNN (particularly the former) to dominate the simpler aggregation approaches, and this does not seem borne out by the data. Although this trend is seemingly reversed on the short text data, muddying the story.
Weaknesses
---
- There are a number of important limitations here, many of which the authors themselves note, which mitigate the implications of the reported results. First, this is a small set of tasks, and results may not hold more generally. It would have been nice to see some work on Seq2Seq tasks, or sequence tagging tasks at least.
- I was surprised to see no mention of the "Fixed-Size Ordinally-Forgetting Encoding Method" (FOFE) proposed by Zhang et al. in 2015, which would seem to be a natural point of comparison here, given that it sits in a sweet spot of being simple and efficient while still expressive enough to preserve word-order information. This actually seems like a pretty glaring omission given that it meets many of the desiderata the authors put forward.
- The interpretability angle discussed seems underdeveloped. I'm not sure that being able to identify individual words (as the authors have listed) meaningfully constitutes "interpretability" -- standard CNNs, e.g., lend themselves to this as well by tracing back through the filter activations.
- Some of the questions addressed seem tangential to the main question of the paper -- e.g., word vector dimensionality seems an orthogonal issue to the composition function, and would influence performance for the more complex architectures as well.
Smaller comments
---
- On page 1, the authors write "By representing each word as a fixed-length vector, these embeddings can group semantically similar words, while explicitly encoding rich linguistic regularities and patterns", but actually I would say that these *implicitly* encode such regularities, rather than explicitly.
- "architecture in Kim 2014; Collobert et al. 2011; Gan et al. 2017" -- citation formatting a bit weird here.
*** Update based on author response ***
I have read the authors response and thank them for the additional details.
Regarding the limited set of problems: of course any given work can only explore so many tasks, but for this to have general implications in NLP I would maintain that a standard (structured) sequence tagging task/dataset should have been considered. This is not about the number of datasets, but rather than diversity of the output spaces therein.
I appreciated the additional details regarding FOFE, which as the authors themselves note in their response is essentially a generalization of SWEM.
Overall, the response has not changed my opinion on this paper: I think this (exploring simple representations and baselines) is an important direction in NLP, but feel that the paper would greatly benefit from additional work.


Standard deep learning systems require thousands or millions of examples to learn a concept, and cannot integrate new concepts easily. By contrast, humans have an incredible ability to do one-shot or few-shot learning. For instance, from just hearing a word used in a sentence, humans can infer a great deal about it, by leveraging what the syntax and semantics of the surrounding words tells us. Here, we draw inspiration from this to highlight a simple technique by which deep recurrent networks can similarly exploit their prior knowledge to learn a useful representation for a new word from little data. This could make natural language processing systems much more flexible, by allowing them to learn continually from the new words they encounter.

Paper Summary
From just seeing a word used in a sentence, humans can infer a lot about this word by leveraging the surrounding words. Based on this idea, this work tries to obtain a better understanding of words in the one-shot or few-shot setting by leveraging surrounding word. They do this by language modeling sentences which contain rarely seen or never seen words. They evaluated their model using percent change in perplexity on test sentences containing new word by varying the number of training sentences containing this word. 3 Proposed Methods to model few-shot words: (1) beginning with random embedding, (2) beginning with zero embedding (3) beginning with the centroid of other words in the sentence. They compare to 2 Baseline Methods: (1) centroid of other words in the sentence, and (2) full training including the sparse words. Their results show that learning from centroids of other words can outperform full training on the new words.
Explanation
The paper is well written, and the experiments are well explained.  It is an interesting paper, and a research topic which is not well studied. The experiments are reasonable. The method seems to work well.
However, the method provides a very marginal difference between the previous method in Lazaridou et al. (2017). They just use backdrop to learn from this starting position. The main contribution of this work is the evaluation section.
Why only use the PTB language modeling task. Why not use the task in Gauthier & Mordatch or Hermann et al. The one task of language modeling shows promising results, but it’s not totally convincing.
One of the biggest caveats is that the experiments are only done in a few words. I’m not sure why more couldn’t have been done. This is discussed in section 4.1, but I think some of these differences could have been alleviated if there were more experiments done. Regardless, the experiments on the 8 words that they did chose were well done.
I don’t think that section 3.3 (embedding similarity) is particularly useful.


Temporal Difference Learning with function approximation is known to be unstable. Previous work like Sutton et al. (2009b) and Sutton et al. (2009a) has presented alternative objectives that are stable to minimize. However, in practice, TD-learning with neural networks requires various tricks like using a target network that updates slowly (Mnih et al., 2015). In this work we propose a constraint on the TD update that minimizes change to the target values. This constraint can be applied to the gradients of any TD objective, and can be easily applied to nonlinear function approximation. We validate this update by applying our technique to deep Q-learning, and training without a target network. We also show that adding this constraint on Baird's counterexample keeps Q-learning from diverging.

This paper proposes adding a constraint to the temporal difference update to minimize the effect of the update on the next state’s value. The constraint is added by projecting the original gradient to the orthogonal of the maximal direction of change of the next state’s value. It is shown empirically that the constrained update does not diverge on Baird’s counter example and improves performance in a grid world domain and cart pole over DQN.
This paper is reasonably readable. The derivation for the constraint is easy to understand and seems to be an interesting line of inquiry that might show potential.
The key issue is that the justification for the constrained gradients is lacking. What is the effect, in terms of convergence, in modifying the gradient in this way? It seems highly problematic to simply remove a whole part of the gradient, to reduce effect on the next state. For example, if we are minimizing the changes our update will make to the value of the next state, what would happen if the next state is equivalent to the current state (or equivalent in our feature space)? In general, when we project our update to be orthogonal to the maximal change of the next states value, how do we know it is a valid direction in which to update?
I would have liked some analysis of the convergence results for TD learning with this constraint, or some better intuition in how this effects learning. At the very least a mention of how the convergence proof would follow other common proofs in RL. This is particularly important, since GTD provides convergent TD updates under nonlinear function approximation; the role for a heuristic constrained TD algorithm given convergent alternatives is not clear.

For the experiments, other baselines should be included, particularly just regular Q-learning. The primary motivation comes from the use of a separate target network in DQN, which seems to be needed in Atari (though I am not aware of any clear result that demonstrates why, rather just from informal discussions). Since you are not running experiments on Atari here, it is invalid to simply assume that such a second network is needed. A baseline of regular Q-learning should be included for these simpler domains.
The results in Baird’s counter example are discouraging for the new constraints. Because we already have algorithms which better solve this domain, why is your method advantageous? The point of showing your algorithm not solve Baird’s counter example is unclear.
There are also quite a few correctness errors in the paper, and the polish of the plots and language needs work, as outlined below.
There are several mistakes in the notation and background section.
1. “If we consider TD-learning using function approximation, the loss that is minimized is the squared TD error.“ This is not true; rather, TD minimizes the mean-squared project Bellman error. Further, L_TD is strangely defined: why a squared norm, for a scalar value?
2. The definition of v and delta_TD w.r.t. to v seems unnecessary, since you only use Q. As an additional (somewhat unimportant) point, the TD-error is usually defined as the negative of what you have.
3. In the function approximation case the value function and q functions parameterized by \theta are only approximations of the expected return.
4. Defining the loss w.r.t. the state, and taking the derivative of the state w.r.t. to theta is a bit odd. Likely what you meant is the q function, at state s_t? Also, are ignoring the gradient of the value at the next step? If so, this further means that this is not a true gradient.
There is a lot of white space around the plots, which could be used for larger more clear figures. The lack of labels on the plots makes them hard to understand at a glance, and the overlapping lines make finding certain algorithm’s performance much more difficult. I would recommend combining the plots into one figure with a drawing program so you have more control over the size and position of the plots.
Examples of odd language choices:
	-	“The idea also does not immediately scale to nonlinear function approximation. Bhatnagar et al. (2009) propose a solution by projecting the error on the tangent plane to the function at the point at which it is evaluated. “ - The paper you give exactly solves for the nonlinear function approximation case. What do you mean does not scale to nonlinear function approximation? Also Maei is the first author on this paper.
	-	“Though they do not point out this insight as we have” - This seems to be a bit overreaching.
- “the gradient at s_{t+1} that will change the value the most”  - This is too colloquial. I think you simply mean the gradient of the value function, for the given s_t, but its not clear.


We propose a novel approach to multimodal sentiment analysis using deep neural networks combining visual recognition and natural language processing. Our goal is different than the standard sentiment analysis goal of predicting whether a sentence expresses positive or negative sentiment; instead, we aim to infer the latent emotional state of the user. Thus, we focus on predicting the emotion word tags attached by users to their Tumblr posts, treating these as "self-reported emotions." We demonstrate that our multimodal model combining both text and image features outperforms separate models based solely on either images or text. Our model's results are interpretable, automatically yielding sensible word lists associated with emotions. We explore the structure of emotions implied by our model and compare it to what has been posited in the psychology literature, and validate our model on a set of images that have been used in psychology studies. Finally, our work also provides a useful tool for the growing academic study of imagesboth photographs and memes-on social networks.

This paper presents a method for classifying Tumblr posts with associated images according to associated single emotion word hashtags.  The method relies on sentiment pre-processing from GloVe and image pre-processing from Inception.

My strongest criticism for this paper is against the claim that Tumblr post represent self-reported emotions and that this method sheds new insight on emotion representation and my secondary criticism is a lack of novelty in the method, which seems to be simply a combination of previously published sentiment analysis module and previously published image analysis module, fused in an output layer.
The authors claim that the hashtags represent self-reported emotions, but this is not true in the way that psychologists query participants regarding emotion words in psychology studies.  Instead these are emotion words that a person chooses to broadcast along with an associated announcement.  As the authors point out, hashtags and words may be used sarcastically or in different ways from what is understood in emotion theory.  It is quite common for everyday people to use emotion words this way e.g. using #love to express strong approval rather than an actual feeling of love.
In their analysis the authors claim:
“The 15 emotions retained were those with high relative frequencies on Tumblr among the PANAS-X scale (Watson & Clark, 1999)”.
However five of the words the authors retain: bored, annoyed, love, optimistic, and pensive are not in fact found in the PANAS-X scale:
Reference: The PANAS-X Scale: https://wiki.aalto.fi/download/attachments/50102838/PANAS-X-scale_spec.pdf Also the longer version that the authors cited:
https://www2.psychology.uiowa.edu/faculty/clark/panas-x.pdf
It should also be noted that the PANAS (Positive and Negative Affect Scale) scale and the PANAS-X (the “X” is for eXtended) scale are questionnaires used to elicit from participants feelings of positive and negative affect, they are not collections of "core" emotion words, but rather words that are colloquially attached to either positive or negative sentiment.  For example PANAS-X includes words like:“strong” ,“active”, “healthy”, “sleepy” which are not considered emotion words by psychology.
If the authors stated goal is "different than the standard sentiment analysis goal of predicting whether a sentence expresses positive or negative sentiment" they should be aware that this is exactly what PANAS is designed to do - not to infer the latent emotional state of a person, except to the extent that their affect is positive or negative.
The work of representing emotions had been an field in psychology for over a hundred years and it is still continuing.  https://en.wikipedia.org/wiki/Contrasting_and_categorization_of_emotions.
One of the most popular theories of emotion is the theory that there exist “basic” emotions: Anger, Disgust, Fear, Happiness (enjoyment), Sadness and Surprise (Paul Ekman, cited by the authors).  These are short duration sates lasting only seconds.  They are also fairly specific, for example “surprise” is sudden reaction to something unexpected, which is it exactly the same as seeing a flower on your car and expressing “what a nice surprise.”  The surprise would be the initial reaction of “what’s that on my car?  Is it dangerous?” but after identifying the object as non-threatening, the emotion of “surprise” would likely pass and be replaced with appreciation.
The Circumplex Model of Emotions (Posner et al 2005) the authors refer to actually stands in opposition to the theories of Ekman.  From the cited paper by Posner et al :
"The circumplex model of affect proposes that all affective states arise from cognitive interpretations of core neural sensations that are the product of two independent neurophysiological systems. This model stands in contrast to theories of basic emotions, which posit that a discrete and independent neural system subserves every emotion."
From my reading of this paper, it is clear to me that the authors do not have a clear understanding of the current state of psychology’s view of emotion representation and this work would not likely contribute to a new understanding of the latent structure of peoples’ emotions.
In the PCA result, it is not "clear" that the first axis represents valence, as "sad" has a slight positive on this scale and "sad" is one of the emotions most clearly associated with negative valence.
With respect to the rest of the paper, the level of novelty and impact is "ok, but not good enough."  This analysis does not seem very different from Twitter analysis, because although Tumblr posts are allowed to be longer than Twitter posts, the authors truncate the posts to 50 characters.  Additionally, the images do not seem to add very much to the classification.  The authors algorithm also seems to be essentially a combination of two other, previously published algorithms.
For me the novelty of this paper was in its application to the realm of emotion theory, but I do not feel there is a contribution here.  This paper is more about classifying Tumblr posts according to emotion word hashtags than a paper that generates a new insights into emotion representation or that can infer latent emotional state.


ON THE REGULARIZATION OF WASSERSTEIN GANS
Since their invention, generative adversarial networks (GANs) have become a popular approach for learning to model a distribution of real (unlabeled) data. Convergence problems during training are overcome by Wasserstein GANs which minimize the distance between the model and the empirical distribution in terms of a different metric, but thereby introduce a Lipschitz constraint into the optimization problem. A simple way to enforce the Lipschitz constraint on the class of functions, which can be modeled by the neural network, is weight clipping. Augmenting the loss by a regularization term that penalizes the deviation of the gradient norm of the critic (as a function of the network's input) from one, was proposed as an alternative that improves training. We present theoretical arguments why using a weaker regularization term enforcing the Lipschitz constraint is preferable. These arguments are supported by experimental results on several data sets.

This paper proposes a novel regularization scheme for Wasserstein GAN based on a relaxation of the constraints on the Lipschitz constant of 1. The proposed regularization penalize the critic function only when its gradient has a norm larger than one using some kind of squared hinge loss. The reasons for this choice are discussed and linked to theoretical properties of OT. Numerical experiments suggests that the proposed regularization leads to better posed optimization problem and even a slight advantage in terms of inception score on the CIFAR-10 dataset.
The paper is interesting and well written, the proposed regularization makes sens since it is basically a relaxation of the constraints and the numerical experiments also suggest it's a good idea. Still as discussed below the justification do not address a lots of interesting developments and implications of the method and should better discuss the relation with regularized optimal transport.
Discussion:
+ The paper spends a lot of time justifying the proposed method by discussing the limits of the "Improved training of Wasserstein GAN" from Gulrajani et al. (2017). The two limits (sampling from marginals instead of optimal coupling and differentiability of the critic) are interesting and indeed suggest that one can do better but the examples and observations are well known in OT and do not require proof in appendix. The reviewer believes that this space could be better spend discussing the theoretical implication of the proposed regularization (see next).
+ The proposed approach is a relaxation of the constraints on the dual variable for the OT problem. As a matter of fact we can clearly recognize a squared hinge loss is the proposed loss. This approach (relaxing a strong constraint) has been used for years when learning support vector machines and ranking and a small discussion or at least reference to those venerable methods would position the paper on a bigger picture.
+ The paper is rather vague on the reason to go from Eq. (6) to Eq. (7). (gradient approximation between samples to gradient on samples). Does it lead to better stability to choose one or the other?
 How is it implemented in practice? recent NN toolbox can easily compute the exact gradient and use it for the penalization but this is not clearly discussed even in appendix. Numerical experiments comparing the two implementation or at least a discussion is necessary.
+ The proposed approach has a very strong relations to the recently proposed regularized OT (see [1] for a long list of regularizations) and more precisely to the euclidean regularization. I understand that GANS (and Wasserstein GAN) is a relatively young community and that references list can be short but their is a large number of papers discussing regularized optimal transport and how the resulting problems are easier to solve. A discussion of the links is necessary and will clearly bring more theoretical ground to the method. Note that a square euclidean regularization leads to a regularization term in the dual of the form max(0,f(x)+f(y)-|x-y|)^2 that is very similar to the proposed regularization. In other words the authors propose to do regularized OT (possibly with a new regularization term) and should discuss that.
+ The numerical experiments are encouraging but a bit short. The 2D example seem to work very well and the convergence curves are far better with the proposed regularization. But the real data CIFAR experiments are much less detailed with only a final inception score (very similar to the competing method) and no images even in appendix. The authors should also define (maybe in appendix) the conditional and unconditional inception scores and why they are important (and why only some of them are computed in Table 1).
+ This is more of a suggestion. The comparison of the dual critic to the true Wasserstein distance is very interesting. It would be nice to see the behavior for different values of lambda.
[1] Dessein, A., Papadakis, N., & Rouas, J. L. (2016). Regularized Optimal Transport and the Rot Mover's Distance. arXiv preprint arXiv:1610.06447.
Review update after reply:
The authors have responded to most of my concerns and I think the paper is much stronger now and discuss the relation with regularized OT. I change the rating to Accept.


Published as a conference paper at ICLR 2018 LEARNING TO REPRESENT PROGRAMS WITH GRAPHS
Learning tasks on source code (i.e., formal languages) have been considered recently, but most work has tried to transfer natural language methods and does not capitalize on the unique opportunities offered by code's known sematics. For example, long-range dependencies induced by using the same variable or function in distant locations are often not considered. We propose to use graphs to represent both the syntactic and semantic structure of code and use graph-based deep learning methods to learn to reason over program structures. In this work, we present how to construct graphs from source code and how to scale Gated Graph Neural Networks training to such large graphs. We evaluate our method on two tasks: VARNAMING, in which a network attempts to predict the name of a variable given its usage, and VARMISUSE, in which the network learns to reason about selecting the correct variable that should be used at a given program location. Our comparison to methods that use less structured program representations shows the advantages of modeling known structure, and suggests that our models learn to infer meaningful names and to solve the VARMISUSE task in many cases. Additionally, our testing showed that VARMISUSE identifies a number of bugs in mature open-source projects.

Summary:  The paper applies graph convolutions with deep neural networks to the problem of "variable misuse" (putting the wrong variable name in a program statement) in graphs created deterministically from source code.  Graph structure is determined by program abstract syntax tree (AST) and next-token edges, as well as variable/function name identity, assignment and other deterministic semantic relations.  Initial node embedding comes from both type and tokenized name information.  Gated Graph Neural Networks (GGNNs, trained by maximum likelihood objective) are then run for 8 iterations at test time.
The evaluation is extensive and mostly very good.  Substantial data set of 29m lines of code.  Reasonable baselines.  Nice ablation studies.  I would have liked to see separate precision and recall rather than accuracy.  The current 82.1% accuracy is nice to see, but if 18% of my program variables were erroneously flagged as errors, the tool would be useless.  I'd like to know if you can tune the threshold to get a precision/recall tradeoff that has very few false warnings, but still catches some errors.
Nice work creating an implementation of fast GGNNs with large diverse graphs.  Glad to see that the code will be released.  Great to see that the method is fast---it seems fast enough to use in practice in a real IDE.
The model (GGNN) is not particularly novel, but I'm not much bothered by that.  I'm very happy to see good application papers at ICLR.  I agree with your pair of sentences in the conclusion: "Although source code is well understood and studied within other disciplines such as programming language research, it is a relatively new domain for deep learning. It presents novel opportunities compared to textual or perceptual data, as its (local) semantics are well-defined and rich additional information can be extracted using well-known, efficient program analyses."  I'd like to see work in this area encouraged.  So I recommend acceptance.  If it had better (e.g. ROC curve) evaluation and some modeling novelty, I would rate it higher still.
Small notes:
The paper uses the term "data flow structure" without defining it.
Your data set consisted of C# code.  Perhaps future work will see if the results are much different in other languages.