A natural solution for one-shot learning is to augment training data to handle the data deficiency problem. However, directly augmenting in the image domain may not necessarily generate training data that sufficiently explore the intra-class space for one-shot classification. Inspired by the recent vocabulary-informed learning, we propose to generate synthetic training data with the guide of the semantic word space. Essentially, we train an auto-encoder as a bridge to enable the transformation between the image feature space and the semantic space. Besides directly augmenting image features, we transform the image features to semantic space using the encoder and perform the data augmentation. The decoder then synthesizes the image features for the augmented instances from the semantic space. Experiments on three datasets show that our data augmentation method effectively improves the performance of one-shot classification. Extensive study shows that data augmented from semantic space are complementary with those from the image space, and thus boost the classification accuracy dramatically. Source code and dataset will be available.

Summary:
This paper proposes a data augmentation method for one-shot learning of image classes. This is the problem where given just one labeled image of a class, the aim is to correctly identify other images as belonging to that class as well.
The idea presented in this paper is that instead of performing data augmentation in the image space, it may be useful to perform data augmentation in a latent space whose features are more discriminative for classification. One candidate for this is the image feature space learned by a deep network. However they advocate that a better candidate is what they refer to as "semantic space" formed by embedding the (word) labels of the images according to pre-trained language models like word2vec. The reasoning here is that the image feature space may not be semantically organized so that we are not guaranteed that a small perturbation of an image vector will yield image vectors that correspond to semantically similar images (belonging to the same class). On the other hand, in this semantic space, by construction, we are guaranteed that similar concepts lie near by each other. Thus this space may constitute a better candidate for performing data augmentation by small perturbations or by nearest neighbour search around the given vector since 1) the augmented data is more likely to correspond to features of similar images as the original provided image and 2) it is more likely to thoroughly capture the intra-class variability in the augmented data.
The authors propose to first embed each image into a feature space, and then feed this learned representation into a auto-encoder that handles the projection to and from the semantic space with its encoder and decoder, respectively. Specifically, they propose to perform the augmentation on the semantic space representation, obtained from the encoder of this autoencoder. This involves producing some additional data points, either by adding noise to the projected semantic vector, or by choosing a number of that vector's nearest neighbours. The decoder then maps these new data points into feature space, obtaining in this way the image feature representations that, along with the feature representation of the original (real) image will form the batch that will be used to train the one-shot classifier.
They conduct experiments in 3 datasets where they experiment with augmentation in the image feature space by random noise, as well as the two aforementioned types of augmentation in the semantic space. They claim that these augmentation types provide orthogonal benefits and can be combined to yield superior results.
Overall I think this paper addresses an important problem in an interesting way, but there is a number of ways in which it can be improved, detailed in the comments below.
Comments:
-- Since the authors are using a pre-trained VGG for to embed each image, I'm wondering to what extent they are actually doing one-shot learning here. In other words, the test set of a dataset that is used for evaluation might contain some classes that were also present in the training set that VGG was originally trained on. It would be useful to clarify whether this is happening. Can the VGG be instead trained from scratch in an end-to-end way in this model?
-- A number of things were unclear to me with respect to the details of the training process: the feature extractor (VGG) is pre-trained. Is this finetuned during training?  If so, is this done jointly with the training of the auto-encoder? Further, is the auto-encoder trained separately or jointly with the training of the one-shot learning classifier?
-- While the authors have convinced me that data augmentation indeed significantly improves the performance in the domains considered (based on the results in Table 1 and Figure 5a), I am not convinced that augmentation in the proposed manner leads to a greater improvement than just augmenting in the image feature domain. In particular, in Table 2, where the different types of augmentation are compared against each other, we observe similar results between augmenting only in the image feature space versus augmenting only in the semantic feature space (ie we observe that "FeatG" performs similarly as "SemG" and as "SemN"). When combining multiple types of augmentation the results are better, but I'm wondering if this is because more augmented data is used overall. Specifically, the authors say that for each image they produce 5 additional "virtual" data points, but when multiple methods are combined, does this mean 5 from each method? Or 5 overall? If it's the former, the increased performance may merely be attributed to using more data. It is important to clarify this point.
-- Comparison with existing work: There has been a lot of work recently on one-shot and few-shot learning that would be interesting to compare against. In particular, mini-ImageNet is a commonly-used benchmark for this task that this approach can be applied to for comparison with recent methods that do not use data augmentation. Some examples are:
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. (Finn et al.)
- Prototypical Networks for Few-shot Learning (Snell et al.)
- Matching Networks for One-shot Learning (Vinyals et al.)
- Few-Shot Learning Through an Information Retrieval Lens (Triantafillou et al.)
-- A suggestion: As future work I would be very interested to see if this method can be incorporated into common few-shot learning models to on-the-fly generate additional training examples from the "support set" of each episode that these approaches use for training.


Workshop track -ICLR 2018 FAST AND ACCURATE TEXT CLASSIFICATION: SKIM- MING, REREADING AND EARLY STOPPING
Recent advances in recurrent neural nets (RNNs) have shown much promise in many applications in natural language processing. For most of these tasks, such as sentiment analysis of customer reviews, a recurrent neural net model parses the entire review before forming a decision. We argue that reading the entire input is not always necessary in practice, since a lot of reviews are often easy to classify, i.e., a decision can be formed after reading some crucial sentences or words in the provided text. In this paper, we present an approach of fast reading for text classification. Inspired by several well-known human reading techniques, our approach implements an intelligent recurrent agent which evaluates the importance of the current snippet in order to decide whether to make a prediction, or to skip some texts, or to re-read part of the sentence. Our agent uses an RNN module to encode information from the past and the current tokens, and applies a policy module to form decisions. With an end-to-end training algorithm based on policy gradient, we train and test our agent on several text classification datasets and achieve both higher efficiency and better accuracy compared to previous approaches.

This paper proposes to augment RNNs for text classification with a mechanism that decides whether the RNN should re-read a token, skip a number of tokens, or stop and output the prediction. The motivation is that one can stop reading before the end of the text and/or skip some words and still arrive to the same answer but faster.
The idea is intriguing, even though not entirely novel. Apart from the Yu et al. (2017) cited, there is older work trying to save computational time in NLP, e.g.:
Dynamic Feature Selection for Dependency Parsing.
He He, Hal Daumé III and Jason Eisner.
Empirical Methods in Natural Language Processing (EMNLP), 2013
that decides whether to extract a feature or not.
However, what is not clear to me what is achieved here. In the example shown in Figure 5 it seems like what happens is that by not reading the whole text the model avoids passages that might be confusing it. This could improve predictive accuracy (as it seems to do), as long as the model can handle better the earlier part of the text. But this is just an assumption, which is not guaranteed in any way. It could be that the earlier parts of the text are hard for the model. In a way, it feels more like we are addressing a limitation of RNN models in understanding text.
Pros:
- The idea is intersting and if evaluated thoroughly it could be quite influential.
Cons:
- the introduction states that there are two kinds of NLP problems, sequence2sequence and sequence2scalar. I found this rather confusing since text classification falls in the latter presumably, but the output is a label. Similarly, PoS tagging has a linear chain as its output, can't see why it is sequence2scalar. I think there is a confusion between the methods used for a task, and the task itself. Being able to apply a sequence-based model to a task, doesn't make it sequential necessarily.
- the comparison in terms of FLOPs is a good idea. But wouldn't the relative savings depend on the architecture used for the RNN and the RL agent? E.g. it could be that the RL network is more complicated and using it costs more than what it saves in the RNN operations.
- While table 2 reports the savings vs the full reading model, we don't know how much worse the model got for these savings.
- Having a trade-off between savings and accuracy is a good idea too. I would have liked to see an experiment showing how many FLOPs we can save for the same performance, which should be achievable by adjusting the alpha parameter.
- The experiments are conducted on previously published datasets. It would be good to have some previously published results on them to get a sense of how good the RNN model used is.
- Why not use smaller chunks? 20 words or one sentence at the time is rather coarse. If anything, it should help the model proposed achieve greater savings. How much does the choice of chunk matter?
- it is stated in the conclusion that the advantage actor-critic used is beneficial, however no experimental comparison is shown. Was it used for the Yu et al. baseline too?
- It is stated that model hardly relies on any hyperparameters; in comparison to what? It is better to quantify such statements,


One of the main challenges of deep learning methods is the choice of an appropriate training strategy. In particular, additional steps, such as unsupervised pretraining, have been shown to greatly improve the performances of deep structures. In this article, we propose an extra training step, called post-training, which only optimizes the last layer of the network. We show that this procedure can be analyzed in the context of kernel theory, with the first layers computing an embedding of the data and the last layer a statistical model to solve the task based on this embedding. This step makes sure that the embedding, or representation, of the data is used in the best possible way for the considered task. This idea is then tested on multiple architectures with various data sets, showing that it consistently provides a boost in performance.

Summary:
Based on ideas within the context of kernel theory, the authors consider post-training of NNs as an extra training step, which only optimizes the last layer of the network.
This additional step makes sure that the embedding, or representation, of the data is used in the best possible way for the considered task (which is also reflected in the experiments).
According to the authors, the contributions are the following:
1. Post-training step: keeping the rest of the NN frozen (after training), the method trains the last layer in order to "make sure" that the representation learned is used in the most efficient way.
2. Highlighting connections with kernel techniques and RKHS optimization (like kernel ridge regression).
3. Experimental results.
Clarity:
The paper is well-written, the main ideas well-clarified.
Importance:
While the majority of papers nowadays focuses on the representation part (i.e., how we get to \Phi_{L-1}(x)), this paper assumes this is given and proposes how to optimize the weights in the final step of the algorithm. This by itself is not enough to boost the performance universally (e.g., if \Phi_{L-1} is not well-trained, the problem is deeper than training the last layer); however, it proposes an additional step that can be used in most NN architectures. From that front (i.e., proposing to do something different than simply training a NN), I find the paper interesting, that might attract some attention at the conference.
On the other hand, to my humble opinion, the experimental results do not show a significant gain in the performances of all networks (esp. Figure 3 and Table 1 are within the range of statistical error). In order to state something like this universally, either one needs to perform experiments with more than just MNIST/CIFAR datasets, or even more preferably, prove that the algorithm performs better.
Originality:
It would be great to have some more theory (if any) for the post-training step, or investigate more cases, rather than optimizing only the last layer.
Comments:
1. I assume the authors focused in the last layer of the NN for simplicity, but is there a reason why one might want to focus only on the last layer? One reason is convexity in W of the problem (2). Any other?
2. Have the authors considered (even in practice only) to include training of the last 2 layers of the NN? The authors state this question in the future direction, but it would make the paper more complete to consider it here.


Generative adversarial networks (GANs) have enjoyed great success, however often suffer instability during training which motivates many attempts to resolve this issue. Theoretical explanation for the cause of instability is provided in Wasserstein GAN (WGAN), and wasserstein distance is proposed to stablize the training. Though WGAN is indeed more stable than previous GANs, it takes more iterations and time to train. This is because the ways to ensure Lipschitz condition in WGAN (such as weight-clipping) significantly limit the capacity of the network. In this paper, we argue that it is beneficial to ensure Lipschitz condition as well as maintain sufficient capacity and expressiveness of the network. To facilitate this, we develop both theoretical and practical building blocks, using which one can construct different neural networks using a large range of metrics, as well as ensure Lipschitz condition and sufficient capacity of the networks. Using the proposed building blocks, and a special choice of a metric called Dudley metric, we propose Dudley GAN that outperforms the state of the arts in both convergence and sample quality. We discover a natural link between Dudley GAN (and its extension) and empirical risk minimization, which gives rise to generalization analysis.

The authors propose a different type of GAN--the Dudley GAN--that is related to the Dudley metric. In fact, it is very much like the WGAN, but rather than just imposing the function class to have a bounded gradient, they also impose it to be bounded itself. This is argued to be more stable than the WGAN, as gradient clipping is said not necessary for the Dudley GAN. The authors empirically show that the Dudley GAN achieves a greater LL than WGAN for the MNIST and CIFAR-10 datasets.
The main idea [and its variants] looks solid, but with the plethora of GANs in the literature now, after reading I'm still left wondering why this GAN is significantly better than others [BEGAN, WGAN, etc.]. It is clear that imposing the quadratic penalty in equation (3) is really the same constraint as the Dudley norm? The big contribution of the paper seems to be that adding some L_inf regularization to the function class helps preclude gradient clipping, but after reading I'm unsure why this is "the right thing" to do in this case. We know that convergence in the Wasserstein metric is stronger than the Dudley metric, so why is using the weaker metric overweighed by the benefits in training?
Nits: Since the function class is parameterized by a NN, the IPM is not actually the Dudley metric between the two distributions. One would have to show that the NN is dense in Dudley unit ball w.r.t. L_inf norm, but this sort of misnaming had started with the "Wasserstein" GAN.


In cooperative multi-agent reinforcement learning (MARL), how to design a suitable reward signal to accelerate learning and stabilize convergence is a critical problem. The global reward signal assigns the same global reward to all agents without distinguishing their contributions, while the local reward signal provides different local rewards to each agent based solely on individual behavior. Both of the two reward assignment approaches have some shortcomings: the former might encourage lazy agents, while the latter might produce selfish agents. In this paper, we study reward design problem in cooperative MARL based on packet routing environments. Firstly, we show that the above two reward signals are prone to produce suboptimal policies. Then, inspired by some observations and considerations, we design some mixed reward signals, which are off-the-shelf to learn better policies. Finally, we turn the mixed reward signals into the adaptive counterparts, which achieve best results in our experiments. Other reward signals are also discussed in this paper. As reward design is a very fundamental problem in RL and especially in MARL, we hope that MARL researchers can rethink the rewards used in their systems.

The authors suggest using a mixture of shared and individual rewards within a MARL environment to induce cooperation among independent agents. They show that on their specific application this can lead to a better overall global performance than purely sharing the global signal, or using just the independent rewards.
The paper is a little too focused on the packet routing example domain and fails to deliver much in terms of a general theory of reward design for cooperative behaviours beyond showing that mixed rewards can lead to improved results in their domain. They discuss what and how rewards, and this could be made more formal, as well as (at the very least) some guiding principles to follow when mixing rewards. It feels like there is a missing section between sections 2 and 3, where this methodological content could be described.
The rest of the paper has similar issues, with key intuition and concepts either missing entirely or under-represented. The technical content often assumes that the reader is familiar with certain terms, and it is difficult to see what meaningful conclusions can be drawn from the evaluation.
On a minor note, the use of the term cooperative in this paper could be better defined. In game theory, cooperative games are those in which agents share rewards. Non-cooperative (game theory) games are those where agents have general reward signals (not necessarily cooperature or adversarial). Conventionally (yes there is existing reward design/shaping literature for MARL) people have used the same terms in MARL. Perhaps the authors could define their approach as weakly cooperative, or emergent cooperation.
The related work could be better described. There are existing papers on MARL and the issues with cooperation among independent learners, and this could be referenced. This includes reward shaping and reward potential. I would also have expected to see brief mention of empowerment in this section too (the agent favouring states where it has the power to control outcomes in an information theoretic sense), as an underyling principle for intrinsic reward. However, more importantly, the authors really needed to do more to synthesize this into an overall picture of what principles are at play and what ideas/methods exist that have tried to exploit some of these principles.
Detailed comments:
  • [p2] the authors say "We set the meta reward signals as 1 - max(U l ).", before they define what U_l is.
  • [p2] we have "As many applications in the real world can be modeled using similar
methods, we expect that other fields can also benefit from this work." This statement is too vague, and the authors could do more to identify which application areas might benefit.
  • [p3, first para] "However, the reward design studies for MARL is so limited." Drop the word 'so'. Also, I would argue that there have been quite a few (non-deep) discussions about reward design in MARL, cooperative, non-cooperative and competitive domains.
  • [p3, sec 2.2] "This makes the diligent agents confuse about..." should be "confused", and I would advise against anthropomorphism at least when the meaning is obscured.
  • [p3, sec 3] "After having considered several other options, we finally choose the Packet Routing Domain as our experimental environments." Not sure what useful information is being conveyed here.
  • [sec 3] THe domain could be better described with intuition and formal descriptions, e.g. link utilization ratio, etc, before.
  • [p6] "Importantly, the proposed blR seems to have similar capacity with dlR," The discussion here is all in terms of the reward acronyms with very little call on intuition or other such assistance to the reader.
  • [p7] "We firstly try gR without any thinking" The language could be better here.


We investigate in this paper the architecture of deep convolutional networks. Building on existing state of the art models, we propose a reconfiguration of the model parameters into several parallel branches at the global network level, with each branch being a standalone CNN. We show that this arrangement is an efficient way to significantly reduce the number of parameters while at the same time improving the performance. The use of branches brings an additional form of regularization. In addition to splitting the parameters into parallel branches, we propose a tighter coupling of these branches by averaging their log-probabilities. The tighter coupling favours the learning of better representations, even at the level of the individual branches, as compared to when each branch is trained independently. We refer to this branched architecture as "coupled ensembles". The approach is generic and can be applied to almost any neural network architecture. With coupled ensembles of DenseNet-BC and parameter budget of 25M, we obtain error rates of 2.92%, 15.68% and 1.50% on CIFAR-10, CIFAR-100 and SVHN respectively. For the same parameter budget, DenseNet-BC has an error rate of 3.46%, 17.18%, and 1.8% respectively. With ensembles of coupled ensembles, of DenseNet-BC networks, with 50M total parameters, we obtain error rates of 2.72%, 15.13% and 1.42% respectively on these tasks.

This paper presents a deep network architecture which processes data using multiple parallel branches and combines the posterior from these branches to compute the final scores; the network is trained in end-to-end, thus training the parallel branches jointly. Existing literature with branching architecture either employ a 2 stage training approach, training branches independently and then training the fusion network, or the branching is restricted to local regions (set of contiguous layers). In effect, this paper extends the existing literature suggesting end-to-end branching. While the technical novelty, as described in the paper, is relatively limited, the thorough experimentation together with detailed comparisons between intuitive ways to combine the output of the parallel branches is certainly valuable to the research community.
+ Paper is well written and easy to follow.
+ Proposed branching architecture clearly outperforms the baseline network (same number of parameters with a single branch) and thus offer yet another interesting choice while creating the network architecture for a problem
+ Detailed experiments to study and analyze the effect of various parameters including the number of branches as well as various architectures to combine the output of the parallel branches.
+ [Ease of implementation] Suggested architecture can be easily implemented using existing deep learning frameworks.
- Although joint end-to-end training of branches certainly brings value compared to independent training, but the increased resource requirements may limits the applicability to large benchmarks such as ImageNet. While authors suggests a way to circumvent such limitations by training branches on separate GPUs but this would still impose limits on the number of branches as well as its ease of implementation.
- Adding an overview figure of the architecture in the main paper (instead of supplementary) would be helpful.
- Branched architecture serve as a regularization by distributing the gradients across different branches; however this also suggests that early layers on the network across branches would be independent. It would helpful if authors would consider an alternate archiecture where early layers may be shared across branches, suggesting a delayed branching, with fusion at the final layer.
- One of the benefits of architectures such as DenseNet is their usefulness as a feature extractor (output of lower layers) which generalizes even to domain other that the dataset; the branched architecture could potentially diminish this benefit.
Minor edits: Page 1. 'significantly match and improve' => 'either match or improve'
Additional notes:
- It would interesting to compare this approach with a conditional training pipeline that sequentially adds branches, keeping the previous branches fixed. This may offer as a trade-off between benefits of joint training of branches vs being able to train deep models with several branches.


Published as a conference paper at ICLR 2018 SEMANTIC INTERPOLATION IN IMPLICIT MODELS
In implicit models, one often interpolates between sampled points in latent space. As we show in this paper, care needs to be taken to match-up the distributional assumptions on code vectors with the geometry of the interpolating paths. Otherwise, typical assumptions about the quality and semantics of in-between points may not be justified. Based on our analysis we propose to modify the prior code distribution to put significantly more probability mass closer to the origin. As a result, linear interpolation paths are not only shortest paths, but they are also guaranteed to pass through high-density regions, irrespective of the dimensionality of the latent space. Experiments on standard benchmark image datasets demonstrate clear visual improvements in the quality of the generated samples and exhibit more meaningful interpolation paths.

The paper concerns distributions used for the code space in implicit models, e.g. VAEs and GANs. The authors analyze the relation between the latent space dimension and the normal distribution which is commonly used for the latent distribution. The well-known fact that probability mass concentrates in a shell of hyperspheres as the dimensionality grows is used to argue for the normal distribution being sub-optimal when interpolating between points in the latent space with straight lines. To correct this, the authors propose to use a Gamma-distribution for the norm of the latent space (and uniform angle distribution). This results in more mass closer to the origin, and the authors show both that the midpoint distribution is natural in terms of the KL divergence to the data points, and experimentally that the method gives visually appealing interpolations.
While the contribution of using a standard family of distributions in a standard implicit model setup is limited, the paper does make interesting observations, analyses and an attempt to correct the interpolation issue. The paper is clearly written and presents the theory and experimental results nicely. I find that the paper can be accepted but the incremental nature of the contribution prevents a higher score.


Deep neural networks have become the state-of-the-art models in numerous machine learning tasks. However, general guidance to network architecture design is still missing. In our work, we bridge deep neural network design with numerical differential equations. We show that many effective networks, such as ResNet, PolyNet, FractalNet and RevNet, can be interpreted as different numerical discretizations of differential equations. This finding brings us a brand new perspective on the design of effective deep architectures. We can take advantage of the rich knowledge in numerical analysis to guide us in designing new and potentially more effective deep networks. As an example, we propose a linear multi-step architecture (LM-architecture) which is inspired by the linear multi-step method solving ordinary differential equations. The LM-architecture is an effective structure that can be used on any ResNet-like networks. In particular, we demonstrate that LM-ResNet and LM-ResNeXt (i.e. the networks obtained by applying the LM-architecture on ResNet and ResNeXt respectively) can achieve noticeably higher accuracy than ResNet and ResNeXt on both CIFAR and ImageNet with comparable numbers of trainable parameters. In particular, on both CIFAR and ImageNet, LM-ResNet/LM-ResNeXt can significantly compress (> 50%) the original networks while maintaining a similar performance. This can be explained mathematically using the concept of modified equation from numerical analysis. Last but not least, we also establish a connection between stochastic control and noise injection in the training process which helps to improve generalization of the networks. Furthermore, by relating stochastic training strategy with stochastic dynamic system, we can easily apply stochastic training to the networks with the LM-architecture. As an example, we introduced stochastic depth to LM-ResNet and achieve significant improvement over the original LM-ResNet on CIFAR10.

Summary
- This paper draws analogy from numerical differential equation solvers and popular residual network-like deep learning architectures. It makes connection from ResNet, FractalNet, DenseNet, and RevNet to different numerical solvers such as forward and backward Euler and Runge-Kunta. In addition, inspired by the Linear Multi-step methods (LM), the authors propose a novel LM-ResNet architecture in which the next residual block takes a linear combination of the previous two residual blocks’ activations. They also propose a stochastic version of LM-ResNet that resembles Shake-shake regularization and stochastic depth. In both deterministic and stochastic cases, they show a positive improvement in classification accuracy on standard object classification benchmarks such as CIFAR-10/100 and ImageNet.
Pros
- The intuition is good that connects differential equation and ResNet-like architecture, also explored in some of the related work.
- Building upon the intuition, the author proposes a novel architecture based on a numerical ODE solver method.
- Consistent improvement in accuracy is observed in both deterministic and stochastic cases.
Cons
- The title is a little bit misleading. “Beyond Finite Layer Neural Networks” sounds like the paper proposes some infinite layer neural networks but the paper only studies finite number of layers.
- One thing that needs to be clarified is that, if the network is not targeted at solving certain ODEs, then why is the intuition from ODE matters? The paper does not motivate readers in this perspective.
- Given the widespread use of ResNet in the vision community, the incremental improvement of 1% on ImageNet is less likely to push vision research to switch to a completely different architecture. Therefore, the potential impact of the this paper to vision community is probably limited.
Conclusion
- Based on the comments above, I think the paper is a good contribution which links ODE with Deep Networks and derivation is convincing. The proposed new architecture can be considered in future architecture designs. Although the increase in performance is small, I think it is good enough to accept.


One of the fundamental tasks in understanding genomics is the problem of predicting Transcription Factor Binding Sites (TFBSs). With more than hundreds of Transcription Factors (TFs) as labels, genomic-sequence based TFBS prediction is a challenging multi-label classification task. There are two major biological mechanisms for TF binding: (1) sequence-specific binding patterns on genomes known as "motifs" and (2) interactions among TFs known as co-binding effects. In this paper, we propose a novel deep architecture, the Prototype Matching Network (PMN) to mimic the TF binding mechanisms. Our PMN model automatically extracts prototypes ("motif"-like features) for each TF through a novel prototypematching loss. Borrowing ideas from few-shot matching models, we use the notion of support set of prototypes and an LSTM to learn how TFs interact and bind to genomic sequences. On a reference TFBS dataset with 2.1 million genomic sequences, the PMN significantly outperforms baselines and validates our design choices empirically. To our knowledge, this is the first deep learning architecture that introduces prototype learning and considers TF-TF interactions for large scale TFBS prediction. Not only is the proposed architecture accurate, but it also models the underlying biology.

Summary
This paper proposes a prototype matching network (PMN) to model transcription factor (TF) binding motifs and TF-TF interactions for large scale transcription factor binding site prediction task. They utilize the idea of having a support set of prototypes (motif-like features) and an LSTM from the few shot learning framework to develop this prototype matching network. The input is genomic sequences from 14% of the human genome, each sequence in the dataset is bound by at least one TF. First a Convolutional Neural Network with three convolutional layers is trained to predict single/multiple TF binding. The output of the last hidden layer before sigmoid transformation is used as the LSTM input. A weighted sum of similarity score (sigmoid of cosine similarity, similar to attention mechanism of LSTMs) along with prototype vectors are used to update the read vector. The final output is a sigmoid of the final hidden state concatenated with the read vector. The loss function used is difference of a cross-entropy loss function and a lambda weighted prototype loss function. The latter is the mean square error between the output label and the similarity score.  The authors compare the PMN with different lambda values with CNN with single/multi-label and see marginal improvement in auROC, auPR and Recall at 50% FDR with the PWM. To test that PWN finds biologically relevant TF interactions, the authors perform hierarchical clustering on the prototypes of 86 TFs and compare the clusters found to the known co-regulators from the TRRUST database and find 6 significant clusters.
Pros:
1. The authors utilize the idea of prototypes and few shot learning to the task of TF-binding and cooperation.
2. Attention LSTMs are used to model label interactions.
Just like CNN can be related to discriminative training of PSSM or PWM, the above points demonstrate nicely how ideas/concepts from the recent developments in DL can be adopted/relate (and possibly improve on) to  similar generative modeling approaches used in the past  for learning cooperative TF binding.
Cons:
1. Authors do not compare their model’s performance to the previously published TF binding prediction algorithms (DeepBind, DeepSEA).
2. The authors miss important context and make some inaccurate statements: TF do not just “control if a gene is expressed or not” (p.1). It’s not true that previous DL works did not consider co-binding. Works such as DeepSea combined many filters which can capture cooperative binding to define which sequence is “regulated”. It is true this or DeepBind did not construct a structure a structure over those as learned by an LSTM. The authors do point out a model that does add LSTM (Quang and Xie) but then do not compare to it and make a vague claim about it modeling interactions between features but not labels (p. 6 top). Comparing to it and directly to DeepSee/Bind seems crucial to claim improvements on previous works. Furthermore, the authors acknowledge the existence of vast literature on this specific problem but completely discard it as “loose connection to our TFBS formulation”. In reality though, many works in that area are highly relevant and should be discussed in the context of what the authors are trying to achieve. For example, numerous works by Prof. Saurabh Sinha have focused specifically on joint TF modeling (e.g. Kazemian NAR 2011, He Plos One 2009, Ivan Gen Bio 2008, MORPH Plos Comp Bio 2007). In general, trying to lay claims about significant contributions to a problem, as stated here by the authors, while completely disregarding previous work simply because it’s not in a DL framework (which the authors are clearly more familiar with) can easily alienate reviewers and readers alike.
3. The learning setup seems problematic:
3a. The model may overfit for the genomic sequences that contain TF binding sites as it has never seen genomic sequences without TF binding sites (the genomic sequences that don’t have CHIP peaks are discarded from the dataset). Performance for genome wide scans should definitely include those to assess accuracy.
3b. The train/validation/test are defined by chromosome. There does not seem to be any screening for sequence similarity (e.g. repetitive sequences, paralogs). This may inflate performance, especially for more complicated models which may be able to “memorize” sequences better.
4. The paper claims to have 4 major contributions. The details of second claim that the prototype matching loss learns motif like features is not explained anywhere in the paper. If we look at the actual loss function equation (12), it penalizes the difference between the label and the similarity score but the prototypes are not updated. The fourth claim about the biological relevance of the network is not sufficiently explored. The authors show that it learns co-bindings already known in the literature which is a good sanity check but does not offer any new biological insight.  The actual motifs or the structure of their relations is not shown or explored.
5. PWN offers only marginal improvement over the CNN networks


Published as a conference paper at ICLR 2018 ACTIVE LEARNING FOR CONVOLUTIONAL NEURAL NETWORKS: A CORE-SET APPROACH
Convolutional neural networks (CNNs) have been successfully applied to many recognition and learning tasks using a universal recipe; training a deep model on a very large dataset of supervised examples. However, this approach is rather restrictive in practice since collecting a large set of labeled images is very expensive. One way to ease this problem is coming up with smart ways for choosing images to be labelled from a very large collection (i.e. active learning). Our empirical study suggests that many of the active learning heuristics in the literature are not effective when applied to CNNs in batch setting. Inspired by these limitations, we define the problem of active learning as core-set selection, i.e. choosing set of points such that a model learned over the selected subset is competitive for the remaining data points. We further present a theoretical result characterizing the performance of any selected subset using the geometry of the datapoints. As an active learning algorithm, we choose the subset which is expected to yield best result according to our characterization. Our experiments show that the proposed method significantly outperforms existing approaches in image classification experiments by a large margin.

After reading rebuttals from the authors: The authors have addressed all of my concerns. THe additional experiments are a good addition.
************************
The authors provide an algorithm-agnostic active learning algorithm for multi-class classification. The core technique is to construct a coreset of points whose labels inform the labels of other points.  The coreset construction requires one to construct a set of  points which can cover the entire dataset. While this is NP-hard problem in general, the greedy algorithm is 2-approximate. The authors use a variant of the greedy algorithm along with bisection search to solve a series of feasibility problems to obtain a good cover of the dataset each time.  This cover tells us which points are to be queried. The reason why choosing the cover is a good idea is because under suitable Lipschitz continuity assumption the generalization error can be controlled via an appropriate value of the covering radius in the data space.  The authors use the coreset construction with a CNN to demonstrate an active learning algorithm for multi-class classification.
The experimental results are convincing enough to show that it outperforms other active learning algorithms. However, I have a few major and minor comments.
Major comments:
1. The proof of Lemma 1 is incomplete. We need the Lipschitz constant of the loss function. The loss function is a function of the CNN function and the true label. The proof of lemma 1 only establishes the Lipschitz constant of the CNN function. Some more extra work is needed to derive the lipschitz constant of the loss function from the CNN function.
2. The statement of Prop 1 seems a bit confusing to me. the hypothsis says that the loss on the coreset = 0. But the equation in proposition 1 also includes the loss on the coreset. Why is this term included. Is this term not equal to 0?
3. Some important works are missing.  Especially works related to pool based active learning, and landmark results on labell complexity of agnostic active learning.
UPAL: Unbiased Pool based active learning by Ganti & Gray. http://proceedings.mlr.press/v22/ganti12/ganti12.pdf
Efficient active learning of half-spaces by Gonen et al. http://www.jmlr.org/papers/volume14/gonen13a/gonen13a.pdf
A bound on the label complexity of agnostic active learning. http://www.machinelearning.org/proceedings/icml2007/papers/375.pdf
4.  The authors use L_2 loss as their objective function. This is a bit of a weird choice given that they are dealing with multi-class classification and the output layer is a sigmoid layer, making it a natural fit to work with something like a cross-entropy loss function. I guess the theoretical results do not extend to cross-entropy loss, but the authors do not mention these points anywhere in the paper. For example, the ladder network, which is one of the networks used by the authors is a network that uses cross-entropy for training.
Minor-comment:
1. The feasibility program in (6) is an MILP. However, the way it is written it does not look like an MILP. It would have been great had the authors mentioned that u_j \in {0,1}.
2. The authors write on page 4, "Moreover, zero training error can be enforced by converting average loss into maximal loss". It is not clear to me what the authors mean here. For example, can I replace the average error in proposition 1, by maximal loss? Why can I do that? Why would that result in zero training error?
On the whole this is interesting work and the results are very nice. But, the proof for Lemma 1 seems incomplete to me, and some choices (such as choice of loss function) are unjustified. Also, important references in active learning literature are missing.


Published as a conference paper at ICLR 2018 SYNTAX-DIRECTED VARIATIONAL AUTOENCODER FOR STRUCTURED DATA
Deep generative models have been enjoying success in modeling continuous data. However it remains challenging to capture the representations for discrete structures with formal grammars and semantics, e.g., computer programs and molecular structures. How to generate both syntactically and semantically correct data still remains largely an open problem. Inspired by the theory of compiler where the syntax and semantics check is done via syntax-directed translation (SDT), we propose a novel syntax-directed variational autoencoder (SD-VAE) by introducing stochastic lazy attributes. This approach converts the offline SDT check into on-the-fly generated guidance for constraining the decoder. Comparing to the state-of-the-art methods, our approach enforces constraints on the output space so that the output will be not only syntactically valid, but also semantically reasonable. We evaluate the proposed model with applications in programming language and molecules, including reconstruction and program/molecule optimization. The results demonstrate the effectiveness in incorporating syntactic and semantic constraints in discrete generative models, which is significantly better than current state-of-the-art approaches.

NOTE:
Would the authors kindly respond to the comment below regarding Kekulisation of the Zinc dataset? Fair comparison of the data is a serious concern. I have listed this review as a good for publication due to the novelty of ideas presented, but the accusation of misrepresentation below is a serious one and I would like to know the author's response.
*Overview*
This paper presents a method of generating both syntactically and semantically valid data from a variational autoencoder model using ideas inspired by compiler semantic checking. Instead of verifying the semantic correctness offline of a particular discrete structure, the authors propose “stochastic lazy attributes”, which amounts to loading semantic constraints into a CFG and using a tailored latent-space decoder algorithm that guarantees both syntactic semantic valid. Using Bayesian Optimization, search over this space can yield decodings with targeted properties.
Many of the ideas presented are novel. The results presented are state-of-the art. As noted in the paper, the generation of syntactically and semantically valid data is still an open problem. This paper presents an interesting and valuable solution, and as such constitutes a large advance in this nascent area of machine learning.
*Remarks on methodology*
By initializing a decoding by “guessing” a value, the decoder will focus on high-probability starting regions of the space of possible structures. It is not clear to me immediately how this will affect the output distribution. Since this process on average begins at high-probability region and makes further decoding decisions from that starting point, the output distribution may be biased since it is the output of cuts through high-probability regions of the possible outputs space. Does this sacrifice exploration for exploitation in some quantifiable way? Some exploration of this issue or commentary would be valuable.
*Nitpicks*
I found the notion of stochastic predetermination somewhat opaque, and section 3 in general introduces much terminology, like lazy linking, that was new to me coming from a machine learning background. In my opinion, this section could benefit from a little more expansion and conceptual definition.
The first 3 sections of the paper are very clearly written, but the remainder has many typos and grammatical errors (often word omission). The draft could use a few more passes before publication.


This paper proposes a new model for the rating prediction task in recommender systems which significantly outperforms previous state-of-the art models on a time-split Netflix data set. Our model is based on deep autoencoder with 6 layers and is trained end-to-end without any layer-wise pre-training. We empirically demonstrate that: a) deep autoencoder models generalize much better than the shallow ones, b) non-linear activation functions with negative parts are crucial for training deep models, and c) heavy use of regularization techniques such as dropout is necessary to prevent overfitting. We also propose a new training algorithm based on iterative output re-feeding to overcome natural sparseness of collaborate filtering. The new algorithm significantly speeds up training and improves model performance.

This paper presents a deep autoencoder model for rating prediction. The autoencoder takes the user’s rating over all the items as input and tries to predict the observed ratings in the output with mean squared error. A few techniques are applied to make the training feasible without layer-wise pre-training: 1) SELU activation. 2) dropout with high probability. 3) dense output re-feeding. On the Netflix prize dataset, the proposed deep autoencoder outperforms other state-of-the-art approaches.
Overall, the paper is easy to follow. However, I have three major concerns regarding the paper that makes me decide to reject it.
1. Lack of novelty. The paper is essentially a deeper version of the U-AutoRec (Sedhain et al. 2015) with a few recently emerged innovations in deep learning. The dense output re-feeding is not something particularly novel, it is more or less a data-imputation procedure with expectation-maximization — in fact if the authors intend to seek explanation for this output re-feeding technique, EM might be one of the interpretations. And similar technique (more theoretically grounded) has been applied in image imputation for variational autoencoder (Rezende et al. 2014, Stochastic Backpropagation and Approximate Inference in Deep Generative Models).
2. The experimental setup is also worth questioning. Using a time-split dataset is of course more challenging. However, the underlying assumption of autoencoders (or more generally, latent factor models like matrix factorization) is the all the ratings are exchangeable (conditionally independent given the latent representations), i.e., autoencoders/MF are not capable of inferring the temporal information from the data, Thus it is not even a head-to-head comparison with a temporal model (e.g., RNN in Wu et al. 2017). Of course you can still apply a static autoencoder to time-split data, but what ends up happening is the model will use its capacity to try to explain the temporal signal in the data — a deeper model certainly has more extra capacity to do so. I would suggest the authors comparing on a non-time-split dataset with other static models, like I(U)-AutoRec/MF/CF-NADE (Zheng et al. 2016)/etc.
3. Training deep models on recommender systems data is impressive. However, I would like to suggest we, as a community, start to step away from the task of rating predictions as much as we can, especially in more machine-learning-oriented venues (NIPS, ICML, ICLR, etc.) where the reviewers might be less aware of the shift in recommender systems research. (The task of rating predictions was made popular mostly due to the Netflix prize, yet even Netflix itself has already moved on from ratings.) Training (and evaluating) with RMSE on the observed ratings assumes all the missing ratings are missing at random, which is clearly far from realistic for recommender systems (see Marlin et al. 2007, Collaborative Filtering and the Missing at Random Assumption). In fact, understanding why some of the ratings are missing presents a unique challenge for the recommender systems. See, e.g., Steck 2010, Training and testing of recommender systems on data missing not at random, Liang et al. 2016, Modeling user exposure in recommendation, Schnabel et al. 2016, Recommendations as Treatments: Debiasing Learning and Evaluation. A model with good RMSE in a lot of cases does not translate to good recommendations (Cremonesi et al. 2010, Performance of recommender algorithms on top-n recommendation tasks
). As a first step, at least start to use all the 0’s in the form of implicit feedback and focus on ranking-based metrics other than RMSE.


In this paper, we introduce a method to compress intermediate feature maps of deep neural networks (DNNs) to decrease memory storage and bandwidth requirements during inference. Unlike previous works, the proposed method is based on converting fixed-point activations into vectors over the smallest GF(2) finite field followed by nonlinear dimensionality reduction (NDR) layers embedded into a DNN. Such an end-to-end learned representation finds more compact feature maps by exploiting quantization redundancies within the fixed-point activations along the channel or spatial dimensions. We apply the proposed network architecture to the tasks of ImageNet classification and PASCAL VOC object detection. Compared to prior approaches, the conducted experiments show a factor of 2 decrease in memory requirements with minor degradation in accuracy while adding only bitwise computations.

The method of this paper minimizes the memory usage of the activation maps of a CNN. It starts from a representation where activations are compressed with a uniform scalar quantizer and fused to reduce intermediate memory usage. This looses some accuracy, so the contribution of the paper is to add a pair of convolution layers in the binary domain (GF(2)) that are trained to restore the lost precision.
Overall, this paper seems to be a nice addition to the body of works on network compression.
+ : interesting approach and effective results.
+ : well related to the state of the art and good comparison with other works.
- : somewhat incremental. Most of the claimed 100x compression is due to previous work.
- : impact on runtime is not reported. Since there is a caffe implementation it would be interesting to have an additional column with the comparative execution speeds, even if only on CPU. I would expect the FP32 timings to be hard to beat, despite the claims that it uses only binary operations.
- : the paper is sometimes difficult to understand (see below)
detailed comments:
Equations (3)-(4) are difficult to understand. If I understand correctly, b just decomposes a \hat{x} in {0..2^B-1} into its B bits \tilda{x} \in {0,1}^B, which can be then considered as an additional dimension in the activation map where \hat{x} comes from.
It is not stated clearly whether P^l and R^l have binary weights. My understanding is that P^l has but R^l not.
4.1 --> a discussion of the large mini-batch size (1024) could be useful. My understanding is that large mini-batches are required to use averaged gradients and get smooth updates.
end of 4.1 --> unclear what "equivalent bits" means


We propose a novel attention mechanism to enhance Convolutional Neural Networks for fine-grained recognition. The proposed mechanism reuses CNN feature activations to find the most informative parts of the image at different depths with the help of gating mechanisms and without part annotations. Thus, it can be used to augment any layer of a CNN to extract low-and high-level local information to be more discriminative. Differently, from other approaches, the mechanism we propose just needs a single pass through the input and it can be trained end-to-end through SGD. As a consequence, the proposed mechanism is modular, architecture-independent, easy to implement, and faster than iterative approaches. Experiments show that, when augmented with our approach, Wide Residual Networks systematically achieve superior performance on each of five different finegrained recognition datasets: the Adience age and gender recognition benchmark, Caltech-UCSD Birds-200-2011, Stanford Dogs, Stanford Cars, and UEC Food-100, obtaining competitive and state-of-the-art scores.

The manuscript describes a novel attentional mechanism applied to fine-grained recognition.
On the positive side, the approach seems to consistently improve the recognition accuracy of the baseline  (a wide residual net). The approach is also consistently tested on the main fine-grained recognition datasets (the Adience age and gender recognition benchmark, Caltech-UCSD Birds-200-2011, Stanford Dogs, Stanford Cars, and UEC Food-100).
On the negative side, the paper could be better written and motivated.
First, some claimed are made about how the proposed approach "enhances most of the desirable properties from previous approaches” (see pp 1-2) but these claims are never backed up. More generally since the paper focuses on attention, other attentional approaches should be used as benchmarks beyond the WRN baseline. If the authors want to claim that the proposed approach is "more robust to deformation and clutter” then they should design an experiment that shows that this is the case.
Beyond, the approach seems a little ad hoc. No real rationale is provided for the different mechanisms including the gating etc and certainly no experimental validation is provided to demonstrate the need for these mechanisms. More generally, it is not clear from reading the paper specifically what computational limitation of the CNN is being solved by the proposed attentional mechanism.
Some of the masks shown in Fig 3 seem rather suspicious and prompt this referee to think that the networks are seriously overfitting to the data. For instance, why would attending to a right ear help in gender recognition?
The proposed extension adds several hyperparameters (for instance the number K of attention heads). Apologies if I missed it but I am not clear how this was optimized for the experiments reported. In general, the paper could be clearer. For instance, it is not clear from either the text or Fig 2 how H goes from XxYxK for the attention head o XxYxN for the output head.
As a final point, I would say that while some of the criticisms could be addressed in a revision, the improvements seem relatively modest. Given that the focus of the paper is already limited to fine-grained recognition, it seems that the paper would be better suited for a computer vision conference.
Minor point:
"we incorporate the advantages of visual and biological attention mechanisms” not sure this statement makes much sense. Seems like visual and biological are distinct attributes but visual attention can be biological (or not, I guess) and it is not clear how biological the proposed approach is. Certainly no attempt is made by the authors to connect to biology.
"top-down feed-forward attention mechanism” -> it should be just feed-forward attention. Not clear what "top-down feed-forward” attention could be...


SEMI-PARAMETRIC TOPOLOGICAL MEMORY FOR NAVIGATION
We introduce a new memory architecture for navigation in previously unseen environments, inspired by landmark-based navigation in animals. The proposed semiparametric topological memory (SPTM) consists of a (non-parametric) graph with nodes corresponding to locations in the environment and a (parametric) deep network capable of retrieving nodes from the graph based on observations. The graph stores no metric information, only connectivity of locations corresponding to the nodes. We use SPTM as a planning module in a navigation system. Given only 5 minutes of footage of a previously unseen maze, an SPTM-based navigation agent can build a topological map of the environment and use it to confidently navigate towards goals. The average success rate of the SPTM agent in goal-directed navigation across test environments is higher than the best-performing baseline by a factor of three.

*** Revision: based on the author's work, we have switched the score to accept (7) ***
Clever ideas but not end-to-end navigation.
This paper presents a hybrid architecture that mixes parametric (neural) and non-parametric (Dijkstra's path planning on a graph of image embeddings) elements and applies it to navigation in unseen 3D environments (Doom). The path planning in unseen environments is done in the following way: first a human operator traverses the entire environment by controlling the agent and collecting a long episode of 10k frames that are put into a chain graph. Then loop closures are automatically detected using image similarity in feature space, using a localization feed-forward ResNet (trained using a DrLIM-like triplet loss on time-similar images), resulting in a topological graph where edges correspond to similar viewpoints or similar time points. For a given target position image and agent start position image, a nearest neighbor search-powered Dijkstra path planning is done on the graph to create a list of waypoint images. The pairs of (current image, next waypoint) images are then fed to a feed-forward locomotion (policy) network, trained in a supervised manner.
The paper does not discuss at all the problems arising when the images are ambiguous: since the localisation network is feed-forward, surely there must be images that are ambiguously mapped to different graph areas and are closing loops erroneously? The problem is mitigated by the fact that a human operator controls the agent, making sure that the agent's viewpoint is clear, but the method will probably fail if the agent is learning to explore the maze autonomously, bumping into walls and facing walls. The screenshots on Figure 3 suggest that the walls have a large variety of textures and decorations, making each viewpoint potentially unique, unlike the environments in (Mirowski et al, 2017), (Jaderberg et al, 2017) and (Mnih et al, 2016).
Most importantly, the navigation is not based on RL at all, and ignores the problem of exploration altogether. A human operator labels 10k frames by playing the game and controlling the agent, to show it how the maze looks like and what are the paths to be taken. As a consequence, comparison to end-to-end RL navigation methods is unclear, and should be stressed upon in the manuscript. This is NOT a proper navigation agent.
Additional baselines should be evaluated: 1) a fully Dijkstra-based baseline where the direction of motion of the agent along the edge is retrieved and used to guide the agent (i.e., the policy becomes a lookup table on image pairs) and 2) the same but the localization network replaced by image similarities in pixel space or some image descriptor space (e.g., SURF, ORB, etc…). It seems to me that those baselines would be very strong.
Another baseline is missing: (Oh et al, 2016, “Control of Memory, Active Perception, and Action in Minecraft”).
The paper is not without merit: the idea of storing experiences in a graph and in using landmark similarity rather than metric embeddings is interesting. Unfortunately, that episodic memory is not learned (e.g., Neural Turing Machines or Memory Networks).
In summary, just like the early paper released in 2010 about Kinect-based RGBD SLAM: lots of excitement but potential disappointment when the method is applied on an actual mobile robot, navigating in normal environments with visual ambiguity and white walls. The paper should ultimately be accepted to this conference to provide a baseline for the community  (once the claims are revised), but I street that the claims of learning to navigate in unseen environments are unsubstantiated, as the method is neither end-to-end learned (as it relies on human input and heuristic path planning) nor capable of exploring unseen environments with visual ambiguity.


Interpretability and small labelled datasets are key issues in the practical application of deep learning, particularly in areas such as medicine. In this paper, we present a semi-supervised technique that addresses both these issues simultaneously. We learn dense representations from large unlabelled image datasets, then use those representations to both learn classifiers from small labeled sets and generate visual rationales explaining the predictions. Using chest radiography diagnosis as a motivating application, we show our method has good generalization ability by learning to represent our chest radiography dataset while training a classifier on an separate set from a different institution. Our method identifies heart failure and other thoracic diseases. For each prediction, we generate visual rationales for positive classifications by optimizing a latent representation to minimize the probability of disease while constrained by a similarity measure in image space. Decoding the resultant latent representation produces an image without apparent disease. The difference between the original and the altered image forms an interpretable visual rationale for the algorithm's prediction. Our method simultaneously produces visual rationales that compare favourably to previous techniques and a classifier that outperforms the current state-of-the-art.

The authors address two important issues: semi-supervised learning from relatively few labelled training examples in the presence of many unlabelled examples, and visual rationale generation: explaining the outputs of the classifiier by overlaing a visual rationale on the original image. This focus is mainly on medical image classification but the approach could potentially be useful in many more areas. The main idea is to train a GAN on the unlabeled examples to create a mapping from a lower-dimensional space in which the input features are approximately Gaussian, to the space of images, and then to train an encoder to map the original images into this space minimizing reconstruction error with the GAN weights fixed. The encoder is then used as a feature extractor for classification and regression of targets (e.g. heard disease). The visual rationales are generated by optimizing the encoded representation to simultaneously reconstruct an image close to the original and to minimize the probability of the target class. This gives an image that is similar to the original but with features that caused the classification of the disease removed. The resulting image can be subtracted from the original encoding to highlight problematic areas. The approach is evaluated on an in-house dataset and a public NIH dataset, demonstrating good performance, and illustrative visual rationales are also given for MNIST.
The idea in the paper is, to my knowledge, novel, and represents a good step toward the important task of generating interpretable visual rationales. There are a few limitations, e.g. the difficulty of evaluating the rationales, and the fact that the resolution is fixed to 128x128 (which means discarding many pixels collected via ionizing radiation), but these are readily acknowledged by the authors in the conclusion.
Comments:
1) There are a few details missing, like the batch sizes used for training (it is difficult to relate epochs to iterations without this). Also, the number of hidden units in the 2 layer MLP from para 5 in Sec 2.
2) It would be good to include PSNR/MSE figures for the reconstruction task (fig 2) to have an objective measure of error.
3) Sec 2 para 4: "the reconstruction loss on the validation set was similar to the reconstruction loss on the validation set" -- perhaps you could be a little more precise here. E.g. learning curves would be useful.
4) Sec 2 para 5: "paired with a BNP blood test that is correlated with heart failure" I suspect many readers of ICLR, like myself, will not be well versed in this test, correlation with HF, diagnostic capacity, etc., so a little further explanation would be helpful here. The term "correlated" is a bit too broad, and it is difficult for a non-expert to know exactly how correlated this is. It is also a little confusing that you begin this paragraph saying that you are doing a classification task, but then it seems like a regression task which may be postprocessed to give a classification. Anyway, a clearer explanation would be helpful. Also, if this test is diagnostic, why use X-rays for diagnosis in the first place?
5) I would have liked to have seen some indicative times on how long the optimization takes to generate a visual rationale, as this would have practical implications.
6) Sec 2 para 7: "L_target is a target objective which can be a negative class probability or in the case of heart failure, predicted BNP level" -- for predicted BNP level, are you treating this as a probability and using cross entropy here, or
mean squared error?
7) As always, it would be illustrative if you could include some examples of failure cases, which would be helpful both in suggesting ways of improving the proposed technique, and in providing insight into where it may fail in practical situations.


Performance of Deep Neural Network (DNN) heavily depends on the characteristics of hidden layer representations. Unlike the codewords of channel coding, however, the representations of learning cannot be directly designed or controlled. Therefore, we develop a family of penalty regularizers where each one aims to affect one of the representation's statistical properties such as sparsity, variance, or covariance. The regularizers are extended to perform class-wise regularization, and the extension is found to provide an outstanding shaping capability. A variety of statistical properties are investigated for ten different regularization strategies including dropout and batch normalization, and several interesting findings are reported. Using the family of regularizers, performance improvements are confirmed for MNIST, CIFAR-100, and CIFAR-10 classification problems. But more importantly, our results suggest that understanding how to manipulate statistical properties of representations can be an important step toward understanding DNN, and that the role and effect of DNN regularizers need to be reconsidered.

1. Summary
The authors of the paper compare the learning of representations in DNNs with Shannons channel coding theory, which deals with reliably sending information through channels. In channel coding theory the statistical properties of the coding of the information can be designed to fit the task at hand. With DNNs the representations cannot be designed in the same way. But the representations, learned by DNNs, can be affected indirectly by applying regularization. Regularizers can be designed to affect statistical properties of the representations, such as sparsity, variance, or covariance. The paper extends the regularizers to perform per-class regularization. This makes sense, because, for example, forcing the variance of a representation to go towards zero is undesirable as it would state that the unit always has the same output no matter the input. On the other hand having zero variance for a class is desirable as it means that the unit has a consistent activation for all samples of the same class. The paper compares different regularization techniques regarding their error performance. They find that applying representation regularization outperforms classical approaches such as L1 and L2 weight regularization. They also find, that performing representation regularization on the last layer achieves the best performance. Class-wise methods generally outperform methods that apply regularization on all classes.
2. Remarks
Shannons channel coding theory was used by the authors to derive regularizers, that manipulate certain statistical properties of representations learned by DNNs. In the reviewers opinion, there is no theoretical connection between DNNs and channel theory. For one, DNNs are no channels in the sense that they transmit information. DNNs are rather pipes that transform information from one domain to another, where representations are learned as an intermediate model as the information is being transformed. Noise introduced in the process is not due to a faulty channel but due to the quality of the learned representations themselves. The paper falls short in explaining how DNNs and Shannons channel coding theory fit together theoretically and how they used it to derive the proposed regularizers. Despite the theoretical gap between the two was not properly bridged by the authors, channel coding theory is still a good metaphor for what they were trying to achieve.
The authors recognize that there is similar research being done independently by Belharbi et al. (2017). The similarities and differences between the proposed work and Belharbi et al. should be discussed in more detail.
The authors conclude that it is unclear which statistical properties of representations are generally helpful when being strengthened. It would be nice if they had derived at least a set of rules of thumb. Especially because none of the regularizers described in the paper only target one specific statistical property but multiple. One good example that was provided, is that L1-rep consistently failed to train on CIFAR-100, because too much sparsity can hurt performance, when having many different classes (100 in this case). These kinds of conclusions will make it easier to transfer the presented theory into practice.
3. Conclusion
The comparison between DNNs and Shannons channel coding theory stands on shaky ground. The proposed regularizes are rather simple, but perform well in the experiments. The effect of each regularizer on the statistical properties of the representation and the relations to previous work (especially Belharbi et al. (2017)) should be discussed in more detail.


Workshop track -ICLR 2018 ADVERSARIAL POLICY GRADIENT FOR ALTERNATING MARKOV GAMES
Policy gradient reinforcement learning has been applied to two-player alternateturn zero-sum games, e.g., in AlphaGo, self-play REINFORCE was used to improve the neural net model after supervised learning. In this paper, we emphasize that two-player zero-sum games with alternating turns, which have been previously formulated as Alternating Markov Games (AMGs), are different from standard MDP because of their two-agent nature. We exploit the difference in associated Bellman equations, which leads to different policy iteration algorithms. As policy gradient method is a kind of generalized policy iteration, we show how these differences in policy iteration are reflected in policy gradient for AMGs. We formulate an adversarial policy gradient and discuss potential possibilities for developing better policy gradient methods other than self-play REINFORCE. The core idea is to estimate the minimum rather than the mean for the "critic". Experimental results on the game of Hex show the modified Monte Carlo policy gradient methods are able to learn better pure neural net policies than the REIN-FORCE variants. To apply learned neural weights to multiple board sizes Hex, we describe a board-size independent neural net architecture. We show that when combined with search, using a single neural net model, the resulting program consistently beats MoHex 2.0, the previous state-of-the-art computer Hex player, on board sizes from 9×9 to 13×13.

The paper makes the simple but important observation that (deep) reinforcement learning in alternating Markov games requires a min-max formulation of the Bellman equation as well as careful attention to the way in which one alternates solving for both players' policies in a policy iteration setting.
While some of the core algorithmic insights regarding Algorithms 3 & 4 in the paper stem from previous work (Condon, 1990; Hoffman & Karp, 1966), I was not actually aware of these previous results until I reviewed this paper.
A nice corollary of Algorithms 3 & 4 is that they make for a straightforward adaptation of policy gradient algorithms since when optimizing one policy, the other is fixed to the greedy policy.
In general, it would be nice to have the algorithms specified as formal algorithms as opposed to text-based outlines.  I found myself reading and re-reading descriptions to make sure I understood what math was being implied by the descriptions.
Section 6
> Hex is simpler than Go in the sense that perfect play can
> often be achieved whenever virtual connections are found
> by H-Search
It is not clear here what virtual connections are, what H-Search is, and how these imply perfect play, if perfect play as previously discussed is unknown.
Overall, the results on Hex for AMCPG-A and AMCPG-B vs. standard REINFORCE variants currently used are very encouraging.  That said, empirically it is always a question of whether these results are specific to Hex.  Because this paper is not proposing the best Hex player (i.e., the winning rate against Wolve never exceeds 0.5), I think it is quite reasonable to request the authors to compare AMCPG-A and AMCPG-B to standard REINFORCE variants on other games (they do not need to be as difficult as Hex).
Finally, assuming that the results do generalize to other games, I am left wondering about the significance of the contribution.  On one hand, the authors have introduced me to literature I was not aware of, but on the other hand, their actual novel contribution is a rather straightforward adaptation of ideas in the literature to policy gradients (that could be formalized in a more technically precise way) with an evaluation on a single type of game.  This is a useful contribution no doubt, but I am concerned with whether it meets the significance level that I am used to with accepted ICLR papers in previous years.


New types of compute hardware in development and entering the market hold the promise of revolutionizing deep learning in a manner as profound as GPUs. However, existing software frameworks and training algorithms for deep learning have yet to evolve to fully leverage the capability of the new wave of silicon. In particular, models that exploit structured input via complex and instance-dependent control flow are difficult to accelerate using existing algorithms and hardware that typically rely on minibatching. We present an asynchronous model-parallel (AMP) training algorithm that is specifically motivated by training on networks of interconnected devices. Through an implementation on multi-core CPUs, we show that AMP training converges to the same accuracy as conventional synchronous training algorithms in a similar number of epochs, but utilizes the available hardware more efficiently, even for small minibatch sizes, resulting in shorter overall training times. Our framework opens the door for scaling up a new class of deep learning models that cannot be efficiently trained today.

This paper presents AMPNet, that addresses parallel training for dynamic networks. This is accomplished by building a static graph like IR that can serve as a target for compilation for high-level libraries such as tensor flow. In the IR each node of the computation graph is a parallel worker, and synchronization occurs when a sufficient number of gradients have been accumulated. The IR uses constructs such as concat, split, broadcast,.. allowing dynamic, instance dependent control flow decisions. The primary improvement in training performance is from reducing synchronization costs.
Comments for the author:
The paper proposes a solution to an important problem of model parallel training especially over dynamic batching that is increasingly important as we see more complex models where batching is not straightforward. The proposed solution can be effective. However, this is not really evident from the evaluation. Furthermore, the paper can be a little dense read for the ICLR audience. I have the following additional concerns:
1) The paper stresses new hardware throughout the paper. The paper also alludes to “simulator" of a 1 TFLOPs FPGA in the conclusion. However, your entire evaluation is over CPU. The said simulator is a bunch of sleep() calls (unless some details are skipped). I would encourage the authors to remove these references since these new devices have very different hardware behavior. For example, on a real constrained device, you may not enjoy a large L2 cache which you are benefitting from by doing an entire evaluation over CPUs. Likewise, the vector instruction processing behavior is also very different since these devices have limited power budgets and may not be able to support AVX style instructions. Unless an actual simulator like GEM5 is used, a correct representation of what hardware environment is being used is necessary before making claims that this is ideal for emerging hardware.
2) To continue on the hardware front and the evaluation, I feel for this paper to be accepted or appreciated, a simulated hardware is not necessary. Personally, I found the evaluation with simulated sleep functions more confusing than helpful. An appropriate evaluation for this paper can be just benefits over CPU or GPUs, For example, you have a 7 TFLOPS device (e.g. a GPU or a CPU). Existing algorithms extract X TFLOPs of processing power and using your IR/system one gets Y effective TFLOPs and Y>X. This is all that is required. Currently, looking at your evaluation riddled with hypothetical hardware, it is unclear to me if this is helpful for existing hardware. For example, in Table 1, are Tensorflow numbers only provided over the 1 TFLOPs device (they correspond to the 1 TFLOPs column for all workloads except for MNIST)?  Do you use the parallelism at all in your Tensorflow baseline?  Please clarify.
3) How do you compare for dynamic batching with dynamic IR platforms like pytorch? Furthermore, more details about how dynamic batching is happening in benchmarks mentioned in Table 1 will be nice to have. Finally, an emphasis on the novel contributions of the paper will also be appreciated.
4) Finally, the evaluation appears to be sensitive to the two hyper-parameters introduced. Are they dataset specific? I feel tuning them would be rather cumbersome for every model given how sensitive they are (Figure 5).


Workshop track -ICLR 2018 FEATURE INCAY FOR REPRESENTATION REGULARIZA- TION
Softmax-based loss is widely used in deep learning for multi-class classification, where each class is represented by a weight vector and each sample is represented as a feature vector. Inspired by that weight decay is a common practice to regularize the weight vectors, we investigate how to regularize the feature vectors since representation is also tunable in deep learning. One main observation is that elongating the feature norm of both correctlyclassified and mis-classified feature vectors improves learning: (1) increasing the feature norm of correctly-classified examples enlarges the probability margin among different classes and ensures better generalization. (2) increasing the feature norm of mis-classified examples can up-weight the contribution from hard examples. Accordingly, we propose feature incay to regularize feature vectors by encouraging larger feature norm. Extensive empirical results on MNIST, CI-FAR10, CIFAR100 and LFW demonstrate the effectiveness of feature incay.

The manuscript proposes to increase the norm of the last hidden layer to promote better classification accuracy. However, the motivation is a bit less convincing. Here are a few motivations that are mentioned.
(1) Increasing the feature norm of correctly classified examples helps cross entropy, which is of course correct. However, it only decreases the training loss. How do we know it will not lead to overfitting?
(2) Increasing the feature norm of mis-classified examples will make gradient larger for self-correction. And the manuscript proves it in property 2. However, the proof seems not complete. In Eq (7), increasing the feature norm would also affect the value of the term in parenthesis. As an example, if a negative example is already mis-classified as a positive, and its current probability is very close to 1, then further increasing feature norm would make the probability even closer to 1, leading to saturation and smaller gradient.
(3) Figure 1 shows that examples with larger feature norm tend to be predicted well. However, it is not very convincing since it is only a correlation rather than causality. Let's use simple linear softmax regression as a sanity check, where features to softmax are real features rather than hidden units. Increasing the feature norm seems to be against the best practice of feature normalization in which each feature after normalization is of variance 1.
The manuscript states that the feature norm won't be infinitely increased since there is an upper bound. However, the proof of property 3 seems to only apply to the certain cases where K<2D. In addition, alpha is in the formula of upper bound, but what is the upper bound of alpha?
The manuscript does comprehensive experiments to test the proposed method. The results are good, since the proposed method outperforms other baselines in most datasets. But the results are not impressively strong.
Minor issues:
(1) For proof of property 3, it seems that alpha and beta are used before defined. Are they the radius of two circles?


Deep learning has become the state of the art approach in many machine learning problems such as classification. It has recently been shown that deep learning is highly vulnerable to adversarial perturbations. Taking the camera systems of self-driving cars as an example, small adversarial perturbations can cause the system to make errors in important tasks, such as classifying traffic signs or detecting pedestrians. Hence, in order to use deep learning without safety concerns a proper defense strategy is required. We propose to use ensemble methods as a defense strategy against adversarial perturbations. We find that an attack leading one model to misclassify does not imply the same for other networks performing the same task. This makes ensemble methods an attractive defense strategy against adversarial attacks. We empirically show for the MNIST and the CIFAR-10 data sets that ensemble methods not only improve the accuracy of neural networks on test data but also increase their robustness against adversarial perturbations.

In this manuscript, the authors empirically investigated the robustness of some different deep neural networks ensembles to two types of attacks, namely FGSM and BIM, on two popular datasets, MNIST and CIFAR10. The authors concluded that the ensembles are more accurate on both clean and adversaries samples than a single deep neural network. Therefore, the ensembles are more robust in terms of the ability to correctly classify the adversary attacks.
As the authors stated, an attack that is designed to fool one network does not necessarily fool the other networks in the same way. This is likely why ensembles appear more robust than single deep learners. However, robustness of ensembles to the white-box attacks that are generated from the ensemble is still low for FGS. Generally speaking, although FGS attacks generated from one network can fool less the whole ensembles, generating FGS adversaries from a given ensemble is still able to effectively fool it. Therefore, if the attacker has access to the ensemble or even know the classification system based on that ensemble, then the ensemble-based system is still vulnerable to the attacks generated specifically from it. Simple ensemble methods are not likely to confer significant robustness gains against adversaries.
In contrast to FGS results, surprisingly BIM-Grad1 is able to fool more the ensemble than BIM-Grad2. Therefore, it seems that if the attacker makes BIM adversaries from only a single classifier, then she can simply and yet effectively mislead the whole ensemble. In comparison to BIM-Grad2, BIM-Grad1 results show that BIM attacks from one network (BIM-Grad1) can more successfully fool the other different networks in the ensembles in a similar way! BIM-Grad2 is not that much able to fool the ensemble-based system even this attack generated from the ensemble (white-box attacks). In order to confirm the robustness of the ensembles to BIM attacks, the authors can do more experiments by generating BIM-Grad2 attacks with higher number of iterations.
Indeed, the low number of iterations might cause the lower rate of success for generating adversaries by BIM-Grad2. In fact, BIM adversaries from the ensembles might require more number of iterations to effectively fool the majority of the members in the ensembles. Therefore, increasing the number of iterations can increase the successful rate of generating BIM-Average Grad2 adversaries. Note that in this case, it is recommended to compare the amount of distortion (perturbation) with different number of iterations in order to indicate the effectiveness of the ensembles to white-box BIM attacks.
Despite to averaging the output probabilities to compute the ensemble final prediction, the authors generated the adversaries from the ensemble by computing the sum of the gradients of the classifiers loss. A proper approach would have been to average of these gradients. The fact the sum is not divided by the number of members (i.e., sum of gradients instead of average of gradients) is increasing the step size of the adversarial method proportionally to the ensemble size, raising questions on the validity of the comparison with the single-model adversarial generation.
Overall, I found the paper as having several methodological flaws in the experimental part, and rather light in terms of novel ideas. As noticed in the introduction, the idea of using ensemble for enhancing robustness as already been proposed. Making a paper only to restate it, is too light for acceptation. Moreover, experimental setup using a lot of space for comparing results on standard datasets (i.e., MNIST and CIFAR10), even with long presentation of these datasets. Several issues are raised in the current experiments and require adjustments. Experiments should also be more elaborated to make the case stronger, following at least some of indications provided.