While deep neural networks have shown outstanding results in a wide range of applications, learning from a very limited number of examples is still a challenging task. Despite the difficulties of the few-shot learning, metric-learning techniques showed the potential of the neural networks for this task. While these methods perform well, they don't provide satisfactory results. In this work, the idea of metric-learning is extended with Support Vector Machines (SVM) working mechanism, which is well known for generalization capabilities on a small dataset. Furthermore, this paper presents an end-to-end learning framework for training adaptive kernel SVMs, which eliminates the problem of choosing a correct kernel and good features for SVMs. Next, the one-shot learning problem is redefined for audio signals. Then the model was tested on vision task (using Omniglot dataset) and speech task (using TIMIT dataset) as well. Actually, the algorithm using Omniglot dataset improved accuracy from 98.1% to 98.5% on the one-shot classification task and from 98.9% to 99.3% on the few-shot classification task.

Make SVM great again with Siamese kernel for few-shot learning
** PAPER SUMMARY **
The author proposes to combine siamase networks with an SVM for pair classification. The proposed approach is evaluated on few shot learning tasks, on omniglot and timit.
** REVIEW SUMMARY **
The paper is readable but it could be more fluent. It lacks a few references and important technical aspects are not discussed. It contains a few errors. Empirical contribution seems inflated on omniglot as the authors omit other papers reporting better results. Overall, the contribution is modest at best.
** DETAILED REVIEW **
On mistakes, it is wrong to say that an SVM is a parameterless classifier. It is wrong to cite (Boser et al 92) for the soft-margin SVM. I think slack variables come from (Cortes et al 95). "consistent" has a specific definition in machine learning https://en.wikipedia.org/wiki/Consistent_estimator , you must use a different word in 3.2. You mention that a non linear SVM need a similarity measure, it actually need a positive definite kernel which has a specific definition, https://en.wikipedia.org/wiki/Positive-definite_kernel .
On incompleteness, it is not obvious how the classifier is used at test time. Could you explain how classes are predicted given a test problem? The setup of the experiments on TIMIT is extremely unclear. What are the class you are interested in? How many classes and examples does the testing problems have?
On clarity, I do not understand why you talk again about non-linear SVM in the last paragraph of 3.2. since you mention at the end of page 4 that you will only rely on linear SVMs for computational reasons. You need to mention explicitely somewhere that (w,\theta) are optimized jointly. The sentence "this paper investigates only the one versus rest approach" is confusing, as you have only two classes from the SVM perspective i.e. pairs (x1,x2) where both examples come from the same class and pairs (x1,x2) where they come from different class. So you use a binary SVM, not one versus rest. You need to find a better justification for using L2-SVM than "L2-SVM loss variant is considered to be the best by the author of the paper", did you try classical SVM and found them performing worse? Also could you motivate your choice for L1 norm as opposed to L2 in Eq 3?
On empirical evaluation, I already mentioned that it impossible to understand what the classification problem on TIMIT is. I suspect it might be speaker identification. So I will focus on the omniglot experiments.
Few-Shot Learning Through an Information Retrieval Lens, Eleni Triantafillou, Richard Zemel, Raquel Urtasun, NIPS 2017 [arxiv July'17]
and the reference therein give a few more recent baselines than your table. Some of the results are better than your approach. I am not sure why you do not evaluate on mini-imagenet as well as most work on few shot learning generally do. This dataset offers a clearer experimental setup than your TIMIT setting and has abundant published baseline results. Also, most work typically use omniglot as a proof of concept and consider mini-imagenet as a more challenging set.


The biological plausibility of the backpropagation algorithm has long been doubted by neuroscientists. Two major reasons are that neurons would need to send two different types of signal in the forward and backward phases, and that pairs of neurons would need to communicate through symmetric bidirectional connections. We present a simple two-phase learning procedure for fixed point recurrent networks that addresses both these issues. In our model, neurons perform leaky integration and synaptic weights are updated through a local mechanism. Our learning method extends the framework of Equilibrium Propagation to general dynamics, relaxing the requirement of an energy function. As a consequence of this generalization, the algorithm does not compute the true gradient of the objective function, but rather approximates it at a precision which is proven to be directly related to the degree of symmetry of the feedforward and feedback weights. We show experimentally that the intrinsic properties of the system lead to alignment of the feedforward and feedback weights, and that our algorithm optimizes the objective function.

The manuscript discusses a learning algorithm that is based on the equilibrium propagation method, which can be applied to networks with asymmetric connections. This extension is interesting, but the results seem to be incomplete and missing necessary additional analyses. Therefore, I do not recommend acceptance of the manuscript in its current form. The main issues are:
1) The theoretical result is incomplete since it fails to show that the algorithm converges to a meaningful learning result. Also the experimental results do not sufficiently justify the claims.
2) The paper further makes statements about the performance and biological plausibility of the proposed algorithm that do not hold without additional justification.
3) The paper does not sufficiently discuss and compare the relevant neuroscience literature and related work.
Details to major points:
1) The presentation of the theoretical results is misleading. Theorem 1 shows that the proposed neuron dynamics has a fixed point that coincides with a local minimum of the objective function if the weights are symmetric. However, this was already clear from the original equilibrium propagation paper. The interesting question is whether the proposed algorithm automatically converges to the condition of symmetric weights, which is left unanswered. In Figure 3 experimental evidence is provided, but the results are not convincing given that the weight alignment only improves by ~1° throughout learning (compared to >45° in Lillicrap et al., 2017). It is even unclear to me if this effect is statistically significant. How many trials did the authors average over here? The authors should provide standard statistical significance measures for this plot. Since no complete theoretical guarantees are provided, a much broader experimental study would be necessary to justify the claims made in the paper.
2) Throughout the paper it is claimed that the proposed learning algorithm is biologically plausible. However, this argument is also not sufficiently justified. Most importantly, it is unclear how the proposed algorithm would behave in a biologically realistic recurrent networks and it is unclear how the different learning phases should be realized in the brain.
Neural networks in the brain are abundantly recurrent. Even in the layered structure of the neocortex one finds dense lateral connectivity between neurons on each layer. It is not clear to me how the proposed algorithm could be applied to such networks. In a recurrent network, rolled-out over time, information would need to be passed forward and backwards in time. The proposed algorithm does not seem to provide a solution to this temporal credit assignment problem. Also in the experiments the algorithm is applied only to feedforward architectures. What would happen if recurrent networks were used to learn temporal tasks like TIMIT? Please discuss.
In the discussion on page 8 the authors further argue that the learning phases of the proposed algorithm could be implemented in the cortex through theta waves that modulate long-term plasticity. To support this theory the authors cite the results from Orr et al., 2001, where hippocampal place cells in behaving rats were studied. To my knowledge there is no consensus on the precise nature of this modulation of plasticity. E.g. in Wyble et al. 2003, it was observed that application of learning protocols at different phases of theta waves actually leads to a sign change in learning, i.e. long term potentiation was modulated to depression. It seems to me that the algorithm is not compatible with these other experimental findings, since gradients only point in the correct direction towards the final phase and any non-zero learning rate in other phases would therefore perturb learning. Did the authors try non-optimal learning rate schedules in the experiments (including sign change etc.) to test the robustness of the proposed algorithm? Also to my knowledge, the modulatory effect of theta rhythms has so far only been described in the CA1 region of rodent hippocampus which is a very specialized region of the brain (see Hanslmayr et al., 2016, for a review and a modern hypothesis on the role of theta rhythms in the brain).
Furthermore, the discussion of the possible implementation of the learning algorithm in analog hardware on page 8 is missing an explanation of how the different learning phases of the algorithm are controlled on the chip. One of the advantages of analog hardware is that it does not require global clocking, unlike classical digital hardware, which is expensive in wiring and energy requirement. It seems to me that this advantage would disappear if the algorithm was brought to an analog chip, since global information about the learning phase has to be communicated to each synapse. Is there an alternative to a global wiring scheme to convey this information throughout the whole chip? Please discuss this in more depth.
3) The authors apply the learning algorithm only to the MNIST dataset, which is a relatively simple task. Similar results were also achieved using random feedback alignment (Lillicrap et al., 2017). Also, the evolutionary strategies method (Salimans et al., 2017), was recently used for learning deep networks and applied to complex reinforcement learning problems and could likewise also be applied to simple classification tasks. Both these methods are arguably as simple and biologically plausible as the proposed algorithm. It would be good to try other standard benchmark tasks and report and compare the performance there. Furthermore, the paper is missing a broader related work section that discusses approaches for biologically plausible learning rules for deep neural architectures.
Minor points:
The proposed algorithm uses different learning rates that shrink exponentially with the layer number. Have the authors explored whether the algorithm works for really deep architectures with several tens of layers? It seems to me that the used learning rate heuristic may hinder scalability of equilibrium propagation.
On page 5 the authors write: "However we observe experimentally that the dynamics almost always converges." This needs to be quantified. Did the authors find that the algorithm is very sensitive to initial conditions?
References:
Bradley P. Wyble, Vikas Goyal, Christina A. Rossi, and Michael E. Hasselmo. Stimulation in Hippocampal Region CA1 in Behaving Rats Yields Long-Term Potentiation when Delivered to the Peak of Theta and Long-Term Depression when Delivered to the Trough James M. Hyman. Journal of Neuroscience. 2003.
Simon Hanslmayr, Bernhard P. Staresina, and Howard Bowman. Oscillations and Episodic Memory: Addressing the Synchronization/Desynchronization Conundrum. Trends in Neurosciences. 2016.
Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, Ilya Sutskever. Evolution Strategies as a Scalable Alternative to Reinforcement Learning. Arxiv. 2017.


Machine learning models are usually tuned by nesting optimization of model weights inside the optimization of hyperparameters. We give a method to collapse this nested optimization into joint stochastic optimization of both weights and hyperparameters. Our method trains a neural network to output approximately optimal weights as a function of hyperparameters. We show that our method converges to locally optimal weights and hyperparameters for sufficiently large hypernets. We compare this method to standard hyperparameter optimization strategies and demonstrate its effectiveness for tuning thousands of hyperparameters.

*Summary*
The paper proposes to use hyper-networks [Ha et al. 2016] for the tuning of hyper-parameters, along the lines of [Brock et al. 2017]. The core idea is to have a side neural network sufficiently expressive to learn the (large-scale, matrix-valued) mapping from a given configuration of hyper-parameters to the weights of the model we wish to tune.
The paper gives a theoretical justification of its approach, and then describes several variants of its core algorithm which mix the training of the hyper-networks together with the optimization of the hyper-parameters themselves. Finally, experiments based on MNIST illustrate the properties of the proposed approach.
While the core idea may appear as appealing, the paper suffers from several flaws (as further detailed afterwards):
-Insufficient related work
-Correctness/rigor of Theorem 2.1
-Clarity of the paper (e.g., Sec. 2.4)
-Experiments look somewhat artificial
-How scalable is the proposed approach in the perspective of tuning models way larger/more complex than those treated in the experiments?
*Detailed comments*
-"...and training the model to completion." and "This is wasteful, since it trains the model from scratch each time..." (and similar statement in Sec. 2.1): Those statements are quite debatable. There are lines of work, e.g., in Bayesian optimization, to model early stopping/learning curves (e.g., Domhan2014, Klein2017 and references therein) and where training procedures are explicitly resumed (e.g., Swersky2014, Li2016). The paper should reformulate its statements in the light of this literature.
-"Uncertainty could conceivably be incorporated into the hypernet...". This seems indeed an important point, but it does not appear as clear how to proceed (e.g., uncertainty on w_phi(lambda) which later needs to propagated to L_val); could the authors perhaps further elaborate?
-I am concerned about the rigor/correctness of Theorem 2.1; for instance, how is the continuity of the best-response exploited? Also, throughout the paper, the argmin is defined as if it was a singleton while in practice it is rather a set-valued mapping (except if there is a unique minimizer for L_train(., lambda), which is unlikely to be the case given the nature of the considered neural-net model). In the same vein, Jensen's inequality states that Expectation[g(X)] >= g(Expectation[X]) for some convex function g and random variable X; how does it precisely translate into the paper's setting (convexity, which function g, etc.)?
-Specify in Alg. 1 that "hyperopt" refers to a generic hyper-parameter procedure.
-More details should be provided to better understand Sec. 2.4. At the moment, it is difficult to figure out (and potentially reproduce) the model which is proposed.
-The training procedure in Sec. 4.2 seems quite ad hoc; how sensitive was the overall performance with respect to the optimization strategy? For instance, in 4.2 and 4.3, different optimization parameters are chosen.
-typo: "weight decay is applied the..." --> "weight decay is applied to the..."
-"a standard Bayesian optimization implementation from sklearn": Could more details be provided? (there does not seem to be implementation there http://scikit-learn.org/stable/model_selection.html to the best of my knowledge)
-The experimental set up looks a bit far-fetched and unrealistic: first scalar, than diagonal and finally matrix-weighted regularization schemes. While the first two may be used in practice, the third scheme is not used in practice to the best of my knowledge.
-typo: "fit a hypernet same dataset." --> "fit a hypernet on the same dataset."
-(Franceschi2017) could be added to the related work section.
*References*
(Domhan2014) Domhan, T.; Springenberg, T. & Hutter, F. Extrapolating learning curves of deep neural networks ICML 2014 AutoML Workshop, 2014
(Franceschi2017) Franceschi, L.; Donini, M.; Frasconi, P. & Pontil, M. Forward and Reverse Gradient-Based Hyperparameter Optimization preprint arXiv:1703.01785, 2017
(Klein2017) Klein, A.; Falkner, S.; Springenberg, J. T. & Hutter, F. Learning curve prediction with Bayesian neural networks International Conference on Learning Representations (ICLR), 2017, 17
(Li2016) Li, L.; Jamieson, K.; DeSalvo, G.; Rostamizadeh, A. & Talwalkar, A. Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization preprint arXiv:1603.06560, 2016
(Swersky2014) Swersky, K.; Snoek, J. & Adams, R. P. Freeze-Thaw Bayesian Optimization preprint arXiv:1406.3896, 2014
*********
Update post rebuttal
*********
I acknowledge the fact that I read the rebuttal of the authors, whom I thank for their detailed answers.
My minor concerns have been clarified. Regarding the correctness of the proof, I am still unsure about the applicability of Jensen inequality; provided it is true, then it is important to see that the results seem to hold only for particular hyperparameters, namely regularization parameters (as explained in the new updated proof). This limitation should be exposed transparently upfront in the paper/abstract.
Together with the new experiments and comparisons, I have therefore updated my rating from 5 to 6.


Published as a conference paper at ICLR 2018 GENERATING WIKIPEDIA BY SUMMARIZING LONG SEQUENCES
We show that generating English Wikipedia articles can be approached as a multidocument summarization of source documents. We use extractive summarization to coarsely identify salient information and a neural abstractive model to generate the article. For the abstractive model, we introduce a decoder-only architecture that can scalably attend to very long sequences, much longer than typical encoderdecoder architectures used in sequence transduction. We show that this model can generate fluent, coherent multi-sentence paragraphs and even whole Wikipedia articles. When given reference documents, we show it can extract relevant factual information as reflected in perplexity, ROUGE scores and human evaluations.

The main significance of this paper is to propose the task of generating the lead section of Wikipedia articles by viewing it as a multi-document summarization problem. Linked articles as well as the results of an external web search query are used as input documents, from which the Wikipedia lead section must be generated. Further preprocessing of the input articles is required, using simple heuristics to extract the most relevant sections to feed to a neural abstractive summarizer. A number of variants of attention mechanisms are compared, including the transofer-decoder, and a variant with memory-compressed attention in order to handle longer sequences. The outputs are evaluated by ROUGE-L and test perplexity. There is also a A-B testing setup by human evaluators to show that ROUGE-L rankings correspond to human preferences of systems, at least for large ROUGE differences.
This paper is quite original and clearly written. The main strength is in the task setup with the dataset and the proposed input sources for generating Wikipedia articles. The main weakness is that I would have liked to see more analysis and comparisons in the evaluation.
Evaluation:
Currently, only neural abstractive methods are compared. I would have liked to see the ROUGE performance of some current unsupervised multi-document extractive summarization methods, as well as some simple multi-document selection algorithms such as SumBasic. Do redundancy cues which work for multi-document news summarization still work for this task?
Extractiveness analysis:
I would also have liked to see more analysis of how extractive the Wikipedia articles actually are, as well as how extractive the system outputs are. Does higher extractiveness correspond to higher or lower system ROUGE scores? This would help us understand the difficulty of the problem, and how much abstractive methods could be expected to help.
A further analysis which would be nice to do (though I have less clear ideas how to do it), would be to have some way to figure out which article types or which section types are amenable to this setup, and which are not.
I have some concern that extraction could do very well if you happen to find a related article in another website which contains encyclopedia-like or definition-like entries (e.g., Baidu, Wiktionary) which is not caught by clone detection. In this case, the problem could become less interesting, as no real analysis is required to do well here.
Overall, I quite like this line of work, but I think the paper would be a lot stronger and more convincing with some additional work.
----
After reading the authors' response and the updated submission, I am satisfied that my concerns above have been adequately addressed in the new version of the paper. This is a very nice contribution.


We propose Efficient Neural Architecture Search (ENAS), a faster and less expensive approach to automated model design than previous methods. In ENAS, a controller learns to discover neural network architectures by searching for an optimal path within a larger model. The controller is trained with policy gradient to select a path that maximizes the expected reward on the validation set. Meanwhile the model corresponding to the selected path is trained to minimize the cross entropy loss. On the Penn Treebank dataset, ENAS can discover a novel architecture thats achieves a test perplexity of 57.8, which is state-of-the-art among automatic model design methods on Penn Treebank. On the CIFAR-10 dataset, ENAS can design novel architectures that achieve a test error of 2.89%, close to the 2.65% achieved by standard NAS . Most importantly, our experiments show that ENAS is more than 10x faster and 100x less resource-demanding than NAS.

Summary:
The paper presents a method for learning certain aspects of a neural network architecture, specifically the number of output maps in certain connections and the existence of skip connections. The method is relatively efficient, since it searches in a space of similar architectures, and uses weights sharing between the tested models to avoid optimization of each model from scratch. Results are presented for image classification on Cifar 10 and for language modeling.
Page 3: “for each channel, we only predict C/S binary masks”   -> this seems to be a mistake. Probably “for each operation, we only predict C/S binary masks” is the right wording
Page 4: Stabilizing Stochastic Skip Connections: it seems that the suggested configuration does not enable an identity path, which was found very beneficial in (He. et al., 2016). Identity path does not exist since layers are concatenated and go through 1*1 conv, which does not enable plain identity unless learned by the 1*1 conv.
Page 5:
-	The last paragraph in section 4.2 is not clear to me. What does a compilation failure mean in this context and why does it occur? And: if each layer is connected to all its previous layers by skip connections, what remains to be learned w.r.t the model structure? Isn’t the pattern of skip connection the thing we would like to learn?
-	Some  details of the policy LSTM network are also not clear to me:
o	How is the integer mask (output of the B channel steps) encoded? Using 1-hot encoding over 2^{C/S} output neurons? Or maybe C/S output neurons, used for sampling the C/S bits of the mask? this should be reported in some detail.
o	How is the mask converted to an input embedding for the next step? Is it by linear multiplication with a matrix? Something more complicated? And are there different matrices used/trained for each mask embedding (one for 1*1 conv, one for 3*3 conv, etc..)?
o	What is the motivation for using equation 5 for the sampling of skip connection flags? What is the motivation for averaging the winning anchors as the average embedding for the next stage (to let it ‘know’ who is connected to the previous?). Is anchor j also added to the average?
o	How is equation 5 normalized? That is: the probability is stated to be proportional to an exponent of an inner product, but it is not clear what the constant is and how sampling is done.
Page 6:
-	  Section 4.4: what is the fixed policy used for generating models in the stage of training the shared W parameters? (this is answered at page 7
Experiments:
-	The accuracy figures obtained are impressive, but I’m not convinced the ENAS learning is the important ingredient in obtaining them (rather than a very good baseline)
-	Specifically, in the Cifar -10 example it does not seem that the networks chooses the number of maps in a way which is diverse or different from layer to layer. Therefore we do not have any evidence that the LSTM controller has learnt any interesting rule regarding block type, or relation between block type width and layer index. All we see is that the model does not chose too many maps, thus avoid significant overfit. The relevant baseline here is a model with 64 or 96 maps on each block, each layer.Such a model is likely to do as well as the ENAS model, and can be obtained easily with slight parameter tuning of a single parameter.
-	 Similarly, I’m not convinced the  skip connection pattern found for Cifar-10 is superior to standard denseNet or Resnet pattern. The found configuration was not compared to these baselines. So again we do not have evidence showing the merit of keeping and tuning many parameters with the RINFORCE
-	The experiments with Penn Treebank are described with too less details: for example, what exactly is the task considered (in terms on input-output mapping), what is the dataset size, etc..
-	Also, for the Penn treebank experiments no baseline is given, so it is not possible to understand if the structure learning here is beneficial. Comparison of the results to an architecture with all skip connections, and with a single skip connection per layer is required to estimate if useful structure is being learnt.
Overall:
-	Pro: the method gives high accuracy results
-	Cons:
o	It is not clear if the ENAS search is responsible to the results, or just the strong baseline. The advantage of ENAS over plain hyper parameter choosing was not sufficiently established.
o	The controller was not presented in a clear enough manner. Many of its details stay obscure.
o	The method does not seem to be general. It seems to be limited to choosing a specific set of parameters in very specific scenario (scenario which enable parameter sharing between model. The conditions for this to happen seems to be rather strict, and where not elaborated).
After revision:
The controller is now better presented.
However, the main points were not changed:
   - ENAS seems to be limited to a specific architecture and search space, in which probably the search is already exhausted. For example for the image processing network, it is determining the number of skip connections and structure of a single layer as a combination of several function types. We already know the answers to these search problems (denser skip connection pattern works better, more functions types  in a layer in parallel do better, the number of maps should be adjusted to the complexity and data size to avoid overfit). ENAS does not reveal a new surprising architectures, and it seems that instead of searching in the large space it suggests, one can just tune a 1-2 parameters  (for the image network, it is the number of maps in a layer).
  - Results comparing ENAS results to the simple baseline of just tuning 1-2 hyper parameters were not shown. I hence believe the strong empirical results of ENAS are a property of the search space (the architecture used) and not of the search algorithm.


Published as a conference paper at ICLR 2018 ACTIVATION MAXIMIZATION GENERATIVE ADVER- SARIAL NETS
Class labels have been empirically shown useful in improving the sample quality of generative adversarial nets (GANs). In this paper, we mathematically study the properties of the current variants of GANs that make use of class label information. With class aware gradient and cross-entropy decomposition, we reveal how class labels and associated losses influence GAN's training. Based on that, we propose Activation Maximization Generative Adversarial Networks (AM-GAN) as an advanced solution. Comprehensive experiments have been conducted to validate our analysis and evaluate the effectiveness of our solution, where AM-GAN outperforms other strong baselines and achieves state-of-the-art Inception Score (8.91) on CIFAR-10. In addition, we demonstrate that, with the Inception ImageNet classifier, Inception Score mainly tracks the diversity of the generator, and there is, however, no reliable evidence that it can reflect the true sample quality. We thus propose a new metric, called AM Score, to provide more accurate estimation on the sample quality. Our proposed model also outperforms the baseline methods in the new metric.

This paper is a thorough investigation of various “class aware” GAN architectures. It purposes a variety of modifications on existing approaches and additionally provides extensive analysis of the commonly used Inception Score evaluation metric.
The paper starts by introducing and analyzing two previous class aware GANs - a variant of the Improved GAN architecture used for semi-supervised results (named Label GAN in this work) and AC-GAN, which augments the standard discriminator with an auxiliary classifier to classify both real and generated samples as specific classes.
The paper then discusses the differences between these two approaches and analyzes the loss functions and their corresponding gradients. Label GAN’s loss encourages the generator to assign all probability mass cumulatively across the k-different label classes while the discriminator tries to assign all probability mass to the k+1th output corresponding to a “generated” class. The paper views the generators loss as a form of implicit class target loss.
This analysis motivates the paper’s proposed extension, called Activation Maximization. It corresponds to a variant of Label GAN where the generator is encouraged to maximize the probability of a specific class for every sample instead of just the cumulative probability assigned to label classes. The proposed approach performs strongly according to inception score on CIFAR-10 and includes additional experiments on Tiny Imagenet to further increase confidence in the results.
A discussion throughout the paper involves dealing with the issue of mode collapse - a problem plaguing standard GAN variants. In particular the paper discusses how variants of class conditioning effect this problem. The paper presents a useful experimental finding - dynamic labeling, where targets are assigned based on whatever the discriminator thinks is the most likely label, helps prevent mode collapse compared to the predefined assignment approach used in AC-GAN / standard class conditioning.
I am unclear how exactly predefined vs dynamic labeling is applied in the case of the Label GAN results in Table 1. The definition of dynamic labeling is specific to the generator as I interpreted it. But Label GAN includes no class specific loss for the generator. I assume it refers to the form of generator - whether it is class conditional or not - even though it would have no explicit loss for the class conditional version. It would be nice if the authors could clarify the details of this setup.
The paper additionally performs a thorough investigation of the inception score and proposes a new metric the AM score. Through analysis of the behavior of the inception score has been lacking so this is an important contribution as well.
As a reader, I found this paper to be thorough, honest, and thoughtful. It is a strong contribution to the “class aware” GAN literature.


Recent advances in adversarial Deep Learning (DL) have opened up a new and largely unexplored surface for malicious attacks jeopardizing the integrity of autonomous DL systems. We introduce a novel automated countermeasure called Parallel Checkpointing Learners (PCL) to thwart the potential adversarial attacks and significantly improve the reliability (safety) of a victim DL model. The proposed PCL methodology is unsupervised, meaning that no adversarial sample is leveraged to build/train parallel checkpointing learners. We formalize the goal of preventing adversarial attacks as an optimization problem to minimize the rarely observed regions in the latent feature space spanned by a DL network. To solve the aforementioned minimization problem, a set of complementary but disjoint checkpointing modules are trained and leveraged to validate the victim model execution in parallel. Each checkpointing learner explicitly characterizes the geometry of the input data and the corresponding high-level data abstractions within a particular DL layer. As such, the adversary is required to simultaneously deceive all the defender modules in order to succeed. We extensively evaluate the performance of the PCL methodology against the state-of-the-art attack scenarios, including Fast-Gradient-Sign (FGS), Jacobian Saliency Map Attack (JSMA), Deepfool, and Carlini&WagnerL2. Extensive proof-of-concept evaluations for analyzing various data collections including MNIST, CIFAR10, and ImageNet corroborate the effectiveness of our proposed defense mechanism against adversarial samples.

Summary:
 The paper presents an unsupervised method for detecting adversarial examples of neural networks. The method includes two independent components: an ‘input defender’ which tried to inspect the input, and a ‘latent defender’ trying to inspect a hidden representation. Both are based on the claim that adversarial examples lie outside a certain sub-space occupied by the natural image examples, and modeling this sub-space hence enables their detection. The input defender is based on sparse coding, and the latent defender on modeling the latent activity as a mixture of Gaussians. Experiments are presented on MInst, Cifar10, and ImageNet.

-	Introduction: The motivation for detecting adversarial examples is not stated clearly enough. How can such examples be used by a malicious agent to cause damage to a system? Sketching some such scenarios would help the reader understand why the issue is practically important. I was not convinced it is.
Page 4:
-	Step 3 of the algorithm is not clear:
o	How exactly does HDDA model the data (formally) and how does it estimate the parameters? In the current version, the paper does not explain the HDDA formalism and learning algorithm, which is a main building block in the proposed system (as it provides the density score used for adversarial examples detection). Hence the paper cannot be read as a standalone document. I went on to read the relevant HDDA paper, but it is also not clear which of the model variants presented there is used in this paper.
o	What is the relation between the model learned at stage 2 (the centers c^i) and the model learnt by HDDA? Are they completely different models? Or are the C^I used when learning the HDDA model (and how)?
If these are separate models, how are they used in conjunction to give a final density score? If I understand correctly, only the HDDA model is used to get the final score, and the C^i are only used to make the \phy(x) representation more class-seperable. Is that right?
-	Figure 4, b and c: it is not clear what the (x,y,z) measurements plotted in these 3D drawings are (what are the axis).
Page 5:
-	Section 2: the risk analysis is done in a standard Bayesian way and leads to a ratio of PDFs in equation 5. However, this form is not appropriate for the case presented at this paper, since the method presented only models one of these PDFs (Specifically p(x | W1)  - there is not generative model of p(x|W2)).
-	The authors claim in the last sentence of the section that p(x|W2) is equivalent to 1-p(x|W1), but this is not true: these are two continuous densities, they do not sum to 1, and a model of p(x|W2) is not available (as far as I understand the method)
Page 6:
-	How is equation 7) optimized?
-	Which patchs are extracted from images, for training and at inference time? Are these patchs a dense coverage of the image? Sparsely sampled? Densely sampled with overlaps?
-	Its not clear enough what exactly is the ‘PSNR’ value which is used for the adversarial example detection, and what exactly is ‘profile the PSNR of legitimate samples within each class’. A formal definition of PSNR and’profiling’ is missing (does profiling simply mean finding a threshold for filtering?)
Page 7:
-	Figure 7 is not very informative. Given the ROC curves in figure 8  and table 1 it is redundant.
Page 8:
-	The results in general indicate that the method is much better than chance, but it is not clear if it is practical, because the false alarm rates for high detection are quite high. For example on ImageNet, 14.2% of the innocent images are mistakenly rejected as malicious to get 90% detection rate. I do not think this working point is useful for a real application
-	Given the high flares alarm rate, it is surprising that experiments with multiple checkpoints are not presented (specifically as this case of multiple checkpoints is discussed explicitly in previous sections of the paper).  Experiments with multiple checkpoints are clear required to complete the picture regarding the empirical performance of this method
-	The experiments show that essentially, the latent defenders are stronger than the input defender in most cases. However, an ablation study of the latent defender is missing: Specifcially, it is not clear a) how much does stage 2 (model refinement with clusters)  contribute to the accuracy (how does the model do without it? And 3) how important is the HDDA and the specific variant used (which is not clear) important: is it important to model the Gaussians using a sub-space? Of which dimension?
Overall:
Pros:
-	 A nice idea with some novelty,  based on a non-trivial observation
-	The experimental results how the idea holds some promise
Cons
-	The method is not presented clearly enough: the main component modeling the network activity is not explained (the HDDA module used)
-	The results presented show that the method is probably not suitable for a practical application yet (high false alarm rate for good detection rate)
-	Experimental results are partial: results are not presented for multiple defenders, no ablation experiments
After revision:
Some of my comments were addressed, and some were not.
Specifically, results were presented for multiple defenders and some ablation experiments were highlihgted
Things not addressed:
 - The risk analysis is still not relevant. The authors removed a clearly flawed sentence, but the analysis still assumes that two densities (of 'good' and 'bad' examples) are modeled, while in the work presented only one of them is. Hence this analysis does not add anything to the paper-  it states a general case which does not fit the current scenario and its relation to the work is not clear. It would have been better to omit it and use the space to describe HDDA and the specific variant used in this work, as this is the main tool doing the distinction.
I believe the paper should be accepted.


Published as a conference paper at ICLR 2018 NEURAL SPEED READING VIA SKIM-RNN
Inspired by the principles of speed reading, we introduce Skim-RNN, a recurrent neural network (RNN) that dynamically decides to update only a small fraction of the hidden state for relatively unimportant input tokens. Skim-RNN gives computational advantage over an RNN that always updates the entire hidden state. Skim-RNN uses the same input and output interfaces as a standard RNN and can be easily used instead of RNNs in existing models. In our experiments, we show that Skim-RNN can achieve significantly reduced computational cost without losing accuracy compared to standard RNNs across five different natural language tasks. In addition, we demonstrate that the trade-off between accuracy and speed of Skim-RNN can be dynamically controlled during inference time in a stable manner. Our analysis also shows that Skim-RNN running on a single CPU offers lower latency compared to standard RNNs on GPUs.

Summary: The paper proposes a learnable skimming mechanism for RNN. The model decides whether to send the word to a larger heavy-weight RNN or a light-weight RNN. The heavy-weight and the light-weight RNN each controls a portion of the hidden state. The paper finds that with the proposed skimming method, they achieve a significant reduction in terms of FLOPS. Although it doesn’t contribute to much speedup on modern GPU hardware, there is a good speedup on CPU, and it is more power efficient.
Contribution:
- The paper proposes to use a small RNN to read unimportant text. Unlike (Yu et al., 2017), which skips the text, here the model decides between small and large RNN.
Pros:
- Models that dynamically decide the amount of computation make intuitive sense and are of general interests.
- The paper presents solid experimentation on various text classification and question answering datasets.
- The proposed method has shown reasonable reduction in FLOPS and CPU speedup with no significant accuracy degradation (increase in accuracy in some tasks).
- The paper is well written, and the presentation is good.
Cons:
- Each model component is not novel. The authors propose to use Gumbel softmax, but does compare other gradient estimators. It would be good to use REINFORCE to do a fair comparison with (Yu et al., 2017 ) to see the benefit of using small RNN.
- The authors report that training from scratch results in unstable skim rate, while Half pretrain seems to always work better than fully pretrained ones. This makes the success of training a bit adhoc, as one need to actively tune the number of pretraining steps.
- Although there is difference from (Yu et al., 2017), the contribution of this paper is still incremental.
Questions:
- Although it is out of the scope for this paper to achieve GPU level speedup, I am curious to know some numbers on GPU speedup.
- One recommended task would probably be text summarization, in which the attended text can contribute to the output of the summary.
Conclusion:
- Based on the comments above, I recommend Accept


Model-free deep reinforcement learning algorithms are able to successfully solve a wide range of continuous control tasks, but typically require many on-policy samples to achieve good performance. Model-based RL algorithms are sampleefficient on the other hand, while learning accurate global models of complex dynamic environments has turned out to be tricky in practice, which leads to the unsatisfactory performance of the learned policies. In this work, we combine the sample-efficiency of model-based algorithms and the accuracy of model-free algorithms. We leverage multi-step neural network based predictive models by embedding real trajectories into imaginary rollouts of the model, and use the imaginary cumulative rewards as control variates for model-free algorithms. In this way, we achieved the strengths of both sides and derived an estimator which is not only sample-efficient, but also unbiased and of very low variance. We present our evaluation on the MuJoCo and OpenAI Gym benchmarks.

This paper presents a model-based approach to variance reduction in policy gradient methods.  The basic idea is to use a multi-step dynamics model as a "baseline" (more properly a control variate, as the terminology in the paper uses, but I think baselines are more familiar to the RL community) to reduce the variance of a policy gradient estimator, while remaining unbiased.  The authors also discuss how to best learn the type of multi-step dynamics that are well-suited to this problem (essentially, using off-policy data via importance weighting), and they demonstrate the effectiveness of the approach on four continuous control tasks.
This paper presents a nice idea, and I'm sure that with some polish it will become a very nice conference submission. But right now (at least as of the version I'm reviewing), the paper reads as being half-finished.  Several terms are introduced without being properly defined, and one of the key formalisms presented in the paper (the idea of "embedding" an "imaginary trajectory" remains completely opaque to me.  Further, the paper seems to simply leave out some portions: the introduction claims that one of the contributions is "we show that techniques such as latent space trajectory embedding and dynamic unfolding can significantly boost the performance of the model based control variates," but I see literally no section that hints at anything like this (no mention of "dynamic unfolding" or "latent space trajectory embedding" ever occurs later in the paper).
In a bit more detail, the key idea of the paper, at least to the extent that I understood it, was that the authors are able to introduce a model-based variance-reduction baseline into the policy gradient term.  But because (unlike traditional baselines) introducing it alone would affect the actual estimate, they actually just add and subtract this term, and separate out the two terms in the policy gradient: the new policy gradient like term will be much smaller, and the other term can be computed with less variance using model-based methods and the reparameterization trick.  But beyond this, and despite fairly reasonable familiarity with the subject, I simply don't understand other elements that the paper is talking about.
The paper frequently refers to "embedding" "imaginary trajectories" into the dynamics model, and I still have no idea what this is actually referring to (the definition at the start of section 4 is completely opaque to me).  I also don't really understand why something like this would be needed given the understanding above, but it's likely I'm just missing something here.  But I also feel that in this case, it borders on being an issue with the paper itself, as I think this idea needs to be described much more clearly if it is central to the underlying paper.
Finally, although I do think the extent of the algorithm that I could follow is interesting, the second issue with the paper is that the results are fairly weak as they stand currently.  The improvement over TRPO is quite minor in most of the evaluated domains (other than possibly in the swimmer task), even with substantial added complexity to the approach.  And the experiments are described with very little detail or discussion about the experimental setup.
Nor are either of these issues simply due to space constraints: the paper is 2 pages under the soft ICLR limit, with no appendix.  Not that there is anything wrong with short papers, but in this case both the clarity of presentation and details are lacking.  My honest impression is simply that this is still work in progress and that the write up was done rather hastily.  I think it will eventually become a good paper, but it is not ready yet.


The ability of an agent to discover its own learning objectives has long been considered a key ingredient for artificial general intelligence. Breakthroughs in autonomous decision making and reinforcement learning have primarily been in domains where the agent's goal is outlined and clear: such as playing a game to win, or driving safely. Several studies have demonstrated that learning extramural sub-tasks and auxiliary predictions can improve (1) single human-specified task learning, (2) transfer of learning, (3) and the agent's learned representation of the world. In all these examples, the agent was instructed what to learn about. We investigate a framework for discovery: curating a large collection of predictions, which are used to construct the agent's representation of the world. Specifically, our system maintains a large collection of predictions, continually pruning and replacing predictions. We highlight the importance of considering stability rather than convergence for such a system, and develop an adaptive, regularized algorithm towards that aim. We provide several experiments in computational micro-worlds demonstrating that this simple approach can be effective for discovering useful predictions autonomously.

I really enjoyed reading this paper and stopped a few time to write down new ideas it brought up. Well written and very clear, but somewhat lacking in the experimental or theoretical results.
The formulation of AdaGain is very reminiscent of the SGA algorithm in Kushner & Yin (2003), and more generally gradient descent optimization of the learning rate is not new. The authors argue for the focus on stability over convergence, which is an interesting focus, but still I found the lack of connection with related work in this section a strange.
How would a simple RNN work for the experimental problems? The first experiment demonstrates that the regularization is using fewer features than without, which one could argue does not need to be compared with other methods to be useful. Especially when combined with Figure 5, I am convinced the regularization is doing a good job of pruning the least important GVFs. However, the results in Figure 3 have no context for us to judge the results within. Is this effective or terrible? Fast or slow? It is really hard to judge from these results. We can say that more GVFs are better, and that the compositional GVFs add to the ability to lower RMSE. But I do not think this is enough to really judge the method beyond a preliminary "looks promising".
The compositional GVFs also left me wondering: What keeps a GVF from being pruned that is depended upon by a compositional GVF? This was not obvious to me.
Also, I think comparing GVFs and AdaGain-R with an RNN approach highlights the more general question. Is it generally true that GVFs setup like this can learn to represent any value function that an RNN could have? There's an obvious benefit to this approach which is that you do not need BPTT, fantastic, but why not highlight this? The network being used is essentially a recurrent neural net, the authors restrict it and train it, not with backprop, but with TD, which is very interesting. But, I think there is not quite enough here.
Pros:
Well written, very interesting approach and ideas
Conceptually simple, should be easy to reproduce results
Cons:
AdaGain never gets analyzed or evaluated except for the evaluations of AdaGain-R.
No experimental context, we need a non-trivial baseline to compare with


LOSS-AWARE WEIGHT QUANTIZATION OF DEEP NET- WORKS
The huge size of deep networks hinders their use in small computing devices. In this paper, we consider compressing the network by weight quantization. We extend a recently proposed loss-aware weight binarization scheme to ternarization, with possibly different scaling parameters for the positive and negative weights, and m-bit (where m > 2) quantization. Experiments on feedforward and recurrent neural networks show that the proposed scheme outperforms state-of-the-art weight quantization algorithms, and is as accurate (or even more accurate) than the full-precision network.

This paper proposes a new method to train DNNs with quantized weights, by including the quantization as a constraint in a proximal quasi-Newton algorithm, which simultaneously learns a scaling for the quantized values (possibly different for positive and negative weights).
The paper is very clearly written, and the proposal is very well placed in the context of previous methods for the same purpose. The experiments are very clearly presented and solidly designed.
In fact, the paper is a somewhat simple extension of the method proposed by Hou, Yao, and Kwok (2017), which is where the novelty resides. Consequently, there is not a great degree of novelty in terms of the proposed method, and the results are only slightly better than those of previous methods.
Finally, in terms of analysis of the algorithm, the authors simply invoke a theorem from Hou, Yao, and Kwok (2017), which claims convergence of the proposed algorithm. However, what is shown in that paper is that the sequence of loss function values converges, which does not imply that the sequence of weight estimates also converges, because of the presence of a non-convex constraint ($b_j^t \in Q^{n_l}$). This may not be relevant for the practical results, but to be accurate, it can't be simply stated that the algorithm converges, without a more careful analysis.


Published as a conference paper at ICLR 2018 LEARNING FROM BETWEEN-CLASS EXAMPLES FOR DEEP SOUND RECOGNITION
Deep learning methods have achieved high performance in sound recognition tasks. Deciding how to feed the training data is important for further performance improvement. We propose a novel learning method for deep sound recognition: Between-Class learning (BC learning). Our strategy is to learn a discriminative feature space by recognizing the between-class sounds as between-class sounds. We generate between-class sounds by mixing two sounds belonging to different classes with a random ratio. We then input the mixed sound to the model and train the model to output the mixing ratio. The advantages of BC learning are not limited only to the increase in variation of the training data; BC learning leads to an enlargement of Fisher's criterion in the feature space and a regularization of the positional relationship among the feature distributions of the classes. The experimental results show that BC learning improves the performance on various sound recognition networks, datasets, and data augmentation schemes, in which BC learning proves to be always beneficial. Furthermore, we construct a new deep sound recognition network (EnvNet-v2) and train it with BC learning. As a result, we achieved a performance surpasses the human level 1 .

This manuscript proposes a method to improve the performance of a generic learning method by generating "in between class" (BC) training samples. The manuscript motivates the necessity of such technique and presents the basic intuition. The authors show how the so-called BC learning helps training different deep architectures for the sound recognition task.
My first remark regards the presentation of the technique. The authors argue that it is not a data augmentation technique, but rather a learning method. I strongly disagree with this statement, not only because the technique deals exactly with augmenting data, but also because it can be used in combination to any learning method (including non-deep learning methodologies). Naturally, the literature review deals with data augmentation technique, which supports my point of view.
In this regard, I would have expected comparison with other state-of-the-art data augmentation techniques. The usefulness of the BC technique is proven to a certain extent (see paragraph below) but there is not comparison with state-of-the-art. In other words, the authors do not compare the proposed method with other methods doing data augmentation. This is crucial to understand the advantages of the BC technique.
There is a more fundamental question for which I was not able to find an explicit answer in the manuscript. Intuitively, the diagram shown in Figure 4 works well for 3 classes in dimension 2. If we add another class, no matter how do we define the borders, there will be one pair of classes for which the transition from one to another will pass through the region of a third class. The situation worsens with more classes. However, this can be solved by adding one dimension, 4 classes and 3 dimensions seems something feasible. One can easily understand that if there is one more class than the number of dimensions, the assumption should be feasible, but beyond it starts to get problematic. This discussion does not appear at all in the manuscript and it would be an important limitation of the method, specially when dealing with large-scale data sets.
Overall I believe the paper is not mature enough for publication.
Some minor comments:
- 2.1: We introduce --> We discussion
- Pieczak 2015a did not propose the extraction of MFCC.
- the x_i and t_i of section 3.2.2 should not be denoted with the same letters as in 3.2.1.
- The correspondence with a semantic feature space is too pretentious, specially since no experiment in this direction is shown.
- I understand that there is no mixing in the test phase, perhaps it would be useful to recall it.


Workshop track -ICLR 2018 ACCELERATING NEURAL ARCHITECTURE SEARCH US- ING PERFORMANCE PREDICTION
Methods for neural network hyperparameter optimization and architecture search are computationally expensive due to the need to train a large number of model configurations. In this paper, we show that simple regression models can predict the final performance of partially trained model configurations using features based on network architectures, hyperparameters, and time-series validation performance data. We empirically show that our performance prediction models are much more accurate than prominent Bayesian counterparts, are simpler to implement, and are faster to train. Our models can predict final performance in both visual classification and language modeling domains, are effective for predicting performance of drastically varying model architectures, and can even generalize between model classes. Using these prediction models, we also implement an early stopping method for hyperparameter optimization and architecture search, which obtains a speedup of a factor up to 6x in both hyperparameter optimization and architecture search. Finally, we empirically show that our early stopping method can be seamlessly incorporated into both reinforcement learning-based architecture selection algorithms and bandit based search methods. Through extensive experimentation, we empirically show our performance prediction models and early stopping algorithm are state-of-the-art in terms of prediction accuracy and speedup achieved while still identifying the optimal model configurations.

This paper shows a simple method for predicting the performance that neural networks will achieve with a given architecture, hyperparameters, and based on an initial part of the learning curve.
The method assumes that it is possible to first execute 100 evaluations up to the total number of epochs.
From these 100 evaluations (with different hyperparameters / architectures), the final performance y_T is collected. Then, based on an arbitrary prefix of epochs y_{1:t}, a model can be learned to predict y_T.
There are T different models, one for each prefix y_{1:t} of length t. The type of model used is counterintuitive for me; why use a SVR model? Especially since uncertainty estimates are required, a Gaussian process would be the obvious choice.
The predictions in Section 3 appear to be very good, and it is nice to see the ablation study.
Section 4 fails to mention that its use of performance prediction for early stopping follows exactly that of Domhan et al (2015) and that this is not a contribution of this paper; this feels a bit disingenious and should be fixed.
The section should also emphasize that the models discussed in this paper are only applicable for early stopping in cases where the function evaluation budget N is much larger than 100.
The emphasis on the computational demand of 1-3 minutes for LCE seems like a red herring: MetaQNN trained 2700 networks in 100 GPU days, i.e., about 1 network per GPU hour. It trained 20 epochs for the studied case of CIFAR, so 1-3 minutes per epoch on the CPU can be implemented with zero overhead while the network is training on the GPU. Therefore, the following sentence seems sensational without substance: "Therefore, on a full meta-modeling experiment involving thousands of neural network configurations, our method could be faster by several orders of magnitude as compared to LCE based on current implementations."
The experiment on fast Hyperband is very nice at first glance, but the longer I think about it the more questions I have. During the rebuttal I would ask the authors to extend f-Hyperband all the way to the right in Figure 6 (left) and particularly in Figure 6 (right). Especially in Figure 6 (right), the original Hyperband algorithm ends up higher than f-Hyperband. The question this leaves open is whether f-Hyperband would reach the same performance when continued or not.
I would also request the paper not to casually mention the 7x speedup that can be found in the appendix, without quantifying this. This is only possible for a large number of 40 Hyperband iterations, and in the interesting cases of the first few iterations speedups are very small. Also, do the simulated speedup results in the appendix account for potentially stopping a new best configuration, or do they simply count how much computational time is saved, without looking at performance? The latter would of course be extremely misleading and should be fixed. I am looking forward to a clarification in the rebuttal period.
For relating properly to the literatue, the experiment for speeding up Hyperband should also mention previous methods for speeding up Hyperband by a model (I only know one by the authors' reference Klein et al (2017)).
Overall, this paper appears very interesting. The proposed technique has some limitations, but in some settings it seems very useful. I am looking forward to the reply to my questions above; my final score will depend on these.
Typos / Details:
- The range of the coefficient of determination is from 0 to 1. Table 1 probably reports 100 * R^2? Please fix the description.
- I did not see Table 1 referenced in the text.
- Page 3: "more computationally and" -> "more computationally efficient and"
- Page 3: "for performing final" -> "for predicting final"
Points in favor of the paper:
- Simple method
- Good prediction results
- Useful possible applications identified
Points against the paper:
- Methodological advances are limited / unmotivated choice of model
- Limited applicability to settings where >> 100 configurations can be run fully
- Possibly inflated results reported for Hyperband experiment


We propose a novel way of reducing the number of parameters in the storagehungry fully connected layers of a neural network by using pre-defined sparsity, where the majority of connections are absent prior to starting training. Our results indicate that convolutional neural networks can operate without any loss of accuracy at less than 0.5% classification layer connection density, or less than 5% overall network connection density. We also investigate the effects of pre-defining the sparsity of networks with only fully connected layers. Based on our sparsifying technique, we introduce the 'scatter' metric to characterize the quality of a particular connection pattern. As proof of concept, we show results on CIFAR, MNIST and a new dataset on classifying Morse code symbols, which highlights some interesting trends and limits of sparse connection patterns.

This paper examines sparse connection patterns in upper layers of convolutional image classification networks.  Networks with very few connections in the upper layers are experimentally determined to perform almost as well as those with full connection masks.  Heuristics for distributing connections among windows/groups and a measure called "scatter" are introduced to construct the connectivity masks, and evaluated experimentally on CIFAR-10 and -100, MNIST and Morse code symbols.
While it seems clear in general that many of the connections are not needed and can be made sparse (Figures 1 and 2), I found many parts of this paper fairly confusing, both in how it achieves its objectives, as well as much of the notation and method descriptions.  I've described many of the points I was confused by in more detailed comments below.
Detailed comments and questions:
The distribution of connections in "windows" are first described to correspond to a sort of semi-random spatial downsampling, to get different views distributed over the full image.  But in the upper layers, the spatial extent can be very small compared to the image size, sometimes even 1x1 depending on the network downsampling structure.  So are do the "windows" correspond to spatial windows, and if so, how?  Or are they different (maybe arbitrary) groupings over the feature maps?
Also a bit confusing is the notation "conv2", "conv3", etc.  These names usually indicate the name of a single layer within the network (conv2 for the second convolutional layer or series of layers in the second spatial size after downsampling, for example).  But here it seems just to indicate the number of "CL" layers: 2.  And p.1 says that the "CL" layers are those often referred to as "FC" layers, not "conv" (though they may be convolutionally applied with spatial 1x1 kernels).
The heuristic for spacing connections in windows across the spatial extent of an image makes intuitive sense, but I'm not convinced this will work well in all situations, and may even be sub-optimal for the examined datasets.  For example, to distinguish MNIST 1 vs 7 vs 9, it is most important to see the top-left:  whether it is empty, has a horizontal line, or a loop.  So some regions are more important than others, and the top half may be more important than an equally spaced global view.  So the description of how to space connections between windows makes some intuitive sense, but I'm unclear on whether other more general connections might be even better, including some that might not be as easily analyzed with the "scatter" metric described.
Another broader question I have is in the distinction between lower and upper layers (those referred to as "feature extracting" and "classification" in this paper).  It's not clear to me that there is a crisply defined difference here (though some layers may tend to do more of one or the other function, such as we might interpret).  So it seems that expanding the investigation to include all layers, or at least more layers, would be good:  It might be that more of the "classification" function is pushed down to lower layers, as the upper layers are reduced in size.  How would they respond to similar reductions?
I'm also unsure why on p.6 MNIST uses 2d windows, while CIFAR uses 3d --- The paper mentions the extra dimension is for features, but MNIST would have a features dimension as well at this stage, I think?  I'm also unsure whether the windows are over spatial extent only, or over features.


Published as a conference paper at ICLR 2018 EMERGENT COMMUNICATION THROUGH NEGOTIATION
Multi-agent reinforcement learning offers a way to study how communication could emerge in communities of agents needing to solve specific problems. In this paper, we study the emergence of communication in the negotiation environment, a semi-cooperative model of agent interaction. We introduce two communication protocols -one grounded in the semantics of the game, and one which is a priori ungrounded and is a form of cheap talk. We show that self-interested agents can use the pre-grounded communication channel to negotiate fairly, but are unable to effectively use the ungrounded channel. However, prosocial agents do learn to use cheap talk to find an optimal negotiating strategy, suggesting that cooperation is necessary for language to emerge. We also study communication behaviour in a setting where one agent interacts with agents in a community with different levels of prosociality and show how agent identifiability can aid negotiation.

The authors describe a variant of the negotiation game in which agents of different type, selfish or prosocial, and with different preferences. The central feature is the consideration of a secondary communication (linguistic) channel for the purpose of cheap talk, i.e. talk whose semantics are not laid out a priori.
The essential findings include that prosociality is a prerequisite for effective communication (i.e. formation of meaningful communication on the linguistic channel), and furthermore, that the secondary channel helps improve the negotiation outcomes.
The paper is well-structured and incrementally introduces the added features and includes staged evaluations for the individual additions, starting with the differentiation of agent characteristics, explored with combination of linguistic and proposal channel. Finally, agent societies are represented by injecting individuals' ID into the input representation.
The positive:
- The authors attack the challenging task of given agents a means to develop communication patterns without apriori knowledge.
- The paper presents the problem in a well-structured manner and sufficient clarity to retrace the essential contribution (minor points for improvement).
- The quality of the text is very high and error-free.
- The background and results are well-contextualised with relevant related work.
The problematic:
- By the very nature of the employed learning mechanisms, the provided solution provides little insight into what the emerging communication is really about. In my view, the lack of interpretable semantics hardly warrants a reference to 'cheap talk'. As such the expectations set by the well-developed introduction and background sections are moderated over the course of the paper.
- The goal of providing agents with richer communicative ability without providing prior grounding is challenging, since agents need to learn about communication partners at runtime. But it appears as of the main contribution of the paper can be reduced to the decomposition of the learnable feature space into two communication channels. The implicit relationship of linguistic channel on proposal channel input based on the time information (Page 4, top) provides agents with extended inputs, thus enabling a more nuanced learning based on the relationship of proposal and linguistic channel. As such the well-defined semantics of the proposal channel effectively act as the grounding for the linguistic channel. This, then, could have been equally achieved by providing agents with a richer input structure mediated by a single channel. From this perspective, the solution offers limited surprises. The improvement of accuracy in the context of agent societies based on provided ID follows the same pattern of extending the input features.
- One of the motivating factors of using cheap talk is the exploitation of lying on the part of the agents. However, apart from this initial statement, this feature is not explicitly picked up. In combination with the previous point, the necessity/value of the additional communication channel is unclear.
Concrete suggestions for improvement:
- Providing exemplified communication traces would help the reader appreciate the complexity of the problem addressed by the paper.
- Figure 3 is really hard to read/interpret. The same applies to Figure 4 (although less critical in this case).
- Input parameters could have been made explicit in order to facilitate a more comprehensive understanding of technicalities (e.g. in appendix).
- Emergent communication is effectively unidirectional, with one agent as listener. Have you observed other outcomes in your evaluation?
In summary, the paper presents an interesting approach to combine unsupervised learning with multiple communication channels to improve learning of preferences in a well-established negotiation game. The problem is addressed systematically and well-presented, but can leave the reader with the impression that the secondary channel, apart from decomposing the model, does not provide conceptual benefit over introducing a richer feature space that can be exploited by the learning mechanisms. Combined with the lack of specific cheap talk features, the use of actual cheap talk is rather abstract. Those aspects warrant justification.


Published as a conference paper at ICLR 2018 FIDELITY-WEIGHTED LEARNING
Training deep neural networks requires many training samples, but in practice training labels are expensive to obtain and may be of varying quality, as some may be from trusted expert labelers while others might be from heuristics or other sources of weak supervision such as crowd-sourcing. This creates a fundamental qualityversus-quantity trade-off in the learning process. Do we learn from the small amount of high-quality data or the potentially large amount of weakly-labeled data? We argue that if the learner could somehow know and take the label-quality into account when learning the data representation, we could get the best of both worlds. To this end, we propose "fidelity-weighted learning" (FWL), a semi-supervised studentteacher approach for training deep neural networks using weakly-labeled data. FWL modulates the parameter updates to a student network (trained on the task we care about) on a per-sample basis according to the posterior confidence of its label-quality estimated by a teacher (who has access to the high-quality labels). Both student and teacher are learned from the data. We evaluate FWL on two tasks in information retrieval and natural language processing where we outperform state-of-the-art alternative semi-supervised methods, indicating that our approach makes better use of strong and weak labels, and leads to better task-dependent data representations.

The problem of interest is to train deep neural network models with few labelled training samples. The specific assumption is there is a large pool of unlabelled data, and a heuristic function that can provide label annotations, possibly with varying levels of noises, to those unlabelled data. The adopted learning model is of a student/teacher framework as in privileged learning/knowledge distillation/model compression, and also machine teaching. The student (deep neural network) model will learn from both labelled and unlabelled training data with the labels provided by the teacher (Gaussian process) model. The teacher also supplies an uncertainty estimate to each predicted label. How about the heuristic function? This is used for learning initial feature representation of the student model. Crucially, the teacher model will also rely on these learned features. Labelled data and unlabelled data are therefore lie in the same dimensional space.
Specific questions to be addressed:
1)	Clustering of strongly-labelled data points. Thinking about the statement “each an expert on this specific region of data space”, if this is the case, I am expecting a clustering for both strongly-labelled data points and weakly-labelled data points. Each teacher model is trained on a portion of strongly-labelled data, and will only predict similar weakly-labelled data. On a related remark, the nice side-effect is not right as it was emphasized that data points with a high-quality label will be limited. As well, GP models, are quite scalable nowadays (experiments with millions to billions of data points are available in recent NIPS/ICML papers, though, they are all rely on low dimensionality of the feature space for optimizing the inducing point locations).  It will be informative to provide results with a single GP model.
2)	From modifying learning rates to weighting samples. Rather than using uncertainty in label annotation as a multiplicative factor in the learning rate, it is more “intuitive” to use it to modify the sampling procedure of mini-batches (akin to baseline #4); sample with higher probability data points with higher certainty. Here, experimental comparison with, for example, an SVM model that takes into account instance weighting will be informative, and a student model trained with logits (as in knowledge distillation/model compression).


Modern machine learning models are characterized by large hyperparameter search spaces and prohibitively expensive training costs. For such models, we cannot afford to train candidate models sequentially and wait months before finding a suitable hyperparameter configuration. Hence, we introduce the large-scale regime for parallel hyperparameter tuning, where we need to evaluate orders of magnitude more configurations than available parallel workers in a small multiple of the wall-clock time needed to train a single model. We propose a novel hyperparameter tuning algorithm for this setting that exploits both parallelism and aggressive earlystopping techniques, building on the insights of the Hyperband algorithm (Li et al., 2016). Finally, we conduct a thorough empirical study of our algorithm on several benchmarks, including large-scale experiments with up to 500 workers. Our results show that our proposed algorithm finds good hyperparameter settings nearly an order of magnitude faster than random search.

This paper introduces a simple extension to parallelize Hyperband.
Points in favor of the paper:
* Addresses an important problem
Points against:
* Only 5-fold speedup by parallelization with 5 x 25 workers, and worse performance in the same budget than Google Vizier (even though that treats the problem as a black box)
* Limited methodological contribution/novelty
The paper's methodological contribution is quite limited: it amounts to a straight-forward parallelization of successive halving (SHA). Specifically, whenever a worker frees up, do a new run on it, at the highest rung possible while making sure to not run too many runs for too high rungs. (I am pretty sure that is the idea, even though Algorithm 1, which is supposed to give the details, appears to have a bug in Procedure get_job -- it would always either pick the highest rung or the lowest!)
Empirically, the paper strangely does not actually evaluate a parallel version of Hyperband, but only evaluates the 5 parallel variants of SHA that Hyperband would run, each of them with all workers. The experiments in Section 4.2 show that, using 25 workers, the best of these 5 variants obtains a 5-fold speedup over sequential Hyperband on CIFAR and an 8-fold speedup on SVHN. I am confused: the *best* of 5 SHA variants only achieves a 5-fold speedup using 25 workers? I.e., parallel Hyperband, which would run the 5 SHA variants in parallel, would require 125 workers but only yield a 5-fold speedup? If I understand this correctly, I would clearly call this a negative result.
Likewise, for the large-scale experiment, a single run of Vizier actually yields as good performance as the best of the 5 SHA variants, and it is unknown beforehand which SHA variant works best -- in this example, actually Bracket 0 (which is often the best) stagnates. Parallel Hyperband would run the 5 SHA variants in parallel, so its performance at a budget of 10R with a total of 500 workers can be evaluated by taking the minimum of the 5 SHA variants at a budget of 2R. This would obtain a perplexity of above 90, which is quite a bit worse than Vizier's result of about 82. In general, the performance of parallel Hyperband can be computed by taking the minimum of the SHA variants and multiplying the time taken by 5; this shows that at any time in the plot (Figure 3, left) Vizier dominates parallel Hyperband. Again, this is apparently a negative result. (For Figure 3, right, no results for Vizier are given yet.)
If I understand correctly, the experiment in Section 4.4 does not involve any run of Hyperband, but merely plots predictions of Qi et al.'s Paelo framework of how many models could be evaluated with a growing number of GPUs.
Therefore, all empirical results for parallel Hyperband reported in the paper appear to be negative. This confuses me, especially since the authors seem to take them as positive results.
Because the original Hyperband paper argued that Bayesian optimization does not parallelize as well as random search / Hyperband, and because Hyperband has been reported to work much better than Bayesian optimization on a single node, I would have expected clear improvements of parallel Hyperband over parallel Bayesian optimization (=Vizier in the authors' setup). However, this is not what I see in the results. Am I mistaken somewhere? If not, based on these negative results the paper does not seem to quite clear the bar for ICLR.
Details, in order of appearance in the paper:
- Vizier: why did the authors only use Vizier's default Bayesian optimization algorithm? The Vizier paper by Golovin et al (2017) states that for large budgets other optimizers often perform better, and the budget in the large scale experiments is as high as 5000 function evaluations. Also, isn't there an automatic choice built into Vizier to pick the optimizer expected to be best? I think using a suboptimal version of Vizier would be a problem for the experimental setup.
- Algorithm 1: this needs some improvement; in particular fixing the bug I mentioned above.
- Section 3.1: Li et al (2017) do not analyze any algorithm theoretically. They also do not discuss finite vs. infinite horizon. I believe the authors meant Li et al's arXiv paper (2016) in both of these cases.
- Section 3.1, point 2: this is unclear to me, even though I know Hyperband very well. Can you please make this clearer?
- "A complete theoretical treatment of asynchronous SHA is out of the scope of this paper" -> is some theoretical treatment in scope?
- Section 4.1: It seems very useful to already recommend configurations in each rung of Hyperband, and I am surprised that the methods section does not mention this. From the text in this experiments section, it feels a little like that was always part of Hyperband; I didn't think it was, so I checked the original papers and blog posts, and both the ICLR 2017 and the arXiv 2016 paper state "In fact, the first result returned by HYPERBAND after using a budget of 5R is often competitive with results returned by other searchers after using 50R." and Kevin Jamieson's blog post on Hyperband (https://people.eecs.berkeley.edu/~kjamieson/hyperband.html) explicitly states: "While random and the Bayesian Optimization algorithms output their first recommendation after max_iter iterations, Hyperband does not output anything until about max_iter(logeta(max_iter)+1) iterations [...]"
Therefore, recommending after each rung seems to be a contribution of this paper, and I think it would be nice to read about this in the methods section.
- Experiment 1 (SVM) used dataset size as a budget, which is what Fabolas ("Fast Bayesian optimization on large datasets") is designed for according to Klein et al (2017). On the other hand, Experiments (2) and (3) used the number of epochs as a budget, and Fabolas is not designed for that (one would want to use a different kernel, for epochs, e.g., like Freeze-Thaw Bayesian optimization (FTBO) by Swersky et al (2014), instead of a kernel made for dataset sizes). Therefore, it is not surprising that Fabolas does not work as well in those cases. The case of number of epochs as a budget would be the domain of FTBO. I know that there is no reference implementation of FTBO, so I am not asking for a comparison, but the comparison against Fabolas is misleading for Experiments (2) and (3). This doesn't really change anything for the paper: the authors could still make the case that Fabolas hasn't been designed for this case and that (to the best of my knowledge) there simply isn't an implementation of a BO algorithm that is. Fabolas is arguably the closest thing, so the results could still be reported, just not as an apples-to-apples comparison; probably best as "Fabolas-like, with dataset size kernel" in the figure. The justification to not compare against Fabolas in the parallel regime is clearly valid.
- A clarification question: Section 4.4 does not report on any runs of actual neural networks, does it? And not on any runs of Hyperband, correct? Do I understand the reasoning correctly as pointing out that standard parallelization across multiple GPUs is not great, and that thus, in combination with parallel Hyperband, runs should be done mostly on one GPU only? How does this relate to the results in the cited paper "Accurate, Large-batch SGD: Training ImageNet in 1 Hour" (https://arxiv.org/abs/1706.02677)? Quoting from its abstract: "Using commodity hardware, our implementation achieves ∼ 90% scaling efficiency when moving from 8 to 256 GPUs." That seems like a very good utilization of parallel computing power?
- There is no conclusion / future work.
----------
Edit after author rebuttal:
I thank the reviewers for their rebuttal. This cleared up some points, but some others are still open.
(1) and (2) Unfortunately, I still do not agree that the need for 5*25 workers to get a 5-fold to 8-fold speedup is a positive result. Similarly, I would interpret the results in Figure 3 differently than the authors. For the comparison against Vizier the authors argue that they could just take the lowest 2 brackets of Hyperband; but running both of these two would still be 2x slower than Vizier. And we can't only run the best bracket because the information which one is the best is not available ahead of time. In fact, it is the entire point of Hyperband to hedge across multiple brackets including the one that is random search; one *could* just use the smallest bracket, but that is a heuristic and has no theoretical guarantees of being better (or at least not worse by more than a bounded factor) than random search.
Orthogonally: the comparison to Vizier (or any other baseline) is still missing for the LSTM acoustic model.
(3) Concerning SOTA results, I have to agree with AnonReviewer3: one way to demonstrate success is to show competitive performance on a dataset (e.g., CIFAR) on which other researchers can also evaluate their algorithms on. Getting 17% on CIFAR-10 does not fall into that category. Nevertheless, I agree with the authors that another way to demonstrate success is to show competitive performance on a *combination* of a dataset and a design space, but for that to be something that other researchers can compare to requires the authors making publicly available the implementations they have optimized; without that public availability, due to a host of possible confounding factors, it is impossible to judge whether state-of-the-art performance on such a combination of dataset and design space has been achieved.  I therefore recommend that the authors make the entire code they used for training CIFAR available; I don't expect this to have anything new in there, but it's a useful benchmark.
Likewise, for the LSTM on PTB, DeepMind used Google Vizier (https://arxiv.org/abs/1707.05589) to achieve *perplexities below 60* (compared to the results above 80 reported by the authors). Just as above, I therefore recommend that the authors make their pipeline for LSTB on PTB available. Likewise for the LSTM acoustic model.
(4) I'm confused that Section 4.4 does relate to SHA/Hyperband. Of course, there are some diminishing returns of running an optimizer across multiple GPUs. But similarly, there are diminishing returns of parallelizing SHA (e.g., the 5-fold speedup on 125 workers above). So the natural question that would be nice to answer is which combination of the two will yield the best results. Relatedly, the paper by Goyal et al seems to show that the weak scaling regime leads to almost linear speedups; why do the authors then analyze the strong scaling regime that does not appear to work as well?
Overall, the rebuttal did not change my evaluation and I kept my original score.


DIVIDE-AND-CONQUER REINFORCEMENT LEARNING
Standard model-free deep reinforcement learning (RL) algorithms sample a new initial state for each trial, allowing them to optimize policies that can perform well even in highly stochastic environments. However, problems that exhibit considerable initial state variation typically produce high-variance gradient estimates for model-free RL, making direct policy or value function optimization challenging. In this paper, we develop a novel algorithm that instead partitions the initial state space into "slices", and optimizes an ensemble of policies, each on a different slice. The ensemble is gradually unified into a single policy that can succeed on the whole state space. This approach, which we term divide-and-conquer RL, is able to solve complex tasks where conventional deep RL methods are ineffective. Our results show that divide-and-conquer RL greatly outperforms conventional policy gradient methods on challenging grasping, manipulation, and locomotion tasks, and exceeds the performance of a variety of prior methods. Videos of policies learned by our algorithm can be viewed at https://sites.google.com/view/dnc-rl/.

This paper presents a method for learning a global policy over multiple different MDPs (referred to as different "contexts", each MDP having the same dynamics and reward, but different initial state).  The basic idea is to learn a separate policy for each context, but regularized in a manner that keeps all of them relatively close to each other, and then learn a single centralized policy that merges the multiple policies via supervised learning.  The method is evaluated on several continuous state and action control tasks, and shows improvement over existing and similar approaches, notably the Distral algorithm.
I believe there are some interesting ideas presented in this paper, but in its current form I think that the delta over past work (particularly Distral) is ultimately too small to warrant publication at ICLR.  The authors should correct me if I'm wrong, but it seems as though the algorithm presented here is virtually identical to Distral except that:
1) The KL divergence term regularizes all policies together in a pairwise manner.
2) The distillation step happens episodically every R steps rather than in a pure SGD manner.
3) The authors possibly use a TRPO type objective for the standard policy gradient term, rather than REINFORCE-like approach as in Distral (this one point wasn't completely clear, as the authors mention that a "centralized DnC" is equivalent to Distral, so they may already be adapting it to the TRPO objective? some clarity on this point would be helpful).
Thus, despite better performance of the method over Distral, this doesn't necessarily seem like a substantially new algorithmic development.  And given how sensitive RL tasks are to hyperparameter selection, there needs to be some very substantial treatment of how the regularization parameters are chosen here (both for DnC and for the Distral and centralized DnC variants).  Otherwise, it honestly seems that the differences between the competing methods could be artifacts of the choice of regularization (the alpha parameter will affect just how tightly coupled the control policies actually are).
In addition to this point, the formulation of the problem setting in many cases was also somewhat unclear.  In particular, the notion of the contextual MDP is not very clear from the presentation.  The authors define a contextual MDP setting where in addition to the initial state there is an observed context to the MDP that can affect the initial state distribution (but not the transitions or reward).  It's entirely unclear to me why this additional formulation is needed, and ultimately just seems to confuse the nature of the tasks here which is much more clearly presented just as transfer learning between identical MDPs with different state distributions; and the terminology also conflicts with the (much more complex) setting of contextual decision processes (see: https://arxiv.org/abs/1610.09512).  It doesn't seem, for instance, that the final policy is context dependent (rather, it has to "infer" the context from whatever the initial state is, so effectively doesn't take the context into account at all).  Part of the reasoning seems to be to make the work seem more distinct from Distral than it really is, but I don't see why "transfer learning" and the presented contextual MDP are really all that different.
Finally, the experimental results need to be described in substantially more detail.  The choice of regularization parameters, the precise nature of the context in each setting, and the precise design of the experiments is all extremely opaque in the current presentation.  Since the methodology here is so similar to previous approaches, much more emphasis is required to better understand the (improved) empirical results in this eating.
In summary, while I do think the core ideas of this paper are interesting: whether it's better to regularize policies to a single central policy as in Distral or whether it's better to use joint regularization, whether we need two different timescales for distillation versus policy training, and what policy optimization method works best, as it is right now the algorithmic choices in the paper seem rather ad-hoc compared to Distral, and need substantially more empirical evidence.
Minor comments:
• There are several missing words/grammatical errors throughout the manuscript, e.g. on page 2 "gradient information can better estimated".


Two fundamental problems in unsupervised learning are efficient inference for latent-variable models and robust density estimation based on large amounts of unlabeled data. For efficient inference, normalizing flows have been recently developed to approximate a target distribution arbitrarily well. In practice, however, normalizing flows only consist of a finite number of deterministic transformations, and thus they possess no guarantee on the approximation accuracy. For density estimation, the generative adversarial network (GAN) has been advanced as an appealing model, due to its often excellent performance in generating samples. In this paper, we propose the concept of continuous-time flows (CTFs), a family of diffusion-based methods that are able to asymptotically approach a target distribution. Distinct from normalizing flows and GANs, CTFs can be adopted to achieve the above two goals in one framework, with theoretical guarantees. Our framework includes distilling knowledge from a CTF for efficient inference, and learning an explicit energy-based distribution with CTFs for density estimation.
Experiments on various tasks demonstrate promising performance of the proposed CTF framework, compared to related techniques.

The authors propose the use of first order Langevin dynamics as a way to transition from one latent variable to the next in the VAE setting, as opposed to the deterministic transitions of normalizing flow. The extremely popular Fokker-Planck equation is used to analyze the steady state distributions in this setting. The authors also propose the use of CTF in density estimation, as a generator of samples from the ''true'' distribution, and show competitive performance w.r.t. inception score for some common datasets.
The use of Langevin diffusion for latent transitions is a good idea in my opinion; though quite simple, it has the benefit of being straightforward to analyze with existing machinery. Though the discretized Langevin transitions in \S 3.1 are known and widely used, I liked the motivation afforded by Lemma 2.
I am not convinced that taking \rho to be the sample distribution with equal probabilities at the z samples is a good choice in \S 3.1; it would be better to incorporate the proximity of the langevin chain to a stationary point in the atom weights instead of setting them to 1/K. However to their credit the authors do provide an estimate of the error in the distribution stemming from their choice.
To the best of my knowledge the use of CTF in density estimation as described in \S 4 is new, and should be of interest to the community; though again it is fairly straightforward. Regarding the experiments, the difference in ELBO between the macVAE and the vanilla ones with normalizing flows is only about 2%; I wish the authors included a discussion on how the parameters of the discretized Langevin chain affects this, if at all.
Overall I think the theory is properly described and has a couple of interesting formulations, in spite of being not particularly novel. I think CTFs like the one described here will see increased usage in the VAE setting, and thus the paper will be of interest to the community.