AUTO-CONDITIONED RECURRENT NETWORKS FOR EXTENDED COMPLEX HUMAN MOTION SYNTHESIS
We present a real-time method for synthesizing highly complex human motions using a novel training regime we call the auto-conditioned Recurrent Neural Network (acRNN). Recently, researchers have attempted to synthesize new motion by using autoregressive techniques, but existing methods tend to freeze or diverge after a couple of seconds due to an accumulation of errors that are fed back into the network. Furthermore, such methods have only been shown to be reliable for relatively simple human motions, such as walking or running. In contrast, our approach can synthesize arbitrary motions with highly complex styles, including dances or martial arts in addition to locomotion. The acRNN is able to accomplish this by explicitly accommodating for autoregressive noise accumulation during training. Our work is the first to our knowledge that demonstrates the ability to generate over 18,000 continuous frames (300 seconds) of new complex human motion w.r.t. different styles.

The problem of learning auto-regressive (data-driven) human motion models that have long-term stability
is of ongoing interest. Steady progress is being made on this problem, and this paper adds to that.
The paper is clearly written. The specific form of training (a fixed number of self-conditioned predictions,
followed by a fixed number of ground-truth conditioned steps) is interesting for simplicity and its efficacy.
The biggest open question for me is how it would compare to the equally simple stochastic version proposed
by the scheduled sampling approach of [Bengio et al. 2015].
PROS:  The paper provides a simple solution to a problem of interest to many.
CONS:  It is not clear if it improves over something like scheduled sampling, which is a stochastic predecessor
       of the main idea introduced here. The "duration of stability" is a less interesting goal than
       actually matching the distribution of the input data.
The need to pay attention to the distribution-mismatch problem for sequence prediction problems
has been known for a while. In particular, the DAGGER (see below) and scheduled sampling algorithms (already cited)
target this issue, in addition to the addition of progressively increasing amounts of noise during training
(Fragkiadaki et al). Also see papers below on Professor Forcing, as well as "Learning Human Motion Models
for Long-term Predictions" (concurrent work?), which uses annealing over dropout rates to achieve stable long-term predictions.
  DAGGER algorithm (2011):  http://www.jmlr.org/proceedings/papers/v15/ross11a/ross11a.pdf
  "A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning"
  Professor Forcing (NIPS 2016)
  http://papers.nips.cc/paper/6099-professor-forcing-a-new-algorithm-for-training-recurrent-networks.pdf
  Learning Human Motion Models for Long-term Predictions (2017)
  https://arxiv.org/abs/1704.02827
  https://www.youtube.com/watch?v=PgJ2kZR9V5w

While the motions do not freeze, do the synthesized motion distributions match the actual data distributions?
This is not clear, and would be relatively simple to evaluate.  Is the motion generation fully deterministic?
It would be useful to have probabilistic transition distributions that match those seen in the data.
An interesting open issue (in motion, but also of course NLP domains) is that of how to best evaulate
sequence-prediction models.  The duration of "stable prediction" does not directly capture the motion quality.
Figure 1:  Suggest to make u != v for the purposes of clarity, so that they can be more easily distinguished.
Data representation:
Why not factor out the facing angle, i.e., rotation about the vertical axis, as done by Holden et al, and in a variety of
previous work in general?
The representation is already made translation invariant. Relatedly, in the Training section,
data augmentation includes translating the sequence: "rotate and translate the sequence randomly".
Why bother with the translation if the representation itself is already translation invariant?
The video illustrates motions with and without "foot alignment".
However, no motivation or description of "foot alignment" is given in the paper.
The following comment need not be given much weight in terms of evaluation of the paper, given that the
current paper does not use simulation-based methods. However, it is included for completeness.
The survey of simulation-based methods for modeling human motions is not representative of the body of work in this area
over the past 25 years.  It may be more useful to reference a survey, such as
"Interactive Character Animation Using Simulated Physics: A State‐of‐the‐Art Review" (2012)
An example of recent SOTA work for modeling dynamic motions from motion capture, including many
highly dynamic motions, is "Guided Learning of Control Graphs for Physics-Based Characters" (2016)
More recent work includes "Learning human behaviors from motion capture by adversarial imitation",
"Robust Imitation of Diverse Behaviors", and "Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning", all of which demonstrate imitation of various motion styles to various degrees.
It is worthwhile acknowledging that the synthesized motions are still low quality, particular when rendered with more human-like looking models, and readily distinguishable from the original motions.  In this sense, they are not comparable to the quality of results demonstrated in recent works by Holden et al. or some other recent works.  However, the authors should be given credit for including some results with fully rendered characters, which much more readily exposes motion flaws.
The followup work on [Lee et al 2010 "Motion Fields"] is quite relevant:
"Continuous character control with low-dimensional embeddings"
In terms of usefulness, being able to provide some control over the motion output is a more interesting problem than
being able to generate long uncontrolled sequences.  A caveat is that the methods are not applied to large datasets.


Recently popularized graph neural networks achieve the state-of-the-art accuracy on a number of standard benchmark datasets for graph-based semi-supervised learning, improving significantly over existing approaches. These architectures alternate between a propagation layer that aggregates the hidden states of the local neighborhood and a fully-connected layer. Perhaps surprisingly, we show that a linear model, that removes all the intermediate fully-connected layers, is still able to achieve a performance comparable to the state-of-the-art models. This significantly reduces the number of parameters, which is critical for semi-supervised learning where number of labeled examples are small. This in turn allows a room for designing more innovative propagation layers. Based on this insight, we propose a novel graph neural network that removes all the intermediate fullyconnected layers, and replaces the propagation layers with attention mechanisms that respect the structure of the graph. The attention mechanism allows us to learn a dynamic and adaptive local summary of the neighborhood to achieve more accurate predictions. In a number of experiments on benchmark citation networks datasets, we demonstrate that our approach outperforms competing methods. By examining the attention weights among neighbors, we show that our model provides some interesting insights on how neighbors influence each other.

The paper proposes a semi supervised learning algorithm for graph node classification. The Algorithm is inspired from Graph Neural Networks and more precisely graph convolutional NNs recently proposed by ref (Kipf et al  2016)) in the paper.  These NNs alternate 2 types of layers:  non linear projection and diffusion, the latter incorporates the graph relational information by constraining neighbor nodes to have close representations according to some “graph metrics”. The authors propose a model with simplified projection layers and more sophisticated diffusion ones, incorporating a simple attention mechanism. Experiments are performed on citation textual datasets. Comparisons with published results on the same datasets are presented.
The paper is clear and develops interesting ideas relevant to semi-supervised graph node classification. One finding is that simple models perform as well as more complex ones in this setting where labeled data is scarce. Another one is the importance of integrating relational information for classifying nodes when it is available. The attention mechanism itself is extremely simple, and learns one parameter per diffusion layers. One parameter weights correlations between node embeddings in a diffusion layer. I understand that you tried more complex attention mechanisms, but the one finally selected is barely an attention mechanism and rather a simple “importance” weight. This is not a criticism, but this makes the title somewhat misleading. The experiments show that the proposed model is state of the art for graph node classification. The performance is on par with some other recent models according to table 2. The other tests are also interesting, but the comparison could have been extended to other models e.g. GCN.
You advocate the role of the diffusion layers, and in the experiments you stack 3 to 4 such layers. It would be interesting to have indications on the compromise performance/ number of diffusion layers and on the evolution of these performances when adding such layers.
The bibliography on semi-supervised learning in graphs for classification is light and should be enhanced.
Overall this is an interesting paper with nice findings. The originality is however relatively limited in a field where many recent papers have been proposed, and the experiments need to be completed.


MULTI-SCALE DENSE NETWORKS FOR RESOURCE EFFICIENT IMAGE CLASSIFICATION
In this paper we investigate image classification with computational resource limits at test time. Two such settings are: 1. anytime classification, where the network's prediction for a test example is progressively updated, facilitating the output of a prediction at any time; and 2. budgeted batch classification, where a fixed amount of computation is available to classify a set of examples that can be spent unevenly across "easier" and "harder" inputs. In contrast to most prior work, such as the popular Viola and Jones algorithm, our approach is based on convolutional neural networks. We train multiple classifiers with varying resource demands, which we adaptively apply during test time. To maximally re-use computation between the classifiers, we incorporate them as early-exits into a single deep convolutional neural network and inter-connect them with dense connectivity. To facilitate high quality classification early on, we use a two-dimensional multi-scale network architecture that maintains coarse and fine level features all-throughout the network. Experiments on three image-classification tasks demonstrate that our framework substantially improves the existing state-of-the-art in both settings.

This paper introduces a new model to perform image classification with limited computational resources at test time. The model is based on a multi-scale convolutional neural network similar to the neural fabric (Saxena and Verbeek 2016), but with dense connections (Huang et al., 2017) and with a classifier at each layer. The multiple classifiers allow for a finer selection of the amount of computation needed for a given input image. The multi-scale representation allows for better performance at early stages of the network. Finally the dense connectivity allows to reduce the negative effect that early classifiers have on the feature representation for the following layers.
A thorough evaluation on ImageNet and Cifar100 shows that the network can perform better than previous models and ensembles of previous models with a reduced amount of computation.
Pros:
- The presentation is clear and easy to follow.
- The structure of the network is clearly justified in section 4.
- The use of dense connectivity to avoid the loss of performance of using early-exit classifier is very interesting.
- The evaluation in terms of anytime prediction and budgeted batch classification can represent real case scenarios.
- Results are very promising, with 5x speed-ups and same or better accuracy that previous models.
- The extensive experimentation shows that the proposed network is better than previous approaches under different regimes.
Cons:
- Results about the more efficient densenet* could be shown in the main paper
Additional Comments:
- Why in training you used logistic loss instead of the more common cross-entropy loss? Has this any connection with the final performance of the network?
- In fig. 5 left for completeness I would like to see also results for DenseNet^MT and ResNet^MT
- In fig. 5 left I cannot find the 4% and 8% higher accuracy with 0.5x10^10 to 1.0x10^10 FLOPs, as mentioned in section 5.1 anytime prediction results
- How the budget in terms of Mul-Adds is actually estimated?
I think that this paper present a very powerful approach to speed-up the computational cost of a CNN at test time and clearly explains some of the common trade-offs between speed and accuracy and how to improve them. The experimental evaluation is complete and accurate.


CAN RECURRENT NEURAL NETWORKS WARP TIME?
Successful recurrent models such as long short-term memories (LSTMs) and gated recurrent units (GRUs) use ad hoc gating mechanisms. Empirically these models have been found to improve the learning of medium to long term temporal dependencies and to help with vanishing gradient issues. We prove that learnable gates in a recurrent model formally provide quasiinvariance to general time transformations in the input data. We recover part of the LSTM architecture from a simple axiomatic approach. This result leads to a new way of initializing gate biases in LSTMs and GRUs. Experimentally, this new chrono initialization is shown to greatly improve learning of long term dependencies, with minimal implementation effort.
Recurrent neural networks (e.g. (Jaeger, 2002)) are a standard machine learning tool to model and represent temporal data; mathematically they amount to learning the parameters of a parameterized dynamical system so that its behavior optimizes some criterion, such as the prediction of the next data in a sequence.
Handling long term dependencies in temporal data has been a classical issue in the learning of recurrent networks. Indeed, stability of a dynamical system comes at the price of exponential decay of the gradient signals used for learning, a dilemma known as the vanishing gradient problem (Pascanu et al., 2012;Hochreiter, 1991;Bengio et al., 1994). This has led to the introduction of recurrent models specifically engineered to help with such phenomena.
Use of feedback connections (Hochreiter & Schmidhuber, 1997) and control of feedback weights through gating mechanisms (Gers et al., 1999) partly alleviate the vanishing gradient problem. The resulting architectures, namely long short-term memories (LSTMs (Hochreiter & Schmidhuber, 1997;Gers et al., 1999)) and gated recurrent units (GRUs (Chung et al., 2014)) have become a standard for treating sequential data.
Using orthogonal weight matrices is another proposed solution to the vanishing gradient problem, thoroughly studied in (Saxe et al., 2013;Le et al., 2015;Arjovsky et al., 2016;Wisdom et al., 2016;Henaff et al., 2016). This comes with either computational overhead, or limitation in representational power. Furthermore, restricting the weight matrices to the set of orthogonal matrices makes forgetting of useless information difficult.
The contribution of this paper is threefold:
• We show that postulating invariance to time transformations in the data (taking invariance to time warping as an axiom) necessarily leads to a gate-like mechanism in recurrent models (Section 1). This provides a clean derivation of part of the popular LSTM and GRU architectures from first principles. In this framework, gate values appear as time contraction or time dilation coefficients, similar in spirit to the notion of time constant introduced in (Mozer, 1992).
• From these insights, we provide precise prescriptions on how to initialize gate biases (Section 2) depending on the range of time dependencies to be captured. It has previously been advocated that setting the bias of the forget gate of LSTMs to 1 or 2 provides overall good performance (Gers & Schmidhuber, 2000;Jozefowicz et al., 2015). The viewpoint here explains why this is reasonable in most cases, when facing medium term dependencies, but fails when facing long to very long term dependencies.
• We test the empirical benefits of the new initialization on both synthetic and real world data (Section 3). We observe substantial improvement with long-term dependencies, and slight gains or no change when short-term dependencies dominate.

tl;dr:
 - The paper has a really cool theoretical contribution.
 - The experiments do not directly test whether the theoretical insight holds in practice, but instead a derivate method is tested on various benchmarks.
I must say that this paper has cleared up quite a few things for me. I have always been a skeptic wrt LSTM, since I myself did not fully understand when to prefer them over vanilla RNNs for reasons other than “they empirically work much better in many domains.” and “they are less prone to vanishing gradients”.
Section 1 is a bliss: it provides a very useful candidate explanation under which conditions vanilla RNNs fail (or at least, do not efficiently generalise) in contrast to gated cells. I am sincerely happy about the write up and will point many people to it.
The major problem with the paper, in my eyes, is the lack of experiments specific to test the hypothesis. Obviously, quite a bit of effort has gone into the experimental section. The focus however is comparison to the state of the art in terms of raw performance.
That leaves me asking: are gated RNNs superior to vanilla RNNs if the data is warped?
Well, I don’t know now. I only can say that there is reason to believe so.
I *really* do encourage the authors to go back to the experiments and see if they can come up with an experiment to test the main hypothesis of the paper. E.g. one could make synthetic warpings, apply it to any data set and test if things work out as expected. Such a result would in my opinion be of much more use than the tiny increment in performance that is the main output of the paper as of now, and which will be stomped by some other trick in the months to come. It would be a shame if such a nice theoretical insight got under the carpet because of that. E.g. today we hold [Pascanu 2013] dear not because of the proposed method, but because of the theoretical analysis.
Some minor points.
- The authors could make use of less footnotes, and try to incorporate them into the text or appendix.
- A table of results would be nice.
- Some choices of the experimental section seem arbitrary, e.g. the use of optimiser and to not use clipping of gradients. In general, the evaluation of the hyper parameters is not rigorous.
- “abruplty” -> “abruptly” on page 5, 2nd paragraph
### References
[Pascanu 2013] Pascanu, Razvan, Tomas Mikolov, and Yoshua Bengio. "On the difficulty of training recurrent neural networks." International Conference on Machine Learning. 2013.


Published as a conference paper at ICLR 2018 ROBUSTNESS OF CLASSIFIERS TO UNIVERSAL PERTUR- BATIONS: A GEOMETRIC PERSPECTIVE
Deep networks have recently been shown to be vulnerable to universal perturbations: there exist very small image-agnostic perturbations that cause most natural images to be misclassified by such classifiers. In this paper, we provide a quantitative analysis of the robustness of classifiers to universal perturbations, and draw a formal link between the robustness to universal perturbations, and the geometry of the decision boundary. Specifically, we establish theoretical bounds on the robustness of classifiers under two decision boundary models (flat and curved models). We show in particular that the robustness of deep networks to universal perturbations is driven by a key property of their curvature: there exist shared directions along which the decision boundary of deep networks is systematically positively curved. Under such conditions, we prove the existence of small universal perturbations. Our analysis further provides a novel geometric method for computing universal perturbations, in addition to explaining their properties.

The paper develops models which attempt to explain the existence of universal perturbations which fool neural networks — i.e., the existence of a single perturbation which causes a network to misclassify most inputs. The paper develops two models for the decision boundary:
(a) A locally flat model in which the decision boundary is modeled with a hyperplane and the normals two the hyperplanes are assumed to lie near a low-dimensional linear subspace.
(b) A locally positively curved model, in which there is a positively curved outer bound for the collection of points which are assigned a given label.
The paper works out a probabilistic analysis arguing that when either of these conditions obtains, there exists a fooling perturbation which affects most of the data.
The theoretical analysis in the paper is straightforward, in some sense following from the definition. The contribution of the paper is to posit these two conditions which can predict the existence of universal fooling perturbations, argue experimentally that they occur in (some) neural networks of practical interest.
One challenge in assessing the experimental claims is that practical neural networks are nonsmooth; the quadratic model developed from the hessian is only valid very locally. This can be seen in some of the illustrative examples in Figure 5: there *is* a coarse-scale positive curvature, but this would not necessarily come through in a quadratic model fit using the hessian. The best experimental evidence for the authors’ perspective seems to be the fact that random perturbations from S_c misclassify more points than random perturbations constructed with the previous method.
I find the topic of universal perturbations interesting, because it potentially tells us something structural (class-independent) about the decision boundaries constructed by artificial neural networks. To my knowledge, the explanation of universal perturbations in terms of positive curvature is novel. The paper would be much stronger if it provided an explanation of *why* there exists this common subspace of universal fooling perturbations, or even what it means geometrically that positive curvature obtains at every data point.
Visually, these perturbations seem to have strong, oriented local high-frequency content — perhaps they cause very large responses in specific filters in the lower layers of a network, and conventional architectures are not robust to this?
It would also be nice to see some visual representations of images perturbed with the new perturbations, to confirm that they remain visually similar to the original images.


MULTI-LEVEL RESIDUAL NETWORKS FROM DYNAMI- CAL SYSTEMS VIEW
Deep residual networks (ResNets) and their variants are widely used in many computer vision applications and natural language processing tasks. However, the theoretical principles for designing and training ResNets are still not fully understood. Recently, several points of view have emerged to try to interpret ResNet theoretically, such as unraveled view, unrolled iterative estimation and dynamical systems view. In this paper, we adopt the dynamical systems point of view, and analyze the lesioning properties of ResNet both theoretically and experimentally. Based on these analyses, we additionally propose a novel method for accelerating ResNet training. We apply the proposed method to train ResNets and Wide ResNets for three image classification benchmarks, reducing training time by more than 40% with superior or on-par accuracy.

I enjoyed reading the paper. This is a very well written paper, the authors propose a method for speeding up the training time of Residual Networks based on the dynamical system view interpretation of ResNets. In general I have a positive opinion about the paper, however, I’d like to ask for some clarifications.
I’m not fully convinced by the interpretation of Eq. 5: “… d is inversely proportional to the norm of the residual modules G(Yj)”. Since F(Yj) is not a constant, I think that d is inversely proportional to ||G(Yj)||/||F(Yj)||, however, in the interpretation the dependence on ||F(Yj)|| is ignored. Could the authors comment on that?
Section 4. 1 “ Each cycle itself can be regarded as a training process, thus we need to reset the learning rate value at the beginning of each training cycle and anneal the learning rate during that cycle.” Is there any empirical evidence for this? What would happen if the learning rate is not reset at the beginning of each cycle?
Questions with respect to dynamical systems point of view: Eq. 4 assumes small value of h. However, for ResNet there is no guarantee that the h would be small (e. g. in Appendix C the values between 0.25 and 1 are used). Would the authors be willing to comment on the importance of the value of h? In figure 1, pooling (strided convolutions) are not depicted between network stages. I have one question w.r.t. feature maps dimensionality changes inside a CNN: how does pooling (or strided convolution) fit into dynamical systems view?
Table 3 and 4. I assume that the training time unit is a minute, I couldn’t find this information in the paper. Is the batch size the same for all models (100 for CIFAR and 32 for STL-10)? I understand that the models with different #Blocks have different capacity, for clarity, would it be possible to add # of parameters to each model? For multilevel method, would it be possible to show intermediate results in Table 3 and 4, e. g. at the end of cycle 1 and 2? I see these results in Figure 6, however, the plots are condensed and it is difficult to see the exact number at the end of each cycle.
The citation (E, 2017) seems to be wrong, could the authors check it?


An obstacle that prevents the wide adoption of (deep) reinforcement learning (RL) in control systems is its need for a large number of interactions with the environment in order to master a skill. The learned skill usually generalizes poorly across domains and re-training is often necessary when presented with a new task. We present a framework that combines techniques in formal methods with hierarchical reinforcement learning (HRL). The set of techniques we provide allows for the convenient specification of tasks with logical expressions, learns hierarchical policies (meta-controller and low-level controllers) with well-defined intrinsic rewards using any RL methods and is able to construct new skills from existing ones without additional learning. We evaluate the proposed methods in a simple grid world simulation as well as simulation on a Baxter robot.

The paper argues for structured task representations (in TLTL) and shows how these representations can be used to reuse learned subtasks to decrease learning time.
Overall, the paper is sloppily put together, so it's a little difficult to assess the completeness of the ideas. The problem being solved is not literally the problem of decreasing the amount of data needed to learn tasks, but a reformulation of the problem that makes it unnecessary to relearn subtasks. That's a good idea, but problem reformulation is always hard to justify without returning to a higher level of abstraction to justify that there's a deeper problem that remains unchanged. The paper doesn't do a great job of making that connection.
The idea of using task decomposition to create intrinsic rewards seems really interesting, but does not appear to be explored in any depth. Are there theorems to be had? Is there a connection to subtasks rewards in earlier HRL papers?
The lack of completeness (definitions of tasks and robustness) also makes the paper less impactful than it could be.
Detailed comments:
"learn hierarchical policies" -> "learns hierarchical policies"?
"n games Mnih et al. (2015)Silver et al. (2016),": The citations are a mess. Please proof read.
"and is hardly reusable" -> "and are hardly reusable".
"Skill composition is the idea of constructing new skills with existing skills (" -> "Skill composition is the idea of constructing
new skills out of existing skills (".
"to synthesis" -> "to synthesize".
"set of skills are" -> "set of skills is".
"automatons" -> "automata".
"with low-level controllers can" -> "with low-level controllers that can".
"the options policy π o is followed until β(s) > threshold": I don't think that's how options were originally defined... beta is generally defined as a termination probability.
"The translation from TLTL formula FSA to" -> "The translation from TLTL formula to FSA"?
"four automaton states Qφ = {q0, qf , trap}": Is it three or four?
"learn a policy that satisfy" -> "learn a policy that satisfies".
"HRL, We introduce the FSA augmented MDP" -> "HRL, we introduce the FSA augmented MDP.".
" multiple options policy separately" -> " multiple options policies separately"?
"Given flat policies πφ1 and πφ2 that satisfies " -> "Given flat policies πφ1 and πφ2 that satisfy ".
"s illustrated in Figure 3 ." -> "s illustrated in Figure 2 ."?
", we cam simply" -> ", we can simply".
"Figure 4 <newline> ." -> "Figure 4.".
", disagreement emerge" -> ", disagreements emerge"?
The paper needs to include SOME definition of robustness, even if it just informal. As it stands, it's not even clear if larger
values are better or worse. (It would seem that *more* robustness is better than less, but the text says that lower values are
chosen.)
"with 2 hidden layers each of 64 relu": Missing word? Or maybe a comma?
"to aligns with" -> "to align with".
" a set of quadratic distance function" -> " a set of quadratic distance functions".
"satisfies task the specification)" -> "satisfies the task specification)".
Figure 4: Tasks 6 and 7 should be defined in the text someplace.
"current frame work i" -> "current framework i".
" and choose to follow" -> " and chooses to follow".
" this makes" -> " making".
"each subpolicies" -> "each subpolicy".


We propose a framework to understand the unprecedented performance and robustness of deep neural networks using field theory. Correlations between the weights within the same layer can be described by symmetries in that layer, and networks generalize better if such symmetries are broken to reduce the redundancies of the weights. Using a two parameter field theory, we find that the network can break such symmetries itself towards the end of training in a process commonly known in physics as spontaneous symmetry breaking. This corresponds to a network generalizing itself without any user input layers to break the symmetry, but by communication with adjacent layers. In the layer decoupling limit applicable to residual networks (He et al., 2015a), we show that the remnant symmetries that survive the non-linear layers are spontaneously broken based on empirical results. The Lagrangian for the non-linear and weight layers together has striking similarities with the one in quantum field theory of a scalar. Using results from quantum field theory we show that our framework is able to explain many experimentally observed phenomena, such as training on random labels with zero error (Zhang et al., 2017), the information bottleneck and the phase transition out of it and the following increase in gradient variances (Shwartz-Ziv & Tishby, 2017), shattered gradients (Balduzzi et al., 2017), and many more.

In this paper, an number of very strong (even extraordinary) claims are made:
* The abstract promises "a framework to understand the unprecedented performance and robustness of deep neural networks using field theory."
* Page 8 states that this is "This is a first attempt to describe a neural network with a scalar quantum field theory."
* Page 2 promises the use of the "Goldstone theorem" (no less) to understand phase transition in deep learning
* It also claim that many "seemingly different experimental results can be explained by the presence of these zero eigenvalue weights."
* Three important results are stated as "theorem", with a statement like "Deep feedforward networks learn by breaking symmetries" proven in 5 lines, with no formal mathematics.
These are extraordinary claims, but  when reaching page 5, one sees that the basis of these claims seems to be the Lagrangian of a simple phi-4 theory, and Fig. 1 shows the standard behaviour of the so-called mexican hat in physics, the basis of the second-order transition. Given physicists have been working on neural network for more than three or four decades, I am surprise that this would enough to solve all these problems!
I tried to understand these many results, but I am afraid I cannot really understand or see them. In many case, the explanation seems to be a vague analogy. These are not without interest, and maybe there is indeed something deep in this paper, but it is so far hidden by the hype. Still, I fail to see how the fact that phase transitions and negative direction in the landscape is a new phenomena, and how it explains all the stated phenomenology. Beside, there are quite a lot of things known about the landscape of these problems
Maybe I am indeed missing something, but i clearly suspect the authors are simply overselling physics results.
I have been wrong many times, but I beleive that the authors should probably precise their claim, and clarify the relation between their results and both the physics AND statistics litterature, or better, with the theoretical physics litterature applied to learning, which is ---astonishing-- absent in the paper.
About the content:
The main problem for me is that the whole construction using field theory seems to be used to advocate for the appearence of a phase transition in neural nets and in learning. This rises three comments:
(1) So we really need to use quantum field theory for this? I do not see what should be quantum here (despite the very vague remarks page 12 "WHY QUANTUM FIELD THEORY?")
(2) This is not new. Phase transitions in learning in neural nets are being discussed since aboutn 40 years, see for instance all the pionnering work of Sompolinky et al. one can see for instance the nice review in https://arxiv.org/abs/1710.09553 In non aprticular order, phase transition and symmetry breaking are discussed in
* "Statistical mechanics of learning from examples", Phys. Rev. A 45, 6056 – Published 1 April 1992
* "The statistical mechanics of learning a rule", Rev. Mod. Phys. 65, 499 – Published 1 April 1993
* Phase transitions in the generalization behaviour of multilayer neural networks
http://iopscience.iop.org/article/10.1088/0305-4470/28/16/010/meta
* Note that some of these results are now rigourous, as shown in "Phase Transitions, Optimal Errors and Optimality of Message-Passing in Generalized Linear Models", https://arxiv.org/abs/1708.03395
* The landscape of these problems has been studied quite extensivly, see for instance "Identifying and attacking the saddle point problem in high-dimensional non-convex optimization", https://arxiv.org/abs/1406.2572
(3) There is nothing particular about deep neural net and neural nets about this. Negative direction in the Hessian in learning problems appears in matrix and tensor factorizaion, where phase transition are well understood (even rigorously, see for instance, https://arxiv.org/abs/1711.05424 ) or in problems such as unsupervised learning, as e.g.:
https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.86.2174
https://journals.aps.org/pre/pdf/10.1103/PhysRevE.50.1766
Here are additional comments:
PAGE 1:
* "It has been discovered that the training process ceases when it goes through an information bottleneck (ShwartzZiv & Tishby, 2017)".
While this paper indeed make a nice suggestion, I would not call it a discovery yet as this has never been shown on a large network. Beside, another paper in the conference is claiming exacly the opposite, see : "On the Information Bottleneck Theory of Deep Learning". This is still subject of discussion.
* "In statistical terms, a quantum theory describes errors from the mean of random variables. "
Last time I studied quantum theory, it was a theory that aim to explain the physical behaviours at the molecular, atomic and sub-atomic levels, usinge either on the wave function (Schrodinger) or the Matrix operatir formalism (Hesienbger) (or if you want, the path integral formalism of Feynman).
It is certainly NOT a theory that describes errors from the mean of random variables. This is, i beleive, the field of "statistics" or "probability" for correlated variables. It is certianly used in physics, and heavily both in statistical physics and in quantum thoery, but this is not what the theory is about in the first place.
Beside, there is little quantum in this paper, I think most of what the authors say apply to a statistical field theory ( https://en.wikipedia.org/wiki/Statistical_field_theory )
* "In the limit of a continuous sample space, the quantum theory becomes a quantum field theory."
Again, what is quantum about all this? This true for a field theory, as well for continous theories of, say, mechanics, fracture, etc...
PAGE 2:
* "Using a scalar field theory we show that a phase transition must exist towards the end of training based on empirical results."
So it is a scalar classical field theory after all. This sounds a little bit less impressive that a quantum field theory. Note that the fact that phase transition arises in learning, and in a statistical theory applied to any learning process, is an old topic, with a classical litterature. The authors might be interested by the review "The statistical mechanics of learning a rule", Rev. Mod. Phys. 65, 499 – Published 1 April 1993
PAGE 8:
* "In this work we solved one of the most puzzling mysteries of deep learning by showing that deep neural networks undergo spontaneous symmetry breaking."
I am afraid I fail to see what is so mysterious about this nor what the authors showed about it. In any case, gradient descent break symmetry spontaneously in many systems, including phi-4, the Ising model or (in learning problems) the community detection problem (see eg https://journals.aps.org/prx/abstract/10.1103/PhysRevX.4.011047). I am afraid I miss what is new there...
* "This is a first attempt to describe a neural network with a scalar quantum field theory."
Given there seems to be little quantum in the paper, I fail to see the relevance of the statement. Secondly, I beleive that field theory has been used, many times and in greater lenght, both for statistical and dynamical problems in neural nets, see eg.
* http://iopscience.iop.org/article/10.1088/0305-4470/27/6/016/meta
* https://arxiv.org/pdf/q-bio/0701042.pdf
* http://www.lps.ens.fr/~derrida/PAPIERS/1987/gardner-zippelius-87.pdf
* http://iopscience.iop.org/article/10.1088/0305-4470/21/1/030/meta
* https://arxiv.org/pdf/cond-mat/9805073.pdf


One of the most successful techniques in generative models has been decomposing a complicated generation task into a series of simpler generation tasks. For example, generating an image at a low resolution and then learning to refine that into a high resolution image often improves results substantially. Here we explore a novel strategy for decomposing generation for complicated objects in which we first generate latent variables which describe a subset of the observed variables, and then map from these latent variables to the observed space. We show that this allows us to achieve decoupled training of complicated generative models and present both theoretical and experimental results supporting the benefit of such an approach.

This paper proposed a method called Locally Disentangled Factors for hierarchical latent variable generative model, which can be seen as a hierarchical variant of Adversarially Learned Inference (Dumoulin el atl. 2017). The idea seems to be a valid variant, however, the quality of the paper is not good. The introduction and related works sections read well, but the rest of the paper has not been written well. More specifically, the content in section 3 and experiment section is messy. Also the experiments have not been conducted thoroughly, and the results and the interpretation of the results are not complete.
Introduction:
Although in introduction the author discussed a lot of works on hierarchical latent variable model and some motivating examples, after reading it the reviewer has absolutely no idea what the paper is about (except hierarchical latent variable model), what is the motivation, what is the general idea, what is the contribution of the paper. Only after carefully reading the detailed implementation in section 3.1 and section 5, did I realize that what the authors are actually doing is to use N variables to model N different parts of the observation, and one higher level variable to model the N variables. The paper should really more precisly state what the idea is throughout the paper, instead of causing confusion and ambiguity.
Section 3:
1. The concepts of "disentanglment" and "local connectivity" are really unnecessary and confusing. First, the whole paper and experiments has nothing to do with "local connectivity". Even though you might have the intention to propose the idea, you didn't show any support for the idea. Second, what you actually did is to use top level variable to generate N latent variables. That could hardly called "disentanglement". The mean field factorization in (Kingma & Welling 2013) is on the inference side (Q not P), and as found out in literature, it could not achieve disentanglement.
2. In section 3.2, I understand that you want to say the hierarchical model may require less data sample. But, here you are really off-topic. It would be much better if you can relate to the proposed method, and state how it may require less data.
3. Section 3.1 is more important, and is really major part of your method. Therefore, it need more extensive discussion and emphasis.
Experiment:
This section is really bad.
1. Since in the introduction and related works, there are already so many hierarchical latent variable model listed, the baseline methods should really not just vanilla GAN, but hierarchical latent variable models, such as the Hierachical VAE, Variational Ladder Autoencoder in (Zhao et al. 2017), ALI (not hierarchical, but should be a baseline) in (Dumoulin et al. 2017), etc.
2. Since currently there is still no standard way to evaluate the quality of image generation, by giving only inception score, we can really not judge whether it is good or not. You need to give more metrics, or generation examples, recontruction examples, and so on. And equally importantly, compare and discuss about the results. Not just leave it there.
3. For section 5.2, similar problems as above exist. Baseline methods might be insufficient. The paper only shows several examples, and the reviewer cannot draw any conclusion about it. Nor does the paper discuss any of the results.
4. Section 5.3, second to the last line, typo: "This is shown in 7". Also this result is not available.
5. More importantly, some experiments should be conducted to explicitly show the validity of the proposed hierarchical latent model idea. Show that it exists and works by some experiment explicitly.
Another suggestion the review would like to make is that, instead of proposing the general framework in section 2, it would be better to propose the hierarchical model in the context of section 3.1. That is, instead of saying z_0 -> z_1 ->... ->x, what the paper and experiment is really about is z_0 -> z_{1,1}, z_{1,2} ... z_{1,N} -> x_{1}, x_{2},...,x_{N}, where z_{1,1...N} are distinct variables. How section 2 is related to the learning of this might be concatenating these N distinct variables into one (if that's what you mean). Talking about the joint distribution and inference process in this way might more align with your idea. Also, the paper actually only deals with 2 level. It seems to me that it's meaningless to generalize to n levels in section 2, since you do not have any support of it.
In conclusion, the reviewer thinks that this work is incomplete and does not worth publishing with its current quality.
==============================================================
The reviewer read the response from the authors. However, I do not think the authors resolved the issues I mentioned. And I am still not convinced by the quality of the paper. I would say the idea is not bad, but the paper is still not well-prepared. So I do not change my decision.


Federated Learning is a machine learning setting where the goal is to train a highquality centralized model while training data remains distributed over a large number of clients each with unreliable and relatively slow network connections. We consider learning algorithms for this setting where on each round, each client independently computes an update to the current model based on its local data, and communicates this update to a central server, where the client-side updates are aggregated to compute a new global model. The typical clients in this setting are mobile phones, and communication efficiency is of the utmost importance. In this paper, we propose two ways to reduce the uplink communication costs: structured updates, where we directly learn an update from a restricted space parametrized using a smaller number of variables, e.g. either low-rank or a random mask; and sketched updates, where we learn a full model update and then compress it using a combination of quantization, random rotations, and subsampling before sending it to the server. Experiments on both convolutional and recurrent networks show that the proposed methods can reduce the communication cost by two orders of magnitude.

The authors examine several techniques that lead to low communication updates during distributed training in the context of Federated learning (FL). Under the setup of FL, it is assumed that training takes place over edge-device like compute nodes that have access to subsets of data (potentially of different size), and each node can potentially be of different computational power. Most importantly, in the FL setup, communication is the bottleneck. Eg a global model is to be trained by local updates that occur on mobile phones, and communication cost is high due to slow up-link.
The authors present techniques that are of similar flavor to quantized+sparsified updates. They distinguish theirs approaches into 1) structured updates and 2) sketched updates. For 1) they examine a low-rank version of distributed SGD where instead of communicating full-rank model updates, the updates are factored into two low rank components, and only one of them is optimized at each iteration, while the other can be randomly sampled.
They also examine random masking, eg a sparsification of the updates, that retains a random subset of the entries of the gradient update (eg by zero-ing out a random subset of elements). This latter technique is similar to randomized coordinate descent.
Under the theme of sketched updates, they examine quantized and sparsified updates with the property that in expectation they are identical to the true updates. The authors specifically examine random subsampling (which is the same as random masking, with different weights) and probabilistic quantization, where each element of a gradient update is randomly quantized to b bits.
The major contribution of this paper is their experimental section, where the authors show the effects of training with structured, or sketched updates, in terms of reduced communication cost, and the effect on the training accuracy. They present experiments on several data sets, and observe that among all the techniques, random quantization can have a significant reduction of up to 32x in communication with minimal loss in accuracy.
My main concern about this paper is that although the presented techniques work well in practice, some of the algorithms tested are similar algorithms that have already been proven to work well in practice. For example, it is unclear how the performance of the presented quantization algorithms compares to say  QSGD [1] and Terngrad [2]. Although the authors cite QSGD, they do not directly compare against it in experiments.
As a matter of fact, one of the issues of the presented quantized techniques (the fact that random rotations might be needed when the dynamic range of elements is large, or when the updates are nearly sparse) is easily resolved by algorithms like QSGD and Terngrad that respect (and promote) sparsity in the updates.
A more minor comment is that it is unclear that averaging is the right way to combine locally trained models for nonconvex problems. Recently, it has been shown that averaging can be suboptimal for nonconvex problems, eg a better averaging scheme can be used in place [3]. However, I would not worry too much about that issue, as the same techniques presented in this paper apply to any weighted linear averaging algorithm.
Another minor comment: The legends in the figures are tiny, and really hard to read.
Overall this paper examines interesting structured and randomized low communication updates for distributed FL, but lacks some important experimental comparisons.
[1] QSGD: Communication-Optimal Stochastic Gradient Descent, with Applications to Training Neural Networks https://arxiv.org/abs/1610.02132
[2] TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
https://arxiv.org/abs/1705.07878
[3] Parallel SGD: When does averaging help?
https://arxiv.org/abs/1606.07365


Despite the widespread interest in generative adversarial networks (GANs), few works have studied the metrics that quantitatively evaluate GANs' performance. In this paper, we revisit several representative sample-based evaluation metrics for GANs, and address the important problem of how to evaluate the evaluation metrics. We start with a few necessary conditions for metrics to produce meaningful scores, such as distinguishing real from generated samples, identifying mode dropping and mode collapsing, and detecting overfitting. Then with a series of carefully designed experiments, we are able to comprehensively investigate existing sample-based metrics and identify their strengths and limitations in practical settings. Based on these results, we observe that kernel Maximum Mean Discrepancy (MMD) and the 1-Nearest-Neighbour (1-NN) two-sample test seem to satisfy most of the desirable properties, provided that the distances between samples are computed in a suitable feature space. Our experiments also unveil interesting properties about the behavior of several popular GAN models, such as whether they are memorizing training samples, and how far these state-of-the-art GANs are from perfect.

This paper introduces a comparison between several approaches for evaluating GANs. The authors consider the setting of a pre-trained image models as generic representations of generated and real images to be compared. They compare the evaluation methods based on five criteria termed disciminability, mode collapsing and mode dropping, sample efficiency,computation efficiency, and robustness to transformation. This paper has some interesting insights and a few ideas of how to validate an evaluation method. The topic is an important one and a very difficult one. However, the work has some problems in rigor and justification and the conclusions are overstated in my view.
Pros
-Several interesting ideas for evaluating evaluation metrics are proposed
-The authors tackle a very challenging subject
Cons
-It is not clear why GANs are the only generative model considered
-Unprecedented visual quality as compared to other generative models has brought the GAN to prominence and yet this is not really a big factor in this paper.
-The evaluations rely on using a pre-trained imagenet model as a representation. The authors point out that different architectures yield similar results for their analysis, however it is not clear how the biases of the learned representations affect the results. The use of learned representations needs more rigorous justification
-The evaluation for discriminative metric, increased score when mix of real and unreal increases, is interesting but it is not convincing as the sole evaluation for “discriminativeness” and seems like something that can be gamed.
- The authors implicitly contradict the argument of Theis et al against monolithic evaluation metrics for generative models, but this is not strongly supported.
Several references I suggest:
https://arxiv.org/abs/1706.08500 (FID score)
https://arxiv.org/abs/1511.04581 (MMD as evaluation)


In recent work, it was shown that combining multi-kernel based support vector machines (SVMs) can lead to near state-of-the-art performance on an action recognition dataset (HMDB-51 dataset). In the present work, we show that combining distributed Gaussian Processes with multi-stream deep convolutional neural networks (CNN) alleviate the need to augment a neural network with handcrafted features. In contrast to prior work, we treat each deep neural convolutional network as an expert wherein the individual predictions (and their respective uncertainties) are combined into a Product of Experts (PoE) framework.

This paper, although titled "Distributed Non-Parametric Deep and Wide Networks", is mostly about fusion of existing models for action recognition on the HMDB51 dataset. The fusion is performed with Gaussian Processes, where each of the i=1,..,4 inspected models (TSN-Inception RGB, TSN-Inception Flow, ResNet-LSTM RGB, ResNet-LSTM Flow) returns a (\mu_i, \sigma_i), which are then combined in a product of experts formulation, optimized w.r.t. maximum likelihood.
At its current form this paper is unfit for submission. First, the novelty of the paper is not clear. It is stated that a framework is introduced for independent deep neural networks. However, this framework, the Gaussian Processes, already exists. Also, it is stated that the method can classify video snippets that have heterogeneity regarding camera angle, video quality, pose, etc. This is something characterizes also all other methods that report similar results on the same dataset. The third claim is that deep networks are combined with non-parameteric Bayesian models. That is a good claim, which is also shared between papers at http://bayesiandeeplearning.org/. The last claim is that model averaging taking into account uncertainty is shown to be useful. That is not true, the only result are the final accuracies per GP model, there is no experiment that directly reports any results regarding uncertainty and its contribution to the final accuracy.
Second, it is not clear that the proposed method is the one responsible for the reported improvements in the experiments. Currently, the training set is split into 7 sets, and each set is used to train 4 models, totalling 28 GP experts. It is unclear what new is learned by the 7 GP expert models for the 7 splits. Why is this better than training a single model on the whole dataset? Also, why is difference bigger between ResNet Fusion-1 and Resnet SVM-SingleKernel?
Third, the method reports results only on a single dataset, HMDB51, which is also rather small. Deriving conclusions from results on a single dataset is suboptimal. Other datasets that can be considered are (mini) Kinetics or Charades.
Forth,  the paper does not have the structure of a scientific publication. It rather looks like an unofficial technical report. There is no related work. The methodology section reads more like a tutorial of existing methods. And the discussion section is larger than any other section in the paper.
All in all, there might be some interesting ideas in the paper, specifically how to integrate GPs with deep nets. However, at the current stage the submission is not ready for publication.


Deep reinforcement learning (DRL) algorithms have demonstrated progress in learning to find a goal in challenging environments. As the title of the paper by Mirowski et al. (2016) suggests, one might assume that DRL-based algorithms are able to "learn to navigate" and are thus ready to replace classical mapping and path-planning algorithms, at least in simulated environments. Yet, from experiments and analysis in this earlier work, it is not clear what strategies are used by these algorithms in navigating the mazes and finding the goal. In this paper, we pose and study this underlying question: are DRL algorithms doing some form of mapping and/or path-planning? Our experiments show that the algorithms are not memorizing the maps of mazes at the testing stage but, rather, at the training stage. Hence, the DRL algorithms fall short of qualifying as mapping or path-planning algorithms with any reasonable definition of mapping. We extend the experiments in Mirowski et al. (2016) by separating the set of training and testing maps and by a more ablative coverage of the space of experiments. Our systematic experiments show that the NavA3C+D 1 D 2 L algorithm, when trained and tested on the same maps, is able to choose the shorter paths to the goal. However, when tested on unseen maps the algorithm utilizes a wall-following strategy to find the goal without doing any mapping or path planning.

Science is about reproducible results and it is very commendable from scientists to hold their peers accountable for their work by verifying their results. It is also necessary to inspect claims that are made by researchers to avoid the community straying in the wrong direction. However, any critique needs to be done properly, by 1) attending to the actual claims that were made in the first place, by 2) reproducing the results in the same way as in the original work, 3) by avoiding introducing false claims based on a misunderstanding of terminology and 4) by extensively researching the literature before trying to affirm that a general method (here, Deep RL) cannot solve certain tasks.
This paper is a critique of deep reinforcement learning methods for learning to navigate in 3D environments, and seems to focus intensively on one specific paper (Mirowski et al, 2016, “Learning to Navigate in Complex Environments”) and one of the architectures (NavA3C+D1D2L) from that paper. It conducts an extensive assessment of the methods in the critiqued paper but does not introduce any alternative method. For this reason, I had to carefully re-read the critiqued paper to be able to assess the validity of the arguments made in this submission and to evaluate its merit from the point of view of the quality of the critique. The (Mirowski et al, 2016) paper shows that a neural network-based agent with LSTM-based memory and auxiliary tasks such as depth map prediction can learn to navigate in fixed environments (3D mazes) with a fixed goal position (what they call “static maze”), and in fixed mazes with changing goal environments (what they call “environments with dynamic elements” or “random goal mazes”).
This submission claims that:
[a] “[based on the critiqued paper] one might assume that DRL-based algorithms are able to 'learn to navigate' and are thus ready to replace classical mapping and path-planning algorithms”,
[b] “following training and testing on constant map structures, when trained and tested on the same maps, [the NavA3C+D1D2L algorithm] is able to choose the shorter paths to the goal”,
[c] “when tested on unseen maps the algorithm utilizes a wall-following strategy to find the goal without doing any mapping or path planning”,
[d] “this state-of-the-art result is shown to be successful on only one map, which brings into question the repeatability of the results”,
[e] “Do DRL-based navigation algorithms really 'learn to navigate'? Our results answer this question negatively.”
[f] “we are the first to evaluate any DRL-based navigation method on maps with unseen structures”
The paper also conducts an extensive analysis of the performance of a different version of the NavA3C+D1D2L algorithm (without velocity inputs, which probably makes learning path integration much more difficult), in the same environments but by introducing unjustified changes (e.g., with constant velocities and a different action space) and with a different reward structure (incorporating a negative reward for wall collisions). While the experimental setup does not match (Mirowski et al, 2016), thereby invalidating claim [d], the experiments are thorough and do show that that architecture does not generalize to unseen mazes. The use of attention heat maps is interesting.
The main problem however is that it seems that this submission completely misrepresents the intent of (Mirowski et al, 2016) by using a straw man argument, and makes a rather unacademic and unsubstantiated accusation of lack of repeatability of the results.
Regarding the former, I could not find any claim that the methods in (Mirowski et al, 2017) learn mapping and path planning in unseen environments, that could support claim [a]. More worryingly, when observing that the method of (Mirowski et al, 2017) may not generalize to unseen environments in claim [c], the authors of this submission seem to confuse navigation, cartography and SLAM, and attribute to that work claims that were never made in the first place, using a straw man argument. Navigation is commonly defined as the goal driven control of an agent, following localization, and is a broad skill that involves the determination of position and direction, with or without a map of the environment (Fox 1998, ” Markov Localization: A Probabilistic Framework for Mobile Robot Localization and Navigation”). This widely accepted definition of navigation does not preclude being limited to known environments only.
Regarding repeatability, the claim [d] is contradicted in section 5 when the authors demonstrate that the NavA3C+D1D2L algorithm does achieve a reduction in latency to goal in 8 out of 10 experiments on random goal, static map and random or static spawns. The experiments in section 5.3 are conducted in simple but previously unseen maps and cannot logically contradict results (Mirowski et al, 2016) achieved by training on static maps such as their “I-maze”. Moreover, claim [d] about repeatability is also invalidated by the fact that the experiments described in the paper use different observations (no velocity inputs), different action space, different reward structure, with no empirical evidence to support these changes. It seems, as the authors also claim in [b], that the work of (Mirowski et al, 2017), which was about navigation in known environments, actually is repeatable.
Additionally, some statements made by the authors are demonstrably untrue. First, the authors claim that they are the first to train DRL agents in all random mazes [f], but this has been already shown in at least two publications (Mnih et al, 2016 and Jaderberg et al, 2016).
Second, the title of the submission, “Do Deep Reinforcement Learning Algorithms Really Learn to Navigate” makes a broad statement [e] that cannot be logically invalidated by only one particular set of experiments on a particular model and environment, particularly since it directly targets one specific paper (out of several recent papers that have addressed navigation) and one specific architecture from that paper, NavA3C+D1D2L (incidentally, not the best-performing one, according to table 1 in that paper). Why did the authors not cite and consider (Parisotto et al, 2017, “Neural Map: Structured Memory for Deep Reinforcement Learning”), which explicitly claims that their method is “capable of generalizing to environments that were not seen during training”? It seems that the authors need to revise both their bibliography and their logical reasoning: one cannot invalidate a broad set of algorithms for a broad goal, simply by taking a specific example and showing that it does not fit a particular interpretation of navigation *in previously unseen environments*.


Learning high-quality word embeddings is of significant importance in achieving better performance in many down-stream learning tasks. On one hand, traditional word embeddings are trained on a large scale corpus for general-purpose tasks, which are often sub-optimal for many domain-specific tasks. On the other hand, many domain-specific tasks do not have a large enough domain corpus to obtain high-quality embeddings. We observe that domains are not isolated and a small domain corpus can leverage the learned knowledge from many past domains to augment that corpus in order to generate high-quality embeddings. In this paper, we formulate the learning of word embeddings as a lifelong learning process. Given knowledge learned from many previous domains and a small new domain corpus, the proposed method can effectively generate new domain embeddings by leveraging a simple but effective algorithm and a meta-learner, where the metalearner is able to provide word context similarity information at the domain-level. Experimental results demonstrate that the proposed method can effectively learn new domain embeddings from a small corpus and past domain knowledges
1 . We also demonstrate that general-purpose embeddings trained from a large scale corpus are sub-optimal in domain-specific tasks.

This paper presents a lifelong learning method for learning word embeddings.  Given a new domain of interest, the method leverages previously seen domains in order to hopefully generate better embeddings compared to ones computed over just the new domain, or standard pre-trained embeddings.
The general problem space here -- how to leverage embeddings across several domains in order to improve performance in a given domain -- is important and relevant to ICLR.  However, this submission needs to be improved in terms of clarity and its experiments.
In terms of clarity, the paper has a large number of typos (I list a few at the end of this review) and more significantly, at several points in the paper is hard to tell what exactly was done and why.  When presenting algorithms, starting with an English description of the high-level goal and steps of the algorithm would be helpful.  What are the inputs and outputs of the meta-learner, and how will it be used to obtain embeddings for the new domain?  The paper states the purpose of the meta learning is "to learn a general word context similarity from the first m domains", but I was never sure what this meant.  Further, some of the paper's pseudocode includes unexplained steps like "invert by domain index" and "scanco-occurrence".
In terms of the experiments, the paper is missing some important baselines that would help us understand how well the approach works.  First, besides the GloVe common crawl embeddings used here, there are several other embedding sets (including the other GloVe embeddings released along with the ones used here, and the Google News word2vec embeddings) that should be considered.  Also, the paper never considers concatenations of large pre-trained embedding sets with each other and/or with the new domain corpus -- such concatenations often give a big boost to accuracy, see :
"Think Globally, Embed Locally—Locally Linear Meta-embedding of Words", Bollegala et al., 2017
https://arxiv.org/pdf/1709.06671.pdf
That paper is not peer reviewed to my knowledge so it is not necessary to compare against the new methods introduced there, but their baselines of concatenation of pre-trained embedding sets should be compared against in the submission.
Beyond trying other embeddings, the paper should also compare against simpler combination approaches, including simpler variants of its own approach.  What if we just selected the one past domain that was most similar to the new domain, by some measure?  And how does the performance of the technique depend on the setting of m?  Investigating some of these questions would help us understand how well the approach works and in which settings.
Minor:
Second paragraph, GloVec should be GloVe
"given many domains with uncertain noise for the new domain" -- not clear what "uncertain noise" means, perhaps "uncertain relevance" would be more clear
The text refers to a Figure 3 which does not exist, probably means Figure 2.  I didn't understand the need for both figures, Figure 1 is almost contained within Figure 2
When m is introduced, it would help to say that m < n and justify why dividing the n domains into two chunks (of m and n-m domains) is necessary.
"from the first m domain corpus" -> "from the first m domains"?
"may not helpful" -> "may not be helpful"
"vocabularie" -> "vocabulary"
"system first retrieval" -> "system first retrieves"
COMMENTS ON REVISIONS: I appreciate the authors including the new experiments against concatenation baselines.  The concatenation does fairly comparably to LL in Tables 3&4.  LL wins by a bit more in Table 2.  Given these somewhat close/inconsistent wins, it would help the paper to include an explanation of why and under what conditions the LL approach will outperform concatenation.


Clustering high-dimensional datasets is hard because interpoint distances become less informative in high-dimensional spaces. We present a clustering algorithm that performs nonlinear dimensionality reduction and clustering jointly. The data is embedded into a lower-dimensional space by a deep autoencoder. The autoencoder is optimized as part of the clustering process. The resulting network produces clustered data. The presented approach does not rely on prior knowledge of the number of ground-truth clusters. Joint nonlinear dimensionality reduction and clustering are formulated as optimization of a global continuous objective. We thus avoid discrete reconfigurations of the objective that characterize prior clustering algorithms. Experiments on datasets from multiple domains demonstrate that the presented algorithm outperforms state-of-the-art clustering schemes, including recent methods that use deep networks.

As authors stated, the proposed DCC is very similar to RCC-DR (Shah & Koltun, 2007). The only difference in (3) from RCC-DR is the decoding part, which is replaced by autoencoder instead of linear transformation used in RCC-DR. Authors claimed that there are three major differences. However, due to the highly nonconvex properties of both formulations, the last two differences hardly support the advantages of the proposed DCC comparing with RCC-DR because the solutions obtained by both optimization approaches are local solutions, unless authors can claim that the gradient-based solver is better than alternating approach in RCC-DR. Hence, DCC is just a simple extension of RCC-DR.
In Section 3.2, how does the optimization algorithm handle the equality constraints in (5)? It is unclear why the existing autoencoder solver can be used to solve (3) or (5). It seems that the first term in (5) corresponds to the objective of autoencoder, but the last two terms added lead to different objective with respect to variables y. It is better to clarify the correctness of the optimization algorithm.
Authors claimed that the proposed method avoid discrete reconfiguration of the objective that characterize prior clustering algorithms, and it does not rely on a priori knowledge of the number of ground-truth clusters. However, it seems not true since the graph construction at every epoch depends on the initial parameter delta_2 and the graph is constructed such that f_{i,j}=1 if distance is less than delta_2. As a result, delta_2 is a fixed threshold for graph construction, so it is indirectly related to the number of clusters generated. In the experiments, authors set it as the mean of the bottom 1% of the pairwise distances in E at initialization, and clustering assignment is given by connected component in the last graph. This parameter might be sensitive to the final results.
Many terms in the paper are not well explained. For example, in (1), theta are treated as parameters to optimize, but what is the theta used for? Does the Omega related to encoder and decoder of the parameters in autoencoder. What is the scaled Geman-McClure function? Any reference? Why should this estimator be used?
From the visualization results in Figure 1, it is interesting to see that K-means++ can achieve much better results on the space learned by DCC than that by SDAE from Table 2. In Figure 1, the embedding by SDAE (Figure 1(b)) seems more suitable for kmeans-like algorithm than DCC (Figure 1(c)). That is the reason why connected component is used for cluster assignment in DCC, not kmeans. The results between Table 2 and Figure 1 might be interesting to investigate.


This is an empirical paper which constructs color invariant networks and evaluates their performances on a realistic data set. The paper studies the simplest possible case of color invariance: invariance under pixel-wise permutation of the color channels. Thus the network is aware not of the specific color object, but its colorfulness. The data set introduced in the paper consists of images showing crashed cars from which ten classes were extracted. An additional annotation was done which labeled whether the car shown was red or non-red. The networks were evaluated by their performance on the classification task. With the color annotation we altered the color ratios in the training data and analyzed the generalization capabilities of the networks on the unaltered test data. We further split the test data in red and non-red cars and did a similar evaluation. It is shown in the paper that an pixel-wise ordering of the rgb-values of the images performs better or at least similarly for small deviations from the true color ratios. The limits of these networks are also discussed.

The paper proposes and evaluates a method to make neural networks for image recognition color invariant.
The contribution of the paper is:
 - some proposed methods to extract a color-invariant representation
 - an experimental evaluation of the methods on the cifar 10 dataset
 - a new dataset "crashed cars"
 - evaluation of the best method from the cifar10 experiments on the new dataset
Pros:
 - the crashed cars dataset is interesting. The authors have definitely found an interesting untapped source of interesting images.
Cons:
- The authors name their method order network but the method they propose is not really parts of the network but simple preprocessing steps to the input of the network.
- The paper is incomplete without the appendices. In fact the paper is referring to specific figures in the appendix in the main text.
 - the authors define color invariance as a being invariant to which specific color an object in an image does have, e.g. whether a car is red or green, but they don't think about color invariance in the broader context - color changes because of lighting, shades, ..... Also, the proposed methods aim to preserve the "colorfullness" of a color. This is also problematic, because while the proposed method works for a car that is green or a car that is red, it will fail for a car that is black (or white) - because in both cases the "colorfulness" is not relevant. Note that this is specifically interesting in the context of the task at hand (cars) and many cars being, white, grey (silver), or black.
- the difference in the results in table 1 could well come from the fact that in all of the invariant methods except for "ord" the input is a WxHx1 matrix, but for "ord" and "cifar" the input is a "WxHx3" matrix. This probably leads to more parameters in the convolutions.
- the results in the  figure 4: it's very unlikely that the differences reported are actually significant. It appears that all methods perform approximately the same - and the authors pick a specific line (25k steps) as the relevant one in which the RGB-input space performs best. The proposed method does not lead to any relevant improvement.
Figure 6/7: are very hard to read. I am still not sure what exactly they are trying to say.
Minor comments:
 - section 1: "called for is network" -> called for is a network
 - section 1.1: And and -> And
 - section 1.1: Appendix -> Appendix C
 - section 2: Their exists many -> There exist many
 - section 2: these transformation -> these transformations
 - section 2: what does "the wallpaper groups" refer to?
 - section 2: are a groups -> are groups
 - section 3.2: reference to a non-existing figure
 - section 3.2/Training: 2499999 iterations = steps?
 - section 3.2/Training: longer as suggested -> longer than suggested


Much recent research has been devoted to video prediction and generation, yet a lot of previous work has been focused on short-scale time horizons. The hierarchical video prediction method by Villegas et al. (2017) is an example of a state of the art method for long term video prediction. However, their method has limited applicability in practical settings as it requires a ground truth pose (e.g., poses of joints of a human) at training time. This paper presents a long term hierarchical video prediction model that does not have such a restriction. We show that the network learns its own higher level structure (e.g., pose-equivalent hidden variables) that works better in cases where the ground truth pose does not fully capture all of the information needed to predict the next frame. This method gives sharper results than other video prediction methods which do not require a ground truth pose, and its efficiency is shown on the Humans 3.6M and Robot Pushing datasets.

The paper presents a method for hierarchical future frame prediction in monocular videos. It builds upon the recent method of Villegas et al. 2017, which generates future RGB frames in two stages: in the first stage, it predicts a human body pose sequence, then it conditions on the pose sequence to predict RGB content, using an image analogy network. This current paper, does not constrain the first stage (high level) prediction to be human poses, but instead it can be any high level representation. Thus, the method does not require human annotations.
The method has the following two sub-networks:
1) An image encoder, that given an RGB image, predicts a deep feature encoding.
2) An LSTM predictor, that conditioned on the last observed frame's encoding,  predicts future high level structure p_t. Once enough frames are generated though, it conditions on its own predictions.
3) A visual analogy network (VAN), that given predicted high level structure p_t, it predicts the pixel image I_t, by applying the transformation from the first to tth frame, as computed by the vector subtraction of the corresponding high level encodings (2nd equation of the paper). VAN is trained to preserve parallelogram relationships in the joint RGB image and high level structure embedding.
The authors experiment with  many different neural network connectivities, e.g., not constraining the predicted high level structure to match the encoder's outputs, constraining the predicted high level structure to match the encoder's output (EPEV), and training together the VAN  and predictor so that VAN can tolerate mistakes of the predictor. Results are shown in H3.6m and the pushobject datasets, and are compared against the method of Villegas et all (INDIVIDUAL). The conclusion seems to be that not constraining the predicted high level structure to match the encoder’s output, but biasing the encoder’s output in the observed frames to represent ground-truth pose information, gives the best results.
Pros
1) Interesting alternative training schemes are tested
Cons:
1)Numerous English mistakes, e.g.,  ''an intelligent agents", ''we explore ways generate" etc.
2) Equations are not numbered (and thus is hard to refer to them.) E.g., i do not understand the first equation, shouldn’t it be that e_{t-1} is always fixed and equal to the encoding of the last observed (not predicted) frame? Then the subscript cannot be t-1.
3) In H3.6M, the results are only qualitative. The conclusions from the paper are uncertain, partly  due to the difficulty of evaluating the video prediction results.
Given the difficulty of assessing the experimental results quantitatively (one possibility to do so is asking a set of people of which one they think is the most plausible video completion), and given the limited novelty of the paper, though interesting alternative architectures are tried out, it may not be suitable to be part of  ICLR proceedings as a conference paper.


Vector semantics, especially sentence vectors, have recently been used successfully in many areas of natural language processing. However, relatively little work has explored the internal structure and properties of spaces of sentence vectors. In this paper, we will explore the properties of sentence vectors by studying a particular real-world application: Automatic Summarization. In particular, we show that cosine similarity between sentence vectors and document vectors is strongly correlated with sentence importance and that vector semantics can identify and correct gaps between the sentences chosen so far and the document. In addition, we identify specific dimensions which are linked to effective summaries. To our knowledge, this is the first time specific dimensions of sentence embeddings have been connected to sentence properties. We also compare the features of different methods of sentence embeddings. Many of these insights have applications in uses of sentence embeddings far beyond summarization.

This paper examines a number of sentence and document embedding methods for automatic summarization. It pairs a number of recent sentence embedding algorithms (e.g., Paragraph Vectors and Skip-Thought Vectors) with several simple summarization decoding algorithms for sentence selection, and evaluates the resulting output summary on DUC 2004 using ROUGE, based on the general intuition that the selected summary should be similar to the original document in the vector space induced by the embedding algorithm. It further provides a number of analyses of the sentence representations as they relate to summarization, and other aspects of the summarization process including the decoding algorithm.
The paper was well written and easy to understand. I appreciate the effort to apply these representation techniques in an extrinsic task.
However, the signficance of the results may be limited, because the paper does not respond to a long line of work in summarization literature which have addressed many of the same points. In particular, I worry that the paper may in part be reinventing the wheel, in that many of the results are quite incremental with respect to previous observations in the field.
Greedy decoding and non-redundancy: many methods in summarization use greedy decoding algorithms. For example, SumBasic (Nenkova and Vanderwende, 2005), and HierSum (Haghighi and Vanderwende, 2009) are two such papers. This specific topic has been thoroughly expanded on by the work on greedy decoding for submodular objective functions in summarization (Lin and Bilmes, 2011), as well as many papers which focus on how to optimize for both informativeness and non-redundancy (Kulesza and Taskar, 2012).

The idea that the summary should be similar to the entire document is known as centrality. Some papers that exploit or examine that property include (Nenkova and Vanderwende, 2005; Louis and Nenkova, 2009; Cheung and Penn, 2013)

Another possible reading of the paper is that its novelty lies in the evaluation of sentence embedding models, specifically. However, these methods were not designed for summarization, and I don't see why they should necessarily work well for this task out of the box with simple decoding algorithms without finetuning. Also, the ROUGE results are so far from the SotA that I'm not sure what the value of analyzing this suite of techniques is.

In summary, I understand that this paper does not attempt to produce a state-of-the-art summarization system, but I find it hard to understand how it contributes to our understanding of future progress in the summmarization field. If the goal is to use summarization as an extrinsic evaluation of sentence embedding models, there needs to be better justification of this is a good idea when there are so many other issues in content selection that are not due to sentence embedding quality, but which affect summarization results.
References:
Nenkova and Vanderwende, 2005. The impact of frequency on summarization. Tech report.
Haghighi and Vanderwende, 2009. Exploring content models for multi-document summarization. NAACL-HLT 2009.
Lin and Bilmes, 2011. A class of submodular functions for document summarization. ACL-HLT 2011.
Kulesza and Taskar, 2012. Learning Determinantal Point Processes.
Louis and Nenkova, 2009. Automatically evaluating content selection in summarization without human models. EMNLP 2009.
Cheung and Penn, 2013. Towards Robust Abstractive Multi-Document Summarization: A Caseframe Analysis of Centrality and Domain. ACL 2013.
Other notes:
The acknowledgements seem to break double-blind reviewing.


Published as a conference paper at ICLR 2018 MEASURING THE INTRINSIC DIMENSION OF OBJECTIVE LANDSCAPES
Many recently trained neural networks employ large numbers of parameters to achieve good performance. One may intuitively use the number of parameters required as a rough gauge of the difficulty of a problem. But how accurate are such notions? How many parameters are really needed? In this paper we attempt to answer this question by training networks not in their native parameter space, but instead in a smaller, randomly oriented subspace. We slowly increase the dimension of this subspace, note at which dimension solutions first appear, and define this to be the intrinsic dimension of the objective landscape. The approach is simple to implement, computationally tractable, and produces several suggestive conclusions. Many problems have smaller intrinsic dimensions than one might suspect, and the intrinsic dimension for a given dataset varies little across a family of models with vastly different sizes. This latter result has the profound implication that once a parameter space is large enough to solve a problem, extra parameters serve directly to increase the dimensionality of the solution manifold. Intrinsic dimension allows some quantitative comparison of problem difficulty across supervised, reinforcement, and other types of learning where we conclude, for example, that solving the inverted pendulum problem is 100 times easier than classifying digits from MNIST, and playing Atari Pong from pixels is about as hard as classifying CIFAR-10. In addition to providing new cartography of the objective landscapes wandered by parameterized models, the method is a simple technique for constructively obtaining an upper bound on the minimum description length of a solution. A byproduct of this construction is a simple approach for compressing networks, in some cases by more than 100 times.

This paper proposes an empirical measure of the intrinsic dimensionality of a neural network problem. Taking the full dimensionality to be the total number of parameters of the network model, the authors assess intrinsic dimensionality by randomly projecting the network to a domain with fewer parameters (corresponding to a low-dimensional subspace within the original parameter), and then training the original network while restricting the projections of its parameters to lie within this subspace. Performance on this subspace is then evaluated relative to that over the full parameter space (the baseline). As an empirical standard, the authors focus on the subspace dimension that achieves a performance of 90% of the baseline. The authors then test out their measure of intrinsic dimensionality for fully-connected networks and convolutional networks, for several well-known datasets, and draw some interesting conclusions.
Pros:
* This paper continues the recent research trend towards a better characterization of neural networks and their performance. The authors show a good awareness of the recent literature, and to the best of my knowledge, their empirical characterization of the number of latent parameters is original.
* The characterization of the number of latent variables is an important one, and their measure does perform in a way that one would intuitively expect. For example, as reported by the authors, when training a fully-connected network on the MNIST image dataset, shuffling pixels does not result in a change in their intrinsic dimensionality. For a convolutional network the observed 3-fold rise in intrinsic dimension is explained by the authors as due to the need to accomplish the classification task while respecting the structural constraints of the convnet.
* The proposed measures seem very practical - training on random projections uses far fewer parameters than in the original space (the baseline), and presumably the cost of determining the intrinsic dimensionality would presumably be only a fraction of the cost of this baseline training.
* Except for the occasional typo or grammatical error, the paper is well-written and organized. The issues are clearly identified, for the most part (but see below...).
Cons:
* In the main paper, the authors perform experiments and draw conclusions without taking into account the variability of performance across different random projections. Variance should be taken into account explicitly, in presenting experimental results and in the definition and analysis of the empirical intrinsic dimension itself. How often does a random projection lead to a high-quality solution, and how often does it not?
* The authors are careful to point out that training in restricted subspaces cannot lead to an optimal solution for the full parameter domain unless the subspace intersects the optimal solution region (which in general cannot be guaranteed). In their experiments (FC networks of varying depths and layer widths for the MNIST dataset), between projected and original solutions achieving 90% of baseline performance, they find an order of magnitude gap in the number of parameters needed. This calls into question the validity of random projection as an empirical means of categorizing the intrinsic dimensionality of a neural network.
* The authors then go on to propose that compression of the network be achieved by random projection to a subspace of dimensionality greater than or equal to the intrinsic dimension. However, I don't think that they make a convincing case for this approach. Again, variation is the difficulty: two different projective subspaces of the same dimensionality can lead to solutions that are extremely different in character or quality. How then can we be sure that our compressed network can be reconstituted into a solution of reasonable quality, even when its dimensionality greatly exceeds the intrinsic dimension?
* The authors argue for a relationship between intrinsic dimensionality and the minimum description length (MDL) of their solution, in that the intrinsic dimensionality should serve as an upper bound on the MDL. However they don't formally acknowledge that there is no standard relationship between the number of parameters and the actual number of bits needed to represent the model - it varies from setting to setting, with some parameters potentially requiring many more bits than others. And given this uncertain connection, and given the lack of consideration given to variation in the proposed measure of intrinsic dimensionality, it is hard to accept that "there is some rigor behind" their conclusion that LeNet is better than FC networks for classification on MNIST because its empirical intrinsic dimensionality score is lower.
* The experimental validation of their measure of intrinsic dimension could be made more extensive. In the main paper, they use three image datasets - MNIST, CIFAR-10 and ImageNet. In the supplemental information, they report intrinsic dimensions for reinforcement learning and other training tasks on four other sets.
Overall, I think that this characterization does have the potential to give insights into the performance of neural networks, provided that variation across projections is properly taken into account. For now, more work is needed.
====================================================================================================
Addendum:
The authors have revised their paper to take into account the effect of variation across projections, with results that greatly strengthen their results and provide a much better justification of their approach. I'm satisfied too with their explanations, and how they incorporated them into their revised version. I've adjusted my rating of the paper accordingly.
One point, however: the revisions seem somewhat rushed, due to the many typos and grammatical errors in the updated sections. I would like to encourage the authors to check their manuscript once more, very carefully, before finalizing the paper.
====================================================================================================


Sequence-to-sequence (Seq2Seq) models with attention have excelled at tasks which involve generating natural language sentences such as machine translation, image captioning and speech recognition. Performance has further been improved by leveraging unlabeled data, often in the form of a language model. In this work, we present the Cold Fusion method, which leverages a pre-trained language model during training, and show its effectiveness on the speech recognition task. We show that Seq2Seq models with Cold Fusion are able to better utilize language information enjoying i) faster convergence and better generalization, and ii) almost complete transfer to a new domain while using less than 10% of the labeled training data.

This paper present a simple but effective approach to utilize language model information in a seq2seq framework. The experimental results show improvement for both baseline and adaptation scenarios.
Pros:
The approach is adapted from deep fusion but the results are promising, especially for the off-domain setup. The analysis also well-motivated about why cold-fusion outperform deep-fusion.
Cons:
(1) I have some question about the baseline. Why the decoder is single layer but for LM it is 2 layer? I suspect the LM may add something to it.  For my own Seq2seq model, 2 layer decoder always better than one. Also, what is HMM/DNN/CTC baseline ? Since they use a internal dataset, it's hard to know how was the seq2seq numbers. The author also didn't compare with re-scoring method.
(2) It would be more interesting to test it on more standard speech corpus, for example, SWB (conversational based) and librispeech (reading task). Then it's easier to reproduce and measure the quality of the model.
(3) This paper only report results on speech recognition. It would be more interesting to test it on more area, e.g. Machine Translation.
Missing citation: In (https://arxiv.org/pdf/1706.02737.pdf) section 3.3, they also pre-trained RNN-LM on more standard speech corpus. Also, need to compare with this type of shallow fusion.
Updates:
https://arxiv.org/pdf/1712.01769.pdf (Google's End2End system) use 2-layer LSTM decoder.
https://arxiv.org/abs/1612.02695,  https://arxiv.org/abs/1707.07413 and https://arxiv.org/abs/1506.07503) are small task.
Battenberg et al. paper (https://arxiv.org/abs/1707.07413) use Seq2Seq as a baseline and didn't show any combined results of different #decoder layer vs. different LM integration method. My point is how a stronger decoder affect the results with different LM integration methods. In the paper, it still only compared with deep fusion with one decoder layer.
Also, why it only compared shallow fusion w/ CTC model? I suspect deep decoder + shallow fusion already could provide good results. Or the gain is additive?
Thanks a lot adding Librispeech results. But why use Wav2Letter paper (instead of refer to a peer reviewed paper)? The Wav2letter paper didn't compare with any baseline on librispeech (probably because librispeech isn't a common dataset, but at least the Kaldi baseline is there).
In short, I'm still think this is a good paper but still slightly below the acceptance threshold.