Workshop track -ICLR 2018 ENSEMBLE ROBUSTNESS AND GENERALIZATION OF STOCHASTIC DEEP LEARNING ALGORITHMS
The question why deep learning algorithms generalize so well has attracted increasing research interest. However, most of the well-established approaches, such as hypothesis capacity, stability or sparseness, have not provided complete explanations (Zhang et al., 2017;Kawaguchi et al., 2017). In this work, we focus on the robustness approach (Xu & Mannor, 2012), i.e., if the error of a hypothesis will not change much due to perturbations of its training examples, then it will also generalize well. As most deep learning algorithms are stochastic (e.g., Stochastic Gradient Descent, Dropout, and Bayes-by-backprop), we revisit the robustness arguments of Xu & Mannor, and introduce a new approach -ensemble robustness -that concerns the robustness of a population of hypotheses. Through the lens of ensemble robustness, we reveal that a stochastic learning algorithm can generalize well as long as its sensitiveness to adversarial perturbations is bounded in average over training examples. Moreover, an algorithm may be sensitive to some adversarial examples (Goodfellow et al., 2015) but still generalize well. To support our claims, we provide extensive simulations for different deep learning algorithms and different network architectures exhibiting a strong correlation between ensemble robustness and the ability to generalize.

Summary:
This paper presents an adaptation of the algorithmic robustness of Xu&Mannor'12 to a notion robustness of ensemble of hypothesis allowing the authors to study generalization ability of stochastic learning algorithms for Deep Learning Networks.
Generalization can be established as long as the sensitiveness of the learning algorithm to adversarial perturbations is bounded.
The paper presents learning bounds and an experimental showing correlation between empirical ensemble robustness and generalization error.
Quality:
Globally correct
Clarity:
Paper clear
Originality:
Limited with respect to the original definition of algorithmic robustness
Significance:
The paper provides a new theoretical analysis for stochastic learning of Deep Networks but the contribution is limited in its present form.
Pros:
-New theoretical study for DL algorithms
-Focus on adversarial learning
Cons
-I find the contribution a bit limited
-Some aspects have to be precised/more argumented
-Experimental study could have been more complete
Comments:
---------
*About the proposed framework.
The idea of taking a max over instances of partition C_i (Def 3) already appeared in the proof of results of Xu&Mannor, and the originality of the contribution is essentially to add an expectation over the result of the algorithm.
In Xu&Mannor paper, there is a notion of weak robustness that is proved to be necessary and sufficient to generalize. The contribution of the authors would be stronger if they can discuss an equivalent notion in their context.
The partition considered by the framework is never discussed nor taken into account, while this is an important aspect of the analysis. In particular, there is a tradeoff between \epsilon(s) and K:  using a very fine tiling it is always possible to have a very small \epsilon(s) at the price of a very large K (if you think of a covering number, K can be exponential in the size of the tiling and hard to calculate).
In the context of adversarial examples, this is actually important because it can be very likely that the adversarial example can belong to a partition set different from the set the original example belong to.
In this context, I am not sure to understand the validity of the framework because we can then compare 2 instances of different set which is outside of the framework.
So I wonder if the way the adervarial examples are generates should be taken into account for the definition of the partition.
Additionnally, the result is given in the contect of IID data, and with a multinomial distribution according to the partition set - adversarial generation can violate this IID assumption.
In the experimental setup, the partition set is not explained and we have no guarantee to compare instances of the same set. Nothing is said about $r$ and its impact on the results. This is a clear weak aspect of the experimental analysis
In the experimental setup, as far as I understand the setup, I find the term "generalization error" a bit abusive since it is actually the error on the test set.
Using cross validation or considering multiple training/test sets would be more appropriate.
In the proof of Lemma 2, I am not sure to understand where the term 1/n comes from in the term 2M^2/2 (before "We then bound the term H as follows")


The success of Deep Learning and its potential use in many important safetycritical applications has motivated research on formal verification of Neural Network (NN) models. Despite the reputation of learned NN models to behave as black boxes and the theoretical hardness of proving their properties, researchers have been successful in verifying some classes of models by exploiting their piecewise linear structure. Unfortunately, most of these approaches test their algorithms without comparison with other approaches. As a result, the pros and cons of the different algorithms are not well understood. Motivated by the need to accelerate progress in this very important area, we investigate the trade-offs of a number of different approaches based on Mixed Integer Programming, Satisfiability Modulo Theory, as well as a novel method based on the Branch-and-Bound framework. We also propose a new data set of benchmarks, in addition to a collection of previously released testcases that can be used to compare existing methods. Our analysis not only allows a comparison to be made between different strategies, the comparison of results from different solvers also revealed implementation bugs in published methods. We expect that the availability of our benchmark and the analysis of the different approaches will allow researchers to develop and evaluate promising approaches for making progress on this important topic.

The paper compares some recently proposed method for validation of properties
of piece-wise linear neural networks and claims to propose a novel method for
the same. Unfortunately, the proposed "branch and bound method" does not explain
how to implement the "bound" part ("compute lower bound") -- and has been used
several times in the same application, incl.:
Ruediger Ehlers. Planet. https://github.com/progirep/planet,
Chih-Hong Cheng, Georg Nuhrenberg, and Harald Ruess.  Maximum resilience of artificial neural networks. Automated Technology for Verification and Analysis
Alessio Lomuscio and Lalit Maganti.  An approach to reachability analysis for feed-forward relu neural networks. arXiv:1706.07351
Specifically, the authors say: "In our experiments, we use the result of
minimising the variable corresponding to the output of the network, subject
to the constraints of the linear approximation introduced by Ehlers (2017a)"
which sounds a bit like using linear programming relaxations, which is what
the approaches using branch and bound cited above use. If that is the case,
the paper does not have any original contribution. If that is not the case,
the authors may have some contribution to make, but have not made it in this
paper, as it does not explain the lower bound computation other than the one
based on LPs.
Generally, I find a jarring mis-fit between the motivation (deep learning
for driving, presumably involving millions or billions of parameters) and
the actual reach of the methods proposed (hundreds of parameters).
This reach is NOT inherent in integer programming, per se. Modern solvers
routinely solve instances with tens of millions of non-zeros in the constraint
matrix, but require a strong relaxation. The authors may hence consider
improving the LP relaxation, noting that the big-M constraint are notorious
for producing weak relaxations.


EXPRESSIVE POWER OF RECURRENT NEURAL NET- WORKS
Deep neural networks are surprisingly efficient at solving practical tasks, but the theory behind this phenomenon is only starting to catch up with the practice. Numerous works show that depth is the key to this efficiency. A certain class of deep convolutional networks -namely those that correspond to the Hierarchical Tucker (HT) tensor decomposition -has been proven to have exponentially higher expressive power than shallow networks. I.e. a shallow network of exponential width is required to realize the same score function as computed by the deep architecture. In this paper, we prove the expressive power theorem (an exponential lower bound on the width of the equivalent shallow network) for a class of recurrent neural networks -ones that correspond to the Tensor Train (TT) decomposition. This means that even processing an image patch by patch with an RNN can be exponentially more efficient than a (shallow) convolutional network with one hidden layer. Using theoretical results on the relation between the tensor decompositions we compare expressive powers of the HT-and TT-Networks. We also implement the recurrent TT-Networks and provide numerical evidence of their expressivity.

In this paper, the expressive power of neural networks characterized by tensor train (TT) decomposition, a chain-type tensor decomposition, is investigated. Here, the expressive power refers to the rank of tensor decomposition, i.e., the number of latent components. The authors compare the complexity of TT-type networks with networks structured by CP decomposition, which corresponds to shallow networks. It is proved that the space of TT-type networks with rank O(r)  can be complex as the same as the space of CP-type networks with rank poly(r).
The paper is clearly written and easy to follow.
The contribution is clear and it is distinguished from previous studies.
Though I enjoyed reading this paper, I have several concerns.
1. The authors compare the complexity of TT representation with CP representation (and HT representation). However, CP representation does not have universality (i.e., some tensors cannot be expressed by CP representation with finite rank, see [1]), this comparison may not make sense. It seems the comparison with Tucker-type representation makes much more sense because it has universality.
2. Connecting RNN and TT representation is a bit confusing. Specifically, I found two gaps.
   (a) RNNs reuse the same parameter against all the input x_1 to x_d. This means that G_1 to G_d in Figure 1 are all the same. That's why RNNs can handle size-varying sequences.
   (b) Standard RNNs do not use the multilinear units shown in Figure 3, but use a simple addition of an input and the output from the previous layer (i.e., h_t = f(Wx_t + Vh_{t-1}), where h_t is the t-th hidden unit, x_t is the t-th input, W and V are weights, and f is an activation function.)
Due to the gaps, the analysis used in this paper seems not applicable to RNNs. If this is true, the story of this paper is somewhat misleading. Or, is your theory still applicable?
[1] Hackbusch, Wolfgang. Tensor spaces and numerical tensor calculus. Vol. 42. Springer Science & Business Media, 2012.


Neural embeddings have been used with great success in Natural Language Processing (NLP) where they provide compact representations that encapsulate word similarity and attain state-of-the-art performance in a range of linguistic tasks. The success of neural embeddings has prompted significant amounts of research into applications in domains other than language. One such domain is graph-structured data, where embeddings of vertices can be learned that encapsulate vertex similarity and improve performance on tasks including edge prediction and vertex labelling. For both NLP and graph-based tasks, embeddings in high-dimensional Euclidean spaces have been learned. However, recent work has shown that the appropriate isometric space for embedding complex networks is not the flat Euclidean space, but a negatively curved hyperbolic space. We present a new concept that exploits these recent insights and propose learning neural embeddings of graphs in hyperbolic space. We provide experimental evidence that hyperbolic embeddings significantly outperform Euclidean embeddings on vertex classification tasks for several real-world public datasets.

== Preamble ==
As promised, I have read the updated paper from scratch and this is my revised review. My original review is kept below for reference. My original review had rating "4: Ok but not good enough - rejection".
== Updated review ==
The revised improves upon the original submission in several ways and, in particular, does a much better job at positioning itself within the existing body of literature. The new experiments also indicate that the proposed model offer some improvement over Nickel & Douwe, NIPS 2017).
I do have remaining concerns that unfortunately still prevent me from recommending acceptance:
- Throughout the paper it is argued that we should embed into a hyperbolic space. Such a space is characterized by its metric, but the proposed model do not use a hyperbolic metric. Rather it relies on a heuristic similarity measure that is inspired by the hyperbolic metric. I understand that this may be a practical choice, but then I find it misleading that the paper repeatedly states that points are embedded into a hyperbolic space (which is incorrect). This concern was also raised on this forum prior to the revision.
- The resulting optimization is one of the key selling points of the proposed method as it is unconstrained while Nickel & Douwe resort to a constrained optimization. Clearly unconstrained optimization is to be preferred. However, it is not entirely correct (from what I understand), that the resulting optimization is indeed unconstrained. Nickel & Douwe work under the constraint that |x| < 1, while the proposed model use polar coordinates (r, theta): r in (0, infinity) and theta in (0, 2 pi]. Note that theta parametrize a circle, and therefore wrapping may occur (this should really be mentioned in the paper). The constraints on theta are quite easy to cope with, so I agree with the authors that they have a more simple optimization problem. However, this is only true since points are embedded on the unit disk (2D). Should you want to embed into higher dimensional spaces, then theta need to be confined to live on the unit sphere, i.e. |theta| = 1 (the current setting is just a special-case of the unit sphere). While optimizing over the unit sphere is manageable it is most definitely a constrained optimization problem, and it is far from clear that it is much easier than working under the Nickel & Douwe constraint, |x| < 1.
Other comments:
- The sentence "even infinite trees have nearly isometric embeddings in hyperbolic space (Gromov, 2007)" sounds cool (I mean, we all want to cite Gromov), but what does it really mean? An isometric embedding is merely one that preserves a metric, so this statement only makes sense if the space of infinite trees had a single meaningful metric in the first place (it doesn't; that's a design choice).
- In the "Contribution" and "Conclusion" sections it is claimed that the paper "introduce the new concept of neural embeddings in hyperbolic space". I thought that was what Nickel & Douwe did... I understand that the authors are frustrated by this parallel work, but at this stage, I don't think the present paper can make this "introducing" claim.
- The caption in Figure 2 miss some indication that "a" and "b" refer to subfigures. I recommend "a" --> "a)" and "b" --> "b)".
- On page 4 it is mentioned that under the heuristic similarity measure some properties of hyperbolic spaces are lost while other are retained. From what I can read, it is only claimed that key properties are kept; a more formal argument (even if trivial) would have been helpful.
== Original Review ==
The paper considers embeddings of graph-structured data onto the hyperbolic Poincare ball. Focus is on word2vec style models but with hyperbolic embeddings. I am unable to determine how suitable an embedding space the Poincare ball really is, since I am not familiar enough with the type of data studied in the paper. I have a few minor comments/questions to the work, but my main concern is a seeming lack of novelty:
The paper argues that the main contribution is that this is the first neural embedding onto a hyperbolic space. From what I can see, the paper
  Poincaré Embeddings for Learning Hierarchical Representations
  https://arxiv.org/abs/1705.08039
consider an almost identical model to the one proposed here with an almost identical motivation and application set. Some technicalities appear different, but (to me) it seems like the main claimed novelties of the present paper has already been out for a while. If this analysis is incorrect, then I encourage the authors to provide very explicit arguments for this in the rebuttal phase.
Other comments:
*) It seems to me that, by construction, most data will be pushed towards the boundary of the Poincare ball during the embedding. Is that a property you want?
*) I found it rather surprising that the log-likelihood under consideration was pushed to an appendix of the paper, while its various derivatives are part of the main text. Given the not-so-tight page limits of ICLR, I'd recommend to provide the log-likelihood as part of the main text (it's rather difficult to evaluate the correctness of a derivative when its base function is not stated).
*) In the introduction must energy is used on the importance of large data sets, but it appears that only fairly small-scale experiments are considered. I'd recommend a better synchronization.
*) I find visual comparisons difficult on the Poincare ball as I am so trained at assuming Euclidean distances when making visual comparisons (I suspect most readers are as well). I think one needs to be very careful when making visual comparisons under non-trivial metrics.
*) In the final experiment, a logistic regressor is fitted post hoc to the embedded points. Why not directly optimize a hyperbolic classifier?
Pros:
+ well-written and (fairly) well-motivated.
Cons:
- It appears that novelty is very limited as highly similar work (see above) has been out for a while.


Word embedding is a useful approach to capture co-occurrence structures in a large corpus of text. In addition to the text data itself, we often have additional covariates associated with individual documents in the corpus-e.g. the demographic of the author, time and venue of publication, etc.-and we would like the embedding to naturally capture the information of the covariates. In this paper, we propose a new tensor decomposition model for word embeddings with covariates. Our model jointly learns a base embedding for all the words as well as a weighted diagonal transformation to model how each covariate modifies the base embedding. To obtain the specific embedding for a particular author or venue, for example, we can then simply multiply the base embedding by the transformation matrix associated with that time or venue. The main advantages of our approach is data efficiency and interpretability of the covariate transformation matrix. Our experiments demonstrate that our joint model learns substantially better embeddings conditioned on each covariate compared to the standard approach of learning a separate embedding for each covariate using only the relevant subset of data. Furthermore, our model encourages the embeddings to be "topic-aligned" in the sense that the dimensions have specific independent meanings. This allows our covariate-specific embeddings to be compared by topic, enabling downstream differential analysis. We empirically evaluate the benefits of our algorithm on several datasets, and demonstrate how it can be used to address many natural questions about the effects of covariates.

The authors present a method for learning word embeddings from related groups of data. The model is based on tensor factorization which extends GloVe to higher order co-ocurrence tensors, where the co-ocurrence is of words within subgroups of the text data. These two papers need to be cited:
Rudolph et al., NIPS 2017, "Sturctured Embedding Models for Grouped Data": This paper also presents a method for learning embeddings specific for subgroups of the data, but based on hierarchical modeling. An experimental comparison is needed.
Cotterell et al., EACL 2017 "Explaining and Generalizing Skip-Gram through Exponential Family Principal Component Analysis": This paper also derives a tensor factorization based approach for learning word embeddings for different covariates. Here the covariates are morphological tags such as part-of-speech tags of the words.
Due to these two citations, the novelty of both the problem set-up of learning different embeddings for each covariate and the novelty of the tensor factorization based model are limited.
The writing is ok. I appreciated the set-up of the introduction with the two questions. However, the questions themselves could have been formulated differently:
Q1: the way Q1 is formulated makes it sound like the covariates could be both discrete and continuous while the method presented later in the paper is only for discrete covariates (i.e. group structure of the data).
Q2: The authors mention topic alignment without specifying what the topics are aligned to. It would be clearer if they stated explicitly that the alignment is between covariate-specific embeddings. It is also distracting that they call the embedding dimensions topics.
Also, why highlight the problem of authorship attribution of Shakespear's work in the introduction, if that problem is not addressed later on?
In the model section, the paragraphs "notation" and "objective function and discussion" are clear.  I also liked the idea of having the section "A geometric view of embeddings and tensor decomposition", but that section needs to be improved. For example, the authors describe RandWalk (Arora et al. 2016) but how their work falls into that framework is unclear.
In the third paragraph, starting with "Therefore we consider a natural extension of this model, ..." it is unclear which model the authors are referring to. (RandWalk or their tensor factorization?).
What are the context vectors in Figure 1?  I am guessing the random walk transitions are the ellipsoids? How are they to be interpreted?
In the last paragraph, beginning with "Note that this is essentially saying...", I don't agree with the argument that the "base embeddings" decompose into independent topics. The dimensions of the base embeddings are some kind of latent attributes and each individual dimension could be used by the model to capture a variety of attributes. There is nothing that prevents the model from using multiple dimensions to capture related structure of the data. Also, the qualitative results in Table 3 do not convince me that the embedding dimensions represent topics. For example "horses" has highest value in embedding dimension 99. It's nearest neighbours in the embedding space (i.e. semantically similar words) will also have high values in coordinate 99. Hence, the apparent semantic coherence in what the authors call "topics".
The authors present multiple qualitative and quantitative evaluations. The clustering by weight (4.1.) is nice and convincing that the model learns something useful. 4.2, the only quantitative analysis was missing some details. Please give references for the evaluation metrics used, for proper credit and so people can look up these tasks. Also, comparison needed to fitting GloVe on the entire corpus (without covariates) and existing methods Rudolph et al. 2017 and Cotterell et al. 2017.
Section 5.2 was nice and so was 5.3. However, for the covariate specific analogies (5.3.) the authors could also analyze word similarities without the analogy component and probably see similar qualitative results. Specifically, they could analyze for a set of query words, what the most similar words are in the embeddings obtained from different subsections of the data.
PROS:
+ nice tensor factorization model for learning word embeddings specific to discrete covariates.
+ the tensor factorization set-up ensures that the embedding dimensions are aligned
+ clustering by weights (4.1) is useful and seems coherent
+ covariate-specific analogies are a creative analysis
CONS:
- problem set-up not novel and existing approach not cited (experimental comparison needed)
- interpretation of embedding dimensions as topics not convincing
- connection to Rand-Walk (Aurora 2016) not stated precisely enough
- quantitative results (Table 1) too little detail:
        * why is this metric appropriate?
        * comparison to GloVe on the entire corpus (not covariate specific)
        * no reference for the metrics used (AP, BLESS, etc.?)
- covariate specific analogies presented confusingly and similar but simpler analysis might be possible by looking at variance in neighbours v_b and v_d without involving v_a and v_c (i.e. don't talk about analogies but about similarities)


Some recent work has shown separation between the expressive power of depth-2 and depth-3 neural networks. These separation results are shown by constructing functions and input distributions, so that the function is well-approximable by a depth-3 neural network of polynomial size but it cannot be well-approximated under the chosen input distribution by any depth-2 neural network of polynomial size. These results are not robust and require carefully chosen functions as well as input distributions. We show a similar separation between the expressive power of depth-2 and depth-3 sigmoidal neural networks over a large class of input distributions, as long as the weights are polynomially bounded. While doing so, we also show that depth-2 sigmoidal neural networks with small width and small weights can be wellapproximated by low-degree multivariate polynomials.

This paper proves a new separation results from 3-layer neural networks to 2-layer neural networks. The core of the analysis is a proof that any 2-layer neural networks can be well approximated by a polynomial function with reasonably low degrees. Then the authors constructs a highly non-smooth function can be represented by a 3-layer network, but impossible to approximate by any polynomial-degree polynomial function.
Similar results about polynomial approximation can be found in [1] (Theorem 4). To me, the result proved in [1] is spiritually very similar to propositions 3-4. The authors need to justify the difference.
The main strength of the new separation result is that it holds for a larger class of input distributions. Comparing to Daniely (2017) which requires the input distribution to be spherically uniform, the new result only needs the distribution to be lower bounded by 1/poly(d) in a small ball of radius 1/poly(d). Conceptually I don't think this is a much weaker condition. For a "truly" non-uniform distribution, one should allow its density function to be very close to zero at certain regions of the ball. Nevertheless, the result is a step forward from Daniely (2017) and the paper is well written.
I am still in doubt of the practical value of such kind of separation results. The paper proves the separation by constructing a very specific function that cannot be approximated by 2-layer networks. This function has a super large Lipschitz constant, which we don't expect to see in practice. Consider the function f(x)=cos(Nx). When N is chosen large enough, the function f can not be well approximated by any 2-layer network with polynomial size. Does it imply that the family of cosine functions is rich enough so that it is a better family to learn than 2-layer neural networks? I guess the answer would be negative. In addition, the paper doesn't show that any 2-layer network can be well approximated by a 3-layer network, which is a missing piece in justifying the richness of 3-layer nets.
Finally, the constructed "hard" function has order d^5 Lipschitz constant, but Theorem 7 assumes that the 2-layer networks' weight must be bounded by O(d^2). This assumption is crucial to the proof but not well justified (especially considering the d^5 factor in the function definition).
[1] On the Computational Efficiency of Training Neural Networks, Livni et al., NIPS'14


Many tasks in artificial intelligence require the collaboration of multiple agents. We exam deep reinforcement learning for multi-agent domains. Recent research efforts often take the form of two seemingly conflicting perspectives, the decentralized perspective, where each agent is supposed to have its own controller; and the centralized perspective, where one assumes there is a larger model controlling all agents. In this regard, we revisit the idea of the master-slave architecture by incorporating both perspectives within one framework. Such a hierarchical structure naturally leverages advantages from one another. The idea of combining both perspectives is intuitive and can be well motivated from many real world systems, however, out of a variety of possible realizations, we highlights three key ingredients, i.e. composed action representation, learnable communication and independent reasoning. With network designs to facilitate these explicitly, our proposal consistently outperforms latest competing methods both in synthetic experiments and when applied to challenging StarCraft 1 micromanagement tasks.

This paper investigates multiagent reinforcement learning  making used of a "master slave" architecture (MSA). On the positive side, the paper is mostly well-written, seems technically correct, and there are some results that indicate that the MSA is working quite well on relatively complex tasks. On the negative side, there seems to be relatively limited novelty: we can think of MSA as one particular communication (i.e, star) configuration one could use is a multiagent system. One aspect does does strike me as novel is the "gated composition module", which allows differentiation of messages to other agents based on the receivers internal state. (So, the *interpretation* of the message is learned). I like this idea, however, the results are mixed, and the explanation given is plausible, but far from a clearly demonstrated answer.
There are some important issues that need clarification:
* "Sukhbaatar et al. (2016) proposed the “CommNet”, where broadcasting communication channel among all agents were set up to share a global information which is the summation of all individual agents. [...] however the summed global signal is hand crafted information and does not facilitate an independently reasoning master agent."
-Please explain what is meant here by 'hand crafted information', my understanding is that the f^i in figure 1 of that paper are learned modules?
-Please explain what would be the differences with CommNet with 1 extra agent that takes in the same information as your 'master'.
*This relates also to this:
"Later we empirically verify that, even when the overall in-
formation revealed does not increase per se, an independent master agent tend to absorb the same
information within a big picture and effectively helps to make decision in a global manner. Therefore
compared with pure in-between-agent communications, MS-MARL is more efficient in reasoning
and planning once trained. [...]
Specifically, we compare the performance among the CommNet model, our
MS-MARL model without explicit master state (e.g. the occupancy map of controlled agents in this
case), and our full model with an explicit occupancy map as a state to the master agent. As shown in
Figure 7 (a)(b), by only allowed an independently thinking master agent and communication among
agents, our model already outperforms the plain CommNet model which only supports broadcast-
ing communication of the sum of the signals."
-Minor: I think that the statement "which only supports broadcast-ing communication of the sum of the signals" is not quite fair: surely they have used a 1-channel communication structure, but it would be easy to generalize that.
-Major: When I look at figure 4D, I see that the proposed approach *also* only provides the master with the sum (or really mean) with of the individual messages...? So it is not quite clear to me what explains the difference.
*In 4.4, it is not quite clear exactly how the figure of master and slave actions is created. This seems to suggest that the only thing that the master can communicate is action information? It this the case?
* In table 2, it is not clear how significant these differences are. What are the standard errors?
* The section 3.2 explains standard things (policy gradient), but the details are a bit unclear. In particular, I do not see how the Gaussian/softmax layers are integrated; they do not seem to appear in figure 4?
* I cannot understand figure 7 without more explanation. (The background is all black - did something go wrong with the pdf?)

Details:
* references are wrongly formatted throughout.
* "In this regard, we are among the first to combine both the centralized perspective and the decentralized perspective"
This is a weak statement (E.g., I suppose that in the greater scheme of things all of us will be amongst the first people that have walked this earth...)
* "Therefore they tend to work more like a commentator analyzing and criticizing the play, rather than
a coach coaching the game."
-This sounds somewhat vague. Can it be made crisper?
* "Note here that, although we explicitly input an occupancy map to the master agent, the actual infor-
mation of the whole system remains the same."
This is a somewhat peculiar statement. Clearly, the distribution of information over the agents is crucial. For more insights on this one could refer to the literature on decentralized POMDPs.


Model-free deep reinforcement learning approaches have shown superhuman performance in simulated environments (e.g., Atari games, Go, etc). During training, these approaches often implicitly construct a latent space that contains key information for decision making. In this paper, we learn a forward model on this latent space and apply it to model-based planning in a miniature Real-time Strategy game with incomplete information (MiniRTS [Tian et al. (2017)]). We first show that the latent space constructed from existing actor-critic models contains relevant information of the game, and design training procedure to learn forward models. We also show that our learned forward model can predict meaningful future state and is usable for latent space Monte-Carlo Tree Search (MCTS), in terms of win rates against rule-based agents.

Summary: This paper proposes to use the latent representations learned by a model-free RL agent to learn a transition model for use in model-based RL (specifically MCTS). The paper introduces a strong model-free baseline (win rate ~80% in the MiniRTS environment) and shows that the latent space learned by this baseline does include relevant game information. They use the latent state representation to learn a model for planning, which performs slightly better than a random baseline (win rate ~25%).
Pros:
- Improvement of the model-free method from previous work by incorporating information about previously observed states, demonstrating the importance of memory.
- Interesting evaluation of which input features are important for the model-free algorithm, such as base HP ratio and the amount of resources available.
Cons:
- The model-based approach is disappointing compared to the model-free approach.
Quality and Clarity:
The paper in general is well-written and easy to follow and seems technically correct, though I found some of the figures and definitions confusing, specifically:
- The terms for different forward models are not defined (e.g. MatchPi, MatchA, etc.). I can infer what they mean based on Figure 1 but it would be helpful to readers to define them explicitly.
- In Figure 3b, it is not clear to me what the difference between the red and blue curves is.
- In Figure 4, it would be helpful to label which color corresponds to the agent and which to the rule-based AI.
- The caption in Figure 8 is malformatted.
- In Figure 7, the baseline of \hat{h_t}=h_{t-2} seems strange---I would find it more useful for Figure 7 to compare to the performance if the model were not used (i.e. if \hat{h_t}=h_t) to see how much performance suffers as a result of model error.
Originality:
I am unfamiliar with the MiniRTS environment, but given that it is only published in this year's NIPS (and that I couldn't find any other papers about it on Google Scholar) it seems that this is the first paper to compare model-free and model-based approaches in this domain. However, the model-free approach does not seem particularly novel in that it is just an extension of that from Tian et al. (2017) plus some additional features. The idea of learning a model based on the features from a model-free agent seems novel but lacks significance in that the results are not very compelling (see below).
Significance:
I feel the paper overstates the results in saying that the learned forward model is usable in MCTS. The implication in the abstract and introduction (at least as I interpreted it) is that the learned model would outperform a model-free method, but upon reading the rest of the paper I was disappointed to learn that in reality it drastically underperforms. The baseline used in the paper is a random baseline, which seems a bit unfair---a good baseline is usually an algorithm that is an obvious first choice, such as the model-free approach.


Deep lifelong learning systems need to efficiently manage resources to scale to large numbers of experiences and non-stationary goals. In this paper, we explore the relationship between lossy compression and the resource constrained lifelong learning problem of knowledge transfer. We demonstrate that lossy episodic experience storage can enable efficient knowledge transfer between different architectures and algorithms at a fraction of the storage cost of lossless storage. This is achieved by introducing a generative knowledge distillation strategy that does not store any training examples. As an important extension of this idea, we show that lossy recollections stabilize deep networks much better than lossless sampling in resource constrained settings of lifelong learning while avoiding catastrophic forgetting. For this setting, we propose a novel dual purpose recollection buffer used to both stabilize the recollection generator itself and an accompanying reasoning model.

This paper presents an approach to lifelong learning with episodic experience storage under resource constraints. The key idea of the approach is to store the latent code obtained from a categorical Variational Autoencoder as opposed to the input example itself. When a new task is learnt, catastrophic forgetting is avoided by randomly sampling stored codes corresponding to past experience and adding the corresponding reconstruction to a batch of data from a new problem. The authors show that explicitly storing data provides better results than random sampling from the generative model. Furthermore, the method is compared to other techniques relying on episodic memory and as expected, achieves better results given a fixed effective buffer size due to being able to store more experience.
While the core idea of this paper is reasonable, it provides little insight into how episodic experience storage compares to related methods as an approach to lifelong learning. While the authors compare their method to other techniques based on experience replay, I feel that a comparison to other techniques is important. A natural choice would be a model which introduces task-specific parameters for each problem (e.g.  (Li & Hoiem, 2016) or (Rusu et al., 2016)).
A major concern is the fact that the VAE with categorical latents itself suffers from catastrophic forgetting. While the authors propose to "freeze decoder parameters right before each incoming experience and train multiple gradient descent iterations over randomly selected recollection batches before moving on to the next experience", this makes the approach both less straight-forward to apply and more computationally expensive.
Moreover, the authors only evaluate the approach on simple image recognition tasks (MNIST, CIFAR-100, Omniglot). I feel that an experiment in Reinforcement Learning  (e.g. as proposed in (Rusu et al., 2016)) would provide more insight into how the approach behaves in more challenging settings. In particular, it is not clear whether experience replay may lead to negative transfer when subsequent tasks are more diverse.
Finally, the manuscript lacks clarity. As another reviewer noted, detailed sections of weakly related motivations fail to strengthen the reader's understanding. As a minor point, the manuscript contains several grammar and spelling mistakes.


Reliable measures of statistical dependence could be useful tools for learning independent features and performing tasks like source separation using Independent Component Analysis (ICA). Unfortunately, many of such measures, like the mutual information, are hard to estimate and optimize directly. We propose to learn independent features with adversarial objectives (Goodfellow et al., 2014;Huszar, 2016) which optimize such measures implicitly. These objectives compare samples from the joint distribution and the product of the marginals without the need to compute any probability densities. We also propose two methods for obtaining samples from the product of the marginals using either a simple resampling trick or a separate parametric distribution. Our experiments show that this strategy can easily be applied to different types of model architectures and solve both linear and non-linear ICA problems.

The focus of the paper is independent component analysis (ICA) and its nonlinear variants such as the post non-linear (PNL) ICA model. Motivated by the fact that estimating mutual information and similar dependency measures require density estimates and hard to optimize, the authors propose a Wasserstein GAN (generative adversarial network) based solution to tackle the problem, with illustrations on 6 (synthetic) and 3-dimemensional (audio) examples. The primary idea of the paper is to use the Wasserstein distance as an independence measure of the estimated source coordinates, and optimize it in a neural network (NN) framework.
Although finding novel GAN applications is an exciting topic, I am not really convinced that ICA with the proposed Wasserstein GAN based technique fulfills this goal.

Below I detail my reasons:
1)The ICA problem can be formulated as the minimization of pairwise mutual information [1] or one-dimensional entropy [2]. In other words, estimating the joint dependence of the source coordinates is not necessary; it is worthwhile to avoid it.
2)The PNL ICA task can be efficiently tackled by first 'removing' the nonlinearity followed by classical linear ICA; see for example [3].
3)Estimating information theoretic (IT) measures (mutual information, divergence) is a quite mature field with off-the-self techniques, see for example [4,5,6,8]. These methods do not estimate the underlying densities; it would be superfluous (and hard).
4)Optimizing non-differentiable IT measures can computationally quite efficiently carried out in the ICA context by e.g., Givens rotations [7]; differentiable ICA cost functions can be robustly handled by Stiefel manifold methods; see for example [8,9].
5)Section 3.1: This section is devoted to generating samples from the product of the marginals, even using separate generator networks. I do not see the necessity of these solutions; the subtask can be solved by independently shuffling all the coordinates of the sample.
6)Experiments (Section 6):
i) It seems to me that the proposed NN-based technique has some quite serious divergence issues: 'After discarding diverged models, ...' or 'Unfortunately, the model selection procedure also didn't identify good settings for the Anica-g model...'.
ii) The proposed method gives pretty comparable results to the chosen baselines (fastICA, PNLMISEP) on the selected small-dimensional tasks. In fact, [7,8,9] are likely to provide more accurate (fastICA is a simple kurtosis based method, which is
a somewhat crude 'estimate' of entropy) and faster estimates; see also 2).
References:
[1] Pierre Comon. Independent component analysis, a new concept? Signal Processing, 36:287-314, 1994.
[2] Aapo Hyvarinen and Erkki Oja. Independent Component Analysis: Algorithms and Applications. Neural Networks, 13(4-5):411-30, 2000.
[3] Andreas Ziehe, Motoaki Kawanabe, Stefan Harmeling, and Klaus-Robert Muller. Blind separation of postnonlinear mixtures using linearizing transformations and temporal decorrelation. Journal of Machine Learning Research, 4:1319-1338, 2003.
[4] Barnabas Poczos, Liang Xiong, and Jeff Schneider. Nonparametric divergence: Estimation with applications to machine learning on distributions. In Conference on Uncertainty in Artificial Intelligence, pages 599-608, 2011.
[5] Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Scholkopf, Alexander Smola. A Kernel Two-Sample Test. Journal of Machine Learning Research, 13:723-773, 2012.
[6] Alan Wisler, Visar Berisha, Andreas Spanias, Alfred O. Hero. A data-driven basis for direct estimation of functionals of distributions. TR, 2017. (https://arxiv.org/abs/1702.06516)
[7] Erik G. Learned-Miller, John W. Fisher III. ICA using spacings estimates of entropy. Journal of Machine Learning Research, 4:1271-1295, 2003.
[8] Francis R. Bach. Michael I. Jordan. Kernel Independent Component Analysis. Journal of Machine Learning Research 3: 1-48, 2002.
[9] Hao Shen, Stefanie Jegelka and Arthur Gretton. Fast Kernel-Based Independent Component Analysis, IEEE Transactions on Signal Processing, 57:3498-3511, 2009.


A Boo(n) for Evaluating Architecture Performance
We point out important problems with the common practice of using the best single model performance for comparing deep learning architectures, and we propose a method that corrects these flaws. Each time a model is trained, one gets a different result due to random factors in the training process, which include random parameter initialization and random data shuffling. Reporting the best single model performance does not appropriately address this stochasticity. We propose a normalized expected best-out-of-n performance (Boo n ) as a way to correct these problems.

This paper addresses multiple issues arising from the fact that commonly reported best model performance numbers are a single sample from a performance distribution. These problems are very real, and they deserve significant attention from the ML community.  However, I feel that the proposed solution may actually compound the issues highlighted.
Firstly, the proposed metric requires calculation of multiple test set experiments for every evaluation. In the paper up to 100 experiments were used. This may be reasonable in scenarios where the test set is hidden, and individual test numbers are never revealed. It also may be reasonable if we cynically assume that researchers are already running many test-set evaluations. But I am very opposed to any suggestion that we should relax the maxim that the test set should be used only once, or as close to once as is possible. Even the idea of researchers knowing their test set variance makes me very uneasy.
Secondly, this paper tries to account for variation in results due to different degrees of hyper-parameter tuning. This is certainly an admirable aim, since different research groups have access to very different types of resources. However, the suggested approach relies on randomly picking hyper-parameters from "a range that we previously found to work reasonably well". This randomization does not account for the many experiments that were required to find this range. And the randomization is also not extended to parameters controlling the model architecture (I suspect that a number of experiments went into picking the 32 layers in the ResNet used by this paper). Without a solid and consistent basis for these hyper-parameter perturbations, I worry that this approach will fail to normalize the effect of experiment numbers while also giving researchers an excuse to avoid reporting their experimental process.
I think this is a nice idea and the metric does merge the stability and low variance of mean score with the aspirations of best score. The metric may be very useful at development time in helping researchers build a reasonable expectation of test time performance in cases where the dev and test sets are strongly correlated. However, for the reasons outlined above, I don't think the proposed approach solves the problems that it addresses. Ultimately, the decision about this paper is a subjective one. Are we willing to increase the risk of inadvertent hyper-parameter tuning on the test set for the sake of a more stable metric?


Published as a conference paper at ICLR 2018 IMITATION LEARNING FROM VISUAL DATA WITH MULTIPLE INTENTIONS
Recent advances in learning from demonstrations (LfD) with deep neural networks have enabled learning complex robot skills that involve high dimensional perception such as raw image inputs. LfD algorithms generally assume learning from single task demonstrations. In practice, however, it is more efficient for a teacher to demonstrate a multitude of tasks without careful task set up, labeling, and engineering. Unfortunately in such cases, traditional imitation learning techniques fail to represent the multi-modal nature of the data, and often result in sub-optimal behavior. In this paper we present an LfD approach for learning multiple modes of behavior from visual data. Our approach is based on a stochastic deep neural network (SNN), which represents the underlying intention in the demonstration as a stochastic activation in the network. We present an efficient algorithm for training SNNs, and for learning with vision inputs, we also propose an architecture that associates the intention with a stochastic attention module. Furthermore, we demonstrate our method on real robot visual object reaching tasks, and show that it can reliably learn the multiple behavior modes in the demonstration data. Video results are available at https://vimeo.com/240212286/fd401241b9.

The authors provide a method for learning from demonstrations where several modalities of the same task are given. The authors argue that in the case where several demonstrations exists and a deterministic (i.e., regular network) is given, the network learns some average policy from the demonstrations.
The paper begins with the authors stating the motivation and problem of how to program robots to do a task based only on demonstrations rather on explicit modeling or programming. They put the this specific work in the right context of imitation learning and IRL. Afterward, the authors argue that deterministic network cannot adequately several modalities. The authors cover in Section 2 related topics, and indeed the relevant literature includes behavioral cloning, IRL , Imitation learning, GAIL, and VAEs. I find that recent paper by Tamar et al 2016. on Value Iteration Networks is highly relevant to this work: the authors there learn similar tasks (i.e., similar modalities) using the same network. Even the control task is very similar to the current proposed task in this paper.
The authors argue that their contribution is 3-fold: (1) does not require robot  rollouts, (2) does not require label for a task, (3) work within raw image inputs. Again, Tamar et al. 2016 deals with this 3 points.
I went over the math. It seems right and valid. Indeed, SNN is a good choice for adding (Bayesian) context to a task. Also, I see the advantage of referring only to the "good" quantiles when needed. It is indeed a good method for dealing with the variance.
I must say that I was impressed with the authors making the robot succeed in the tasks in hand (although reaching to an object is fairly simple task).
My concerns are as follows:
1) Seems like that the given trajectories are naturally divided with different tasks, i.e., a single trajectory consists only a single task. For me, this is not the pain point in this tasks. the pain point is knowing when tasks are begin and end.
2) I'm not sure, and I haven't seen evidence in the paper (or other references) that SNN is the only (optimal?) method for this context. Why not adding (non Bayesian) context (not label) to the task will not work as well?
3) the robot task is impressive. but proving the point, and for the ease of comparing to different tasks, and since we want to show the validity of the work on more than 200 trials, isn't showing the task on some simulation is better for understanding the different regimes that this method has advantage? I know how hard is to make robotic tasks work...
4) I’m not sure that the comparison of the suggested architecture to one without any underlying additional variable Z or context (i.e., non-Bayesian setup) is fair. "Vanilla" NN indeed may fail miserably . So, the comparison should be to any other work that can deal with "similar environment but different details".
To summarize, I like the work and I can see clearly the motivation. But I think some more work is needed in this work: comparing to the right current state of the art, and show that in principal (by demonstrating on other simpler simulations domains) that this method is better than other methods.


Human brain function as measured by functional magnetic resonance imaging (fMRI), exhibits a rich diversity. In response, understanding the individual variability of brain function and its association with behavior has become one of the major concerns in modern cognitive neuroscience. Our work is motivated by the view that generative models provide a useful tool for understanding this variability. To this end, this manuscript presents two novel generative models trained on real neuroimaging data which synthesize task-dependent functional brain images. Brain images are high dimensional tensors which exhibit structured spatial correlations. Thus, both models are 3D conditional Generative Adversarial networks (GANs) which apply Convolutional Neural Networks (CNNs) to learn an abstraction of brain image representations. Our results show that the generated brain images are diverse, yet task dependent. In addition to qualitative evaluation, we utilize the generated synthetic brain volumes as additional training data to improve downstream fMRI classifiers (also known as decoding, or brain reading). Our approach achieves significant improvements for a variety of datasets, classification tasks and evaluation scores. Our classification results provide a quantitative evaluation of the quality of the generated images, and also serve as an additional contribution of this manuscript.

The work is motivated by a real challenge of neuroimaging analysis: how to increase the amount of data to support the learning of brain decoding.
The contribution seems to mix two objectives: on one hand to prove that it is possible to do data augmentation for fMRI brain decoding, on the other hand to design (or better to extend) a new model (to be more precise two models).
Concerning the first objective the empirical results do not provide meaningful support that the generative model is really effective. The improvement is really tiny and a statistical test (not included in the analysis) probably wouldn't pass a significant threshold.  This analysis is missing a straw man. It is not clear whether the difference in the evaluation measures is related to the greater number of examples or by the specific generative model.
Concerning the contribution of the model, one novelty is the conditional formulation of the discriminator. The design of the empirical evaluation doesn't address the analysis of the impact of this new formulation. It is not clear whether the supposed improvement is related to the conditional formulation.
Figure 3 and Figure 5 illustrate the brain maps generated for Collection 1952 with ICW-GAN and for collection 503 with ACD-GAN. It is not clear how the authors operated the choices of these figures. From the perspective of neuroscience a reader,  would expect to look at the brain maps for the same collection with different methods. The pairwise brain maps would support the interpretation of the generated data. It is worthwhile to remember that the location of brain activations is crucial to detect whether the brain decoding (classification) relies on artifacts or confounds.
Minor comments
- typos: "a first application or this" => "a first application of this" (p.2)
- "qualitative quality" (p.2)


We study the statistical properties of the endpoint of stochastic gradient descent (SGD). We approximate SGD as a stochastic differential equation (SDE) and consider its Boltzmann Gibbs equilibrium distribution under the assumption of isotropic variance in loss gradients.. Through this analysis, we find that three factors -learning rate, batch size and the variance of the loss gradients -control the trade-off between the depth and width of the minima found by SGD, with wider minima favoured by a higher ratio of learning rate to batch size. In the equilibrium distribution only the ratio of learning rate to batch size appears, implying that it's invariant under a simultaneous rescaling of each by the same amount. We experimentally show how learning rate and batch size affect SGD from two perspectives: the endpoint of SGD and the dynamics that lead up to it. For the endpoint, the experiments suggest the endpoint of SGD is similar under simultaneous rescaling of batch size and learning rate, and also that a higher ratio leads to flatter minima, both findings are consistent with our theoretical analysis. We note experimentally that the dynamics also seem to be similar under the same rescaling of learning rate and batch size, which we explore showing that one can exchange batch size and learning rate in a cyclical learning rate schedule. Next, we illustrate how noise affects memorization, showing that high noise levels lead to better generalization. Finally, we find experimentally that the similarity under simultaneous rescaling of learning rate and batch size breaks down if the learning rate gets too large or the batch size gets too small.

The paper investigates how the learning rate and mini-batch size in SGD impacts the optima that the SGD algorithm finds.
Empirically, the authors argue that it was observed that larger learning rates converge to minima which are more wide,
and that smaller learning rates more often lead to convergence to minima which are narrower, i.e. where the Hessian has large Eigenvalues. In this paper, the authors derive an analytical theory that aims at explaining this phenomenon.
Point of departure is an analytical theory proposed by Mandt et al., where SGD is analyzed in a continuous-time stochastic
formalism. In more detail, a stochastic differential equation is derived which mimicks the behavior of SGD. The advantage of
this theory is that under specific assumptions, analytic stationary distributions can be derived. While Mandt et al. focused
on the vicinity of a local optima, the authors of the present paper assumed white diagonal gradient noise, which allows to
derive an analytic, *global* stationary distribution (this is similar as in Langevin dynamics).
Then, the authors focus again on individual local optima and "integrate out" the stationary distribution around a local optimum, using again a Gaussian assumption. As a result, the authors obtain un-normalized probabilities of getting trapped in a given local optimum. This un-normalized probability depends on the strength of the value of the loss function in the vicinity of the optimum, the gradient noise, and the width of the optima. In the end, these un-normalized probabilities are taken as
probabilities that the SGD algorithm will be trapped around the given optimum in finite time.
Overall assessment:
I find the analytical results of the paper very original and interesting. The experimental part has some weaknesses. The paper could be drastically improved when focusing on the experimental part.
Detailed comments:
Regarding the analytical part, I think this is all very nice and original. However, I have some comments/requests:
1. Since the authors focus around Gaussian regions around the local minima, perhaps the diagonal white noise assumption could be weakened. This is again the multivariate Ornstein-Uhlenbeck setup examined in Mandt et al., and probably possesses an analytical solution for the un-normalized probabilities (even if the noise is multivariate Gaussian). Would the authors to consider generalizing the proof for the camera-ready version perhaps?
2. It would be nice to sketch the proof of theorem 2 in the main paper, rather than to just refer to the appendix. In my opinion, the theorem results from a beautiful and instructive calculation that should provide the reader with some intuition.
3. Would the authors comment on the underlying theoretical assumptions a bit more? In particular, the stationary distribution predicted by the Ornstein-Uhlenbeck formalism is never reached in practice. When using SGD in practice, one is in the initial mode-seeking phase. So, why is it a reasonable assumption to still use results obtained from the stationary (equilibrated) distribution which is never reached?
Regarding the experiments: here I see a few problems. First, the writing style drops in quality. Second, figures 2 and 3 are cryptic. Why do the authors focus on two manually selected optima? In which sense is this statistically significant? How often were the experiments repeated? The figures are furthermore hard to read. I would recommend overhauling the entire experiments section.
Details:
- Typo in Figure 2: ”with different with different”.
- “the endpoint of SGD with a learning rate schedule η → η/a, for some a > 0, and a constant batch size S, should be the same
  as the endpoint of SGD with a constant learning rate and a batch size schedule S → aS.” This is clearly wrong as there are many local minima, and running teh algorithm twice results in different local optima.  Maybe add something that this only true on average, like “the characteristics of these minima ... should be the same”.


Published as a conference paper at ICLR 2018 IMPROVING GAN TRAINING VIA BINARIZED REPRESENTATION ENTROPY (BRE) REGULARIZATION
We propose a novel regularizer to improve the training of Generative Adversarial Networks (GANs). The motivation is that when the discriminator D spreads out its model capacity in the right way, the learning signals given to the generator G are more informative and diverse. These in turn help G to explore better and discover the real data manifold while avoiding large unstable jumps due to the erroneous extrapolation made by D . Our regularizer guides the rectifier discriminator D to better allocate its model capacity, by encouraging the binary activation patterns on selected internal layers of D to have a high joint entropy. Experimental results on both synthetic data and real datasets demonstrate improvements in stability and convergence speed of the GAN training, as well as higher sample quality. The approach also leads to higher classification accuracies in semi-supervised learning.

The paper proposes a regularizer that encourages a GAN discriminator to focus its capacity in the region around the manifolds of real and generated data points, even when it would be easy to discriminate between these manifolds using only a fraction of its capacity, so that the discriminator provides a more informative signal to the generator. The regularizer rewards high entropy in the signs of discriminator activations. Experiments show that this helps to prevent mode collapse on synthetic Gaussian mixture data and improves Inception scores on CIFAR10.
The high-level idea of guiding model capacity by rewarding high-entropy activations  is interesting and novel to my knowledge (though I am not an expert in this space). Figure `1 is a fantastic illustration that presents the core idea very clearly. That said I found the intuitive story a little bit difficult to follow -- it's true that in Figure 1b the discriminator won't communicate the detailed structure of the data manifold to the generator, but it's not clear why this would be a problem -- the gradients should still pull the generator *towards* the manifold of real data, and as this happens and the manifolds begin to overlap, the discriminator will naturally be forced to allocate its capacity towards finer-grained details. Is the implicit assumption that for real, high-dimensional data the generator and data manifolds will *never* overlap? But in that case much of the theoretical story goes out the window. I'd also appreciate further discussion of the relationship of this approach to Wasserstein GANs, which also attempt to provide a clearer training gradient when the data and generator manifolds do not overlap.
More generally I'd like to better understand what effect we'd expect this regularizer to have. It appears to be motivated by improving training dynamics, which is understandably a significant concern. Does it also change the location of the Nash equilibria? (or equivalently, the optimal generator under the density-ratio-estimator interpretation of discriminators proposed by https://arxiv.org/abs/1610.03483). I'd expect that it would but the effects of this changed objective are not discussed in the paper.
 The experimental results seem promising, although not earthshattering. I would have appreciated a comparison to other methods for guiding discriminator representation capacity, e.g. autoencoding (I'd also imagine that learning an inference network (e.g. BiGAN) might serve as a useful auxiliary task?).
Overall this feels like an cute hack, supported by plausible intuition but without deep theory or compelling results on real tasks (yet). As such I'd rate it as borderline; though perhaps interesting enough to be worth presenting and discussing.
A final note: this paper was difficult to read due to many grammatical errors and unclear or misleading constructions, as well as missing citations (e.g. sec 2.1). From the second paragraph alone:
"impede their wider applications in new data domain" -> domains
"extreme collapse and heavily oscillation" -> heavy oscillation
"modes of real data distribution" -> modes of the real data distribution
"while D fails to exploit the failure to provide better training signal to G" -> should be "this failure" to refer to the previously-described generator mode collapse, or rewrite entirely
"even when they are their Jensen-Shannon divergence" -> even when their Jensen-Shannon divergence
 I'm sympathetic to the authors who are presumably non-native English speakers; many good papers contain mistakes, but in my opinion the level in this paper goes beyond what is appropriate for published work. I encourage the authors to have the work proofread by a native speaker; clearer writing will ultimately increase the reach and impact of the paper.


Recent neural network and language models have begun to rely on softmax distributions with an extremely large number of categories. In this context calculating the softmax normalizing constant is prohibitively expensive. This has spurred a growing literature of efficiently computable but biased estimates of the softmax. In this paper we present the first two unbiased algorithms for maximizing the softmax likelihood whose work per iteration is independent of the number of classes and datapoints (and does not require extra work at the end of each epoch). We compare our unbiased methods' empirical performance to the state-of-the-art on seven real world datasets, where they comprehensively outperform all competitors.

The paper presents interesting algorithms for minimizing softmax with many classes. The objective function is a multi-class classification problem (using softmax loss) and with linear model. The main idea is to rewrite the obj as double-sum using the dual formulation and then apply SGD to solve it. At each iteration, SGD samples a subset of training samples and labels. The main contribution of this paper is: 1) proposing a U-max trick to improve the numerical stability and 2) proposing an implicit SGD approach. It seems the implicit SGD approach is better in the experimental comparisons.
I found the paper quite interesting, but meanwhile I have the following comments and questions:
- As pointed out by the authors, the idea of this formulation and doubly SGD is not new. (Raman et al, 2016) has used a similar trick to derive the double-sum formulation and solved it by doubly SGD. The authors claim that  the algorithm in (Raman et al) has an O(NKD) cost for updating u at the end of each epoch. However, since each epoch requires at least O(NKD) time anyway (sometimes larger, as in Proposition 2), is another O(NKD) a significant bottleneck? Also, since the formulation is similar to (Raman et al., 2016), a comparison is needed.
- I'm confused by Proposition 1 and 2. In appendix E.1, the formulation of the update is derived, but why we need Newton to get log(1/epsilon) time complexity? I think most first order methods instead of Newton will have linear converge (log(1/epsilon) time)? Also, I guess we are assuming the obj is strongly convex?
- The step size is selected in one dataset and used for all others. This might lead to divergence of other algorithms, since usually step size depends on data. As we can see, OVE, NCE and IS diverges on Wiki-small, which may be fixed if the step size is chosen for each data (in practice we can choose using subsamples for each data).
- All the comparisons are based on "epochs", but the competing algorithms are quite different and can have very different running time for each epoch. For example, implicit SGD has another iterative solver for each update. Therefore, the timing comparison is needed in this paper to justify that implicit SGD is faster.
- The claim that "implicit SGD never overshoots the optimum" needs more supports. Is it proved in some previous papers?
- The presentation can be improved. I think it will be helpful to state the algorithms explicitly in the main paper.


We present an information-theoretic framework for understanding trade-offs in unsupervised learning of deep latent-variables models using variational inference. This framework emphasizes the need to consider latent-variable models along two dimensions: the ability to reconstruct inputs (distortion) and the communication cost (rate). We derive the optimal frontier of generative models in the two-dimensional rate-distortion plane, and show how the standard evidence lower bound objective is insufficient to select between points along this frontier. However, by performing targeted optimization to learn generative models with different rates, we are able to learn many models that can achieve similar generative performance but make vastly different trade-offs in terms of the usage of the latent variable. Through experiments on MNIST and Omniglot with a variety of architectures, we show how our framework sheds light on many recent proposed extensions to the variational autoencoder family.

EDIT:  I have reviewed the authors revisions and still recommend acceptance.
Summary
This paper proposes assessing VAEs via two quantities: rate R (E[ KLD[q(z|x) || p(z)] ]) and distortion D (E[ log p(x|z) ]), which can be used to bound the mutual information (MI) I(x,z) from above and below respectively (i.e. H[x] - D <= I(x,z) <= R).  This fact then implies the inequality H[x] <= R + D, where H[x] is the entropy of the true data distribution, and allows for the construction of a phase diagram (Figure 1) with R and D on the x and y axis respectively.  Models can be plotted on the diagram to show the degree to which they favor reconstruction (D) or sampling diversity (R).  The paper then reports several experiments, the first being a simulation to show that a VAE trained with vanilla ELBO cannot recover the true rate in even a 1D example.  For the second experiment, 12 models are trained by varying the encoder/decoder strength (CNN vs autoregressive) and prior (fact. Gauss vs autoregressive vs VampPrior).  Plots of the D vs R and ELBO vs R are shown for the models, revealing that the same ELBO value can be decomposed into drastically different R and D values.  The point is further made through qualitative results in Figure 4.
Evaluation
Pros:  While no one facet of the paper is particularly novel (as similar observations and discussion has been made by [1-4]), the paper, as far as I’m aware, is the first to formally decompose the ELBO into the R vs D tradeoff, which is natural.  As someone who works with VAEs, I didn’t find the conclusions surprising, but I imagine the paper would be valuable to someone learning about VAEs.  Moreover, it’s nice to have a clear reference for the unutilized-latent-space-behavior mentioned in various other VAE papers.  The most impressive aspect of the paper is the number of models trained for the empirical investigation.  Placing such varied models (CNN vs autoregressive vs VampPrior etc) onto the same plot from comparison (Figure 3) is a valuable contribution.
Cons:  As mentioned above, I didn’t find the paper conceptually novel, but this isn’t a significant detraction as its value (at least for VAE researchers) is primarily in the experiments (Figure 3).  I do think the paper---as the ‘Discussion and Further Work’ section is only two sentences long---could be improved by providing a better summary of the findings and recommendations moving forward.  Should generative modeling papers be reporting final R and D values in addition to marginal likelihood?  How should an author demonstrate that their method isn’t doing auto-decoding?  The conclusion claims that “[The rate-distortion tradeoff] provides new methods for training VAE-type models which can hopefully advance the state of the art in unsupervised representation learning.”  Is this referring to the constrained optimization problem given in Equation #4?  It seems to me that the optimal R-vs-D tradeoff is application dependant; is this not always true?
Miscellaneous / minor comments:  Figure 3 would be easier to read if the dots better reflected their corresponding tuple (although I realize representing all tuple combinations in terms of color, shape, etc is hard).  I had to keep referring to the legend, losing my place in the scatter plot.  I found sections 1 and 2 rather verbose; I think some text could be cut to make room for more final discussion / recommendations.  For example, I think the first two whole paragraphs could be cut or at least condensed and moved to the related works section, as they are just summarizing research history/trends.  The paper’s purpose clearly starts at the 3rd paragraph (“We are interested in understanding…”).  The references need cleaned-up.  There are several conference publications that are cited via ArXiv instead of the conference (IWAE should be ICLR, Bowman et al. should be CoNLL, Lossy VAE should be ICLR, Stick-Breaking VAE should be ICLR, ADAM should be ICLR, Inv Autoregressive flow should be NIPS, Normalizing Flows should be ICML, etc.), and two different versions of the VAE paper are cited (ArXiv and ICLR).
Conclusions
I found this paper to present valuable analysis of the ELBO objective and how it relates to representation learning in VAEs.  I recommend the paper be accepted, although it could be substantially improved by including more discussion at the end.
1.  S. Zhao, J. Song, and S. Ermon.  “InfoVAE: Information Maximizing Variational Autoencoders.”  ArXiv 2017.
2.  X. Chen, D. Kingma, T. Salimans, Y. Duan, P. Dhariwal, J. Shulman, I. Sutskever, and P. Abbeel.  “Variational Lossy Autoencoder.”  ICLR 2017.
3.  I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner. “Beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework.”  ICLR 2017
4.  S. Bowman, L. Vilnis, O. Vinyas,  A. Dai, R. Jozefowicz, and S. Bengio.  “Generating Sentences from a Continuous Space.”  CoNLL 2016.


Recently, there has been growing interest in methods that perform neural network compression, namely techniques that attempt to substantially reduce the size of a neural network without performance degradation. In this paper, we propose a simple compression algorithm that incorporates pruning within scalar quantization, whose non-probabilistic nature results in minimal change and overhead to standard network training. The key idea in our approach is to modify the original optimization problem by adding K independent Gaussian priors and a sparsityinducing prior over the parameters. We show that our approach is easy to implement using existing neural network libraries, generalizes 1 and 2 regularization, and elegantly enforces parameter tying/pruning constraints on all parameters across the network. Experimentally, we demonstrate that our method yields state-of-the-art compression on several standard benchmarks with minimal loss in accuracy while requiring significantly less hyperparameter tuning compared with related, competing approaches.

As the authors mentioned, weight-sharing and pruning are not new to neural network compression. The proposed method resembles a lot with the deep compression work (Han et. al. 2016), with the distinction of clustering across different layers and a Lasso regularizer to encourage sparsity of the weights. Even though the change seems minimal, the authors has demonstrated the effectiveness on the benchmark.
But the description of the optimization strategy in Section 3 needs some refinement. In the soft-tying stage, why only the regularizer (1) is considered, not the sparsity one? In the hard-tying stage, would the clustering change in each iteration? If not, this has reduced to the constrained problem as in the Hashed Compression work (Chen et. al. 2015) where the regularizer (1) has no effect since the clustering is fixed and all the weights in the same cluster are equal. Even though it is claimed that the proposed method does not require a pre-trained model to initialize, the soft-tying stage seems to take the responsibility to "pre-train" the model.
The experiment section is a weak point. It is much less convincing with no comparison result of compression on large neural networks and large datasets. The only compression result on large neural network (VGG-11) comes with no baseline comparisons. But it already tells something: 1) what is the classification result for reference network without compression? 2) the compression ratio has significantly reduced comparing with those for MNIST. It is hard to say if the compression performance could generalize to large networks.
Also, it would be good to have an ablation test on different parts of the objective function and the two optimization stages to show the importance of each part, especially the removal of the soft-tying stage and the L1 regularizer versus a simple pruning technique after each iteration. This maybe a minor issue, but would be interesting to know: what would the compression performance be if the classification accuracy maintains the same level as that of the deep compression. As discussed in the paper, it is a trade-off between accuracy and compression. The network could be compressed to very small size but with significant accuracy loss.
Some minor issues:
- In Section 1, the authors discussed a bunch of pitfalls of existing compression techniques, such as large number of parameters, local minimum issues and layer-wise approaches. It would be clearer if the authors could explicitly and succinctly discuss which pitfalls are resolved and how by the proposed method towards the end of the Introduction section.
- In Section 4.2, the authors discussed the insensitivity of the proposed method to switching frequency. But there is no quantitative results shown to support the claims.
- What is the threshold for pruning zero weight used in Table 2?
- There are many references and comparisons missing: Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations in NIPS 17 for instance. This paper also considers quantization for compression which is related to this work.


A fundamental, and still largely unanswered, question in the context of Generative Adversarial Networks (GANs) is whether GANs are actually able to capture the key characteristics of the datasets they are trained on. The current approaches to examining this issue require significant human supervision, such as visual inspection of sampled images, and often offer only fairly limited scalability. In this paper, we propose new techniques that employ classification-based perspective to evaluate synthetic GAN distributions and their capability to accurately reflect the essential properties of the training data. These techniques require only minimal human supervision and can easily be scaled and adapted to evaluate a variety of state-of-the-art GANs on large, popular datasets. They also indicate that GANs have significant problems in reproducing the more distributional properties of the training dataset. In particular, the diversity of such synthetic data is orders of magnitude smaller than that of the original data.

Overall comments: Trying to shed light at comparison between different GAN variants, but the metrics introduced are not very novel, results are not comparable with prior work and older version of certain models are used (WGAN instead of Improved WGAN)
Section 2.1: quantifying mode collapse
* This section should mention Inception scores. A model which collapses on only one class will have a low inception score, and this metric also uses a conv net classifier, as the approach is very similar (the method is only mentioned briefly in section 2.3)
* The authors might not be aware of concurrent work published before the ICLR deadline, which introduces a very similar metric: https://arxiv.org/abs/1706.08500
Section 2.2: measuring diversity:
* There is an inherent flaw in this metric, namely it trains one GAN per class. One cannot generalize from this metric on how different GAN models will perform when trained on the entire dataset. One model might be able to capture more diverse distributions, but lose a bit of quality, while another model might be able to create good samples when train on low diversity data. We already know that when looking at other generative models, we can find such examples. VAEs can obtain very good samples on celebA, a dataset with relative low diversity, but not so good samples on cifar.
* The authors compare their experiment with Radford et al. (2015), but that needs to be done with caution. In Radford et al. (2015), the authors use a conditional generative model trained on the entire dataset. In that setting, this test is more suitable since one can test how good well the model has learned the conditioning. For example, for a conditional model trained on cats and dogs, a failure mode is that the model generates only cats. This failure mode can then be captured by this metric. However, when training two models, one on cats and one on dogs, this failure mode is not present since the data is already split into classes.
* The proposed metric is not necessarily a diversity metric, it is also a quality metric:  in a situation where all the models diverge and generate random noise, with high diversity, but without any structure. This metric would capture this issue, because a classifier will not be able to learn the classes, because there is no correlation between the classes and the generated images.
Experimental results:
* Positive insights regarding labels and celeba. Looks like subtle labels on faces are not being captured by GAN models.
Figure 1 is hard to read.
* ALI having higher diversity on celeba is explicitly mentioned in a paper the authors cite, namely “Do GANs actually learn the distribution? An empirical study”. Would be nice to mention that in the paper.
Would like to see:
* A comparison with the Improved Wasserstein GAN model. This model is now the one used by the community, as opposed to the original Wasserstein GAN.
* Models trained on cifar, with the reported inception scores of the models on cifar. That makes the paper comparable with previous work and is a test against bugs in model implementations or other parts of the code. This would also allow to test for claims such as the fact that the Improved GAN has more mode collapse than DCGAN, while the Improved GAN paper says the opposite.
The reason why the authors chose the models they did for comparison. In the BiGAN (same model as ALI) paper the authors report a low inception score, which suggests that their model is not able to capture the subtleties of the Cifar dataset, and this seems to be correlated with the results obtained in this work.


CAN NEURAL NETWORKS UNDERSTAND LOGICAL ENTAILMENT?
We introduce a new dataset of logical entailments for the purpose of measuring models' ability to capture and exploit the structure of logical expressions against an entailment prediction task. We use this task to compare a series of architectures which are ubiquitous in the sequence-processing literature, in addition to a new model class-PossibleWorldNets-which computes entailment as a "convolution over possible worlds". Results show that convolutional networks present the wrong inductive bias for this class of problems relative to LSTM RNNs, treestructured neural networks outperform LSTM RNNs due to their enhanced ability to exploit the syntax of logic, and PossibleWorldNets outperform all benchmarks.

SUMMARY
The paper is fairly broad in what it is trying to achieve, but the approach is well thought out. The purpose of the paper is to investigate the effectiveness of prior machine learning methods with predicting logical entailment and then provide a new model designed for the task. Explicitly, the paper asks the following questions: "Can neural networks understand logical formula well enough to detect entailment?", and "Which architectures are best at inferring, encoding, and relating features in a purely structural sequence-based problem?". The goals of the paper is to understand the learning bias of current architectures when they are tasked with learning logical entailment. The proposed network architecture, PossibleWorldNet, is then viewed as an improvement on an earlier architecture TreeNet.
POSITIVES
The structure of this paper was very well done. The paper attempts to do a lot, and succeeds on most fronts. The generated dataset used for testing logical entailment is given a constructive description which allows for future replication. The baseline benchmark networks are covered in depth and the reader is provided with a deep understanding on the limitations of some networks with regard to exploiting structure in data. The PossibleWorldNets is also given good coverage, and the equations provided show the means by which it operates.
• A clear methodological approach to the research. The paper covers how they created a dataset which can be used for logical entailment learning, and then explains clearly all the previous network models which will be used in testing as well as their proposed model.
• The background information regarding each model was exceptionally thorough. The paper went into great depth describing the pros and cons of earlier network models and why they may struggle with recognizing logical entailment.
• The section describing the creation of a dataset captures the basis for the research, learning logical entailment. They describe the creation of the data, as well as the means by which they increase the difficulty for learning.
• The paper provides an in depth description of their PossibleWorldNet model, and during experimentation we see clear evidence of the models capabilities.
NEGATIVES
One issue I had with the paper is regarding the creation of the logical entailment dataset. Not so much for how they explained the process of creating the dataset, that was very thorough, but the fact that this dataset was the only means to test the previous network models and their new proposed network model. I wonder if it would be better to find non-generated datasets which may contain data that have entailment relationships. It is questionable if their hand crafted network model is learned best on their hand crafted dataset.
The use of a singular dataset for learning logical entailment. The dataset was also created by the researchers for the express purpose of testing neural network capacity to learn logical entailment. I am hesitant to say their proposed network is an incredible achievement since PossibleWorldNet effectively beat out other methods on a dataset that they created expressly for it.
RELATED WORK
The paper has an extensive section dedicated to covering related work. I would say the research involved was very thorough and the researchers understood how their method was different as well as how it was improving on earlier approaches.
CONCLUSION
Given the thorough investigation into previous networks’ capabilities in logical entailment learning, I would accept this paper as a valid scientific contribution. The paper performs a thorough analysis on the limitations that previous networks face with regard to exploiting structure from data. The paper also covers results of the experiments by not only pointing out their proposed network’s success, but by analyzing why certain earlier network models were able to achieve competitive learning results. The structure of the PossibleWorldNet was also explained well, and during ex- perimentation demonstrated its ability to learn structure from data. The paper would have been improved through testing of multiple datasets, and not just on there self generated dataset, but the contribution of their research on their network and older networks is still justification enough for this paper.


In the past decade, many urban areas in China have suffered from serious air pollution problems, making air quality forecast a hot spot. Conventional approaches rely on numerical methods to estimate the pollutant concentration and require lots of computing power. To solve this problem, we applied the widely used deep learning methods. Deep learning requires large-scale datasets to train an effective model. In this paper, we introduced a new dataset, entitled as AirNet 1 , containing the 0.25 degree resolution grid map of mainland China, with more than two years of continued air quality measurement and meteorological data. We published this dataset as an open resource for machine learning researches and set up a baseline of a 5-day air pollution forecast. The results of experiments demonstrated that this dataset could facilitate the development of new algorithms on the air quality forecast.

This paper's main contribution is in the building of a spatio-temporal data set on air pollution indicators as the title states.
The data set is built from open source data to comprise pollutants measured at a number of stations and meteorological data. Then, an air pollutant predictor is built as a baseline machine learning model with a reducedLSTM model.
Most of the first part's work is in the extraction of the public data from the above mentioned sources, aligning of the two source data and sampling considerations.
The paper lacks detailed explanation of the problem it is actually addressing by omitting the current systems' performance: simply stating: 1.1/page 2 "Thus it became essential and urgent to set up a larger scale training dataset to enhance the accuracy of the forecast results."
It also lacks definition of certain application domain area terms and acronyms (PM2:5).
Certain paragraphs need rewriting:
     - 2.2/Page 3: "Latitude ranges from 75 degrees to 132 degrees and the north latitude range of is from 18 degrees to 51 degrees".
     - 3.1/Page 4: "We converted the problem of the pollutant prediction as time sequential prediction problems, as in the case of giving the past pollutant concentration x0 to xt􀀀1.".
Also, Table 1: GFS Field Description contains 6 features  not 7 as stated in 2.1
For air pollutant prediction a baseline machine learning model is built with a reducedLSTM model.
Results seem promising but lack serious comparison with currently obtained results by other approaches as mentioned above. The statement in 5./Page 7:"Furthermore, reduced LSTM is improved than LSTM, we assumed this is because our equation considered air pollutant dynamics, thus we gave more information to model than LSTM while keeping LSTMs advantage." attributes the enhanced results to extra data (quantity) fed to the model rather than the fact (quality) as stated in the paper that the meteorological conditions (dispersion etc.) influence the air pollutant presence/ concentrations in nearby stations.
A rewriting and clarification of certain paragraphs is therefore recommended.


Domain adaptation is critical for success in new, unseen environments. Adversarial adaptation models applied in feature spaces discover domain invariant representations, but are difficult to visualize and sometimes fail to capture pixel-level and low-level domain shifts. Recent work has shown that generative adversarial networks combined with cycle-consistency constraints are surprisingly effective at mapping images between domains, even without the use of aligned image pairs. We propose a novel discriminatively-trained Cycle-Consistent Adversarial Domain Adaptation model. CyCADA adapts representations at both the pixel-level and feature-level, enforces cycle-consistency while leveraging a task loss, and does not require aligned pairs. Our model can be applied in a variety of visual recognition and prediction settings. We show new state-of-the-art results across multiple adaptation tasks, including digit classification and semantic segmentation of road scenes demonstrating transfer from synthetic to real world domains.

This paper essentially uses CycleGANs for Domain Adaptation. My biggest concern is that it doesn't adequately compare to similar papers that perform adaptation at the pixel level (eg. Shrivastava et al-'Learning from Simulated and Unsupervised Images through Adversarial Training' and Bousmalis et al - 'Unsupervised Pixel-level Domain Adaptation with GANs', two similar papers published in CVPR 2017 -the first one was even a best paper- and available on arXiv since December 2016-before CycleGANs). I believe the authors should have at least done an ablation study to see if the cycle-consistency loss truly makes a difference on top of these works-that would be the biggest selling point of this paper. The experimental section had many experiments, which is great. However I think for semantic segmentation it would be very interesting to see whether using the adapted synthetic GTA5 samples would improve the SOTA on Cityscapes. It wouldn't be unsupervised domain adaptation, but it would be very impactful. Finally I'm not sure the oracle (train on target) mIoU on Table 2 is SOTA, and I believe the proposed model's performance is really far from SOTA.
Pros:
* CycleGANs for domain adaptation! Great idea!
* I really like the work on semantic segmentation, I think this is a very important direction
Cons:
* I don't think Domain separation networks is a pixel-level transformation-that's a feature-level transformation, you probably mean to use Bousmalis et al. 2017. Also Shrivastava et al is missing from the image-level papers.
* the authors claim that Bousmalis et al, Liu & Tuzel and Shrivastava et al ahve only been shown to work for small image sizes. There's a recent work by Bousmalis et al. (Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping) that shows these methods working well (w/o cycle-consistency) for settings similar to semantic segmentation at a relatively high resolution. Also it was mentioned that these methods do not necessarily preserve content, when pixel-da explicitly accounts for that with a task loss (identical to the semantic loss used in this submission)
* The authors talk about the content similarity loss on the foreground in Bousmalis et al. 2017, but they could compare to this method w/o using the content similarity or using a different content similarity tailored to the semantic segmentation tasks, which would be trivial.
* Math seems wrong in (4) and (6). (4) should be probably have a minus instead of a plus. (6) has an argmin of a min, not sure what is being optimized here. In fact, I'm not sure if eg you use the gradients of f_T for training the generators?
* The authors mention that the pixel-da approach cross validates with some labeled data. Although I agree that is not an ideal validation, I'm not sure if it's equivalent or not the authors' validation setting, as they don't describe what that is.
* The authors present the semantic loss as novel, however this is the task loss proposed by the pixel-da paper.
* I didn't understand what pixel-only and feat-only meant in tables 2, 3, 4. I couldn't find an explanation in captions or in text
=====
Post rebuttal comments:
Thanks for adding content in response to my comments. The cycle ablation is still a sticky point for me, and I'm still left not sure if cycle-consistency really offers an improvement. Although I applaud your offering examples of failures for when there's no cycle-consistency, these are circumstantial and not quantitative.  The reader is still left wondering why and when is the cycle-consistency loss is appropriate. As this is the main novelty, I believe this should be in the forefront of the experimental evaluation.