DATA-DEPENDENT CORESETS FOR COMPRESSING NEURAL NETWORKS WITH APPLICATIONS TO GENER- ALIZATION BOUNDS
We present an efficient coresets-based neural network compression algorithm that sparsifies the parameters of a trained fully-connected neural network in a manner that provably approximates the network's output. Our approach is based on an importance sampling scheme that judiciously defines a sampling distribution over the neural network parameters, and as a result, retains parameters of high importance while discarding redundant ones. We leverage a novel, empirical notion of sensitivity and extend traditional coreset constructions to the application of compressing parameters. Our theoretical analysis establishes guarantees on the size and accuracy of the resulting compressed network and gives rise to generalization bounds that may provide new insights into the generalization properties of neural networks. We demonstrate the practical effectiveness of our algorithm on a variety of neural network configurations and real-world data sets.
4. Analytical results establishing guarantees on the approximation accuracy, size, and generalization of the compressed neural network.
5. Evaluations on real-world data sets that demonstrate the practical effectiveness of our algorithm in compressing neural network parameters and validate our theoretical results.

Given an additively decomposable function F(X, Q) = sum_over_x_in_X cost(x, Q), one can approximate it using either random sampling of x in X (unbiased, possibly high variance), or using importance sampling and replace the sum_over_x with a sum_over_coreset importance_of_a_point * cost(x, Q) which if properly defined can be both unbiased and have low variance [1]. In this work the authors consider the weighted sum of activations as F and suggest that for each neuron we can subsample the incoming edges. To construct the importance sampling strategy the authors adapt the classic notion of sensitivity from the coreset literature. Then, one has to carefully balance the approximation quality from one layer to the next and essentially union bound the results over all layers and all sampled points. The performed analysis is sound (up to my knowledge).
Pro:
- I commend the authors for a clean and polished writeup.
- The analysis seems to be sound (apart from the issues discussed below)
- The experimental results look promising, at least in the limited setup.
Con:
- There exists competing work with rigorous guarantees, for example [2].
- The analysis hinges on two assumptions which, in my opinion, make the problem feasible: having (sub) exponential tails allows for strong concentration results, and with proper analysis (as done by the authors), the fact that the additively decomposable function can be approximated given well-behaving summands is not surprising. The analysis is definitely non-trivial and I commend the authors for a clean writeup.
- While rigorous guarantees are lacking for some previous work, previously introduced techniques were shown to be extremely effective in practice and across a spectrum of tasks. As the guarantees arguably stem from the assumptions 1 and 2, I feel that it’s unfair to not compare to those results empirically. Hence, failing to compare to results of at least [2, 3] is a major drawback of this work.
- The result holds for n points drawn from P. However, in practice the network might receive essentially arbitrary input from P at inference time. Given that we need to decide on the number of edges to preserve apriori, what are the implications?
- The presented bounds should be discussed on an intuitive level (i.e. the number of non zero entries is approximately cubic in L).
I consider this to be a well-executed paper which brings together the main ideas from the coreset literature and shows one avenue of establishing provable results. However, given that no comparison to the state-of-the-art techniques is given I'm not confident that the community will apply these techniques in practice. On the other hand, the main strength -- the theoretical guarantees -- hinge on the introduced assumptions. As such, without additional empirical results demonstrating the utility with respect to the state-of-the-art methods (for the same capacity in terms of NNZ) I cannot recommend acceptance.
[1] https://arxiv.org/abs/1601.00617
[2] papers.nips.cc/paper/6910-net-trim-convex-pruning-of-deep-neural-networks-with-performance-guarantee
[3] https://arxiv.org/abs/1510.00149
========
Thank you for the detailed responses. Given the additional experimental results and connections to existing work, I have updated my score from 5 to 6.


Using variational Bayes with neural networks, we develop an algorithm capable of accumulating knowledge into a prior from multiple different tasks. This results in a rich prior capable of few-shot learning on new tasks. The posterior can go beyond the mean field approximation and yields good uncertainty on the performed experiments. Analysis on toy tasks show that it can learn from significantly different tasks while finding similarities among them. Experiments on Mini-Imagenet reach state of the art with 74.5% accuracy on 5 shot learning. Finally, we provide two new benchmarks, each showing a failure mode of existing meta learning algorithms such as MAML and prototypical Networks.

The authors state that their goal with this paper is manifold:
They want to learn a prior over neural networks for multiple tasks. The posterior should go beyond mean field inference and yield good results.  The authors claim in their paper that they learn an 'expressive transferable prior over the weights of a network' for multi-task settings, which they denote with the unfortunate term 'deep prior'.
In sec. 2.1 the authors introduce the idea of a hierarchical probabilistic model of weights for a neural network p(W|a) conditioned on task latent variables p(a). They realize that one might want to generate those weights with a function which conditions on variable "z" and has parameters "a". They continue their argument in Sec 2.2 that since the weight scoring can be canceled out in the ELBO, the score of the model does not depend on weights "w" explicitly anymore.
This, of course, is wrong, since the likelihood term in the ELBO still is an expectation over the posterior of q(w|z)q(z).
However, the authors also realize this and continue their argumentation as follows:
In this case -according to the authors- one may drop the entire idea about learning distributions over weights entirely.
The math says: p(y|x ; a) = int_z p(z) int_w p(w|z ; a) p(y|x, w)dw dz.
So the authors claim that a model p(y|x, z) which only conditions on 'z' is the same as the full Bayesian Model with marginalized weights. They then suggest to just use any neural network with parameters "a" to model this p(y|x, z ;a) directly with z being used as an auxiliary input variable to the network with parameters "a" and claim this is doing the same. This is of course utterly misleading, as the parameter "a" in the original model indicated a model mapping a low dimensional latent variable to weights, but now a maps to a neural network mapping a latent variable and an input vector x to an output vector y. As such, these quantities are different and the argument does not hold. Also a point estimate of said mapping will not be comparable to the marginalized p(y|x).
What is more concerning is that the authors claim this procedure is equivalent to learning a distribution over weights and call the whole thing a deep prior, while this paper contains no work on trying to perform the hard task of successfully parametrizing a high-dimensional conditional distribution over weights p(w|z) (apart from a trivial experiment generating all of them at once  from a neural network for a single layer in a failed experiment) but claims to succeed in doing so by circumventing it entirely.
In their experiments, the authors also do not actually successfully try to really learn a full distribution over the weights of a neural network. This alone suffices to realize that the paper appears to be purposefully positioned in a highly misleading way and makes claims about weight priors that are superficially discussed in various sections but never actually executed on properly in the paper.
This is a disservice to the hard work many recent and older papers are doing in actually trying to derive structured hierarchical weight distributions for deep networks, which this paper claims is a problem they find to be 'high dimensional and noisy', which is exactly why it is a valid research avenue to begin with that should not be trivially subsumed by work such as this.
When reducing this paper to the actual components it provides, it is a simple object: A deterministic neural network with an auxiliary, task-dependent latent variable which provides extra inputs to model conditional densities.
Such ideas have been around for a while and the authors do not do a good job of surveying the landscape of such networks with additional stochastic input variables.
One example is "Learning Stochastic Feedforward Neural Networks" by Tang and Salakhutdinov, NIPS 2013, a more recent one is "Uncertainty Decomposition in Bayesian Neural Networks with Latent Variables" by Depeweg et al 2017.
An obvious recent example of multi-task/meta/continual learning comparators would be "VARIATIONAL CONTINUAL LEARNING" by Nguyen et al. and other work from the Cambridge group that deals with multi-task and meta-learning and priors for neural networks.
Another weakness of the paper is that the main driver of success in the paper's experiment regarding classification is the prototypical network idea, rather than anything else regarding weight uncertainty which seems entirely disentangled from the core theoretical statements of the paper.
All in all, I find this paper unacceptably phrased with promises it simply does not even attempt to keep and a misleading technical section that would distort the machine learning literature without actually contributing to a solution to the technical problems it claims to tackle (in relation to modeling weight uncertainty/priors on NN). Paired with the apparent disinterest of the authors to cite recent and older literature executing strongly related underlying ideas combining neural networks with auxiliary latent variables, I can only recommend that the authors significantly change the writing and the attribution of ideas in this paper for a potential next submission focusing on multi-task learning and clarify and align the core ideas in the theory sections and the experiment sections.


We investigate Relational Graph Attention Networks, a class of models that extends non-relational graph attention mechanisms to incorporate relational information, opening up these methods to a wider variety of problems. A thorough evaluation of these models is performed, and comparisons are made against established benchmarks. To provide a meaningful comparison, we retrain Relational Graph Convolutional Networks, the spectral counterpart of Relational Graph Attention Networks, and evaluate them under the same conditions. We find that Relational Graph Attention Networks perform worse than anticipated, although some configurations are marginally beneficial for modelling molecular properties. We provide insights as to why this may be, and suggest both modifications to evaluation strategies, as well as directions to investigate for future work.

This paper presented a relational graph attention networks that could consider both node
features and relational information (edge features) to perform node-level and graph-level
classifications. The basic idea is to combine the graph attention networks (Veličković et
al. 2017) and the relational graph networks (Schlichtkrull et al. 2018) to derive a hybrid
networks. This paper is generally easy to follow and written clearly. Several experiments
are conducted to demonstrate the performance of the proposed model. Although some promising
results have been achieved, I think there are several limitations regarding the novelty and
significance of the proposed model.
i) The proposed architecture is mainly adopted from the graph attention networks (Veličković
et al. 2017) and the relational graph networks (Schlichtkrull et al. 2018). Such a simple
combination is a good attempt to incorporate both node features and edge features but the
novelty is quite limited.
ii) In table 2, I don’t really see any promising results compared to baselines. There are
little improvements over the baselines or even significantly worse. More importantly,
compared two schemes of this work, the ones with attentions are “almost” identical with ones
without attentions, which implies that the proposed attentions mechanism is not really useful
in practice. For most of newly proposed graph embedding algorithms, it is hard to convince
it is indeed better without some significant improvements (at least 2% absolute accuracy more).
iii) For MUTAG dataset, the statistical information of this dataset is quite different from
what I used to use. MUTAG is a standard dataset for testing graph-level classification for
both graph kernels and graph networks. MUTAG is a dataset of 188 mutagenic aromatic and
heteroaromatic nitro compounds with 7 discrete node labels. Each chemical compound is labeled
according to whether it has mutagenic effect on the gram- negative bacterium Salmonnella
Typhimurium. Could you explain why your MUTAG is now a single graph and is cast as node
classification problem?


WASSERSTEIN BARYCENTER MODEL ENSEMBLING
In this paper we propose to perform model ensembling in a multiclass or a multilabel learning setting using Wasserstein (W.) barycenters. Optimal transport metrics, such as the Wasserstein distance, allow incorporating semantic side information such as word embeddings. Using W. barycenters to find the consensus between models allows us to balance confidence and semantics in finding the agreement between the models. We show applications of Wasserstein ensembling in attribute-based classification, multilabel learning and image captioning generation. These results show that the W. ensembling is a viable alternative to the basic geometric or arithmetic mean ensembling.

Paper overview: Model ensembling techniques aim at improving machine learning model prediction results by i) executing several different algorithms on the same task and ii) solving the discrepancies in the responses of all the algorithms, for each task. Some common methods are voting and averaging (arithmetic or geometric average) on the results provided by the different algorithms.
Since averaging amounts to computing barycenters with different distance functions, this paper proposes to use the Wassertein barycenter instead of the L2 barycenter (arithmetic average) or the extended KL barycenter (geometric mean).
Remarks, typos and experiences that would be interesting to add:
     1) Please define the acronyms before using them, for instance DNN (in first page, 4th line), KL (also first page), NLP, etc.
    2) In practice, when ensembling different methods, the geometric and arithmetic mean are not computed with equal weights ($\lambda_l$ in Definition 1). Instead, these weights are computed as the optimal values for a given small dev-set. It would be interesting to see how well does the method compare to these optimal weighted averages, and also if it improves is we also compute the optimal $\lambda_l$ for the Wasserstein barycenter.
    3) How computationally expensive are these methods?
    4) So the output of the ensembling method is a point in the word embedding space, but we know that not all points in this space have an associated word, thus, how are the words chosen?
    5) The image captioning example of Fig.4 is very interesting (although the original image should be added to understand better the different results), can you show also some negative examples? That is to say, when is the Wassertein method is failing but not the other methods.
Points in favor:
     1)Better results: The proposed model is not only theoretically interesting, but it also improves the arithmetic and geometric mean baselines.
    2) Interesting theoretical and practical properties: semantic accuracy, diversity and robustness (see Proposition 1).
Points against: The paper is not easy to read. Ensembling methods are normally applied to the output of a classifier or a regression method, so it is not evident to understand why the 'underlying geometry' is in the word embedding space (page 2 after the Definition 1). I think this is explained in the second paragraph of the paper, but that paragraph is really not clear. I assume that is makes sense to use the word-embedding space for the image caption generation or other ML tasks where the output is a word, but I am not sure how this is used in other cases.
Conclusion: The paper proposes a new method for model assembling by rethinking other popular methods such as the arithmetic and geometric average. It also shows that it improves the current methods. Therefore, I think it presents enough novelties to be accepted in the conference.


LEARNING TO REPRESENT EDITS
We introduce the problem of learning distributed representations of edits. By combining a "neural editor" with an "edit encoder", our models learn to represent the salient information of an edit and can be used to apply edits to new inputs. We experiment on natural language and source code edit data. Our evaluation yields promising results that suggest that our neural network models learn to capture the structure and semantics of edits. We hope that this interesting task and data source will inspire other researchers to work further on this problem.

The authors state nicely and clearly the main contributions they see in their work (Intro, last paragraph). Specifically the state the paper: 1) present a new and important machine learning task, 2) present a family of models that capture the structure of edits and compute efficient representations, 3) create a new source code edit dataset, 4) perform a set of experiments on the learned edit representations and present promising empirical evidence that the models succeed in capturing the semantics of edits.
We decided to organize this review by commenting on the above-stated contributions one at a time:
“A new and important machine learning task”
Regarding “new task”:
PRO: We are unfamiliar with past work which presents this precise task; the task is new. Section 5 makes a good case for the novelty of this work.
CON: None.
Regarding “important task”:
PRO: The authors motivate the task with tantalizing prospective applications-- automatically editing text and code, e.g. for grammar, clarity, and style. Conceptualizing edits as NLP objects of interest that can be concretely represented, clustered, and used for prediction is an advance.
CON: Many text editors, office suites, and coding IDEs already include features which automatically suggest or apply edits for grammar, clarity, and style. The authors do not describe shortcomings in existing tools that might be better addressed using distributed representations of edits. Consequently, the significance of the proposed contribution is unclear.
“A family of models that capture the structure of edits and compute efficient representations”
Regarding “a family of models”:
PRO: The family of models presented by the authors clearly generalizes: such models may be utilized for computational experiments on datasets and edit types beyond those specifically utilized in this evaluation. The authors apply well-utilized neural network architectures that may be trained and applied to large datasets. The architecture of the neural editor permits evaluation of the degree to which the editor successfully predicts the correct edit given a pre-edit input and a known representation of a similar edit.
CON: The authors do not propose any scheme under which edit representations might be utilized for automatically editing text or code when an edit very similar to the desired edit is not already known and its representation available as input. Hence, we find the authors do not sufficiently motivate the input scheme of their neural editor. The input scheme of the neural editor makes trivial the case in which no edit is needed, as the editor would learn during training that the output x+ should be the same as the input x- when the representation of the “zero edit” is given as input. While the authors discuss the importance of “bottlenecking” the edit encoder so that it does not simply learn to encode the desired output x+, they do not concretely demonstrate that the edit encoder has done otherwise in the final experiments. Related to that: If the authors aimed to actually solve automated edits in text/code then it seems crucial their data contained "negative examples" i.e. segments which require no edits. In such an evaluation one would test also when the algorithm introduces unnecessary/erroneous edits.
Regarding “capture structure of edits”:
PRO: The authors present evidence that edit encoders tightly cluster relatively simple edits which involve adding or removing common tokens. The authors present evidence that relatively simple edits completed automatically by a “fixer” often cluster together, i.e. a known signal is retained in clustering. The authors present evidence that the nearest neighbors of edits in an edit-representation space often are semantically or structurally similar, as judged by human annotators. Section 4.3 includes interesting observations comparing edit patterns better captured by the graph or seq edit encoders.
CON: The details of the human annotation tasks which generated the numerical results in Tables 1 and 2 are unclear: were unbiased third parties utilized? Were the edits stripped of their source-encoder label when evaluated? Objectively, what separates an “unrelated” from a “similar” edit, and what separates a “similar” from a “same” edit? Did multiple human annotators undertake this task in parallel, and what was their overall concordance (e.g. “intercoder reliability”)? Without concrete answers to these questions, the validity and significance of the DCG/NDCG results reported in Tables 1 and 2 are unclear. It is not clear from the two examples given in Table 1 that the three nearest neighbors embedded by the Seq encoder are “better”, i.e. overall more semantically and/or syntactically similar to the example edit, than those embedded by the Bag of Words model. It is unclear which specific aspects of “edit structure” are better captured by the Seq encoder than the Bag of Words model. The overall structure of Tables 1 and 2 is awkward, with concrete numerical results dominated by a spatially large section containing a small number of examples.
“create a new source code edit dataset”
PRO: The authors create a new source code edit dataset, an important contribution to the study of this new task.
CON: Minor: is the provided dataset large enough to do more than simple experiments? See note below on sample size.
“present promising empirical evidence that the models succeed in capturing the semantics of edits”
PRO: The experiment results show how frequently the end-to-end system successfully predicted the correct edit given a pre-edit input and a known representation of a similar edit. Gold standard accuracies of more than 70%, and averaged transfer learning accuracies of more than 30%, suggest that this system shows promise for capturing the semantics of edits.
CON: Due to concerns expressed above about the model design and evaluation of the edit representations, it remains unclear to what degree the models succeed in capturing the semantics of edits. Table 11 shows dramatic variation in success levels across fixer ID in the transfer learning task, yet the authors do not propose ways their end-to-end system might be adjusted to address areas of weak performance. The authors do not discuss the impact of training set size on their evaluation metrics. The authors do not discuss the degree to which their model training task would scale to larger language datasets such as those needed for the motivating applications.
##############
Based on the authors' response, revisions, and disucssions we have updated the review and the score.


LEARNING WHEN TO COMMUNICATE AT SCALE IN MULTIAGENT COOPERATIVE AND COMPETITIVE TASKS
Learning when to communicate and doing that effectively is essential in multi-agent tasks. Recent works show that continuous communication allows efficient training with back-propagation in multiagent scenarios, but have been restricted to fullycooperative tasks. In this paper, we present Individualized Controlled Continuous Communication Model (IC3Net) which has better training efficiency than simple continuous communication model, and can be applied to semi-cooperative and competitive settings along with the cooperative settings. IC3Net controls continuous communication with a gating mechanism and uses individualized rewards for each agent to gain better performance and scalability while fixing credit assignment issues. Using variety of tasks including StarCraft BroodWars TM explore and combat scenarios, we show that our network yields improved performance and convergence rates than the baselines as the scale increases. Our results convey that IC3Net agents learn when to communicate based on the scenario and profitability.

This work is an extension to the work of Sukbaatar et al. (2016) with two main differences:
1) Selective communication: agents are able to decide whether they want to communicate.
2) Individualized reward: Agents receive individual rewards; therefore, agents are aware of their contribution towards the goal.
These two new extensions enable their model to work in either cooperative or a mix of competitive and competitive/collaborative settings. The authors also claim these two extensions enable their model to converge faster and better.
The paper is well written, easy to follow, and everything has been explained quite well. The experiments are competent in the sense that the authors ran their model in four different environments (predator and prey, traffic junction, StarCraft explore, and StarCraft combat). The comparison between their model with three baselines was extensive; they reported the mean and variance over different runs. I have some concerns regarding their method and the experiments which are brought up in the following:

Method:
In a non-fully-cooperative environment, sharing hidden state entirely as the only option for communicate is not very reasonable; I think something like sending a message is a better option and  more realistic (e.g., something like the work of Mordatch & Abbeel, 2017)
Experiment:
The experiment "StarCraft explore" is similar to predator-prey; therefore, instead of explaining StarCraft explore, I would like to see how the model works in StarCraft combat. Right now, the authors explain a bit about the model performance in Starcraft combat, but I found the explanation confusing.

Authors provide 3 baselines:
1) no communication, but IR
2) no communication, no IR
3) global communication, no IR (commNet)
I think having a baseline that has global communication with IR can show the effect of selective communication better.
There are some questions in the experiment section that have not been addressed very well. For example:
 Is there any difference between the results of table 1, if we look at the cooperative setup?
Does their model outperform a model which has global communication with IR?
Why do IRIC and IC work worst in the medium in comparison to hard in TJ in table1?
Why is CommNet work worse than IRIC and IC in table 2?


Model-based reinforcement learning (RL) methods can be broadly categorized as global model methods, which depend on learning models that provide sensible predictions in a wide range of states, or local model methods, which iteratively refit simple models that are used for policy improvement. While predicting future states that will result from the current actions is difficult, local model methods only attempt to understand system dynamics in the neighborhood of the current policy, making it possible to produce local improvements without ever learning to predict accurately far into the future. The main idea in this paper is that we can learn representations that make it easy to retrospectively infer simple dynamics given the data from the current policy, thus enabling local models to be used for policy learning in complex systems. We evaluate our approach against other model-based and model-free RL methods on a suite of robotics tasks, including a manipulation task on a real Sawyer robotic arm directly from camera images.

In this work the authors propose an end to end approach for model based reinforcement learning from images, where the main building blocks are locally-linear dynamical systems and variational auto-encoders (VAE). Specifically, it is assumed that the input features (i.e., the images) are generated from a low dimensional latent representation mapped through parametric random functions; the latter are modeled via neural networks. A recognition model based on convolutional neural networks operates on the reverse way and is responsible for projecting the input features to the latent space, in order to proceed with the reinforcement learning task. The variational framework is employed in order to jointly learn the VAE and the linear dynamics on the latent state. As a final step, once the model is fitted a linear quadratic system (LQS) is solved in order to learn the cost function and the optimal policy.
* The paper is well motivated and tries to solve an interesting problem, that of data-efficient reinforcement learning. The experiments are well picked and demonstrate the advantages of the proposed approach towards solving the task, however, the method is only evaluated on few environments and compared against only a couple of other methods. I would expect a broader evaluation and/or comparison against more methods. Since the model is able to reach TRPO’s performance in much less steps it would be nice to see how it performs against PPO from [Schulman et al. 2017] (at least on the simulated environments). Also, would it make sense to compare against [Levine et al. 2016] that has been evaluated on similar tasks?
[Schulman et al 2017] “Proximal Policy Optimization Algorithms”.
[Levine et al. 2016] “End-to-End Training of Deep Visuomotor Policies”.
* Methodologically, the paper is sound. The model part (as the authors point out) is based on [Johnson et al. 2016] and is well explained. On the other hand, the policy part, and in particular the policy update in Section 4.2 has some issues regarding readability. There is a strong interplay between Section 4.2, Section 2.1 and Appendix D and the authors did not manage to nicely explain what exactly is happening during the update phase. In the beginning the reader has the impression that we are finding the optimal policy via the closed-form LQS. Later on we switch to constrained optimisation for the cost by accounting for the KL divergence between the policy on two episodes. Finally, in the appendix we are back to the original quadratic cost. The authors need to clarify all the above. Also, they need to explicitly mention why they opt for stochastic optimisation (is it because of minibatching?)
* To continue with the policy, in Section 4.2 the authors argue that although the optimal policy can be found in closed form this is not desirable because the policy will overfit the model and will not generalise well in the real environment. I disagree with this statement. If this happens it effectively means that the learned model or the assumption/learning of the linear dynamics is not right. The authors seem to also agree with this since they clearly state in the the experimental section that “... our method does not heavily rely on an accurate model...”. To my understanding, this means that we need to refine the modelling strategy and not learn a sub-optimal policy. I am really interested in the authors opinion on that.
* The above argument is also directly related to the recognition model and learning of the policy in the latent state (I completely agree with that). The recognition network, which in this case is a convolutional neural network, is used as an inference mechanism to project the observations to the latent space. We learn the (variational) parameters of the recognition model by optimising the likelihood’s lower bound. This means that we are “allowed” to overfit the variational parameters as long as the bound gets tighter. This can possibly result in degraded performance during the policy update. Furthermore, the variational distribution of the latent state, i.e., q(z_t | s_t) is assumed to be mean field across time (independent z’s), while clearly this is not the case in the posterior. You somehow mitigate that by augmenting the observed state (feeding consecutive frames to the network), but still this is not ideal. Finally, is there a reason why we only use the mean of the recognition model to fit the cost on the projected latent states? Why are we throwing away the uncertainty? Especially since you do not use an exact solver and follow a stochastic gradient.
* In the end of Section 2.1, the authors argue regarding the fact that the prior work assumes access to a compact low-dimensional representation which does not allow them to perform well on images. Reference is needed.
* In the related work the authors mention modelling bias as a downside of prior work. Can you please elaborate on that? Where does the bias come from and, more importantly, how does your approach overcome this issue?
* In the experiment and specifically in Figure 4 am I right in assuming that the distance to target is measured in actual pixels? Furthermore, why the relevant plot for the reacher task is depicting rewards instead of the distance to target. To me this suggests that the task is not solved. In general what I find very upsetting in the field are plots that only depict accumulated reward for a specific task. There are many situations where the agent learns a weird behaviour that happens to give good rewards (e.g., spinning around the cart-pole), and unfortunately such behaviours are not spotted on the reward plots.
Overall, the paper is nicely presented and definitely an interesting work. However, given the fact that methodologically we have not learned anything new from this paper and in combination with the not satisfying experimental evaluation I warrant for rejection.


DOM-Q-NET: GROUNDED RL ON STRUCTURED LANGUAGE
Building agents to interact with the web would allow for significant improvements in knowledge understanding and representation learning. However, web navigation tasks are difficult for current deep reinforcement learning (RL) models due to the large discrete action space and the varying number of actions between the states. In this work, we introduce DOM-Q-NET, a novel architecture for RL-based web navigation to address both of these problems. It parametrizes Q functions with separate networks for different action categories: clicking a DOM element and typing a string input. Our model utilizes a graph neural network to represent the tree-structured HTML of a standard web page. We demonstrate the capabilities of our model on the MiniWoB environment where we can match or outperform existing work without the use of expert demonstrations. Furthermore, we show 2x improvements in sample efficiency when training in the multi-task setting, allowing our model to transfer learned behaviours across tasks.

Caveat: I am an emergency reviewer filling in for someone that fell through on their commitment to review for ICLR.  The framing of this paper is quite outside my typical area, so I am not super familiar with the related work here, nor do I have time to get familiar with it for this last-minute review.

This paper presents a new model for deep reinforcement learning on web pages, where the system is given a goal (stated in text) and is supposed to interact with the web page (through clicking and entering text) in order to achieve that goal.  The supervision is a positive reward when the sequence of actions taken matches the goal.  The novel model presented in this paper is a modular Q function that incorporates graph embeddings of the web page's DOM, as well as similarity scores between elements in the DOM with words in the goal.

Just judging the presentation of the paper, it looks sound.  The methods seem reasonable (very similar to methods that are known to work well on related problems; more on that below), and the experiments look to be well done.  The paper is reasonably well written.  I don't know the RL community well enough to know how impactful this particular piece of work would be there - it's a new model architecture, basically, that gives improved performance.  I'd probably give a similar paper in my area a 3.5-4 out of 5 for an ACL conference.  The one major drawback I see in this paper is that it is _so_ similar to work on semantic parsing, but doesn't realize it.

I am not a "reinforcement learning" researcher, though I am a "semantic parsing" researcher.  The problem statement in this paper reads to me exactly like a semantic parsing problem: map a piece of text to a statement in some formal language.  In this case, the "statement" is a sequence of actions on the DOM of a web page.  The web page is possibly unseen at test time (the particulars of the data setup weren't totally clear to me), so the model has to be able to handle linking words in the sentence to pieces of the DOM in a way that doesn't rely on having seen those DOM elements during training.  This setup seems almost identical to the WikiTableQuestions dataset (Pasupat and Liang 2015), which has seen several RL-inspired works recently (e.g., https://arxiv.org/abs/1807.02322).  The way that the authors propose to use attention scores in the "global module" is _very_ similar to the linking mechanism proposed by Krishnamurthy, Dasigi and Gardner (EMNLP 2017) for WikiTableQuestions, and the way that the "word-token selection" only allows words in the goal sentence is very reminiscent of Chen Liang's language for parsing questions in WikiTableQuestions, which has similar restrictions for similar reasons.

I think the main difference between what we call "weakly-supervised semantic parsing" and what you call "deep reinforcement learning" is that semantic parsing leverages the fact that we know the language we're parsing into, so we don't need to use model-free RL methods like Q-learning.  We know the model, so we can be much smarter about learning.  Again, I'm not super familiar with the tasks you're looking at here, but I'm pretty sure there are much better _supervised_ learning techniques that you could apply to these problems.

All of this is to say that the methods proposed here look _very_ similar to methods that have been studied for quite a while in the semantic parsing literature (I gave only recent references above, but the basic problems go back decades; e.g., http://aclweb.org/anthology/P09-1010, or http://www.cs.utexas.edu/~ml/papers/senior-aaai-2008.pdf).  Yet this paper only cites recent deep RL papers.  I think the authors would benefit greatly from familiarizing themselves with this literature.  I think the semantic parsing community would also benefit from this, as there are surely ideas in the deep RL community that we could benefit from, too.  But the two communities don't really talk to each other much, it seems, even though in some cases we are working on _very_ similar problems.

So, to summarize: the paper seems reasonable enough.  I'm guessing that the RL community would find it at least moderately interesting, and it appears well written and well executed.  My one concern is that it's totally oblivious to the fact that it's sitting right next to a well-established literature that could probably teach it a thing or two about mapping language to actions.
--------------
After seeing the authors engage at least a little with the related semantic parsing literature, I've increased my score to a 7.


In this paper, we describe the "implicit autoencoder" (IAE), a generative autoencoder in which both the generative path and the recognition path are parametrized by implicit distributions. We use two generative adversarial networks to define the reconstruction and the regularization cost functions of the implicit autoencoder, and derive the learning rules based on maximum-likelihood learning. Using implicit distributions allows us to learn more expressive posterior and conditional likelihood distributions for the autoencoder. Learning an expressive conditional likelihood distribution enables the latent code to only capture the abstract and high-level information of the data, while the remaining information is captured by the implicit conditional likelihood distribution. For example, we show that implicit autoencoders can disentangle the global and local information, and perform deterministic or stochastic reconstructions of the images. We further show that implicit autoencoders can disentangle discrete underlying factors of variation from the continuous factors in an unsupervised fashion, and perform clustering and semi-supervised learning.

This paper introduces the implicit autoencoder, which purports to be a VAE with an implicit encoding and decoding distribution.
My principle problem with the paper and reason for my strong rejection is that there appears to be a complete separation between the discussion and theory of the paper and the actual experiments run.  The paper's discussion and theory all centers around rewriting the ordinary ELBO lower bound on the marginal likelihood in equations (4) through (7) where it is shown that this can be recast in the form of two KL divergences, one between the representational joint q(x,z) = p_data(x) encoder(z|x) and the 'reconstruction joint' r(x,z) = encoder_marginal(z) decoder(x|z), and one between the encoding marginal q(z) and the generative prior p(z).   The entire text of the paper then discusses the similarities between this formulation of the objective and some of the alternatives as well as discussing how this objective might behave in various limits.
However, this is not the objective that is actually trained. In the  "Training Process" section is it revealed that an ordinary GAN discriminator is trained.  The ordinary GAN objective does not minimize a KL divergence, it is a minimax formulation of a Jensen Shannon divergence as the original GAN paper notes.  More specifically, you can optimize a KL divergence with a GAN, as shown in the f-GAN paper (1606.00709) but this requires attention be paid to the functional form of the loss and structure of the discriminator.  No such care was taken in this case.  As such the training process does not minimize the objective derived or discussed.  Not to mention that in practice a further hack is employed wherein only the negative example passes gradients to the generator.
While is not specified in the training process section, assuming the ordinary GAN objective (Equation 1) is used, according to their own reference (AVB) the optimal decoder should be:  D = 1/(1 + r(z,x)/q(z,x))  for which we have that what they deem the 'generative loss of the reconstruction GAN' is T = log(1 + r(z,x)/q(z,x))  .   When we take unbiased gradients of the expectation of this quantity, we do not obtain an unbiased gradient of the KL divergence between q(z,x) and r(z,x).
Throughout the paper, factorized Gaussian distributions are equated with tractable variational approximations.  While it is common to use a mean field gaussian distribution for the decoder in VAEs this is by no means required.  Many papers have investigated the use of more powerful autoregressive or flow based decoders, as this paper itself cites (van der Oord et al. 2016).  The text further misrepresents the current literature when it claims that the IAE uniquely "generalizes the idea of deterministic reconstruction to stochastic reconstruction by learning a decoder distribution that learns to match to the inverse encoder distribution".  All VAEs have employ stochastic reconstruction, if the authors again here meant to distinguish a powerful implicit decoder from a mean field gaussian one, the choice of language here is wrong.
Given that there are three joint distributions in equation (the generative model, the representational joint and the reconstruction joint), the use of Conditional entropy H(x|z) and mutual information I(x, z) are ambiguous.  While the particular joint distribution is implied by context in the equations, please spell it out for the reader.
The "Global vs. Local Decomposition of Information in IAEs" section conflates dimensionality with information capacity.  While these are likely correlated for real neural networks, at least fundamentally an arbitrary amount of information could be stored in even a 1 dimensional continuous random variable.  This is not addressed.
The actual experiments look nice, its just that objective used to train the resulting networks is not the one presented in the paper.
------
In light of the author's response I am changing my review from 2 to 3.  I still feel as though the paper should be rejected.  While I appreciate that there is a clear history of using GANs to target otherwise intractable objectives, I still feel like those papers are all very explicit about the fact that they are modifying the objective when they do so.  I find this paper confusing and at times erroneous.  The added appendix on the bits back argument for instance I believe is flawed.
"It first transmits z, which ideally would only require H(z) bits; however, since the code is designed
under p(z), the sender has to pay the penalty of KL(q(z)kp(z)) extra bits"
False.  The sender is not trying to send an unconditional latent code, they are trying to send the code for a given image, z \sim q(z|x).  Under usual communication schemes this would be sent via an entropic code designed for the shared prior at the cost of the cross entropy \int q(z|x) \log p(z) and the excess bits would be KL(q(z|x) | p(z)), not Kl(q(z)|p(z)).
The appendix ends with "IAE only minimizes the extra number of bits required for transmitting x, while the VAE minimizes the total number of bits required for the transmission" but the IAE = VAE by Equation (4-6).  They are equivalent, how can one minimizing something the other doesn't?  In general the paper to me reads at times as VAE=IAE but IAE is better.  While it very well might be true that the objective trained in the paper (a joint GAN objective attempting to minimize the Jensen Shannon divergence between both  (1) the joint data density q(z,x) and the aggregated reconstruction density r(z,x) and (2) the aggregated posterior q(z) and the prior p(z)) is better than a VAE (as the experiments themselves suggest), the rhetoric of the paper suggests that the IAE referred to throughout is Equation (6).  Equation 6 is equivalent to a VAE.
I think the paper would greatly benefit from a rewriting of the central story.  The paper has a good idea in it, I just feel as though it is not well presented in its current form and worry that if accepted in this form might cause more confusion than clarity.  Combined with what I view as some technical flaws especially in the appendices I still must vote for a rejection.


As the pace of deep learning innovation accelerates, it becomes increasingly important to organize the space of problems by relative difficultly. Looking to other fields for inspiration, we see analogies to the Chomsky Hierarchy in computational linguistics and time and space complexity in theoretical computer science. As a complement to prior theoretical work on the data and computational requirements of learning, this paper presents an empirical approach. We introduce a methodology for measuring validation error scaling with data and model size and test tasks in natural language, vision, and speech domains. We find that power-law validation error scaling exists across a breadth of factors and that model size scales sublinearly with data size, suggesting that simple learning theoretic models offer insights into the scaling behavior of realistic deep learning settings, and providing a new perspective on how to organize the space of problems. We measure the power-law exponent-the "steepness" of the learning curve-and propose using this metric to sort problems by degree of difficulty. There is no data like more data, but some tasks are more effective at taking advantage of more data. Those that are more effective are easier on the proposed scale. Using this approach, we can observe that studied tasks in speech and vision domains scale faster than those in the natural language domain, offering insight into the observation that progress in these areas has proceeded more rapidly than in natural language.

The paper proposes an empirical solution to coming up with a hierarchy of deep learning tasks or in general machine learning tasks. They propose a two-way analysis where power-law relations are assumed between (a) validation loss and training set size, and (b) the number of parameters of the best model and training set size. The first power-law exponent, \beta_g, indicates how much can more training data be helpful for a given task and is used for ordering the hardness of problems. The second power-law exponent, \beta_p, indicates how effectively does the model use extra parameters with increasing training set (can also be thought of as how good the model is at compression). From experiments across a range of domains, the authors find that indeed on tasks where much of the progress has been made tend to be ones with smaller \beta_g (and \beta_p). It's arguable as to how comparable these power-law exponents are across domains because of differences in losses and other factors, but it's definitely a good heuristic to start working in this direction.
Clarifications needed:
(a) Why was full training data never used? The plots/analysis would have looked more complete if the whole training data was used, wondering why certain thresholds for data fraction were chosen.
(b) What exactly does dividing the training set into independent shards mean? Are training sets of different sizes created by random sampling without replacement from the whole training set?
(c) How exactly is the "Trend" line fitted? From the right sub-figure in Figure 1, it seems that fitting a straight line in only the power-law region makes sense. But that would require determining the start and end points of the power-law region. So some clarification on how exactly is this curve fitting done? For the record, I'm satisfied with the curve fitting done in the plots but just need the details.
Major issues:
(a) Very difficult to know in Figure 3 (right) what s(m)'s are associated with which curve, except the one for RHNs maybe.
(b) Section 4.1: In the discussion around Figure 2, I found some numbers a little off. Firstly, using the left plot, I would say that at even 8 images the model starts doing better than random instead of <25 that's stated in the text.  Secondly, the 99.9% classification error rate for top-5 is wrong, it's 99.5% for top-5 ("99.9% classification error rate for top-1 and top-5").
(c) Section 5: The authors use the phrase "low dimensional natural language data" which is quite debatable, to say the least. The number of possible sentences of K length with vocabulary |V| scale exponentially |V|^K where |V| is easily in 10K's most of the time. So to say that this is low dimensional is plain wrong. Just think about what is the (input, output) space of machine translation compared to image classification.
Typos/Suggestions:
(a) Section 3: "Depending the task" -> "Depending on the task"
(b) Section 4.3: "repeatably" -> "repeatability"
(c) Figure 4: Specify number of params in millions. The plot also seems oddly big compared to other plots. Also, proper case the axis labels, like other plots.
(d) Section 4.2 and 4.2.1 can be merged because the character LM experiments are not discussed in the main text or at least not clearly enough for me.  The values of \beta_g seem to include the results of character LM experiments. So either mention the character LM experiments in more detail or just point to results being in appendix.


We study many-class few-shot (MCFS) problem in both supervised learning and meta-learning scenarios. Compared to the well-studied many-class many-shot and few-class few-shot problems, MCFS problem commonly occurs in practical applications but is rarely studied. MCFS brings new challenges because it needs to distinguish between many classes, but only a few samples per class are available for training. In this paper, we propose "memory-augmented hierarchical-classification network (MahiNet)" for MCFS learning. It addresses the "many-class" problem by exploring the class hierarchy, e.g., the coarse-class label that covers a subset of fine classes, which helps to narrow down the candidates for the fine class and is cheaper to obtain. MahiNet uses a convolutional neural network (CNN) to extract features, and integrates a memory-augmented attention module with a multi-layer perceptron (MLP) to produce the probabilities over coarse and fine classes. While the MLP extends the linear classifier, the attention module extends a KNN classifier, both together targeting the "few-shot" problem. We design different training strategies of MahiNet for supervised learning and meta-learning. Moreover, we propose two novel benchmark datasets "mcfsImageNet" (as a subset of ImageNet) and "mcfsOmniglot" (re-splitted Omniglot) specifically for MCFS problem. In experiments, we show that MahiNet outperforms several state-of-the-art models on MCFS classification tasks in both supervised learning and meta-learning scenarios.

This paper presents methods for (1) adding inductive bias to a classifier through coarse-to-fine prediction along a class hierarchy and (2) learning a memory-based KNN classifier through an intuitive procedure that keeps track of mislabeled instances during learning. Further, the paper motivates focused work on the many class / few shot classification scenario and creates new benchmark datasets from subsets of imagenet and omniglot that match this scenario. Experimental results show gains over popular competing methods on these benchmarks.
Overall, I like the motivation that this paper provides for many class / few shot and find some of the methods proposed interesting. Yet there are issues with clarity of presentation that made it somewhat difficult to fully understand the exact procedures that were implemented. The model figure is useful, but could be refined to add additional clarity -- particularly in the case of the KNN learning procedure.
I'm not entirely familiar with recent work in this sub-field, so it is difficult for me to judge the novelty of the proposed procedure. Is it really true that class-hierarchies have never been used to perform coarse-to-fine inference in past work? If so, this should be state clearly. If not, related work should be mentioned and compared against. Finally, while the procedures are intuitive -- the takeaway of this paper could be substantially improved if even simple theoretical analysis were provided. For example, in the limit of infinite data, does the memory-based KNN learning procedure actually produce the right classifier?
Misc comments questions:
-The paper says at least twice that coarse classification will be performed with an MLP, while fine classification will use a KNN -- yet, the model section also state that both coarse and fine use both MLP and KNN. It is unclear to me which model setup was used in experiments.
-Can anything theoretical be shown about the class hierarchy based classification technique? Intuitively, it does add inductive bias through a manually defined taxonomy, but can something more precise be said about how it restricts the hypothesis space? This procedure is simple enough that I would be surprised if similar techniques had not be studied thoroughly in the statistical learning theory literature.
-The procedure for updating the KNN memory is intuitive, but can anything more be said about it? In isolation, is the KNN learning procedure at least consistent -- i.e. in the limit of large data does it converge to the correct classifier? Maybe this is trivial to prove, but is worth including.


The weak contraction mapping is a self mapping that the range is always a subset of the domain, which admits a unique fixed-point. The iteration of weak contraction mapping is a Cauchy sequence that yields the unique fixed-point. A gradientfree optimization method as an application of weak contraction mapping is proposed to achieve global minimum convergence. The optimization method is robust to local minima and initial point position.

This paper proposes an approach to zeroth order optimization based on the Banach fixed point theorem for contractive maps. They define a "weak contraction map," argue that it will have a unique fixed point, and use this to propose a zeroth order optimization algorithm which iteratively identifies sublevel sets of the objective until convergence to the optimum.
At each iteration $t$, the $f(x_t)$-sublevel set of $f$ is found using a root-finding algorithm, and the next point $x_{t+1}$ is calculated by averaging a collection of points on the boundary of the sublevel set.
My main concern about this paper is that the optimization algorithm works neither in theory nor in realistic practical scenarios. There are two main issues:
(1) Identifying the sublevel sets requires solving equations of the form $f(x) = L$, which is just as hard as optimizing $f$ in the first place! In many realistic scenarios, e.g. machine learning problems, you know what the minimum value of the function is, so you could just solve $f(x) = f^*$ and be done in one step! Even if you don't know the optimal value, you could do some version of binary search. Also, for the two or three dimensional problems with relatively simple expressions that the authors experimented on, finding roots might be possible, but for higher dimensions or more complicated functions, finding these roots would require numerical optimization--which is the problem we are trying to solve in the first place.
(2) The authors seem to imply that this algorithm would work for any $f$, however, consider the function in 1 dimension $f(x) = 1$ for all $x \neq x^*$, and $f(x^*) = 0$. For this function, the $f(x_t)$-sublevel sets are the entire domain until $x_t = x^*$. It is unclear what "points on the contours" would mean in this case, but whatever those contours are, the algorithm would never converge on this function because the function value of $x_0, x_1, ...$ would all be the same, so the contours would remain the same. This function might seem a little ridiculous, but continuous or even Lipschitz version of this counterexample could be constructed by smoothing things out around $x^*$, and a function such as this could be obfuscated by writing it down with a long, complicated expression making it hard to identify $x^*$ by inspection.
There are some typos/typesetting issues. It seems that all of the theorem and lemma statements in Section 2 are missing the bold "Theorem" and "Lemma" heading. In the paragraph after equation (2), "weak contraction mapping" is defined twice, I believe the first definition should be just a "contraction mapping."


We explore some mathematical features of the loss landscape of overparameterized neural networks. A priori one might imagine that the loss function looks like a typical function from R n to R -in particular, nonconvex, with discrete global minima. In this paper, we prove that in at least one important way, the loss function of an overparameterized neural network does not look like a typical function. If a neural net has n parameters and is trained on d data points, with n > d, we show that the locus M of global minima of L is usually not discrete, but rather an n − d dimensional submanifold of R n . In practice, neural nets commonly have orders of magnitude more parameters than data points, so this observation implies that M is typically a very high-dimensional subset of R n .

This paper considers over-parametrized neural networks (n > d), where n is the number of parameters and d is the number of data, in the regression setting. It consists of three main results:
a) (Theorem 2.1) If the output of the network is a smooth function of its parameters and global minimum with zero squared loss is attainable, then the locus of global minima is a smooth $n - d$ dimensional submanifold of R^n.
b) (Lemma 3.3) A neural network with a hidden layer as wide as the number of data points can attain global minima with zero loss.
c) (Theorem 4.1) If a neural network has a hidden layer as wide as the number of data points and is a smooth function of its parameters, then the locus of global minima with loss zero is a smooth $n - d$ dimensional submanifold. This is just a combination of a) and b).
I think the only contribution of this paper is Theorem 2.1. It is already well-known (e.g., Zhang et al. 16’) that a network with a hidden layer as wide as the number of data can easily attain zero empirical error. The authors claim that it is an extension over ReLU, but the extension is almost trivial. Theorem 4.1 is just a corollary from Theorem 2.1 and Lemma 3.3.
Theorem 2.1 is an interesting observation, and its implication that the Hessian at minima has only d positive eigenvalues is interesting and explains empirical observations.
From my understanding, however, the n - d dimensional submanifold argument can be applied to any regular values of H(w,b) (not necessarily global minima), so the theorem is basically saying there are a lot of equivalence classes in the parameter space that outputs exactly the same outcome. If my understanding is correct, this result does not give us any insight into why SGD can find global minima easily, because we can apply the same proof to spurious local minima and say, “look, the locus of this spurious local minimum is a high-dimensional submanifold, so it is easy to get stuck at poor local minima.”
Moreover, the notation used in this paper is not commonly used in deep learning community, thus might confuse the readers. $n$ is more commonly used for the number of data points and $d$ is used for some dimension. Also, there are overloading uses of $p$:
* In Section 2.1 it is used for the dimension of the input space
* In the proof of Prop 2.3 it is now a vector in R^n
* In section 3 it is now a parameter vector of a neural network
* Back to the input dimension in Lemma 3.3 and Section 4
Also, the formulation of neural networks in Section 3 is very non-standard. If you are to end up showing some result on standard fully-connected feed-forward networks, why not just use, for example, $W_2 \sigma(W_1 x_i + b_1) + b_2$? To me, this graph theoretical formulation only adds confusion. Also at the end of page 4, there is a typo: the activation function (\sigma) must be applied after the summation.
Overall, I believe this paper does not meet the standards of ICLR at the moment. I would recommend rejection.


DIFFERENTIABLE LEARNING-TO-NORMALIZE VIA SWITCHABLE NORMALIZATION
We address a learning-to-normalize problem by proposing Switchable Normalization (SN), which learns to select different normalizers for different normalization layers of a deep neural network. SN employs three distinct scopes to compute statistics (means and variances) including a channel, a layer, and a minibatch. S-N switches between them by learning their importance weights in an end-to-end manner. It has several good properties. First, it adapts to various network architectures and tasks (see Fig.1). Second, it is robust to a wide range of batch sizes, maintaining high performance even when small minibatch is presented (e.g. 2 images/GPU). Third, SN does not have sensitive hyper-parameter, unlike group normalization that searches the number of groups as a hyper-parameter. Without bells and whistles, SN outperforms its counterparts on various challenging benchmarks, such as ImageNet, COCO, CityScapes, ADE20K, and Kinetics. Analyses of SN are also presented. We hope SN will help ease the usage and understand the normalization techniques in deep learning. The code of SN has been released in https://github.com/switchablenorms/.

In this work, the authors propose Switchable Normalization (SN), which *learns* to switch / select different normalization algorithms (including batch normalization (BN), Instance Normalization (IN), Layer Normalization (LN)) in layers of the networks and in different applications. The idea is motivated by observations (shown in Fig 1) that, 1) different tasks tend to have applied different normalization methods; 2) some normalization methods (e.g. BN) are fragile to very small batch size.
The authors propose a general form for different normalization methods, which is a Gaussian normalization and then scale and shift by scalars. Different normalization methods utilize different statistics as the mean and the variance of the Gaussian normalization. The authors further propose to learn the combination weights on mean and variance, which is w_k and w'_k in Eqn (3). To avoid duplicate computation, the authors also do some careful simplification on computing mean and variance with all of the three normalization methods.
In the experiment part, the authors demonstrate the effectiveness of the proposed SN method on various kinds of tasks, including ImageNet classification, object detection and instance segmentation in COCO, semantic image parsing and video recognition. In all of the tasks tested, which also cover the common application in computer vision, SN shows superior and robust performance.
Pros:
+ Neat motivation;
+ Extensive experiments;
+ Clear illustration;
Cons
- There are still some experiment results missing, as the authors themselves mentioned in the Kinetics section (but the reviewer thinks it would be ready);
- In Page 3 the training section and Page 4, the first paragraph, it mentioned Θ and Φ (which are the weights for different normalization methods) are jointly trained and different from the previous iterative meta-learning style method. The authors attribute "In contrast, SN essentially prevents overfitting by choosing normalizers to improve both learning and generalization ability as discussed below". The reviewer does not see it is well justified and the reviewer thinks optimizing them jointly could lead to instability in the training (but it did not happen in the experiments). The authors should justify the jointly training part better.
- Page 5 the final paragraph, the reviewer does not see the point there. "We would also like to acknowledge the contributions of previous work that explored spatial region (Ren et al., 2016) and conditional normalization (Perez et al., 2017). "  Please make it a bit more clear how these works are related.


We present a differentiable multi-prototype word representation model that disentangles senses of polysemous words and produces meaningful sense-specific embeddings without external resources. It jointly learns how to disambiguate senses given local context and how to represent senses using hard attention. Unlike previous multi-prototype models, our model approximates differentiable discrete sense selection via a modified Gumbel softmax. We also propose a novel human evaluation task that quantitatively measures (1) how meaningful the learned sense groups are to humans and (2) how well the model is able to disambiguate senses given a context sentence-an evaluation ignored by previous models. Our model not only discovers distinct, interpretable embeddings but is competitive against previous models on word similarity tasks.
Moreover, these evaluations ignore a key component of learning sense inventories: do they make sense to a human? Previous multisense embedding papers present nearest neighbors to claim their representations are interpretable and useful. Like topic models, these implicit interpretability claims need to be rigorously verified. In Section 6, we adapt techniqes for evaluating topic models (Chang et al., 2009) to measure whether learned sense groups are internally coherent and whether humans can consistently match a learned sense vector to a word in context. Just like topic models, word embedding models that win conventional evaluations do not always make sense to humans.
We present a simple method that not only correlates well with traditional word similarity evaluations (Section 5) but also discovers interpretable (measured by human evaluations) sense embeddings (Section 6). Our model extends the Skip-Gram Word2Vec model and simultaneously learns (1) automatic sense induction given local context and (2) sense-specific embeddings. To learn disentangled sense representations (i.e., avoid sense mixing), we approximate hard attention and preserve Figure 1: Network struture with an example of our GASI model which learns a set of global context embeddings C and a set of sense embeddings S differentiability via a scaled variant of the Gumbel Softmax function (Section 3.2). This modeling contribution-Scaled Gumbel Softmax-is critical for disambiguating senses.

The paper presents a method for deriving multi sense word embeddings. The key idea behind this method is to learn a sense embedding tensor using a skip-gram style training objective. The objective defines the probability of contexts marginalised over latent sense embeddings. The paper uses Gumbel-softmax reparametrization trick to approximate sampling from the discrete sense distributions. The method also uses a separate hyperparameter to help scale the dot product appropriately.
Strengths:
1. The technique is a well-motivated solution for a hard problem that builds on the skip-gram model for learning word embeddings.
2. A new manual evaluation approach for comparing sense induction approaches.
3. The empirical advance while relatively modest appears to be significant since the technique seems to yield better results than multiple baselines across a range of tasks.
Suggestions:
1. The number of senses is fixed to three. This is a bit arbitrary, even though it is following some precedence. I like the information in the appendix that shows how to handle cases when there are duplicate senses induced for words that dont have many senses. It would be useful to know how to handle the cases where a word can have more than three senses. Given that the authors have a way of pruning duplicate senses, it would have been interesting to try a few basic methods that select the number of senses per word dynamically.
2. The evaluation includes word similarity task and crowdsourcing for sense intrusion and sense selection. These provide a measure of intrinsic quality of the sense based embeddings. However, as Li and Jurafsky (2015) point out, typically applications use more powerful models that use a wide context. It is not clear how these improvements to sense embeddings will translate in these settings. It would have been useful to have at least one or two end applications to illustrate this.
3. Given that the empirical gains are not quite consistent, I would encourage the authors to specifically argue why this particular method should be favoured over other existing methods. The related work discussion merely highlights methodological differences. For example, the contrast with Lee and Chen (2017) seems to be only that of differentiability. Is the claim that differentiability is desirable because this allows for fine tuning in applications? If this is the case then it will be nice to have this verified.
4. The lower bound on the log likelihood objective is good but what are we supposed to take away from it? Is it that there is an interpretation that allows us to get away with negative sampling?
Overall I like the paper. It presents an application of the Gumbel-softmax trick for sense embeddings induction and shows some empirical evidence for the usefulness of this idea, including some manual evaluation.
I think the evaluation could be strengthened with some end applications and much crisper arguments on why the method is preferable over other methods that achieve comparable performance.
References:
[Li and Jurafsky., EMNLP 2015] Do Multi-Sense Embeddings Improve Natural Language Understanding?


Multi-label classification (MLC) is the task of assigning a set of target labels for a given sample. Modeling the combinatorial label interactions in MLC has been a long-haul challenge. Recurrent neural network (RNN) based encoder-decoder models have shown state-of-the-art performance for solving MLC. However, the sequential nature of modeling label dependencies through an RNN limits its ability in parallel computation, predicting dense labels, and providing interpretable results. In this paper, we propose Message Passing Encoder-Decoder (MPED) Networks, aiming to provide fast, accurate, and interpretable MLC. MPED networks model the joint prediction of labels by replacing all RNNs in the encoder-decoder architecture with message passing mechanisms and dispense with autoregressive inference entirely. The proposed models are simple, fast, accurate, interpretable, and structure-agnostic (can be used on known or unknown structured data). Experiments on seven real-world MLC datasets show the proposed models outperform autoregressive RNN models across five different metrics with a significant speedup during training and testing time.

As a reviewer I am expert in learning in structured data domains. Because of that I completely disagree that the proposed title of the paper is not misleading. In fact, both the input and the output of the proposed system are not graphs. Moreover, the intermediate representations are always complete graphs, so there is no graph to graph transformation here. It is the internal topology of the encoder and decoder that corresponds to a complete graph and not the nature of the processed data.
The main intended contribution of the paper is to define a system able to capture the dependencies among input features as well as output labels, so to improve the multi-label classification task addressed by the system. This is obtained by defining a recurrent model with a complete graph topology to both encode the input and decode the output. The decoding part starts from the assumption of independence among the output labels and then, via interaction with the encoded representation of the input, eventually turns to an output where relevant statistical dependences among output labels emerge with decoding. Since both encoding and decoding are recurrent models (with no enforced guarantee to have stable points), the paper proposes to unfold the recursion for a fixed predefined number of time steps.
Presentation of the proposal is generally good, although there are some issues that are not clear. For example, the same weights indices are used for matrices belonging to the encoding and decoding, making the reader to believe that such matrices are shared. In addition, the sentence about model parameters at page 5 is a bit ambiguous and it is not sufficient to resolve the presentation problem.
The discussion at the end of page 4 on the fact that a sequential representation for the input components is not natural is actually out of place for the specific application task selected for presentation. In fact, words in a sentence have an order. The fact that such order is lost with the bag-of-word representation is a problem of preprocessing, not of the nature of the data. In general, however, it is true that forcing an order is not natural.
Going in the merit of the proposal, the number of parameters for the decoder scales quadratically with the number of output labels (fully connected graph). In domains with a large numbers of labels (e.g. thousands) there may be concerns on two different aspects: i) computational burden may grow significantly even if the average number of labels per item is small; ii) proper propagation of information on dependencies among labels may require to use a large value for T (graph hops), i.e. there is a dependency between size of label graph and "useful" value for T.  On this issue, by the way, figures 3 and 4 seem to report incongruent results since, because of symmetries in the model topology, equal and  reciprocal influences between input components (and output labels) would have been expected, but these are not observed in the figures.
Analogous considerations could be done for the encoder when the size of the input is large.
Concerning experimental results, no statistical significance test is performed, so it is not clear to me if the shown improvements are actually significant. Speed-up in training and testing seem at least to give some advantage with respect to other competing approaches, however the scaling problem described above for the decoder (and encoder) may lead to much worst performances in those special cases.
The addressed problem is covered by a large literature, involving many different approaches. It would have been nice to report, for the selected datasets, the best performance (and computation times) obtained by, for example,  probabilistic graphical models or SVM-based models.
The paper seems to refer most of the relevant recent neural-based approaches.
I think the paper is relevant for ICLR (although there is no explicit analysis of the obtained hidden representations) and of interest for a good portion of attendees.
Minor issues:
- two rows before Section 2.2.1: \mathbb{h}_*^2  should be \mathbb{h}_*^1
- equations 4, 5, 9, 10, 14: matrices W are indexed in such a way to assume that each input word/label is associated to a different matrix (i.e., set of parameters). Is this really the case ? How is then managed the fact that different inputs may have a different number of components ? how is a specific matrix assigned to a specific word ? I guess this is a presentation mistake, otherwise there are relevant issues that are completely not addressed by the presentation.
- equation (10): since the output should be interpreted as a probability, why not using a softmax? sigmoidal units by themselves do not guarantee that the outputs sum to 1. I guess you do not have this problem because you adopt batch normalisation. This however is conceptually not nice since there is no uniformity across the dataset. Moreover, the softmax function has a nice probabilistic interpretation in the family of the exponential distributions.
- "[...] we use add a positional encoding..."
- Multi-head Attention: apart for the not so clear description, the equation involving the softmax is missing.
- "[..] the the attention and feedforward layers."
- "[..] the the sum of the total true..."


Variational Autoencoders (VAEs) are a popular generative model, but one in which conditional inference can be challenging. If the decomposition into query and evidence variables is fixed, conditional VAEs provide an attractive solution. To support arbitrary queries, one is generally reduced to Markov Chain Monte Carlo sampling methods that can suffer from long mixing times. In this paper, we propose an idea we term cross-coding to approximate the distribution over the latent variables after conditioning on an evidence assignment to some subset of the variables. This allows generating query samples without retraining the full VAE. We experimentally evaluate three variations of cross-coding showing that (i) they can be quickly optimized for different decompositions of evidence and query and (ii) they quantitatively and qualitatively outperform Hamiltonian Monte Carlo.  (Girolami & Calderhead, 2011;Daniel Levy, 2018) are difficult to adapt to these problems, and can suffer from long mixing times as we show empirically.
To remedy the limitations of existing methods for conditional inference in VAEs, we aim to approximate the distribution over the latent variables after conditioning on an evidence assignment through a variational Bayesian methodology. In doing this, we reuse the decoder of the VAE and show that the error of the distribution over query variables is controlled by that over latent variables via a fortuitous cancellation in the KL-divergence. This avoids the computational expense of re-training the decoder as done by the CVAE approach. We term the network that generates the conditional latent distribution the cross-coder.
We experiment with two cross-coding alternatives: Gaussian variational inference via a linear transform (GVI) and Normalizing Flows (NF). We also provide some comparison to a fully connected network (FCN), which suffers from some technical and computational issues but provides a useful point of reference for experimental comparison purposes. Overall, our results show that the GVI and NF variants of cross-coding can be optimized quickly for arbitrary decompositions of query and evidence and compare favorably against a ground truth provided by rejection sampling for low latent dimensionality. For high dimensionality, we observe that HMC often fails to mix despite our systematic efforts to tune its parameters and hence demonstrates poor performance compared to cross-coding in both quantitative and qualitative evaluation.
1 Under review as a conference paper at ICLR 2019 Algorithm 1 Conditional Inference via Cross-coding. Input (a) Pre-trained VAE p(z)p θ (t|z) with p θ (t|z) based on Decoder θ (z). (Encoder ignored.) (b) Single query x (any subset of t) for which to predict y. (Rest of t.) Optimize Define q( )q ψ (z| ) with q ψ (z| ) based on XCoder ψ ( ). Find ψ to maximize C-ELBO[q ψ (Z) p θ (Z, x)] (Defined in Theorem 2). Estimate stochastic gradients by drawing random ∼ q( ) and using the reparameterization trick. Predict Draw a sample {z m } M m=1 ∼ q ψ (z) by setting z m = XCoder ψ ( m ) for m ∼ q( ). Predict p θ (y|x) ≈

(apologies for this belated review)
Summary
The authors consider the task of imputing missing data using variational auto-encoders. To do so, they assume a fixed pre-trained generative model, perform variational inference to infer a posterior on latent variables given a partial image, and then use this approximate posterior to predict missing pixels. They compare a variety of parameterizations of the variational distribution to HMC inference, and evaluate on MNIST, Celeb-A and the Anime data.
Comments
There are many things about this paper that I don’t understand. My main concern is that I fail to follow why the authors are interested in this task. In what settings would we be interested in performing non-autoencoding variational inference in order to impute missing data? Moreover, in cases where are interested in performing such imputations, what would we like to use the results for? This paper seems like a nice demo, but I’m not entirely convinced I see a compelling application.
My second concern is about the baselines that are considered. If I were interested in carrying out this inference task, my inclination would not be to run an HMC chain to convergence, but instead to do something like annealed importance sampling (AIS), where at each step I run an iteration of HMC on a large batch of samples on a sequence of target densities that interpolate between the prior and full joint p(x, Z). If computational cost is a concern, I imagine this would not be more expensive than training a density estimator. Moreover, whereas HMC is generally not known to be a good method for estimating marginal likelihoods, AIS methods generally perform much better.
Finally I find the language used in this paper confusing. Cross-coding seems a misnomer for the technique that the authors propose. Isn’t this simply a form of variational inference in which qψ(Z) approximates pθ(Ζ | x)? The term “-coding” suggests that we somehow define an encoder that accepts the query as input. Moreover, isn’t the XCoder network just a neural density estimator?
Finally, Lemma 1 seems like a really roundabout way of deriving a lower bound. The authors could instead just write:
	log p(x)
	>=
	E_q(Z,Y)[log p(x, Y, Z) - log q(Z, Υ)]
	=
	E_q(Z,Y)[log p(x | Ζ) + log p(Y | Z) + log p(Z) - log q(Z) - log p(Υ | Z)]
	=
	E_q(Z,Y)[log p(x | Ζ) + log p(Z) - log q(Z)]
	=
	E_q(Z)[log p(x, Ζ) - log q(Z)]
This avoids confusing terminology such as cross-coding, and shows that what the authors are doing is in fact just variational inference. Am I missing something here?
I am also confused about how the comparison to HMC is set up. If you’re training qψ(Z), then you presumably need generate a certain number samples at training time. Shouldn’t you add this number of samples number of samples you generate in HMC, in order to get a more apples to apples comparison in terms of the amount of computation performed? As it stands, it is hard to evaluate whether these methods are given a similar number of samples.
Finally, I am not quite sure what to make of the experimental evaluation. We see some scatter plots on MNIST with a 2D latent space, and some faces of celebrities in which there is arguably some sample diversity, although most of this diversity arises in blurry looking hairstyles. However, since the authors condition on the eyes, rather than, say, the nose or mouth, it is hard to know how good a job the network is doing at generalizing to multiple plausible faces.
Overall, I find it difficult to judge the merit of this paper. Is this task in fact hard? Is it useful? Are the results good? Maybe the authors can give us some additional guidance on these questions.
Questions
- I’m a bit worried that not all the samples that we see in Figure 6 may have equally high probability under the posterior. Could the authors compute and report importance weights?
	W = p(x, Z) / q(Z)

- Could the authors say something about the effective sample size that we obtain when using the learned distribution q(Z) as a proposal?

	ESS = (Σ_k w^k)^2 / (Σ_k (w^k)^2)
- Should it be the case that the ESS is low, and the weights are high variance, could the authors generate a sufficient number of samples to ensure the the ESS = 25 (i.e. the number of images in the figure) and then show the 25 highest-weight samples (or resample 25 images with probability proportional to their weight)?

Minor
- Equation (3): There’s an extra p_θ in the first integral
- In the proof in Appendix 6.1
	KL[ qψ(Z) ‖ pθ(Z | x) ] + KL[ qψ(Y | Z) ‖ pθ(Y | Z, x)]
it would be clearer to explicitly denote the expectation over qψ(Z)
	KL[ qψ(Z) ‖ pθ(Z | x) ] + E_qψ(Z)[ KL[ qψ(Y | Z) ‖ pθ(Y | Z, x)] ]

(I had to google lecture notes to find out that this expectation is sometimes implicit, which
as far as I know is not very standard).


Many notions of fairness may be expressed as linear constraints, and the resulting constrained objective is often optimized by transforming the problem into its Lagrangian dual with additive linear penalties. In non-convex settings, the resulting problem may be difficult to solve as the Lagrangian is not guaranteed to have a deterministic saddle-point equilibrium. In this paper, we propose to modify the linear penalties to second-order ones, and we argue that this results in a more practical training procedure in non-convex, large-data settings. For one, the use of secondorder penalties allows training the penalized objective with a fixed value of the penalty coefficient, thus avoiding the instability and potential lack of convergence associated with two-player min-max games. Secondly, we derive a method for efficiently computing the gradients associated with the second-order penalties in stochastic mini-batch settings. Our resulting algorithm performs well empirically, learning an appropriately fair classifier on a number of standard benchmarks.

The authors propose a method to generate predictions under fairness constraints. The main idea is to take linear fairness constraints, and replace them with weak squared penalties plugged into the objectives, which supposed to help in cases where the loss function is not convex. The penalty coefficients are chosen by cross-validation, and the effectiveness of this approach is demonstrated empirically.
In Sec. 3.1, the authors point out several shortcomings of using linear penalties (using Lagrange multipliers) for non-convex losses. These seem valid. Sec. 3.2, however, is not clear on why exactly replacing the linear penalties with quadratic penalties solves these issues. I'm hoping the authors can clarify the following points:
1) The authors note that, for quadratic penalties, \lambda->0 means no constraints, and \lambda->\infty means hard constraints. Isn't this also true for linear constraints?
2a) Why do linear penalties have unique \lambda_k for each constraint k, but the quadratic objective has only a single \lambda for all constraints?
2b) Why can CV over \lambda be used for quadratic constraints - what is the justification? And, more importantly, why *can't* it be used with linear constraints? If it can, then this should be one of the baselines compared to in the experiments.
3) What is the criterion optimized for by CV - accuracy or the constraints? Different parts of the paper give different answers to this question. For example, "... may be easily determined via standard hyperparameter optimization methods" vs. "tuning \lambda to directly satisfy the desired fairness metrics". Or even more unclear - "choose \lambda ... so that the final solution gives the desired fairness-accuracy trade-off". How is the desired trade-off defined?
4) If there is a trade-off between fairness and accuracy, and no clear-cut criterion for evaluation is pre-defined, then the evaluation procedure should compare methods across this trade-off (similarly to precision-recall analysis).
5) The authors differentiate between cases where the loss is either convex or non-convex. This is confusing - most losses are convex, and non-linearity appears when they are composed with non-linear predictors. Is this the case here? If so, the fairness constraints are no longer linear, and they're quadratic counterpart is no longer quadratic. It would be helpful if the authors specify where the non-linearity comes from, and what they assume about the loss and predictors.
6) Why is it important to show that the quadratic constraints can be written as an expectation? Isn't the square of an expectation always an expectation of pairs? How does the double summation/integral effect runtime?
7) It would be helpful if the authors differentiate between loss/constraints over the entire distribution vs. over a given sample set.


This paper studies the problem of domain division which aims to segment instances drawn from different probabilistic distributions. This problem exists in many previous recognition tasks, such as Open Set Learning (OSL) and Generalized Zero-Shot Learning (G-ZSL), where the testing instances come from either seen or unseen/novel classes with different probabilistic distributions. Previous works only calibrate the confident prediction of classifiers of seen classes (W-SVM Scheirer et al. (2014)) or taking unseen classes as outliers Socher et al. (2013). In contrast, this paper proposes a probabilistic way of directly estimating and fine-tuning the decision boundary between seen and unseen classes. In particular, we propose a domain division algorithm to split the testing instances into known, unknown and uncertain domains, and then conduct recognition tasks in each domain. Two statistical tools, namely, bootstrapping and KolmogorovSmirnov (K-S) Test, for the first time, are introduced to uncover and fine-tune the decision boundary of each domain. Critically, the uncertain domain is newly introduced in our framework to adopt those instances whose domain labels cannot be predicted confidently. Extensive experiments demonstrate that our approach achieved the state-of-the-art performance on OSL and G-ZSL benchmarks.

This paper describes an approach to domain separation based on bootstrapping to identify similarity cutoff thresholds for known classes, followed by a Kolmogorov-Smirnoff test to refine the bootstrapped in-distribution zones. The authors apply these techniques to two general recognition problems: open-world recognition, and Generalized Zero-shot Learning (GZSL). Experimental results are given on a variety of datasets, and a thorough comparative evaluation on GZSL is performed.
The paper has the following strong points:
 1. The motivations for each element of the proposed approaches are fairly well presented and are compelling.

 2. The experimental results on GZSL are impressive, especially compared to established approaches.
The paper also has the following weak points:
 1. The writing is a bit rough throughout, though not to extreme distraction. The abstract starts right off with the awkward "This paper studies the problem of domain division problem..." The manuscript needs more careful revision for clarity.
 2. Related to the previous point, there are several elements of the technical exposition that are lacking in clarity. For example, it is unclear what eq. 9 is defining exactly. It seems to be the final decision rule, but it is not clear how seen, unseen, and uncertain samples are determined. Algorithm 1 is clear, but there is never a clear, complete definition of the end-to-end decision pipeline given. I feel like it would be difficult to reproduce the results of this article without significant trial-and-error.
 3. The authors seems to have relied on the Xian et al. paper to extract data for their comparative evaluation on GZSL. There are more recent works from 2018 that should be included, such as:
 Arora, Gundeep, Vinay Kumar Verma, Ashish Mishra and Piyush Rai. “Generalized Zero-Shot Learning via Synthesized Examples." CVPR 2018.
In summary, this paper has many interesting ideas, and the GZSL results are indeed impressive (with respect to the results in the comparison). However, there are many problems with clarity and missing, recent work in the comparative evaluation.


BA-NET: DENSE BUNDLE ADJUSTMENT NETWORKS
This paper introduces a network architecture to solve the structure-from-motion (SfM) problem via feature-metric bundle adjustment (BA), which explicitly enforces multi-view geometry constraints in the form of feature-metric error. The whole pipeline is differentiable, so that the network can learn suitable features that make the BA problem more tractable. Furthermore, this work introduces a novel depth parameterization to recover dense per-pixel depth. The network first generates several basis depth maps according to the input image, and optimizes the final depth as a linear combination of these basis depth maps via feature-metric BA. The basis depth maps generator is also learned via end-to-end training. The whole system nicely combines domain knowledge (i.e. hard-coded multi-view geometry constraints) and deep learning (i.e. feature learning and basis depth maps learning) to address the challenging dense SfM problem. Experiments on large scale real data prove the success of the proposed method.

I believe that the authors have a solid contribution that can be interesting for the ICLR community.
Therefore, I recommend to accept the paper but after revision because the presentation and explanation of the ideas contain multiple typos and lacking some details (see bellow).
Summary:
The authors propose a new method called BA-Net to solve the SfM problem by explicitly incorporating geometry priors into a machine learning task. The authors focus on the Bundle Adjustment process.
Given several successive frames of a video sequence (2 frames but can be extended up to 5), BA-Net jointly estimates the depth of the first frame and the relative camera motion (between the first frame and the next one).
The method is based on a convolutional neural network which extracts the features of the different pyramid levels of the two images and in parallel computes the depth map of the first frame. The proposed network is based on the DRN-54 (Yu et al., 2017) as a feature extractor.
This is complemented by the linear combination of depth bases obtained from the first image.
The features and the initial depth then passed to the optimization layer called BA-layer where the feature re-projection error is minimized by the modified LM algorithm.
The authors adapt the standard multi-view geometry constraints by a new concept of feature re-projection error in the BA framework (BA-layer) which they made differentiable.
Differentiable optimization of camera motion and image depth via LM algorithm is now possible and can be used in various other DL architectures (ex. MVS-Net can probably benefit from BA-layer).
The authors also propose a novel depth parametrization in the form of linear combination of depth bases which reduces the number of parameters for the learning task,
enables integration into the same backbone net as used or feature pyramids and makes it possible to jointly train the depth generator and the BA-layer.
Originally the proposed approach depicts the network operating in the two-view settings. The extensibility to more views is also possible and, as shown by authors, proved to improve performance. It is, however, limited by the GPU capacity.
Overall, the authors came up with an interesting approach to the standard BA problem. They have managed to inject the multi-view geometry priors and BA into the DL architecture.
Major comments regarding the paper:
It would be interesting to know the evaluation times for the BA-net and more importantly to have some implementation details to ensure reproducibility.
Minor comments regarding the paper:
-	The spacing between sections is not consistent.
-	Figures 1 is way too abstract given the complicated set-up of the proposed architecture. It would be nice to see more details on the subnet for depth estimator and output of the net.
Overall it would be helpful for reproducibility if authors can visualize all the layers of all the different parts of the network as it is commonly done in the DL papers.
-	Talking about proposed formulation of BA use either of the following and be consistent across the paper:
Featuremetric BA / Feature-metric BA / Featuremetric BA / ‘Feature-metric BA’
-	Talking about depth parametrization use ‘basis’ or ‘bases’ not both and clearly defined the meaning of this important notion.
-	Attention should be given to the notation in formulas (3) and (4). The projection function there is no longer accepts a 3D point parametrized by 3 variables. Instead only depth is provided.
In addition, the subindex ‘1’ of the point ‘q’ is not explained.
-	More attention should be given to the evaluation section. Specifically to the tables (1 and 2) with quantitative results showing the comparison to other methods.
It is not clear how the depth error is measured and it would be nicer to have the other errors explained exactly as they referred in the tables (e.g. ATE?).
-	How the first camera pose is initialized?
-	In Figure 2.b I’m surprised by the difference obtained in the feature maps for images which seems very similar (only the lighting seems to be different). Is it three consecutive frames?
-	Attention should be given to the grammar, formatting in particular the bibliography.