We introduce backdrop, a flexible and simple-to-implement method, intuitively described as dropout acting only along the backpropagation pipeline. Backdrop is implemented via one or more masking layers which are inserted at specific points along the network. Each backdrop masking layer acts as the identity in the forward pass, but randomly masks parts of the backward gradient propagation. Intuitively, inserting a backdrop layer after any convolutional layer leads to stochastic gradients corresponding to features of that scale. Therefore, backdrop is well suited for problems in which the data have a multi-scale, hierarchical structure. Backdrop can also be applied to problems with non-decomposable loss functions where standard SGD methods are not well suited. We perform a number of experiments and demonstrate that backdrop leads to significant improvements in generalization.

This paper introduces a data dependent strategy to mask parts of the partial derivatives in the chain rule computation.
Typically with papers proposing modifications of the training regime of the neural network one would expect one of three outcomes:
 - a well justified, mathematically sound method, well tested in simple cases and with some proof of concept results on proper tasks
 - a more heuristic, empirical driven research, where strong results on proper tasks
 - method, however justified, allows us to do something previously impossible, removing some limitations/constraints (like biologically plausible learning etc.)
In its current form paper seems to lack any of these characteristics. On one hand method lacks any guarantees and on the other paper does not present significant improvements under any approved metrics, nor it introduces new which can be properly quantified. In fact, authors explicitly claim that empirical section "Note that in these experiments, the purpose is not to achieve state of the art performance, but to exemplify how backdrop can be used and what measure of performance gains one can expect.".
With methods like this it is almost obvious that resulting update is not an unbiased gradient estimator of any function. Consequently convergence/learning guarantees that we have for GD or SGD no longer apply. Do authors have any thoughts on how bad can it get? As noted in the text, other methods of "dropping" data (such as dropout) don't have this issue as they still estimate proper gradients. Here, since dropping is done inside the network only on backwards pass, resulting estimates could, in principle, lead to oscilations, divergence and other issues. If these are not encountered in practice it might be interesting to understand why.
If authors prefer to go through more empirical path, one would expect at least to see some baselines for tasks proposed, rather than comparing Backdrop to SGD. There are many methods that could be applied in scenarios like this, including dozens forms of dropout (which, as authors note, is not aimed at the same goals, but this does not mean that it will not shine under the metrics introduced, as they are non-standard and so - noone tested them in this exact regime).
I am happy to revisit my rating given authors restructure paper towards one of these paths (or other one which is not listed here).


We propose a novel generative model architecture designed to learn representations for images that factor out a single attribute from the rest of the representation. A single object may have many attributes which when altered do not change the identity of the object itself. Consider the human face; the identity of a particular person is independent of whether or not they happen to be wearing glasses. The attribute of wearing glasses can be changed without changing the identity of the person. However, the ability to manipulate and alter image attributes without altering the object identity is not a trivial task. Here, we are interested in learning a representation of the image that separates the identity of an object (such as a human face) from an attribute (such as 'wearing glasses'). We demonstrate the success of our factorization approach by using the learned representation to synthesize the same face with and without a chosen attribute. We refer to this specific synthesis process as image attribute manipulation. We further demonstrate that our model achieves competitive scores, with state of the art, on a facial attribute classification task.

In this paper, the authors introduce a neural network architecture that has three components.
First a VAE is used to encode images in to two latent states \hat{y} and \hat{z}, with \hat{z}
intended to be class (e.g. face attribute) agnostic. The decoder reconstructs images from \hat{y}
and \hat{z} concatenated together. A GAN style discriminator attempts to distinguish the
decoded image from the original input image as real or fake, allowing the decoder to produce
higher quality decoded images. An auxiliary network A attempts to classify the face attribute y
from the class agnostic features \hat{z}, with the idea being that the encoder should try to produce
\hat{z} vectors from which the class cannot be predicted. An additional classifier is trained
using a classification loss \hat{L}_{class} on the encoded reconstructed image, the use of which
I don't understand.
I think additional work on section 2.5 through section 3 would be helpful to improve clarity.
As one example, "y" is unnecessarily overloaded: y denotes a specific attribute, \hat{y}
denotes a latent vector that is intended to not be class agnostic, \tilde{y} denotes the
prediction of an auxiliary network on an intended class-agnostic latent vector \hat{z} of
the presence of the original attribute y, and \hat{\hat{y}} denotes the non agnostic latent
vector achieved by passing the decoded image back through the encoder.
This notational complexity is compounded by the fact that a number of steps in the method are
not well motivated in the text, and left to the reader to understand their purpose. For example,
the authors state that "we incorporate a classification model into the encoder so that our model may
easily be used to perform classification tasks." What does this mean? In the diagram (Figure 1),
where is this classification model? Why in the GAN loss is there a term that compares the
fake loss with the result of classifying a decoded z vector? Is this z \hat{z}, or a latent vector
drawn from a distribution p(z)? If it is the former, how does this term differ from the second
term in the GAN loss. If it is the latter, then shouldn't it be concatenated with some y in order to
be used as input to the decoder D_{\theta}?
Why is it important to extract \hat{\hat{y}} from \hat{x}? In the paper you state that the loss
"provides a gradient containing label information to the decoder," but why can't we use the known label y
of the original input x to ensure that the encoder and decoder preserve this information if it is used as \hat{y}?
Later in the paper, you explicitly state that \hat{\mathcal{L}_{class}} "does not provide any clear benefit."
If that is the case, then you should ideally include it neither in the model nor in the paper. If it was
included primarily because previous models included it, then I would recommend you introduce its use
in a background section on Bao et al., 2017 rather than including it in your model description with an
explanation like "so that our model may easily be used to perform classification tasks."
Ultimately, this last point brings us to a good summary of my concerns with the model: the inclusion
of too many moving parts, some of which the authors explicitly say later on provide no benefit.
Moving on to experimental results, I think this is another area where I have a few concerns. First, in
Figure 2, the authors argue that your model is "better for 6 out of 10 attributes" and comparable results for most others. The authors include a gap of 0.1 in the "Gray_hair" category as "better" but label a gap of 0.5
in the Black hair category as "comparable." I think results in several of the categories are sufficiently close
that error bars would be necessary to draw actual conclusions. If "better" were to mean "better by 0.5" for example,
then the authors method is better on 4 tasks (smiling, blonde hair, heavy makeup, mustache) and worse on 3 (black hair, brown hair, wavy hair).
With respect to the actual attribute editing, my main concern here is a lack of comparison to models other than Bao et al., despite the fact that face attribute changing is an exhaustively studied task. A number of papers like Perarnau et al., 2016, Upchurch et al., 2017, Lample et al., 2017 and others study this task from machine learning perspectives, and in some cases can perform photorealistic image attribute editing without complicated machinery on megapixel face
images. At least the images in Figure 3 and 4 are substantially downsampled from the typical resolution found in the Celeba dataset, suggesting that there was some failure mode on full resolution images.
----
Edit: I've reviewed the authors' addressing my concerns in their paper and am happy to increase my rating as a result.


There is a stark disparity between the learning rate schedules used in the practice of large scale machine learning and what are considered admissible learning rate schedules prescribed in the theory of stochastic approximation. Recent results, such as in the 'super-convergence' methods which use oscillating learning rates, serve to emphasize this point even more. One plausible explanation is that non-convex neural network training procedures are better suited to the use of fundamentally different learning rate schedules, such as the "cut the learning rate every constant number of epochs" method (which more closely resembles an exponentially decaying learning rate schedule); note that this widely used schedule is in stark contrast to the polynomial decay schemes prescribed in the stochastic approximation literature, which are indeed shown to be (worst case) optimal for classes of convex optimization problems. The main contribution of this work shows that the picture is far more nuanced, where we do not even need to move to non-convex optimization to show other learning rate schemes can be far more effective. In fact, even for the simple case of stochastic linear regression with a fixed time horizon, the rate achieved by any polynomial decay scheme is suboptimal compared to the statistical minimax rate (by a factor of condition number); in contrast the "cut the learning rate every constant number of epochs" provides an exponential improvement (depending only logarithmically on the condition number) compared to any polynomial decay scheme. Finally, it is important to ask if our theoretical insights are somehow fundamentally tied to quadratic loss minimization (where we have circumvented minimax lower bounds for more general convex optimization problems)? Here, we conjecture that recent results which make the gradient norm small at a near optimal rate, for both convex and non-convex optimization, may also provide more insights into learning rate schedules used in practice.

The paper studies the effect of learning-rate choices for stochastic optimization, focusing on least-mean-squares with decaying stepsizes. The main result is showing that exponentially decaying stepsizes can yield improved rates of convergence of the final iterate in terms of dependence on the condition number. The proposed learning rate schedule depends on the condition number and the number of iterations. This positive result is complemented by showing that without prior knowledge of the time horizon, any stepsize sequence will frequently yield suboptimal solutions.
I have mixed feelings about the paper. On the positive side, the particular observation that exponential learning-rate schedules lead to faster convergence for SGD in linear least-squares problems indeed seems to be a novel result, and the lower bound also appears to be new and interesting. The analysis seems to be technically correct as well.
On the other hand, I have several concerns about the presentation of the results:
- The abstract and the introduction sets up a misleading narrative around the results: the authors seem to suggest that their work somehow explains why certain learning-rate schedules work better than others for deep learning applications / non-convex optimization, although the actual results exclusively concern the classical problem of linear least-squares regression. This presentation is completely uncalled for as the authors themselves admit that it is unclear how the results would generalize to other convex optimization settings, let alone non-convex optimization. Also, I think that this presentation style is rather harmful as it suggests that learning-theory results concerning classical setups are somehow embarrassing, so they need to be sold through some made-up connections to trendy topics in deep learning. I would suggest that the authors completely "rethink" the presentation of the paper and write it in a style that is consistent with the actual results: as a learning theory paper, without the irrelevant deep learning experiments (that only show well-known phenomena anyway).
- The paper misrepresents a large body of work on stochastic/online optimization. Specifically, the authors suggest that the stochastic optimization literature exclusively suggests the use of polynomially decaying stepsizes. This picture is grossly inaccurate for multiple reasons:
*** It has been known for a while that the de facto optimal tuning of SGD for least squares involves a large constant stepsize and iterate averaging (see, e.g., Bach and Moulines, NIPS 2013). This approach is only mentioned in passing without any discussion, even though it yields convergence rates that do not involve *any* dependence on the condition number in the leading term---thus achieving a much more significant improvement than the learning-rate schedule studied in this paper. In light of these results, learning-rate schedules are already being "re-thought" as we speak, and studying the behavior of the last iterate has received less attention in the past couple of years. If anything, the present paper only provides further evidence (through the negative result) that the individual iterates are ill-behaved in general and it is better to average the iterates instead. I would consider this negative result as an interesting addition to the stochastic-optimization literature, had it been presented in a completely different narrative (e.g., augmenting the discussion in "Stochastic gradient descent for non-smooth optimization: Convergence results and optimal averaging schemes" by Shamir and Zhang, 2013).
*** Exponentially decaying (or "constant-and-cut", as they are called here) schedules have actually been studied before in the paper "Beyond the Regret Minimization Barrier: Optimal Algorithms for Stochastic Strongly-Convex Optimization" by Hazan and Kale (JMLR 2014). This significantly weakens the main intended selling point of the paper which was being the first-ever study of such learning-rate schedules. The results in said paper are of a somewhat different nature, but they have arguably as little to do with deep learning as the results of the present paper has. Notably, both the present paper and the cited work rely on *strong convexity* of the objective (through assuming prior knowledge of the condition number), so I would expect that none of these results would explain anything in the context of deep learning.
On the technical side, the proofs appear to be correct but presented somewhat sloppily, with most of the notation appearing without proper definitions. For instance, the proof of Theorem 2 seems to import notation from the proof of Theorem 1, although without explicitly mentioning that the covariance matrix is assumed to be diagonal(ized). The proof of Theorem 3 then seems to again replace this previously (non-)established notation by another one (e.g., v becomes err and \eta becomes \gamma). The proofs also involve long sequences of inequalities without explanation, and only bound the variances (w_k-w^*_k)^2 without mentioning how this quantity is related to the excess risk. (The relation is well-known but not obvious at all for first-time readers of such proofs.)
One technical limitation of the results is that they assume a simple additive-noise model for the gradients, which the authors conveniently call "fairly natural" and incorrectly claim to hold for linear regression with well-specified models (footnote 8). In reality, the gradient noise in this setting also depends on the current iterate w_t, which makes analysis significantly harder. (To see the difference, just compare the complexity of the proofs of Lemma 1 and Theorem 2 that correspond to these different settings in "Harder, Better, Faster, Stronger Convergence Rates for Least-Squares Regression" by Dieleveut, Flammarion and Bach, 2017.)
Overall, I don't think that this paper is fit for publication in its present form. Once again, I would suggest that in a future version, the authors focus solely on discussing the actual results without attempting to draw disproportionate conclusions from them.
Detailed comments
=================
- pp.1, abstract: the first half of the abstract is completely irrelevant to the rest of the paper, so I'd suggest removing it.
- pp.1, "learning-rate schedules for SGD is a rather enigmatic topic"---"enigmatic" feels like a bit of a strong adjective here, given that there are many aspects of learning-rate tuning that are actually pretty well-understood.
- pp.2: The second paragraph on page 2 is again irrelevant to the actual technical content of the paper.
- pp.2, "all the works in stochastic approximation try to bound the error of each iterate of SGD"---This is simply not true, given the growing literature concerning the behavior of the *averaged iterates*.
- pp.4, first display: poor typesetting.
- pp.6, Eqs. 1--3: ditto.
- pp.8, last paragraph: Singling out the particular setting of gradient-norm minimization feels arbitrary and poorly justified.
- pp.11: the first and second displays should be switched for better readability (otherwise the first one comes without explanation). Also note that this form is not just due to the algorithm design, but also to the simplified noise model.
- pp.12, App B:
*** It appears that you forgot to mention here that you're working in the coordinate system induced by the eigenvectors, and also forgot to define the eigenvalues, etc.
*** The indices (1) and (k) are incoherent in the first display.
*** Although you promise you'll prove the inequality in the second display, you eventually prove something else.
*** It is not very clear on first sight that \ell^* actually exists and falls within the scope of \ell---you should explain that it exists due to the choice of the number of phases. (Which, by the way, should be rounded up to allow this property?)
*** The sequence of inequalities in the last display seems correct but unnecessarily hard to verify due to the lack of explanations.


Many deep-learning based methods such as Bayesian deep learning (DL) and deep reinforcement learning (RL) have heavily relied on the ability of a model being able to efficiently explore via Bayesian sampling. Particle-optimization sampling (POS) is a recently developed technique to generate high-quality samples from a target distribution by iteratively updating a set of interactive particles, with a representative algorithm the Stein variational gradient descent (SVGD). Though obtaining significant empirical success, the non-asymptotic convergence behavior of SVGD remains unknown. In this paper, we generalize POS to a stochasticity setting by injecting random noise in particle updates, called stochastic particle-optimization sampling (SPOS). Notably, for the first time, we develop non-asymptotic convergence theory for the SPOS framework, characterizing convergence of a sample approximation w.r.t. the number of particles and iterations under both convex-and noncovexenergy-function settings. Interestingly, we provide theoretical understanding of a pitfall of SVGD that can be avoided in the proposed SPOS framework, i.e., particles tend to collapse to a local mode in SVGD under some particular conditions. Our theory is based on the analysis of nonlinear stochastic differential equations, which serves as an extension and a complementary development to the asymptotic convergence theory for SVGD such as (Liu, 2017). With such theoretical guarantees, SPOS can be safely and effectively applied on both Bayesian DL and deep RL tasks. Extensive results demonstrate the effectiveness of our proposed framework.

This paper considers the problem of Bayesian inference using particle optimization sampler. Similarly to SGLD, authors propose Stochastic Particle Optimization Sampler (SPOS), augmenting Stein Variational Gradient Descent (SVGD) with diminishing Gaussian noise, replacing the hard-to-compute term of the Chen et al. (2018) formulation. Various theoretical results are given.
This paper was a pleasant read until I decided to check the proof of Theorem 3. I was not able to understand transitions in some of the steps and certain statements in the proof seem wrong.
Theorem 3:
"Note that $\theta^i_t$ and $\hat \theta^i_t$ are initialized with the same initial distribution µ0 = ν0 and we can also set $\theta^i_0$ to be independent of $\hat \theta^i_0$, we can have $\gamma(0) = 0$. $\gamma(0) = E \|\theta^i_0 - \hat \theta^i_0 \|^2$." - this doesn't seem right to me. Expectation of squared difference of two independent and identically distributed random variables is not 0, assuming expectation is with respect to their joint density.
"Then according to the Gronwall Lemma, we have" - I don't see how the resulting inequality was obtained. When I tried applying Gronwall Lemma, it seems that authors forgot to multiply by $t$ and  $\lambda_1$. Could you please elaborate how exactly Gronwall Lemma was used in this case.
"... some positive constants c1 and c2 independent of (M, d)$ - in the proof authors introduce additional assumption "We can tune the bandwidth of the RBF kernel to make ∇K ≤ H_∇K, which is omitted in the Assumption due to the space limit." First, there is a missing norm, since ∇K is a vector and H_∇K is I believe a scalar constant. Second, c1 = H_∇K + H_F, which both bound norm of d-dimensional vector and hence depend on d. I also suggest that all assumptions are included in the theorem statements, especially since authors have another assumption requiring large bandwidth. Additionally, feasibility of these both assumptions being satisfied should be explored (it seems to me that they can hold together, but it doesn't mean that part of assumptions can be moved to the supplement).
I find using Wasserstein-1 metric misleading in the theorem statement . This is not what authors really bound - from the proof it can be seen that they bound W_1 with W_2 and then with just an expectation of l2 norm. Moreover I don't understand the meaning of this bound. Theorem is concerned with W_1 distance between two atomic measures. What is the expectation over? Note that atom locations are supposed to be fixed for the W_1 to make sense in this context (and the expectation is over the coupling of discrete measures defined by weights of the atoms, not atom locations).
"Note the first bullet indicates U to be a convex function and W to be ... " I think it should be K, not W.
Theorems 3-6 could be lemmas, while there should be a unifying theorem for the bound.
Finally, I think notation should be changed - same letter is used for Wasserstein distance and Wiener process.
Other comments:
Example in Figure 1 is somewhat contrived - clearly gradient based particle sampler will never escape the mode since all modes are disconnected by regions with 0 density. Proposed method on the other hand will eventually jump out due to noise, but it doesn't necessarily mean it produces better posterior estimate. Something more realistic like a mixture of Gaussians, with density bounded away from zero across domain space, will be more informative.
It is not sufficient to report RMSE and test log likelihood for BNNs. One of the key motivating points is posterior uncertainty estimation. Hence important metric, when comparing to other posterior inference techniques, is to show high uncertainty for out of distribution samples and low for training/test data.


Sequence-to-Sequence (Seq2Seq) neural models have become popular for text generation problems, e.g. neural machine translation (NMT) Britz et al., 2017), text summarization (Nallapati et al., 2017;Wang & Ling, 2016), and image captioning (Venugopalan et al., 2015;. Though sequential modeling has been shown to be effective, the dependency graph among words contains additional semantic information, and thus can be utilized for sentence modeling. In this paper, we propose a Graph-Sequence-to-Sequence (GraphSeq2Seq) model to fuse the dependency graph among words into the traditional Seq2Seq framework. For each sample, the sub-graph of each word is encoded to a graph representation, which is then utilized to sequential encoding. At last, a sequence decoder is leveraged for output generation. Since above model fuses different features by contacting them together to encode, we also propose a variant of our model that regards the graph representations as additional annotations in attention mechanism  by separately encoding different features. Experiments on several translation benchmarks show that our models can outperform existing state-of-the-art methods, demonstrating the effectiveness of the combination of Graph2Seq and Seq2Seq.

[Summary]
This paper proposes a Graph-Sequence-to-Sequence (GraphSeq2Seq) model to fuse the dependency graph among words into the traditional Seq2Seq framework.
[clarity]
This paper is basically well written though there are several grammatical errors (I guess the authors can fix them).
Motivation and goal are clear.
[originality]
Several previous methods have already tackled to integrate graph structures into seq2seq models.
Therefore, from this perspective, this study is incremental rather than innovative.
However, the core idea of the proposed method, that is, combining the word representation, sub-graph state, incoming and outgoing representations seems to be novel.
[significance]
The experimental setting used in this paper is slightly out of the current main stream of NMT research.
For example, the current top-line NMT systems uses subword unit for input and output sentences, but this paper doesn’t.
Moreover, the experiments were performed only on the very small datasets, IWSLT-2014 and 2015, which have at most 153K training parallel sentences.
Therefore, it is unclear whether the proposed method has essential effectiveness to improve the performance on the top-line NMT baselines.
Comparing on the small datasets, the proposed method seems to significantly improve the performance over current best results of NPMT+LM.
Overall, I like the idea of utilizing sub-graphs for simplicity and saving the computational cost to encode a structural (grammatical or semantic) information.
However, I really wonder if this type of technique really works well on the large training datasets...


OVERCOMING CATASTROPHIC FORGETTING FOR CONTINUAL LEARNING VIA MODEL ADAPTATION
Learning multiple tasks sequentially is important for the development of AI and lifelong learning systems. However, standard neural network architectures suffer from catastrophic forgetting which makes it difficult for them to learn a sequence of tasks. Several continual learning methods have been proposed to address the problem. In this paper, we propose a very different approach, called Parameter Generation and Model Adaptation (PGMA), to dealing with the problem. The proposed approach learns to build a model, called the solver, with two sets of parameters. The first set is shared by all tasks learned so far and the second set is dynamically generated to adapt the solver to suit each test example in order to classify it. Extensive experiments have been carried out to demonstrate the effectiveness of the proposed approach.

This paper proposes a method for continual learning. The model has three components: a) a data generator to be used at training time to replay past examples, b) a parameter generator that takes the input observation to produce parameters for c) the actual classifier. The authors demonstrate the method on simple datasets with a stream of 2 or 3 tasks.
Strenghts:
- the combination of components is novel
- the method does not rely on task descriptors neither at training nor test time
Weaknesses:
- the paper needs a major rewrite to improve fluency and to better organize and describe the proposed approach
- the empirical validation is weak.
Relevance
Learning in a continual setting is certainly very relevant for this venue.
Novelty
While each component is by itself not very novel (replay methods for continual learning have already been used, networks predicting parameters have also become a fairly common approach in meta-learning literature), the proposed combination is novel in this sub-field.
Clarity
Clarity is very poor and definitely does not meet the acceptance bar for this conference. I believe that the authors would need to make a major revision to address this issue. While ICLR allows authors to revise papers, I think the revision needed to fix this draft goes beyond the acceptable limit, as reviewers would then need to make a whole new revision.
First, fluency is very poor. There are lots of grammatical errors (see first sentence of introduction "neural networks suffers.."),  a plethora of un-necessary acronyms which force the reader to go back and forth to figure out what they refer to (MA, DG, DPG, DPG&S, ...), and several sentences are not well formed (e.g., read first sentence of introduction).
Second, some statements are contradictory; e.g., the authors define "basic unit" as "simple MLP with one hidden layer", but then say it "is an activation function plus a matrix transformation"..
Third, graphics and formulas are too small and not legible.
Fourth, the organization of the paper is poor, it is very wordy yet vague. For instance, the authors should precisely describe how the data generator is trained in sec. 2.3. The authors should provide an algorithm summarizing how the different components interplay both at training and test time. At present, I am making educated guesses about how this system works.
For instance, how are real and generated examples interleaved? how is forgetting prevented in the data generator?
References to prior work
While there are lots of references in sec. 4, they are not sufficiently well described - see third paragraph of sec. 4 where the authors cite almost 20 papers by simply saying they are "some other approaches".
Also, I did not find mention to methods predicting parameters in the meta-learning community but also others like:
Denil et al. "Predicting network parameters in deep learning" NIPS 2013

Empirical validation
The empirical evaluation does show an advantage of the proposed approach on some simple streams composed by up to three tasks. However, a) the tasks are really simple because of the small number of tasks considered and b) the baselines are weak. For instance, EWC is now a bit out-dated as there are variants that work a bit better, like:
Chaundry et al. "Riemannian Walk ..." ECCV 2018
and there are other methods like "Progress and Compress" Schwartz ICML 2018 the authors could have compared against.
Besides, the authors do not mention anything about memory and time cost both at training and test time, possibly including the time to cross validate all the hyper-parameters of this method.
Overall, I am left with the sense that the proposed approach will be hard to scale to many more tasks and more realistic images (for which we do not quite know how to train efficiently and well generators).
Other comments
I did not find the formalization in eq. 9 very useful. The first and last term in that equation can be very big and there is no sense of how lose this bound is.
Also, it is not clear whether there is a general principle to partition the set of parameters (to determine which ones should be shared).


QUERY-EFFICIENT HARD-LABEL BLACK-BOX ATTACK: AN OPTIMIZATION-BASED APPROACH
We study the problem of attacking machine learning models in the hard-label black-box setting, where no model information is revealed except that the attacker can make queries to probe the corresponding hard-label decisions. This is a very challenging problem since the direct extension of state-of-the-art white-box attacks (e.g., C&W or PGD) to the hard-label black-box setting will require minimizing a non-continuous step function, which is combinatorial and cannot be solved by a gradient-based optimizer. The only two current approaches are based on random walk on the boundary (Brendel et al., 2017) and random trials to evaluate the loss function (Ilyas et al., 2018), which require lots of queries and lacks convergence guarantees. We propose a novel way to formulate the hard-label black-box attack as a real-valued optimization problem which is usually continuous and can be solved by any zeroth order optimization algorithm, such as randomized gradientfree method (Nesterov & Spokoiny, 2017). We demonstrate that our proposed method outperforms the previous stochastic approaches to attacking convolutional neural networks on MNIST, CIFAR, and ImageNet datasets. More interestingly, the proposed algorithm can also be used to attack other discrete and non-continuous machine learning models, such as Gradient Boosting Decision Trees.

This paper addresses black-box classifier attacks in the “hard-label” setting, meaning that the only information the attacker has access to is single top-1 label predictions. Relative to even the standard black-box setting where the attacker has access to the per-class logits or probabilities, this setting is difficult as it makes the optimization landscape non-smooth. The proposed approach reformulates the optimization problem such that the outer-loop optimizes the direction using approximate gradients, and the inner-loop estimates the distance to the nearest attack in a given direction. The results show that the proposed approach successfully finds both untargeted and targeted adversarial examples for classifiers of various image datasets (including ImageNet), usually with substantially better query-efficiency and better final results (lower distance and/or higher success rate) than competing methods.
=====================================
Pros:
Very well-written and readable paper with good background and context for those (like me) who don’t closely follow the literature on adversarial attacks. Figs. 1-3 are nice visual aids for understanding the problem and optimization landscape.
Novel formulation and approach that appears to be well-motivated from the literature on randomized gradient-free search methods. Novel theoretical analysis in Appendix that generalizes prior work to approximations (although, see notes below).
Good empirical results showing that the method is capable of query-efficiently finding attacks of classifiers on real-world datasets including ImageNet. Also shows that the model needn’t be differentiable to be subject to such attacks by demonstrating the approach on a decision-tree based classifier. Appears to compare to and usually outperform appropriate baselines from prior work (though I’m not very familiar with the literature here).
=====================================
Cons/questions/suggestions/nitpicks:
Eq 4-5: why \texttt argmin? Inconsistent with other min/maxes.
Eq 4-5: Though I understood the intention, I think the equations are incorrect as written: argmin_{\lambda} { F(\lambda) } of a binary-valued function F would produce the set of all \lambdas that make F=0, rather than the smallest \lambda that makes F=1. I think it should be something like:
min_{\lambda>0} {\lambda}
s.t. f(x_0+\lambda \theta/||\theta||) != y_0
Sec 3.1: why call the first search “fine-grained”? Isn’t the binary search more fine-grained? I’d suggest changing it to “coarse-grained” unless I’m misunderstanding something.
Algorithm 2: it would be nice if this included all the tricks described as “implementation details” in the paragraph right before Sec. 4 -- e.g. averaging over multiple sampled directions u_t and line search to choose the step size \eta. These seem important and not necessarily obvious to me.
Algorithm 2: it could be interesting to show how performance varies with number of sampled directions per step u_t.
Sec: 4.1.2: why might your algorithm perform worse than boundary-attack on targeted attacks for CIFAR classifiers? Would like to have seen at least a hypothesis on this.
Sec 6.3 Theorem 1: I think the theorem statement is a bit imprecise. There is an abuse of big-O notation here -- O(f(n)) is a set, not a quantity, so statements such as \epsilon ~ O(...) and \beta <= O(...) and “at most O(...)” are not well-defined (though common in informal settings) and the latter two are redundant given the meaning of O as an upper bound. The original theorem from [Nesterov & Spokoiny 2017] that this Theorem 1 would generalize doesn’t rely on big-O notation -- I think following the same conventions here might improve the theorem and proof.
=====================================
Overall, this is a good paper with nice exposition, addressing a challenging but practically useful problem setting and proposing a novel and well-motivated approach with strong empirical results.


The generalization ability of deep neural networks (DNNs) is intertwined with model complexity, robustness, and capacity. Through establishing an equivalence between a DNN and a noisy communication channel, we characterize generalization and fault tolerance for unbounded adversarial attacks in terms of informationtheoretic quantities. Invoking rate-distortion theory, we suggest that excess capacity is a significant cause of vulnerability to adversarial examples.

This paper considers the trade-off between the prediction accuracy of deep neural networks (DNNs) and sensitivity to adversarial examples.
Reviewing the (Gaussian) channel capacity and rate-distortion theory, i.e., the information bottleneck, the authors discuss their implications on the generalization performance of DNNs. The experiments demonstrate the SNR of gradients, information plane, the generalization gap, and fault tolerance against adversarial examples.
While the interpretations of DNN learning by the information theoretic concepts are interesting, most of them are already known results, and hence provide little novel theoretical knowledge.
The discussions in Section 3 are superficial. It is not clear how they are related to the main arguments of this paper.
While the experiment of fault tolerance is interesting, the implications obtained from experiments are somewhat trivial.
minor comments:
p.2, l.15: h, w, and c are undefined.
Section 2: Rate-distortion theory is usually explained by the sphere covering argument instead of sphere packing.
Section 4.3.1: It is not explained what zero and one-shot transfer learning is.
Pros:
The experiment of fault tolerance is interesting.
Cons:
Theoretical parts are basic results of information theory.
The implications of experiments are somewhat trivial.


VARIANCE REDUCTION FOR REINFORCEMENT LEARN- ING IN INPUT-DRIVEN ENVIRONMENTS
We consider reinforcement learning in input-driven environments, where an exogenous, stochastic input process affects the dynamics of the system. Input processes arise in many applications, including queuing systems, robotics control with disturbances, and object tracking. Since the state dynamics and rewards depend on the input process, the state alone provides limited information for the expected future returns. Therefore, policy gradient methods with standard state-dependent baselines suffer high variance during training. We derive a bias-free, input-dependent baseline to reduce this variance, and analytically show its benefits over state-dependent baselines. We then propose a meta-learning approach to overcome the complexity of learning a baseline that depends on a long sequence of inputs. Our experimental results show that across environments from queuing systems, computer networks, and MuJoCo robotic locomotion, input-dependent baselines consistently improve training stability and result in better eventual policies.

Summary: This work considers the problem of learning in input-driven environments -- which are characterized by an addition stochastic variable z that can affect the dynamics of the environment and the associated reward the agent might see. The authors show how the PG theorem still applied for a input-aware critic and then they show that the best baseline one can use in conjecture with this critic is a input-dependent one. My main concerns are highlighted in points (3) and (4) in the detailed comments below.
Clarity: Generally it reads good, although I had to go back-and-forth between the main text and appendix several times to understand the experimental side. Even with the supplementary material, examples in Section 3 and Sections 6.2 could be improved in explanation and discussion.
Originality and Significance: Limited in this version, but could be improved significantly by something like point (3)&(4) in detailed comments. Fairly incremental extension of the PG (and TRPO) with the conditioning on the potentially (unobserved) input variables. The fact that a input-aware critic could benefit from a input-aware baseline is not that surprising. The fact that it reduces variance in the PG update is an interesting result; nevertheless I strongly feel the link or comparison needed is with the standard PG update.
Disclaimer: I have not checked the proofs in the appendix.
Detailed comments:
1) On learning the input-dependent baselines: Generalising over context via a parametric functional approximation, like UVFAs [1] seems like a more natural first choice. Also these provide a zero-shot generalisation, bypassing the need for a burn-in period of the task. Can you comment on why something like that was not used at least as baseline?
2) Motivating example. The exposition of this example lacks a bit of clarity and can use some more details as it is not a standard MDP example, so it’s harder to grasp the complexity of this task or how standard methods would do on it and where would they struggle. I think it’s meant to be an example of high variance but the performance in Figure 2 seems to suggest this is actually something manageable for something like A2C. It is also not clear in this example how the comparison was done. For instance, are the value functions used, input-dependent? Is the policy input aware?
3) Input-driven MDP. Case 1/Case 2 : As noted by the authors, in case 1 if both s_t and z_t are observed, this somewhat uninteresting as it recovers a particular structured state variable of a normal MDP. I would argue that the more interesting case here, is where only s_t is observed and z_t is hidden, at least in acting. This might still be information available in hindsight and used in training, but won’t be available ‘online’ -- similar to slack variable, or privileged information at training time.  And in this case it’s not clear to me if this would still result in a variance reduction in the policy update. Case 2 has some of that flavour, but restricts z_t to an iid process. Again, I think the more interesting case is not treated or discussed at all and in my opinion, this might add the best value to this work.

4) Now, as mentioned above the interesting case, at least in my opinion, is when z is hidden. From the formulae(eq. (4),(5)), it seems to be that the policy is unaware of the input variables. Thus we are training a policy that should be able to deal with a distribution of inputs z. How does this compare with the normal PG update, that would consider a critic averaged over z-s and a z-independent baseline? Is the variance of the proposed update always smaller than that of the standard PG update when learning a policy that is unaware of z?
References:
[1] Schaul, T., Horgan, D., Gregor, K. and Silver, D., 2015, June. Universal value function approximators. In International Conference on Machine Learning (pp. 1312-1320).
[POST-rebuttal] I've read the author's response and it clarified some of the concerns. I'm increase the score accordingly.


TRAINING FOR FASTER ADVERSARIAL ROBUSTNESS VERIFICATION VIA INDUCING RELU STABILITY
We explore the concept of co-design in the context of neural network verification. Specifically, we aim to train deep neural networks that not only are robust to adversarial perturbations but also whose robustness can be verified more easily. To this end, we identify two properties of network models -weight sparsity and socalled ReLU stability -that turn out to significantly impact the complexity of the corresponding verification task. We demonstrate that improving weight sparsity alone already enables us to turn computationally intractable verification problems into tractable ones. Then, improving ReLU stability leads to an additional 4-13x speedup in verification times. An important feature of our methodology is its "universality," in the sense that it can be used with a broad range of training procedures and verification approaches.

The paper presents several ways to regularize plain ReLU networks to optimize 3 things
- the adversarial robustness, defined as the fraction of examples for which adversarial perturbation exists
- the provable adversarial robustness, defined as the fraction of examples for which some method can show that there exists no adversarial example within a certain time budget
- the verification speed, i.e. the amount of time it takes some method to verify whether there is an adversarial example or not

Overall, the ideas are sound and the analysis is solid. My main concern is the comparison between the authors method and the 'certification' methods, both conceptually and regarding performance.
The authors note that their method falls under 'verification', whereas many competing methods fall under 'certification'. They point to two advantages of verification over certification: (1) the ability to provide true negatives, i.e. prove that an adversarial example exists when it does, and (2) certification requires that 'models must be trained and optimized for a specific certification method'. However, neither argument convinces me regarding the utility of the authors method.
Regarding (2): The authors method also requires training the network in a specific way (with RS loss), and it is only compatible with verifiers that care about ReLU stability.
Regarding (1): It is not clear that this would be helpful at all. Is it really that much better if method A has 80% proven robustness and 20% proven non-robustness versus method B that has 80% proven robustness and 20% unknown? One could make the case that method B is actually even better.
So overall, I think one has to compare the authors method and the certification methods head-to-head. And in table 3, where this is done, Dvijotham comes out on top 2 out of 2 times and Wong comes out on top 2 out of 4 times. That does not seem convincing. Also, what about the performance numbers form other papers discussed in section 2?
-------
Other issues:
At first glance, the fact that the paper only deals with (small) plain ReLU networks seems to be a huge downside. While I'm not familiar with the verification / certification literature, from reading the paper, I suspect that all the other verification / certification methods also only deal with that or highly similar architectures. However, I will defer to the other reviewers if this is not the case.
To expand upon my comment above, I think the paper should discuss true adversarial accuracy on top of provable adversarial robustness. Looking at table 1, for instance, for rows 2, 3 and 4, it seems that the verifier used much less than 120 seconds on average. Does that mean the verifier finished for all test examples? And wouldn't that mean that the verifier determined for each test example exactly whether an adversarial example existed or not? In that case, I would write "true adversarial accuracy" instead of "provable adversarial accuracy" as column header. If the verifiers did not finish, I would include in the paper for how many examples the result was "adverarial example exists" and for how many the result was "timeout". I would also include that information in table 3, and I would also include proving / certification times there.
Based on the paper, I'm not quite sure whether the idea of training with L1 regularization and/or small weight pruning and/or ReLU pruning for the purpose of improving robustness / verifiability was an original idea of this paper. In either case, this should be made clear. Also, the paper seems to use networks with adversarial training, small weight pruning, L1 and ReLU pruning as its baseline in most cases (all figures except table 1). If some of these techniques are original contributions, this might not be an appropriate baseline to use, even if it is a strong baselines.
Why are most experiments presented outside of the "experiments" section? This seems to be bad presentation.
I would include all test set accuracy values instead of writing "its almost as high". Also, in table 3, it appears as if using RS loss DOES in fact reduce test error significantly, at least for CIFAR. Why is that?
While, again, I'm not familiar with the background work on verification / certification, it appears to me from reading this paper that all known verification algorithms perform terribly and are restricted to a narrow range of network architectures. If that is the case, one has to wonder whether that line of research should be encouraged to continue.
--------
Minor issues:
- "our focus will be on the most common architecture for state-of-the-art models: k-layer fully-connected feed-forward DNN classifiers" Citation needed. Otherwise, I would suggest removing this statement.
- "such models can be viewed as a function f(.,W)" - you also need to include the bias in the formula I think
- "convolutional layers can be represented as fully-connected layers". I think what you mean is "convolutional layers can be represented as matrix multiplication"
- could you make the difference between co-design and co-training more clear?
- The paper could include in the appendix a section outlining the verification method of Tjeng


EFFICIENTLY TESTING LOCAL OPTIMALITY AND ESCAPING SADDLES FOR RELU NETWORKS
We provide a theoretical algorithm for checking local optimality and escaping saddles at nondifferentiable points of empirical risks of two-layer ReLU networks. Our algorithm receives any parameter value and returns: local minimum, secondorder stationary point, or a strict descent direction. The presence of M data points on the nondifferentiability of the ReLU divides the parameter space into at most 2 M regions, which makes analysis difficult. By exploiting polyhedral geometry, we reduce the total computation down to one convex quadratic program (QP) for each hidden node, O(M ) (in)equality tests, and one (or a few) nonconvex QP. For the last QP, we show that our specific problem can be solved efficiently, in spite of nonconvexity. In the benign case, we solve one equality constrained QP, and we prove that projected gradient descent solves it exponentially fast. In the bad case, we have to solve a few more inequality constrained QPs, but we prove that the time complexity is exponential only in the number of inequality constraints. Our experiments show that either benign case or bad case with very few inequality constraints occurs, implying that our algorithm is efficient in most cases.

This paper proposes an efficient method to test whether a point is a local minimum in a 1-hidden-layer ReLU network. If the point is not a local minimum, the algorithm also returns a direction for descending the value of the loss function.
The tests include a first-order stationary point test (FOSP), and a second-order stationary point test (SOSP). As these test can be written as QPs, the core challenge is that if there are M boundary points in the dataset, i.e.,  data points on a non-differentiable region of the ReLU function, then the FOSP test requires 2^M tests of extreme rays -- each boundary partition the whole space into at least two parts. This paper observes that since the feasible sets are pointed polyhedral cones. Therefore checking only these extreme rays suffices. This results in an efficient test with only 2M tests.
Lastly, the paper performs experiments on synthetic data. It turns out there are surprisingly many boundary points.
Comments:
This paper proposes an interesting method of testing whether a given point is a local minimum or not in a ReLU network. The technique is non-trivial and requires some key observation to make it computationally efficient. However, I have the following concerns:
1) such a test may need very high numeric precision. For instance, you cannot make sure whether a floating point number is strictly greater than 0 or not. The small error may critically affect the property of a point.
2) boundary points of a ReLU network should have measure 0 (correct me if not). The finding in the experiment shows surprisingly many boundary points. This is counter-intuitive. Is it because of numeric issues? You might misclassify non-boundary points.
3) Usefulness.
    a. The paper claims that such a test would be very useful in practice. However, they cannot even perform an experiment on real datasets.
    b. Such a method only works for one-hidden layer network. It is not clear deeper network admit similar structure.
    c. Practical training of neural-network usually trains the network using SGD, which always obtain a solution with a non-zero gradient. In this sense, there is no need for such a testing.
    d. It seems like it is much easier to perform a test with different activation function, e.g., sigmoid.

If the authors can address these concerns convincingly, I would be happy to change the rating.


AUXILIARY VARIATIONAL MCMC
We introduce Auxiliary Variational MCMC, a novel framework for learning MCMC kernels that combines recent advances in variational inference with insights drawn from traditional auxiliary variable MCMC methods such as Hamiltonian Monte Carlo. Our framework exploits low dimensional structure in the target distribution in order to learn a more efficient MCMC sampler. The resulting sampler is able to suppress random walk behaviour and mix between modes efficiently, without the need to compute gradients of the target distribution. We test our sampler on a number of challenging distributions, where the underlying structure is known, and on the task of posterior sampling in Bayesian logistic regression. Code to reproduce all experiments is available at https://github.com/AVMCMC.

This paper proposes an auxiliary variable MCMC scheme involving variational inference for efficient MCMC. Given a target distribution p(x), the authors introduce an auxiliary variable a, and learn conditional distributions p(a|x) and q(a|x) by minimizing the KL divergence between p(x)p(a|x) and q(a)q(x|a), with q(a) something simple (the authors use Gaussian). A MH proposal step involves simulating x givea the current MCMC sample x (from p(a|x), taking a step in A-space, and then returning back to the X space (using q(x|a)).  The authors show how to calculate the acceptance probability.
I think the idea is nice and useful (I'm surprised people haven't thought of this before), though I think the paper presents this in a less clear way (as an extension of ideas from Agakov and Barber's "Auxiliary variational method"). While this is correct and perhaps more general, in my mind it slightly obscures the main idea, as well as the strong ties with variational autoencoders: express a complex distribution as a (learnt) transformation of a simple distribution (this is the actual approach taken in the experiments).
The motivation of the approach is that the nonlinear encoding network can transform the complex p(x) into a simpler q(a).
For this reason, I think an important baseline is the independent MH sampler from equation 8 (I think this essentially uses a trained VAE generative model as a proposal distribution). The authors talk about how producing independent proposals can be sub-optimal, yet it seems to me that if the encoder and decoder neural networks are powerful enough, this should do a good job. I think excluding this baseline hurts the paper a bit.
The proof of correctness while correct is a bit unclear, can perhaps be simplified if you view the MCMC algorithm as operating on an augmented space (x,a,x') with stationary distribution p(x)q(a|x)q(x'|a) (writing writing q for \tilde(q)). This clearly has the right distribution over x. Each MCMC iteration starts with x and proceeds as follow:
  1) Given x, sample a and x' from q(a|x) and q(x'|a)
  2) Make a deterministic proposal on the augmented space to swap (x,x'). The acceptance probability is now equation 2.
  3) Discard a,x'.
In figure 4, the authors use HMC as an "improved MCMC algorithm", yet this is not an algorithm that deals with multimodality well. More useful would be to include some tempering algorithm like serial or parallel tempering.
While I like the idea, I unfortunately don't think the experiments are very convincing (and the authors barely discuss their results). Other than mixture of Gaussians, HMC (which involves no training) appears to be superior. With some tempering, I expect it to outperform the proposed method for the MoG case
Table 2 left: since HMC involves no training, does this mean that, taking training time into account, HMC is 5-6 orders of magnitude more efficient. L?ke I mentioned earlier, these results need more discussion.
It would also help to provide absolute training and run times, so the reader can better understand whether the proposed method of ANICE is better.
Figure 3: why don't the authors also plot the histogram of values in the auxiliary space, p(a). It would be interesting to see how Gaussian this is (this is what variational inference is trying to achieve). Also, does Figure 3(a) mean that conditioned on x, p(a|x) is basically a delta function? This would suggest that the encoder is basically learning a deterministic transformation to a simpler low-dimensional space? There is some work in this direction in the statistics literature, e.g.
"Variable transformation to obtain geometric ergodicity in the random-walk Metropolis algorithm"
The authors some refers to the distribution of a|x as q(a|x) sometimes (in section 2.1) and sometimes as p(a|x) which is a bit confusing.
Figure 2: the labels are wrong.


ARM: AUGMENT-REINFORCE-MERGE GRADIENT FOR STOCHASTIC BINARY NETWORKS
To backpropagate the gradients through stochastic binary layers, we propose the augment-REINFORCE-merge (ARM) estimator that is unbiased, exhibits low variance, and has low computational complexity. Exploiting variable augmentation, REINFORCE, and reparameterization, the ARM estimator achieves adaptive variance reduction for Monte Carlo integration by merging two expectations via common random numbers. The variance-reduction mechanism of the ARM estimator can also be attributed to either antithetic sampling in an augmented space, or the use of an optimal anti-symmetric "self-control" baseline function together with the REINFORCE estimator in that augmented space. Experimental results show the ARM estimator provides state-of-the-art performance in auto-encoding variational inference and maximum likelihood estimation, for discrete latent variable models with one or multiple stochastic binary layers. Python code for reproducible research is publicly available.

Overview.
The authors present an algorithm for lowering the variance of the score-function gradient estimator in the special case of stochastic binary networks. The algorithm, called Augment-REINFORCE-merge proceeds by augmenting binary random variables. ARM combines Rao-Blackwellization and common random numbers (equivalent to antithetic sampling in this case, due to symmetry) to produce what the authors claim to be a lower variance gradient estimator. The approach is somewhat novel. I have not seen other authors attempt to apply REINFORCE in an augmented space and with antithetic samples / common random numbers, and Rao-Blackwellization. This combination of techniques may be a good idea in the case of Bernoulli random variables. However, due to a number of issues discussed below, this claim is not possible to evaluate from the paper.
Issues/Concerns
- I assess the paper in its current form as too far below the acceptable standard in writing and in clarity of presentation, setting aside other conceptual issues which I discuss below. The paper contains many typos and a few run-on sentences that span 5-7 lines. This hinders understanding substantially. A number key terms are not explained, irregularly. Although the paper assumes that readers do not know the mean and a variance of a Bernoulli random variable, or theof  definition of an indicator function, it does not explain what random variable augmentation means. The one sentence that comes close to explaining it seems to have a typo: "From (5) it becomes clear that the Bernoulli random variable z ∼ Bernoulli(σ(φ)) can be reparameterized by racing two augmented exponential random variables ...". It is not clear what is meant by "racing," here, and I do not find it clear from equation (5) what is going on. Unfortunately, in the abstract, the paper claims that variance reduction is achieved by "data augmentation," which has a very specific meaning in machine learning unrelated to augmented random variables, further obfuscating meaning. Similarly, the term "merge" is not explained, despite the subheading 2.3.
- Computational issues are not addressed in the paper. Whether or not this method is useful in practice depends on computational complexity
- No effort is made to diagnose the source of the variance reduction, other than in the special case of analytically comparing with the Augment-REINFORCE estimator, which does not appear in any of the experiments.
- No effort is made to empirically characterize the variance of the gradient estimator, unlike Tucker et al (2017) and Grathwohl et al. (2018).
- The algorithm presented in the appendix appears to only address single-layer stochastic binary networks, which are uninteresting in practice.
- Figure 2 (d), (e), and (f) all show that ARM was stopped early. Given that RELAX and REBAR overfit, this is a little troubling. Overal, these results are not very convincing that ARM is better, particularly in the absence of variance analysis (empirically, or other than w.r.t. the same algorithm without the merge step). All algorithms should be run for the same number of steps, particularly in cases where they may be prone to overfitting.
- Figure 1 I believe contains an error for the REINFORCE figure. In my own research I have run these experiments myself, with a value of p close to the one used by the authors. REBAR and RELAX both reduce to a REINFORCE gradient estimator with a control variate that is differentiably reparametrizable, and so the erratic behaviour of the REINFORCE estimator in this case is likely wrong.
- There is a mysterious sentence on page 6 that refers to ARM adjusting the "frequencies, amplitudes, and signs of its gradient estimates with larger and more frequent spikes for larger true gradients"
-The value to the community of another gradient estimator for binary random variables is low, given the plethora of other methods available. Given the questions remaining about this methodology and its experiments, I recommend against publication on this basis also.
- Table 2 compares results that mix widely different architectures against each other, some taken directly from papers, others possibly retrained. This is not a valid comparison to make when evaluating a new gradient estimator, where the model must be fixed.
* EDIT: I have re-evaluated the careful and comprehensive response to my concerns by the authors. I thank them for their effort in this. As many of the concerns were related to communication and have been addressed in the most recent draft, I think it is appropriate to move my review upwards. The revisions make this paper quite different from the original, and I am happy to re-evaluate on that basis--this is a peculiarity of the ICLR open review procedure, but I consider it a strength.
I note that "data augmentation" in machine learning appears to have collided with a term in the Bayesian statistics literature, and the authors have provide a number of citations to support this. I strongly recommend "variable augmentation" going forward, as that is an accurate description (you are augmenting a random variable, rather than the input data domain). This appears to be one of the growing pains of the field of ML which has distinct and often orthogonal concerns to classical statistics around density approximation and computational issues.*


PHASE-AWARE SPEECH ENHANCEMENT WITH DEEP COMPLEX U-NET
Most deep learning-based models for speech enhancement have mainly focused on estimating the magnitude of spectrogram while reusing the phase from noisy speech for reconstruction. This is due to the difficulty of estimating the phase of clean speech. To improve speech enhancement performance, we tackle the phase estimation problem in three ways. First, we propose Deep Complex U-Net, an advanced U-Net structured model incorporating well-defined complex-valued building blocks to deal with complex-valued spectrograms. Second, we propose a polar coordinate-wise complex-valued masking method to reflect the distribution of complex ideal ratio masks. Third, we define a novel loss function, weighted source-to-distortion ratio (wSDR) loss, which is designed to directly correlate with a quantitative evaluation measure. Our model was evaluated on a mixture of the Voice Bank corpus and DEMAND database, which has been widely used by many deep learning models for speech enhancement. Ablation experiments were conducted on the mixed dataset showing that all three proposed approaches are empirically valid. Experimental results show that the proposed method achieves state-of-the-art performance in all metrics, outperforming previous approaches by a large margin 1 .

This paper tackles one of important speech enhancement issues of how to predict phase information. The authors work on this problem based on three novel techniques, one is to use complex U-net, second is to propose a new complex mask representation, which is well bounded and well model complex mask distribution, and the last is an objective function motivated by SDR. The paper is well written, and also shows the experimental effectiveness of the proposed method by analyzing these three novel techniques and also by comparing the method with other speech enhancement methods. My major concern about this paper is that this paper is a little bit too specific to the speech enhancement applications, which will not be accepted with so many researches in the major ICLR community. My suggestion is to describe some potential applications of this method to the other (speech) applications including speech separation, noise-robust front-end for ASR, TTS, or other speech analysis, and also discuss the possibility of extending this method for multichannel input. I’m more interested in the multichannel enhancement because the phase (difference) is critical in this scenario.
Comments:
- Introduction: It’s better to cite and discuss the paper of “E. Hakan et al, “Phase-sensitive and recognition-boosted speech separation using deep recurrent neural networks,” Proc. ICASSP’15, pp. 708--712 (2015). This paper is one of the first studies tries to incorporate the phase information to DNN based speech enhancement.
- Several researchers prefer to use LSTM based enhancement method. Please discuss wether this method (objective function and complex masks) can be applied to complex extensions of LSTMs instead of complex U-net.
- Page 2, the first paragraph: You may also refer https://arxiv.org/abs/1810.01395
- Page 3, it’s better to explicitly mention that h = x + i y
- Section 3.3: discuss how we treat STFT/iSTFT operations under a computational graph representation. It is not so obvious.
- Section 3.3: again it’s better to mention E. Hakan’s method here.
- Page 6 footnote: I cannot access to the URL. Please check it.
- Experiments: I think it would be more interesting to add SDR (using speech and noise as a source) to the experimental measure. Some people use SDR as a speech enhancement measure, and I’m expecting that this method can have more reasonable performance since it is optimized based on wSDR.


When autonomous agents interact in the same environment, they must often cooperate to achieve their goals. One way for agents to cooperate effectively is to form a team, make a binding agreement on a joint plan, and execute it. However, when agents are self-interested, the gains from team formation must be allocated appropriately to incentivize agreement. Various approaches for multi-agent negotiation have been proposed, but typically only work for particular negotiation protocols. More general methods usually require human input or domain-specific data, and so do not scale. To address this, we propose a framework for training agents to negotiate and form teams using deep reinforcement learning. Importantly, our method makes no assumptions about the specific negotiation protocol, and is instead completely experience driven. We evaluate our approach on both non-spatial and spatially extended team-formation negotiation environments, demonstrating that our agents beat hand-crafted bots and reach negotiation outcomes consistent with fair solutions predicted by cooperative game theory. Additionally, we investigate how the physical location of agents influences negotiation outcomes.

Note: This is an emergency review. I managed not to look at existing comments/ratings for this paper before writing my review.
Summary
---
This paper studies deep multi-agent RL in settings where all of the agents must cooperate to accomplish a task (e.g., search and rescue, multi-player video games). It uses simple cooperative weighted voting games 1) to study the efficacy of deep RL in theoretically hard environments and 2) to compare solutions found by deep RL to a fair solution concept known in the literature on cooperative game theory.
In a weighted voting game each agent is given a weight and the agents attempt to form teams. The first team whose total weights exceed a known threshold get the total reward, which is distributed amongst the team members. Given such a game, the __shapely value__ of an agent measures the importance of that agent. How much does it contribute to a team from this set of agents? How much payoff should it get? These have existed in the literature for over 60 years and appear to be widely known and used.
All of this is agnostic to how the agents communicate to form teams: i.e., the communication protocol or the actions available in the environment. The protocol matters because it can allow certain teams to form more or less easily than others, even though the same team would get the same reward regardless of protocol. This can make an agent more or less effective under different protocols. Here two protocols are considered - one where agents suggest proposed teams directly and another where they suggest teams by congregating on a 2d plane. Both protocols result in games whose Nash equilibria are computationally intractable.
The paper shows 4 results:
1) It considers a hand-designed bot similar to models from the game theory literature. Relative to a group of RL agents, an additional RL bot will outputperform a hand-designed bot in terms of average reward it receives.
2) The average reward of a bot is strongly correlated with that bots shapely value.
3) In the negotiation by congregation environment, a bot's spatial position can affect its ability to negotiate.
4) Shapely values can be predicted quite accurately from the weights and threshold that define a cooperative voting game, though these predictions have high variance.
The paper concludes that deep RL is effective at learning agents for cooperative games in multiple ways:
1) Deep agents are better than a hand-designed agent.
2) Deep agents easily extend across negotiation protocols (something hand-designed agents don't do).
3) A popular result in cooperative game theory predicts how effective agents should be. Deep agents are just about that effective.
Strengths
---
* The paper does a pretty good job of reviewing relevant work from game theory.
* Some of the organization is nice (e.g., the list of reasons classic game theory doesn't extend to practice; one section per experiment).
Weaknesses mentioned in individual sections...
Quality
---
Overall, things were well thought through, but I would have liked more out of the experiment 4 section and I think a few minor details might have been missed.
Details:
Section 4.5/Experiment 4: The Shapely value comparison is the most important part of the paper.  This section is important because it tries to explain those results, but it seems like there's more work to be done here. I'm not sure capacity is eliminated as a concern, and there might be other concerns not listed like optimization error.
* I'm not sure what conclusion to take from experiment 4. Shapely values can be computed from the cooperative games directly, independent of protocol. We're interested in __policies__ that get exactly the shapley values as their average reward. Policies depend on the protocol. Does being able to predict shapley values mean that a model with similar capacity can learn a policy that will have the desired shapley value? Was that the desired conclusion?
Other comments:
* The current hand-designed baseline uses weights to form a probability distribution. There should be another baseline that uses Shapley values instead of weights.
* It's not clear exactly what the spatial nature of the Team Patches environment adds. It is good to try another environment just to have an additional notion of generalization.
Clarity
---
Overall, the motivation could be clearer. Is the point to do work on cooperative games or to compare to Shapley values?
Presentation details:
* The paper does not get to specific examples of agents acting in environments until about page 4. Providing a simple, brief example which leaves out some details at the beginning would go a long way toward aiding intuitions about the abstract concepts discussed. Here are some clarity issues I had that might have been helped with an example:
    * What exactly is it about a task which requires agents to form teams? How necessary are those teams?
    * What exactly is a negotiation protocol?
    * What does it mean to distribute/share a reward across agents?
* When talking about shapely values, fairness seems to be emphasized somewhat often, but no concrete intuition about what fairness means in this setting is provided.
* Intro para 4: What does the human data measure? And thus how might it be useful?
* Intro para 7: People in the ICLR community will be more familiar with this work. What is the difference between communication and team forming?
* The section on Shapley values should provide more intuition about what they're thought about as measuring. (An agent's importance or what payoff it should expect, according to wikipedia.)
* Instead of measuring correlation to Shapley values, the paper measures whether average reward approximates Shapley values. It seems like the two are on a different scale. Average reward is unbounded and Shapley values are in [0, 1]. How are they comparable?
* The paper mentions how results vary over different types of boards (ones with higher and lower variance in the sampled weights). It does not show results to support this discussion. A conditional analysis of performance would be interesting and relevant, perhaps conditional versions of Fig. 3.
Originality
---
I do not know much about game theory and I'm only somewhat familiar with multi-agent deep RL, so I am not in a great position to judge novelty. Nonetheless, Given existing work in multi-agent RL, it is unsurprising that deep RL agents learn reasonable policies in these environments.
As far as I know, the comparison of average reward to shapely values has not been done before.
Significance
---
Most work in multi-agent RL evaluates by 1) comparing to baselines or 2) measuring some environment/task-specific metric. The best thing about this work is that it evaluates by comparing actual performance to some external theory that suggests how well an agent should be able to do, falling into a 3rd category.  It's not alone in this category (e.g., paper compare to theoretically optimal baselines if they can), but it is interesting to see another example of this kind of evaluation.
The community might possibly start to focus more on cooperative games because of this paper. A more interesting result would occur if others are inspired to implement more comparisons to how agents __should__ perform in theory.
Justification for Final Rating
---
I am unsure about novelty. As described above, the paper is lacking in clarity and quality (esp. section 4.5), but I don't think these concerns would invalidate the main result. I think the contribution is significant because of the kind of evaluation, but I'm not sure it will ultimately have a large impact. Thus I think some of the concerns above should be addressed before publication, but I would not be very disappointed if it were published as is.


In this paper we develop an approach based on deep reinforcement learning (DRL) to address dynamic pricing problem on E-commerce platform. We models realworld E-commerce dynamic pricing problem as Markov Decision Process. Environment state are defined with four groups of different business data. We make several main improvements on the state-of-the-art DRL-based dynamic pricing approaches: 1. We first extend the application of dynamic pricing to a continuous pricing action space. 2. We solve the unknown demand function problem by designing different reward functions. 3. The cold-start problem is addressed by introducing pre-training and evaluation using the historical sales data. Field experiments are designed and conducted on real-world E-commerce platform, pricing thousands of SKUs of products lasting for months. The experiment results shows that, on E-commerce platform, the difference of the revenue conversion rates (DRCR) is a more suitable reward function than the revenue only, which is different from the conclusion from previous researches. Meanwhile, the proposed continuous action model performs better than the discrete one.

This paper presents a reinforcement learning approach to dynamically set the price of items on sale on an e-commerce website, based on a state description consisting of several kinds of features: price, sales, customer traffic and competitiveness. The actions (possible prices) are constrained to lie in item-specific lower and upper bounds. Against a proposed method relying a continuous action space (an implementation of the Deep Deterministic Policy Gradient of Lillicrap et al., 2015), alternatives relying on multi-armed bandits and Deep Q Networks are evaluated. Experimental results from an online deployment are presented.
The paper is at times hard to understand. In particular, it would benefit from a thorough review of English grammar and style. For instance, the alternating descriptions between the past and present tenses (e.g. the abstract and the introduction) are quite non-natural and somewhat irritating [see specific instances in the detailed comments, below].
From an algorithmic standpoint, the paper does not introduce new methods and relies on well-known reinforcement learning techniques. The proposed methodology of applying specific RL techniques such as DDPG to pricing appears novel. However, there is an abundant literature on optimal pricing and discounting in operations research, much of it based on dynamic programming techniques, and more links to this literature could be provided.
One could question the choice of the reward function chosen (eq. 1): since it is a ratio, it is extremely sensitive to variations in the denominator, and this could severely impact convergence.
The primary weakness of the paper lies in the experimental evaluation: at least the following informations are missing to truly understand the methodological impact and economic benefits of the proposed approach:
1. A complete description of the experimental setting, i.e. how many SKUs (stock keeping units) were evaluated, over which product categories, over what time horizon. Of those, how are they distributed in the fast-mover / slow-mover plane (Syntetos et al., 2005)? What special events were material during the time period? How many of those were predicted and incorporated into the state representation?
2. One or more tables of results giving expected utility gains over baseline of the proposed methods, along with confidence intervals.
3. For an ICLR submission, there should be additional investigations as to the structure of the learned representations, at least by the DQN and the DDPG models — is the state embedding learned by the networks somewhat suggestive of economically meaningful properties of the items or the sales environment?
As such, even though the paper is interesting, it is in too early a state to recommend acceptance at ICLR.
Detailed comments:
* p. 1: many past tenses that should be in the present tense, e.g. (just in the abstract): modeled ==> models, defined ==> defines, then it introduced ==> it then introduces, were designed ==> are designed. Many other cases in the rest of the paper.
* p. 1: has draw ==> has drawn
* p. 2: Forth ==> Fourth
* p. 2: overtime ==> over time
* p. 2: is assumed, to ==> is assumed to
* p. 2: described ==> describe
* p. 3: “looks deep inside learning while earning approaches” ==> sentence not clear
* p. 3: this is not clear: “since the number of page visitors may ﬂuctuate dramatically and this could lead to non-concavity“ ==> why would it lead to non-concavity?
* p. 4: The whole paragraph before eq. (1) is not clear
* p. 5: The D in eq. (5) should be explained immediately, not after eq. (6).
* p. 5: In eq. (5), $\theta’$ is not explained: how does it differ from $\theta$ ?
* p. 5: In eq. (6), how is $\theta^{Q’}$ different from $\theta^{Q}$ ?
* p. 6: in eq. (8), the denominator r_t could be a small number, leading to a noisy error; this should be discussed.
* p. 6: Below eq. (12), these sentences are not clear: “For dynamic pricing problem, we particularly concern the outcome from changing between prices. To have a well knowledge between two prices before and after pricing.”


In traditional neural networks for image processing, the inputs of the neural networks should be the same size such as 224×224×3. But how can we train the neural net model with different input size? A common way to do is image deformation which accompany a problem of information loss (e.g. image crop or wrap). In this paper we propose a new network structure called Attention Incorporate Network(AIN). It solve the problem of different size of input images and extract the key features of the inputs by attention mechanism, pay different attention depends on the importance of the features not rely on the data size. Experimentally, AIN achieve a higher accuracy, better convergence comparing to the same size of other network structure.

This paper presents a strategy to overcome the limitation of fixed input image sizes in CNN classifiers. To this end, the authors incorporate some local and local attention modules, which fit inputs of arbitrary size to the fixed-size fully connected layer of a  CNN. The method is evaluated on three public classification benchmarks: CIFAR-10, ImageNet and Kaggle-Furniture128.
The results are better than those of the baseline architecture with fixed input size.
Even though the need of handling arbitrary input size is an interesting problem, I have several major concerns about this paper:
- One of the main problems of this paper is its presentation, both the writing and methodology. The writing is very poor, with continuous errors and many wrong definitions and concepts. For example, authors talk about ‘data argumentation’, ‘pooling reduces the size of the hidden layers’,’back-to-back convolutional layers’
Further, the paper is not well structured, which makes it very hard to follow.
Methodology:
Another major concern is that I do not see how this approach allows the network to be input-size independent. If one looks at table 1, in both AIN-121 and AIN-169 the GAIL module employs kernel sizes equal to M/32xN/32, with M and N denoting the input image sizes. In this case, for each image, the kernel size will be different and, consequently, the number of learnable parameters. It is not clear to me how this is solved in this paper, as it ultimately results in a ‘different’ architecture for each different input size.
When doing the sum on the proposed module, what does the result represent? absolute sum? mean of the sum? I also believe that a lot of information is lost when performing this operation (for example going from 32 to 1), in addition of the other spatial reductions during the network forward pass. Please comment on this and give a more detailed information about the proposed module.
Evaluation:
In CIFAR-10, authors say that ‘keep MOST of the setting similar to ResNet’. What is then difference with the training with ResNet? For a fair comparison both settings should remain the same. In addition, what is the benefit of evaluating this approach on CIFAR-10, as the images are all of the same size? Furthermore, improvement is marginal with respect to the baselines (and it is not clear what is the reason behind the improvement), while increasing the model complexity by nearly 50%.
Kaggle-Furniture128: Why the learning is stopped exactly at epochs 38 and 53? Is this the same for all the networks? DenseNet and ResNet are pre-trained with what dataset?
ImageNet: In table 4, while the results for the baselines are evaluated on the validation set, the test set is used for evaluating the proposed approach. Furthermore, some results on the test set are obtained with ‘augmentations’. The reported values should correspond to the original test set without any kind of modification.
Minor comments:
The authors assess the input fixed-size problem as a main problem in image processing. Despite being a limitation, some other image processing tasks, such as semantic segmentation, do not suffer from this problem, as CNNs are fully convolutional, and can accommodate images of arbitrary size.
Many inconsistencies between terms: LAIL and then LAIN and GAIL and GAIN.'


Probabilistic models are often trained by maximum likelihood, which corresponds to minimizing a specific form of f -divergence between the model and data distribution. We derive an upper bound that holds for all f -divergences, showing the intuitive result that the divergence between two joint distributions is at least as great as the divergence between their corresponding marginals. Additionally, the f -divergence is not formally defined when two distributions have different supports. We thus propose a noisy version of f -divergence which is well defined in such situations. We demonstrate how the bound and the new version of f -divergence can be readily used to train complex probabilistic generative models of data and that the fitted model can depend significantly on the particular divergence used.

The paper proposes a method for training generative models with general f-divergences between the model and empirical distribution (the VAE and GAN objectives are captured as specific instantiations). The trick that leads to computational tractability involves utilizing a latent variable and optimizing the f-divergence between joint distributions which is an upper bound to the (desired) f-divergence between the marginal distributions. Distribution support issues are handled by convolving the data space with a blurring function. Empirical results on image datasets demonstrate that the additional training flexibility results in qualitatively different learned models. Specifically, optimizing the reverse KL (and/or Jensen-Shannon divergence) leads to sharper images, perhaps at the loss of some variance in the data distribution.
I liked the simplicity of the core idea, and appreciated the exposition being generally easy to follow. The application of upper bounds to general f-divergences for training generative models is novel as far as I know. My two issues are with the practicality of the implementation and evaluation methodology, both potentially affecting the significance of the work. Regarding practicality, it appears the training details are specific to each experiment. This begs the question of how sensitive the results are to these settings. Regarding the methodology, it would have been nice to see the method of Nowozin et al. (2016) applied in all experiments since this is a direct competitor to the proposed method. Moreover, the subjective nature of the results in the real dataset experiments makes it difficult to judge what the increased flexibility in training really provides - although I do note that the authors make this same point in the paper. Finally, given that model training is the paper's focus, an explicit discussion of computational cost was missed.
Even with these issues, however, I believe the paper makes a contribution to the important problem of fitting expressive generative models. In my opinion, a more rigorous and thorough experimental exploration would increase the value, but the paper demonstrates that training with alternative f-divergences is feasible.


Combinatorial optimization is a common theme in computer science. While in general such problems are NP-Hard, from a practical point of view, locally optimal solutions can be useful. In some combinatorial problems however, it can be hard to define meaningful solution neighborhoods that connect large portions of the search space, thus hindering methods that search this space directly. We suggest to circumvent such cases by utilizing a policy gradient algorithm that transforms the problem to the continuous domain, and to optimize a new surrogate objective that renders the former as generic stochastic optimizer. This is achieved by producing a surrogate objective whose distribution is fixed and predetermined, thus removing the need to fine-tune various hyper-parameters in a case by case manner. Since we are interested in methods which can successfully recover locally optimal solutions, we use the problem of finding locally maximal cliques as a challenging experimental benchmark, and we report results on a large dataset of graphs that is designed to test clique finding algorithms. Notably, we show in this benchmark that fixing the distribution of the surrogate is key to consistently recovering locally optimal solutions, and that our surrogate objective leads to an algorithm that outperforms other methods we have tested in a number of measures.

## Summary ##
The authors apply policy gradients to combinatorial optimization problems. They suggest a surrogate reward function that mitigates the variance in the reward, and hence the update size. They demonstrate performance on a clique-finding problem.
## Assessment ##
I don't think Cakewalk is different enough from the cross-entropy method to warrant acceptance in ICLR.
 I also have concerns about the independence assumption in their sampling distribution (Section 3.2), and the fact that their experiments use the same set of (untuned) hyperparameters for each method.
They both approximate the reward CDF from K samples and use this to construct a surrogate reward. The difference is that Cakewalk uses the CDF directly, while CE uses a threshold function on the CDF.
## Specific Comments and Questions ##
1. Cakewalk is *very* closely related to the cross-entropy method. The authors acknowledge this connection, but I think they should begin by introducing CE and then explain how Cakewalk generalizes it. Both Cakewalk and CE approximate the reward CDF from K samples and use this to construct a surrogate reward. The difference is that Cakewalk uses the CDF directly, while CE uses a threshold function on the CDF.
2. The distribution proposed in section 3.2 assumes independence between the elements $x_j$. This seems problematic for some relatively simple problems. Consider $x$ a binary vector and reward equal to the parity $S(x) = \sum{x_j} % 2$.
3. In the experiments, there are large discrepancies between different optimizers on Cakewalk (e.g. SGA vs AdaGrad, Table 4). Is there any explanation for this?
4. How were the hyperparameters (learning rate, AdaGrad $\delta$, Adam $\beta_1, \beta_2$) chosen? It seems like a large assumption that the same learning rate would work for different methods, especially when some of them are normalizing the objective function. I would suggest tuning these values for each method independently.
5. It would be nice to see experimental results on more than one problem. The authors discuss their results on k-medoids in the appendices, but it seems like these results aren't quite complete yet.
6. In Table 3, the figure in bold is not the lowest (best) in the table. The reason for this is only given in a single sentence at the end of Section 6, so it is a little confusing. I would replace these values with N/A or something similar.


IMPOSING CATEGORY TREES ONTO WORD- EMBEDDINGS USING A GEOMETRIC CONSTRUCTION
We present a novel method to precisely impose tree-structured category information onto word-embeddings, resulting in ball embeddings in higher dimensional spaces (N -balls for short). Inclusion relations among N -balls implicitly encode subordinate relations among categories. The similarity measurement in terms of the cosine function is enriched by category information. Using a geometric construction method instead of back-propagation, we create large N -ball embeddings that satisfy two conditions: (1) category trees are precisely imposed onto word embeddings at zero energy cost; (2) pre-trained word embeddings are well preserved. A new benchmark data set is created for validating the category of unknown words. Experiments show that N -ball embeddings, carrying category information, significantly outperform word embeddings in the test of nearest neighborhoods, and demonstrate surprisingly good performance in validating categories of unknown words. Source codes and data-sets are free for public access https://github.com/GnodIsNait/nball4tree.git and https://github.com/GnodIsNait/bp94nball.git.

Attention!!! This submission contains Github and Google Drive links to author-related accounts (see e.g. the abstract). I do not think this is permitted or standard. I leave the decision regarding "automatic rejection" of the submission to meta-reviewers of the paper.
------------------------------------------------
The paper presents a method for tweaking existing vector embeddings of categorical objects (such as words), to convert them to ball embeddings that follow hierarchies. Each category is represented as a Eucldiean norm ball in high dimensional space, with center and radii adaptable to data. Next, inclusion and exclusion constraints on each pair of  balls are imposed based on the hierarchical structure. These constraints are imposed via an algorithmic approach.  The empirical study includes investigating the consistency of the representation with the hierarchy and demonstrating nearest neighbors for a set of words.
On the positive side, the paper addresses an important problem. It is readable and well organized. The related work could be improved by adding a number of representative related works such as [3,4].
The major concern about the paper is the originality of the method. Encoding hierarchies with high dimensional balls and encoding inclusion and exclusion as constraints on those balls is a neat and powerful idea from modeling perspective. However, it is not novel, since the approach is already established for example in [1, 2 Chapter 5].
The next major concern is regarding the evaluation of the quality of embeddings.
The empirical evaluation does not sufficiently evaluate the quality of tweaked embeddings. In contrast, the quantitative evaluation is more concerned with if the embeddings being consistent with the given hierarchy. In particular, not enough quantitative evidence that the proposed embeddings are actually effective in capturing semantics or in prediction tasks is provided. It should be noted that, the first aspect, ie consistency of the feasible solutions with hierarchy, can be theoretically established (see e.g. [1]).  The first paragraph of 3.2 seems unclear or wrong. See for example [2] for a gradient based solution for the problem.
Finally, using an algorithmic approach as opposed to learning method for constructing embeddings, makes the method not directly related to the topic of ICLR conference.
Overall, due to the above reasons, I vote the paper to be rejected. (The poor anonymization makes it a strong case for a reject.)
[1] Mirzazadeh, F., Ravanbakhsh S., Ding N., Schuurmans D.,  "Embedding inference for structured multilabel prediction", NIPS 2015.
[2] Mirzazadeh, F."Solving Association Problems with Convex Co-embedding", PhD thesis, 2017. (Chapter 5)
[3] Vilnis, Luke, and Andrew McCallum. "Word representations via gaussian embedding.", ICLR 2015.
[4] Vendrov, I., Kiros, R., Fidler, S., Urtasun, R. "Order-embeddings of images and language." ICLR 2016.