Min-Max Propagation
We study the application of min-max propagation, a variation of belief propagation, for approximate min-max inference in factor graphs. We show that for "any" highorder function that can be minimized in O(ω), the min-max message update can be obtained using an efficient O(K(ω + log(K)) procedure, where K is the number of variables. We demonstrate how this generic procedure, in combination with efficient updates for a family of high-order constraints, enables the application of min-max propagation to efficiently approximate the NP-hard problem of makespan minimization, which seeks to distribute a set of tasks on machines, such that the worst case load is minimized.

As the title suggests, this paper is interested in
      solving min-max problems using message-passing algorithms. The
      starting point is Eq. 1: You want to optimize a vector x such
      that the max over a discrete set of functions is minimize. The
      set of functions should be finite.
      It's well known that changing the semiring can change the message-passing algorithm from a joint optimization
      (max product) algorithm to a marginalization algorithm (sum
      product). So, just like the (+,*) semiring yields the sum-product
      algorithm and the (max,*) semiring yields the max-product
      algorithm, this paper uses the (min,max) semi-ring. Then,
      Eqs. 2-3 are exactly the standard BP messages on factors and
      single variables, just with the new semi-ring. (It would be good
      if the paper would emphasize more strongly that Eqs. 2 and 3 are
      exactly analogous-- I had to work out the details myself to
      verify this.)
      Since that's a semi-ring you get all the normal properties
      (exactness on trees, running times). However, you don't seem to
      inherit much of the folk theorems around using BP on loopy
      graphs when you change the semi-ring (where most applications
      probably lie) so this paper investigates
      that experimentally.
      The paper assumes that solving the minimization problem on any particular
      factor can be solved in a fixed amount of time. This is
      reasonable to make analysis tractable, but it's worth pointing
      out that in many applications, doing this might itself involve
      running another entire inference algorithm (e.g. max-product
      BP!) There is also an algorithmic specialization to speed up
      message-computation for the case
      where there is a special "choose-one" form for the factors,
      similar to some factor forms that are efficient with different
      semigroups.
      Experimentally, it is compared to a previous algorithm that runs
      a bisection search with a CSP solver in each iteration. This
      appears to perform slightly better, although apparently at
      higher cost.
      As far as I know, this particular semi-ring variant of belief
      propagation has not been investigated before. I don't see strong
      evidence of a huge practical contribution in the expeirments
      (though that might reflect my lack of expertise on these
      problems). Nevertheless, the algorithm is extremely simple in
      concept, and it certainly seems worth noting that there is
      another semi-ring for BP that solves another possibly
      interesting set of problems. Even if the results here aren't
      that strong in the current form, this might be a first step
      towards using some of the many improvements that have been made
      to other semi-ring versions of BP.
      (I was surprised that this particular semi-ring hasn't been
      proposed before, but I'm not aware of such a use, and a brief
      search did not find anything. If this is not actually novel, I
      would downgrade my rating of this paper.)
      - Fig. 5 is very hard to read-- illegible in print
      - In eq. 13 I believe you want a -> m?
EDIT AFTER REBUTTAL:
I've read the rebuttal and other reviews, see no reason to change my recommendation. (I remain positive.)


What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
There are two major types of uncertainty one can model. Aleatoric uncertainty captures noise inherent in the observations. On the other hand, epistemic uncertainty accounts for uncertainty in the model -uncertainty which can be explained away given enough data. Traditionally it has been difficult to model epistemic uncertainty in computer vision, but with new Bayesian deep learning tools this is now possible. We study the benefits of modeling epistemic vs. aleatoric uncertainty in Bayesian deep learning models for vision tasks. For this we present a Bayesian deep learning framework combining input-dependent aleatoric uncertainty together with epistemic uncertainty. We study models under the framework with per-pixel semantic segmentation and depth regression tasks. Further, our explicit uncertainty formulation leads to new loss functions for these tasks, which can be interpreted as learned attenuation. This makes the loss more robust to noisy data, also giving new state-of-the-art results on segmentation and depth regression benchmarks.

I have read the other reviews and the rebuttal.
Additional comment: The authors asked what I mean by 'numerical uncertainty'. I mean what is called 'algorithmic uncertainty' in this Wikipedia article: https://en.wikipedia.org/wiki/Uncertainty_quantification . Deep learning models are very high dimensional optimisation problems and the optimisers does not converge in almost all cases. Hence, this numerical error, i.e. the difference between the true minimum and the output of the optimiser, is a very significant source of error, which we are uncertain about (because we do not know the solution). Additionally, even rounding errors can add to the uncertainty of deep learning, as is e.g. discussed in this paper: http://proceedings.mlr.press/v37/gupta15.pdf .
Hence, I think that the question in the title "What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?" is not really answered. Although the paper is apart from this very good, this limits my rating at 'good paper, accept'. In my opinion, the authors should make clearer which kinds of uncertainty they consider (and which they ignore) or why they believe that the uncertainties considered are the dominant ones (which I find completely unclear).
% Brief Summary
The paper is concerned with evaluating which types of uncertainty are needed for computer vision. The authors restrict their analysis to aleatoric and epistemic uncertainty (leaving out numerical uncertainty). Aleatoric uncertainty includes the uncertainty from statistical noise in data. Epistemic uncertainty is usually another term for ignorance, i.e. things one could in theory know but doesn't know in practice. In this paper however, the authors use epistemic uncertainty as a synonym for model uncertainty (or structural uncertainty, i.e. ignorance over the true physical model which created the data). The paper raises the question how these two sources of uncertainty can be jointly quantified, and when which source of uncertainty is dominant.  In particular, this is discussed in the setting of computer vision with deep learning. While classically it was not possible to capture uncertainty in deep learning, Bayesian neural networks (BNN) have made it possible to capture at least epistemic uncertainty. The authors aim to jointly treat epistemic and aleatoric uncertainty in BNNs.
To expand the existing literature, the authors propose to include aleatoric uncertainty over pixel output sigma by fixing a Laplace likelihood which contains sigma, which is to be minimised with respect to the parameter vector theta. They also observe that this inclusion of aleatoric uncertainty can be interpreted as loss attenuation.
This novel method is then applied to pixel-wise depth regression and semantic segmentation. For semantic segmentation performance, a notable increase in performance is achieved by modelling both aleatoric and epistemic uncertainty.
In Chapter 5, an analysis of the effects of epistemic and aleatoric uncertainty follows. Notably, the authors experimentally validate the well-known claims that aleatoric uncertainty cannot be reduced with more data, and that only epistemic uncertainty increases for out-of-data inputs. The paper concludes by claiming that aleatoric uncertainty is particularly important for large data situations and real-time applications, while epistemic uncertainty is particularly important for safety-critical applications and small datasets.
% Comment
I am skeptical about the use of the classifications of uncertainties which can arise in this setting in this paper. While I agree with the definition of aleatoric uncertainty, I do not agree that epistemic uncertainty is the same as model uncertainty. For me, model uncertainty is the ignorance about the true physical model which creates the data. Epistemic uncertainty is supposed to capture everything which is unknown, but is deterministic and could in theory be known. So, I think that the numerical uncertainty is missing here. In large dimensional optimisation problems (as they arise in deep learning), it is completely unclear whether the optimisation algorithm has converged. I would even expect that the numerical uncertainty could be a dominant source of error, in particular when the computation has too be performed online, e.g. when the self-driving car learns while it's driving. Hence, I would like to ask the authors on their thoughts on this.
The second issue I want to arise is the following: The authors convincingly argue that epistemic uncertainty is reduced when more data is available. In statistics, people talk about the statistical consistency in model. A model is statistically consistent when its output is a Dirac measure on the ground truth, in the limit of infinite data. It would be very interesting to see whether statistical consistency results for deep learning (if they exist) can be connected to the paper's pitch on the relative unimportance of epistemic uncertainty in big data situations.
Since I think (as explained above) that the classifications of uncertainties is missing the numerical uncertainty of the optimiser, I would argue that the paper does not completely live up to comprehensively answer the question asked in the title. While it is fine, that this is not part in the paper; I think it should be discussed in the introduction. Since, the paper is---a part from this---good, I vote for 'accept'.


Variational Inference via χ Upper Bound Minimization
Variational inference (VI) is widely used as an efficient alternative to Markov chain Monte Carlo. It posits a family of approximating distributions q and finds the closest member to the exact posterior p. Closeness is usually measured via a divergence D(q||p) from q to p. While successful, this approach also has problems. Notably, it typically leads to underestimation of the posterior variance. In this paper we propose CHIVI, a black-box variational inference algorithm that minimizes D χ (p||q), the χ-divergence from p to q. CHIVI minimizes an upper bound of the model evidence, which we term the χ upper bound (CUBO). Minimizing the CUBO leads to improved posterior uncertainty, and it can also be used with the classical VI lower bound (ELBO) to provide a sandwich estimate of the model evidence. We study CHIVI on three models: probit regression, Gaussian process classification, and a Cox process model of basketball plays. When compared to expectation propagation and classical VI, CHIVI produces better error rates and more accurate estimates of posterior variance.

Continuing on the recent research trend of proposing alternative divergences to perform variational inference, the author proposed a new variational objective CUBO for VI. Inference is equivalent to minimizing CUBO with respect to the variational distributions. CUBO is a proxy to the Chi-divergence between true posteriors and variational distributions, and is also an upper bound of the model's log-evidence.
The author claimed that the key advantages of CHIVI over standard VI is the EP-like mass covering property of the resulted variational approximations that tend to over-estimate posterior variance instead of under-estimation as with standard VI. Compared to EP, CHIVI optimizes a global objective function that is guaranteed to converge. The author also claimed that CUBO and ELBO together form a sandwich bound of the marginal likelihood that can be used for model selection. The technical content of the paper appears to be correct albeit some small careless mistakes that I believe are typos instead of technical flaw (see #4 below).
The idea of having a sandwich bound for the log-marginal likelihood is certainly good. While the author did demonstrate that the bound does indeed contains the log-marginal likelihood as expected, it is not entirely clear that the sandwich bound will be useful for model selection. This is not demonstrated in the experiment despite being one of the selling point of the paper. It's important to back up this claim using simulated data in experiment.
Another key feature of CHIVI is the over-dispersed approximation that it produces, which the author claimed lead to better estimates of the posterior. I believe a more accurate claim would be 'lead to more conservative estimates of the posterior uncertainty', as the goodness of approximation is fundamentally limited by the richness of the variational distributions, which is orthogonal to the choice of the variational objective function. The author provided some evidence for the claim in the Bayesian probit regression, GPC and the basketball player experiments. However, while it is certainly believable that the 'goodness of posterior approximations' plays a role in the test classification error of the first two experiments, the evidence is a little weak because of the discrete nature of the models' outputs. The author can perhaps make a stronger case by demonstrating the superior posterior uncertainty approximation through Bayesian neural network regression, in which the test LL would be more sensitive to the goodness of approximation, or through active learning experiment in which good posterior uncertainty estimates is crucial. There are examples in both https://arxiv.org/pdf/1502.05336.pdf and https://arxiv.org/pdf/1602.02311.pdf.
The basketball player example provides pretty good visualisations of the posterior uncertainty, but I'm not entirely sure how to interpret the quantitative results in Table 3 confidently. (e.g., Is the difference of 0.006 between CHIVI and BBVI for 'Curry' a big difference? What is the scale of the numbers?)
One aspect that I find lacking in the paper is how computationally demanding CHIVI is compared to standard VI/BBVI? A useful result to present would be side-by-side comparisons of wall clock time required for CHIVI, BBVI and EP in the first two experiments. The variance of the stochastic gradients would certainly play a role in the speed of convergence, and as the author mentioned in the conclusion, the gradients can have high variance. I am quite curious if the reparameterization trick would actually result in lower variance in the stochastic gradients compared to score function gradient when optimizing CUBO?
One important use of the ELBO in standard VI is to learn the model's parameters/hyper-parameters by maximizing ELBO wrt the parameters. This appears to be impossible with CUBO as it's an upper bound of the log-marginal likelihood. The author used grid search to find suitable hyper-parameters for the GPC example. However, this is clearly problematic even for a moderately large number of hyper-parameters. How can this be addressed under the CUBO framework? What is the relationship between the grid search metric (I assume this is cross-validation error?) and the sandwich bound for the parameters on the grid?
While the paper is pretty readable, there is certainly room for improvements in the clarity of the paper. I find paragraphs in section 1 and 2 to be repetitive. It is clear enough from the Introduction that the key advantages of CHIVI are the zero avoiding approximations and the sandwich bound. I don't find it necessary to be stressing that much more in section 2. Other than that, many equations in the paper do not have numbers. The references to the appendices are also wrong (There is no Appendix D or F). There is an extra period in line 188.
The Related Work section is well-written. Good job!
Other concerns/problems:
1. In the paper, a general CUBO_n is proposed where n appears to be a hyper-parameter of the bound. What was the choice of n in the experiments? How was it selected? How sensitive are the results to the choice of n? What property should one expect in the resulted approximation given a specific n? Is there any general strategy that one should adopt to select the 'right' n?
2. In line 60, what exactly is an 'inclusive divergence'?
3. In line 118 and 120, the author used the word 'support' to describe the spread of the distributions. I think a more suitable word here would be 'uncertainty'.
4. The equation in line 125 appears to be wrong. Shouldn't there be a line break before the last equal sign, and shouldn't the last expression be equal to E_q[(\frac{p(z,x)}{q(z)})^2]?
5. Line 155: 'call' instead of 'c all'
6. Line 189: The proposed data sub-sampling scheme is also similar to the scheme first proposed in the Stochastic Variational Inference paper by Hoffman et. al., and should be cited? (http://jmlr.org/papers/volume14/hoffman13a/hoffman13a.pdf)
7. Line 203: Does KLVI really fail? Or does it just produce a worse approximations compared to Laplace/EP? Also , 'typically' instead of 'typical'.
8. Line 28/29: KLVI faces difficulties with light-tailed posteriors when the variational distribution has heavier tails. Does CHIVI not face the same difficulty given the same heavy-tailed variational distributions?


Safe and Nested Subgame Solving for Imperfect-Information Games
In imperfect-information games, the optimal strategy in a subgame may depend on the strategy in other, unreached subgames. Thus a subgame cannot be solved in isolation and must instead consider the strategy for the entire game as a whole, unlike perfect-information games. Nevertheless, it is possible to first approximate a solution for the whole game and then improve it in individual subgames. This is referred to as subgame solving. We introduce subgame-solving techniques that outperform prior methods both in theory and practice. We also show how to adapt them, and past subgame-solving techniques, to respond to opponent actions that are outside the original action abstraction; this significantly outperforms the prior state-of-the-art approach, action translation. Finally, we show that subgame solving can be repeated as the game progresses down the game tree, leading to far lower exploitability. These techniques were a key component of Libratus, the first AI to defeat top humans in heads-up no-limit Texas hold'em poker.

The authors present algorithms and experimental results for solving zero-sum imperfect information extensive-form games, with heads-up no-limit Texas Hold'em poker as the motivating application.
They begin by reviewing the distinction between "unsafe" and "safe" subgame solving.  In unsafe subgame solving, the opponent's strategy outside the subgame, and hence the distribution of the player's state inside the initial information set, is held fixed.  In safe subgame solving, the opponent's strategy in the subgame's head is allowed to vary as well, and hence the distribution of true states is responsive to the player's strategy.
They then introduce "reach" subgame solving, in which a "gift" (the additional value that the opponent could have obtained by choosing a different subgame) is computed for each path that leads to the subgame.  These gifts are added to the margin for each information set.
Finally, they introduce nested subgame solving as a method of dealing with large action spaces: the trunk strategy is computed for an abstracted version of the game, but any time the opponent plays an action that is not in the abstraction, a new subgame is generated and solved on-the-fly that contains the action.
Experimental results suggest that the two methods combined are extremely effective in practice.
Overall I found the exposition well organized and easy to follow, and the performance of the algorithm seems impressive.  I have only minor comments about the presentation.
1. Gifts are motivated as "allowing P2 to be less concerned with P1 reaching the subgame along that path" (L.173-174).  My guess is that the gift can be added to the margin because it's a measure of how unattractive a branch is to the opponent, and the goal of maxmargin solving is to make the subgame as unattractive as possible to the opponent; but a little more concreteness / hand-holding would be helpful here.
2. At L.297-303, the authors note that in spite of having better theoretical guarantees, the no-split variants seem to always outperform the split variants.  They note that this is not necessarily the case if the gifts are scaled up too aggressively; however, that does not seem directly relevant to the point (I don't believe the experimental results include scaling rather than just failure to divide).
   If the full gift is simply awarded to every subgame, with no division but also no scaling, is exploitability ever worse in practice?  If so, can the authors provide an example?  If not, do the authors have an intuition for why that might be?

   At L.210 the authors say that any split is sufficient for the theoretical guarantees, but it seems clear that some splits will leave more on the table than others (as an extreme example, imagine awarding the entire gift to the lowest-probability subgame; this might be close to not using Reach at all).
3. At L.88: "Figure 4a" probably should be "Figure 1".


Deep Hyperspherical Learning
Convolution as inner product has been the founding basis of convolutional neural networks (CNNs) and the key to end-to-end visual representation learning. Benefiting from deeper architectures, recent CNNs have demonstrated increasingly strong representation abilities. Despite such improvement, the increased depth and larger parameter space have also led to challenges in properly training a network. In light of such challenges, we propose hyperspherical convolution (SphereConv), a novel learning framework that gives angular representations on hyperspheres. We introduce SphereNet, deep hyperspherical convolution networks that are distinct from conventional inner product based convolutional networks. In particular, SphereNet adopts SphereConv as its basic convolution operator and is supervised by generalized angular softmax loss -a natural loss formulation under SphereConv. We show that SphereNet can effectively encode discriminative representation and alleviate training difficulty, leading to easier optimization, faster convergence and comparable (even better) classification accuracy over convolutional counterparts. We also provide some theoretical insights for the advantages of learning on hyperspheres. In addition, we introduce the learnable SphereConv, i.e., a natural improvement over prefixed SphereConv, and SphereNorm, i.e., hyperspherical learning as a normalization method. Experiments have verified our conclusions.

This paper presents a novel architecture, SphereNet, which replaces the traditional dot product with geodesic distance as the convolution operators and fully-connected layers. SphereNet also regularizes the weights for softmax to be norm 1 for angular softmax. The results show that SphereNet can achieve superior performance in terms of accuracy and convergence rate as well as mitigating the vanishing/exploding gradients in deep networks.
Novelty: Replacing dot product similarity with angular similarity has widely existed in the deep learning literature. With that being said, most works focus on using angular similarity for Softmax or loss functions. This paper introduces spherical operation for convolution, which is novel in the literature as far as I know.
Significance: The spherical operations the paper introduces achieve faster convergence rate and better accuracy performance. It also mitigates the vanishing/exploding gradients brought by dot products, which is a long standing problem. It will be interesting to lots of people in ML community.
Improvement: I have several suggestions and questions as listed below:
- For angular Softmax, the bias is removed from the equation. However, the existence of bias is very important for calibrating the output, especially for imbalanced datasets. It would be great if the authors could add that back. Otherwise, it's better to try current model on an imbalanced dataset to measure the effect.
- In the first two subfigures of Figure 4, the baseline methods such as standard CNN actually converges faster at the very beginning. But there is a big accuracy drop in the middle. What is the root cause for that? The paper seems no giving any explanation.
- Though the experimental results show faster convergence rate in terms of accuracy vs iteration, it's unknown whether this observation still hold true for accuracy vs time. Traditional dot product operators leverage fast matrix multiplication libraries for speedup, while there is much less support for angular operators.
Minor on grammar:
- Line 216, "this problem will automatically solved" -> "be solved"
- Line 319, "The task can also viewed as a ..." -> "be viewed"


Stochastic Mirror Descent in Variationally Coherent Optimization Problems
In this paper, we examine a class of non-convex stochastic optimization problems which we call variationally coherent, and which properly includes pseudo-/quasiconvex and star-convex optimization problems. To solve such problems, we focus on the widely used stochastic mirror descent (SMD) family of algorithms (which contains stochastic gradient descent as a special case), and we show that the last iterate of SMD converges to the problem's solution set with probability 1. This result contributes to the landscape of non-convex stochastic optimization by clarifying that neither pseudo-/quasi-convexity nor star-convexity is essential for (almost sure) global convergence; rather, variational coherence, a much weaker requirement, suffices. Characterization of convergence rates for the subclass of strongly variationally coherent optimization problems as well as simulation results are also presented.

This paper analyzes the stochastic mirror descent (SMD) algorithm for a specific class of non-convex functions the variational coherence (VC) assumption. The authors study mirror descent by establishing a correspondence between the SMD iterates to an ordinary differential equation. They introduce a new divergence measure named Fenchel coupling and exploit its monotonicity property (due to the VC assumption) to demonstrate the SMD algorithm decreases its objective value until it gets close to the optimum.
The paper is clearly written and - omitting certain details -the proof is relatively easy to follow. The paper however has some significant problems in its presentation:
1) Interest for machine learning community: The main contribution of this paper is a proof of asymptotic convergence of SMD for certain non-convex functions. Although this is an interesting result for the optimization community, I’m however not sure this will be of a broad interest for the ML community which is typically more interested in deriving convergence rates. The authors do not give examples of ML problems where this new analysis could be relevant. There is also no experiment results. Can you please try to address this issue in the rebuttal?
2) Related work: The paper does not cover existing work in optimization for non-convex functions, not does it cover prior work relating ODEs to numerical optimization. I think existing proofs of convergence for stochastic gradient descent and proximal methods should be mentioned, e.g. Ghadimi, Saeed, and Guanghui Lan. "Stochastic first-and zeroth-order methods for nonconvex stochastic programming." SIAM Journal on Optimization 23.4 (2013): 2341-2368.
For ODEs, see references in [18] as well as
Stochastic Approximation and Recursive Algorithms and Applications
Kushner, Harold, Yin, George
I’m sure the authors are familiar with this line of work so please cite it appropriately.
Fenchel coupling
The paper uses the Fenchel coupling to quantify the distance between primal and dual variables thereby serving as an energy function to prove convergence. Can the authors comment on the difficulty of directly using the discrete primal variable to prove convergence? e.g. using a proof similar to SGD for non-convex functions (Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming, Saeed Ghadimi, Guanghui Lan)
Can the authors comment on whether other types of divergences would be appropriate for the analysis? It seems to me that the monotonicity property is due to the variational coherence assumption (as can be seen in E 3.39 in the appendix), can you confirm this? If so, then I wonder if the role of the Fenchel coupling is really that important for the analysis to hold?
Convergence rate: As mentioned earlier, the ML community would probably be more interested in a convergence rate. Can the authors elaborate on the difficulty of establishing a rate using their ODE technique?
Theorem 3.4: This result was at first puzzling to me. It says that the SMD algorithm oscillates in a region near the optimum. This is known to happen for SGD with constant step-size while using a decreasing step size (+ assumptions mentioned in the paper) will then guarantee convergence to an optimum. Can the authors clarify this?
Analysis:
Page 5 appendix above line 92, should be sum_k \alpha_k = + \infty
Line 174: you are citing a book, could you give a more precise reference?
--------------
Post-rebuttal:
I find the answers in the rebuttal convincing and updated my recommendation score accordingly. Overall, I think the paper would benefit from some clarifications which I think is easily do-able between now and the camera ready version. This includes:

1) Importance of the Fenchel coupling. The use of the Fenchel coupling (instead of the commonly used Bregman divergence) is a nice trick to get convergence for the type of non-convex functions considered in the paper. Also the ODE technique used in this paper could potentially push the development of similar techniques for different optimization techniques so this could really be valuable for the ML community. The submitted version of the paper did not explain this well but the answer provided in the rebuttal is convincing. I would recommend developing this aspect even more in the revised version.
2) Interest regarding the ML community, they should develop this more in the introduction. Adding experimental results as they promised to R3 would be valuable as well
3) As R2 pointed out, the intuition behind the analysis is not always clear. Given the rather convincing answers in the rebuttal, I think the authors can easily improve this aspect in the revised version.
4) Minor modifications: Fix the numbering and clarify the use of certain terms whose meaning might differ for different communities, e.g. linear decrease, last iterate.


Minimal Exploration in Structured Stochastic Bandits
This paper introduces and addresses a wide class of stochastic bandit problems where the function mapping the arm to the corresponding reward exhibits some known structural properties. Most existing structures (e.g. linear, Lipschitz, unimodal, combinatorial, dueling, . . . ) are covered by our framework. We derive an asymptotic instance-specific regret lower bound for these problems, and develop OSSB, an algorithm whose regret matches this fundamental limit. OSSB is not based on the classical principle of "optimism in the face of uncertainty" or on Thompson sampling, and rather aims at matching the minimal exploration rates of sub-optimal arms as characterized in the derivation of the regret lower bound. We illustrate the efficiency of OSSB using numerical experiments in the case of the linear bandit problem and show that OSSB outperforms existing algorithms, including Thompson sampling.

Problem Studied: The paper introduces and studies a wide range of stochastic bandit problem with known structural properties. This includes linear, lipschitz, unimodal, combinatorial, dueling bandits.
Results: The paper derives an instance specific regret lower bound based on the structure and gives an algorithm which matches the lower bound. The algorithm needs to solve a semi-infinite LP in the general case, but they show that it can be efficiently done for special cases (which follows from previous literature).
Comments: The paper introduces a nice general framework for several of existing known results for several bandit problems such as linear, lipschitz, unimodal, combinatorial, dueling bandits. This encapsulates all these in very broad theoretical framework which can be useful for any new bandit problems.


Learning to See Physics via Visual De-animation
We introduce a paradigm for understanding physical scenes without human annotations. At the core of our system is a physical world representation that is first recovered by a perception module and then utilized by physics and graphics engines. During training, the perception module and the generative models learn by visual de-animation -interpreting and reconstructing the visual information stream. During testing, the system first recovers the physical world state, and then uses the generative models for reasoning and future prediction. Even more so than forward simulation, inverting a physics or graphics engine is a computationally hard problem; we overcome this challenge by using a convolutional inversion network. Our system quickly recognizes the physical world state from appearance and motion cues, and has the flexibility to incorporate both differentiable and non-differentiable physics and graphics engines. We evaluate our system on both synthetic and real datasets involving multiple physical scenes, and demonstrate that our system performs well on both physical state estimation and reasoning problems. We further show that the knowledge learned on the synthetic dataset generalizes to constrained real images.

This paper presents an approach to physical scene understanding which utilizes the combination of physics engines and renderers.  The approach trains a network to predict a scene representation based on a set of input frames and subject to the constraints of the physics engine.  This is trained using either SGD or reinforcement learning depending on whether a differentiable physics engine is used.  The approach is applied to several simple scenarios with both real and synthetic data.
Overall the idea in the paper is interesting and promising, however, the paper as it's written has several issues.  Some of these may be able to be addressed in a rebuttal.
First, there are some significant missing references, specifically related to using physical models in human tracking.  Kyriazis and Argyros (CVPR2013, CVPR 2014); Pham et al (PAMI 2016); Brubaker et al (ICCV 2009, IJCV 2010); Vondrak et al (SIGGRAPH 2012, PAMI 2013).  In terms of simple billiards-like scenarios, see also Salzmann and Urtasun (ICCV 2011).  This is a significant blind spot in the related works and should be addressed.
Second, the paper is missing some important technical details.  In particular, it is never specified how the system actually works at test time.  Is it simply run on successive sets of three input frames independently using a forward pass of the perception model?  Does this mean that the physics and graphics rendering engines are not used at all at test time?  If this is the case, why not use the physics engine at test time?  It could, for instance, be used as a smoothing prior on the per-frame predictions of the perception model?  This issue is particularly relevant for prediction of stability in the block world scenario.  Is the network only run on a single set of three frames?  This makes it particularly difficult to assess the technical aspects of this work.
Other comments:
 - Absolute mass is never predictable without knowing something about what an object is made of.  Only relative mass plays a role in the physics and visual perception can only tell you volume (assuming calibration), not density.  Any network which appears to "learn" to predict absolute mass is really just learning biases about mass/density from the training set.  This is fine, but important to recognize so that overboard claims aren't made.
 - The table entries and captions are inconsistent in Fig 7(b-c).  The table used "full" and "full+", whereas the caption references "joint" and "full".


Unified representation of tractography and diffusion-weighted MRI data using sparse multidimensional arrays
Recently, linear formulations and convex optimization methods have been proposed to predict diffusion-weighted Magnetic Resonance Imaging (dMRI) data given estimates of brain connections generated using tractography algorithms. The size of the linear models comprising such methods grows with both dMRI data and connectome resolution, and can become very large when applied to modern data. In this paper, we introduce a method to encode dMRI signals and large connectomes, i.e., those that range from hundreds of thousands to millions of fascicles (bundles of neuronal axons), by using a sparse tensor decomposition. We show that this tensor decomposition accurately approximates the Linear Fascicle Evaluation (LiFE) model, one of the recently developed linear models. We provide a theoretical analysis of the accuracy of the sparse decomposed model, LiFE SD , and demonstrate that it can reduce the size of the model significantly. Also, we develop algorithms to implement the optimization solver using the tensor representation in an efficient way. *

The paper propose an efficient approximation of the "Linear Fascicle Evaluation model" (LiFE), which is used for tractography. The key issue with the LiFE model is that it computationally scales poorly such that it is impractical to analyze MRI data at full resolution. The authors propose a sparse approximation based on a Tucker model (tensor model). The authors prove that the approximation is both good and achieves a high compression; this is further evaluated experimentally. The model appears solid and likewise for the analysis. My key objection to the paper is that the actual evaluation of the model for tractography is treated very superficially (just a couple of lines of text). I understand that within the tight boundaries of a NIPS paper, the authors does not have much space for such an evaluation, so I'm willing to forgive this limitation of the paper.
Other than that, the paper is very well written with intuitive figures guiding the reader along the way. As a non-expert, I found the paper a please to read. I do stress that I'm not an expert in neither tractography or tensor models.
A few minor comments (not important in the rebuttal):
*) In line 36, the authors point out that tractography is poorly understood and that competing methods can give wildly different results. The authors then argue that this motivates convex models. I don't buy this argument -- the reason different models give different results is more likely due to different modeling assumptions, rather than whether a global optimum has been found or not.
*) After the propositions: the authors take the time to explain in plain words the key aspects of their propositions. Most authors neglect this, and I just want to say that such efforts make the paper much easier to read. So: thank you!
*) In line 195, you write: "...dataset Fig. 4.a." There's something wrong with that sentence. Perhaps the figure reference should be in a parenthesis?
*) In line 202: typo: "aproach" --> "approach".
== Post rebuttal ==
I have read the other reviews and the rebuttal, and stand by my evaluation that this paper should be accepted.


Bridging the Gap Between Value and Policy Based Reinforcement Learning
We establish a new connection between value and policy based reinforcement learning (RL) based on a relationship between softmax temporal value consistency and policy optimality under entropy regularization. Specifically, we show that softmax consistent action values correspond to optimal entropy regularized policy probabilities along any action sequence, regardless of provenance. From this observation, we develop a new RL algorithm, Path Consistency Learning (PCL), that minimizes a notion of soft consistency error along multi-step action sequences extracted from both on-and off-policy traces. We examine the behavior of PCL in different scenarios and show that PCL can be interpreted as generalizing both actor-critic and Q-learning algorithms. We subsequently deepen the relationship by showing how a single model can be used to represent both a policy and the corresponding softmax state values, eliminating the need for a separate critic. The experimental evaluation demonstrates that PCL significantly outperforms strong actor-critic and Q-learning baselines across several benchmarks.
By contrast, value based methods, such as Q-learning [44,22,30,42,21], can learn from any trajectory sampled from the same environment. Such "off-policy" methods are able to exploit data from other sources, such as experts, making them inherently more sample efficient than on-policy methods [10]. Their key drawback is that off-policy learning does not stably interact with function approximation [35,Chap.11]. The practical consequence is that extensive hyperparameter tuning can be required to obtain stable behavior. Despite practical success [22], there is also little theoretical understanding of how deep Q-learning might obtain near-optimal objective values.
Ideally, one would like to combine the unbiasedness and stability of on-policy training with the data efficiency of off-policy approaches. This desire has motivated substantial recent work on off-policy actor-critic methods, where the data efficiency of policy gradient is improved by training an offpolicy critic [19,21,10]. Although such methods have demonstrated improvements over on-policy actor-critic approaches, they have not resolved the theoretical difficulty associated with off-policy learning under function approximation. Hence, current methods remain potentially unstable and require specialized algorithmic and theoretical development as well as delicate tuning to be effective in practice [10,41,8].
In this paper, we exploit a relationship between policy optimization under entropy regularization and softmax value consistency to obtain a new form of stable off-policy learning. Even though entropy regularized policy optimization is a well studied topic in RL [46,39,40,47,5,4,6,7]-in fact, one that has been attracting renewed interest from concurrent work [25,11]-we contribute new observations to this study that are essential for the methods we propose: first, we identify a strong form of path consistency that relates optimal policy probabilities under entropy regularization to softmax consistent state values for any action sequence; second, we use this result to formulate a novel optimization objective that allows for a stable form of off-policy actor-critic learning; finally, we observe that under this objective the actor and critic can be unified in a single model that coherently fulfills both roles.

SUMMARY:
The paper considers entropy regularized discounted Markov Decision Process (MDP), and shows the relation between the optimal value, action-value, and policy. Moreover, it shows that the optimal value function and policy satisfy a temporal consistency in the form of Bellman-like equation (Theorem 1), which can also be extended to its n-step version (Corollary 2).
The paper introduces Path Consistent Learning by enforcing the temporal consistency, which is essentially a Bellman residual minimization procedure (Section 5).
SUMMARY OF EVALUATION:
Quality: Parts of the paper is sound (Section 3 and 4); parts of it is not (Section 5)
Clarity: The paper is well-written.
Originality: Some results seem to be novel, but similar ideas and analysis have been proposed/done before.
Significance: It might become an important paper in the entropy regularized RL literature.
EVALUATION:
This is a well-written paper that has some interesting results regarding the relation between V, Q, and pi for the entropy regularized MDP setting. That being said, I have some concerns, which I hope the authors clarify.
- Enforcing temporal consistency by minimizing its l2 error is minimizing the empirical Bellman error. Why do we need a new terminology (Path consistency) for this?
- The empirical Bellman error, either in the standard form or in the new form in this paper, is a biased estimate of the true Bellman error. Refer to the discussion in Section 3 of
Antos, Szepesvari, Munos, “Learning near-optimal policies with Bellman-residual minimization based fitted policy iteration and a single sample path,” Machine Learning Journal, 2008.
Therefore, for stochastic dynamics, minimizing Equation (15) does not lead to the desired result. This is my most crucial comment, and I hope the authors provide an answer to it.
- Claiming that this paper is bridging the gap between the value and policy based RL is beyond what has actually been shown. The paper shows the relation between value and policy for a certain type of reward function (which has the entropy of policy as an additional reward term).
- On LL176-177, it is claimed that a new actor-critic paradigm is presented where the policy is not distinct from the values. Isn’t this exactly the same thing in the conventional value-based approach for which the optimal policy has a certain relationship with the optimal value function? In the standard setting, the greedy policy is optimal; here we have relationship of Equation (10).
- It might be noted that relations similar to Equations (7), (8), (9), and (10) have been observed in the context of Maximum Entropy Inverse Optimal Control. See Corollary 2 and the discussion after that in the following paper:
Ziebart, et al., “The Principle of Maximum Causal Entropy for Estimating Interacting Processes,” IEEE Transactions on Information Theory, 2013.
Here the Bellman equations are very similar to what we have in this paper.
Ziebart et al.’s framework is extended by the following paper to address problems with large state spaces:
Huang, et al., “Approximate MaxEnt Inverse Optimal Control and its Application for Mental Simulation of Human Interactions,” AAAI, 2016.
This latter paper concerns with the inverse optimal control (i.e., RL), but the approximate value iteration algorithms used for solving the IOC essentially use the same Bellman equations as we have here. The paper might want to compare the difference between these two approaches.
- One class of methods that can bridge the gap between value and policy based RL algorithms is classification-based RL algorithms. The paper ignores these approaches in its discussion.


Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
The recently proposed Temporal Ensembling has achieved state-of-the-art results in several semi-supervised learning benchmarks. It maintains an exponential moving average of label predictions on each training example, and penalizes predictions that are inconsistent with this target. However, because the targets change only once per epoch, Temporal Ensembling becomes unwieldy when learning large datasets. To overcome this problem, we propose Mean Teacher, a method that averages model weights instead of label predictions. As an additional benefit, Mean Teacher improves test accuracy and enables training with fewer labels than Temporal Ensembling. Without changing the network architecture, Mean Teacher achieves an error rate of 4.35% on SVHN with 250 labels, outperforming Temporal Ensembling trained with 1000 labels. We also show that a good network architecture is crucial to performance. Combining Mean Teacher and Residual Networks, we improve the state of the art on CIFAR-10 with 4000 labels from 10.55% to 6.28%, and on ImageNet 2012 with 10% of the labels from 35.24% to 9.11%.

Summary-
This paper shows that an exponential moving average (in parameter space) of
models produced during SGD training of neural networks can be used as a good
teacher model for the student model which corresponds to the last (current) SGD
iterate. The teacher model is used to provide a target (softmax probabilities)
which is regressed to by the student model on both labelled and unlabelled
data. This additional loss serves as a regularizer. In other words, the model
is being trained to be consistent with its Polyak-averaged version.
Furthermore, noise is injected into both the student and teacher models to
increase the regularization effect (similar motivation as dropout and other
related methods). This is shown to be more effective than (1) Just using noise
(and no moving average), and (2) using a moving average over softmax
probabilities (not parameters) which is updated once every epoch.
Strengths-
- The proposed teacher model is a convenient one to maintain. It just requires keeping a moving average.
- The additional computational cost is just that of a forward prop and benefits seem to be worth it (at least for small amounts of labelled data).
- The supplementary material provides enough experimental details.
Weakness-
- Comparison to other semi-supervised approaches : Other approaches such as variants of Ladder networks would be relevant models to compare to.
Questions/Comments-
- In Table 3, what is the difference between \Pi and \Pi (ours) ?
- In Table 3, is EMA-weighting used for other baseline models ("Supervised", \Pi, etc) ? To ensure a fair comparison, it would be good to know that all the models being compared to make use of the EMA benefits.
- The proposed model benefits from two factors : noise and keeping an exponential moving average. It would be good to see how much each factor contributes on its own. The \Pi model captures just the noise part, so it would be useful to know how much gain can be obtained by just using a noise-free exponential moving average.
- If averaging in parameter space is being used, it seems that it should be possible to apply the consistency cost in the intermediate layers of the model as well. That could potentially provide a richer consistency gradient. Was this tried ?
Minor comments and typos-
- In the abstract : "The recently proposed Temporal Ensembling has  ...  ": Please cite.
- "when learning large datasets." -> "when learning on large datasets."
- "zero-dimensional data points of the input space": It may not be accurate to say that the data points are zero-dimensional.
- "barely applying",  " barely replicating" : "barely" -> "merely"
- "softmax output of a model does not provide a good Bayesian approximation outside training data". Bayesian approximation to what ? Please explain. Any model will have some more generalization error outside training data. Is there another source of error being referred to here ?
Overall-
The paper proposes a simple and effective way of using unlabelled data and
improving generalization with labelled data. The most attractive property is
probably the low overhead of using this in practice, so it is quite likely that
this approach could be impactful and widely used.


Coded Distributed Computing for Inverse Problems
Computationally intensive distributed and parallel computing is often bottlenecked by a small set of slow workers known as stragglers. In this paper, we utilize the emerging idea of "coded computation" to design a novel error-correcting-code inspired technique for solving linear inverse problems under specific iterative methods in a parallelized implementation affected by stragglers. Example machinelearning applications include inverse problems such as personalized PageRank and sampling on graphs. We provably show that our coded-computation technique can reduce the mean-squared error under a computational deadline constraint. In fact, the ratio of mean-squared error of replication-based and coded techniques diverges to infinity as the deadline increases. Our experiments for personalized PageRank performed on real systems and real social networks show that this ratio can be as large as 10 4 . Further, unlike coded-computation techniques proposed thus far, our strategy combines outputs of all workers, including the stragglers, to produce more accurate estimates at the computational deadline. This also ensures that the accuracy degrades "gracefully" in the event that the number of stragglers is large.

The paper considers a distributed computing scenario where a bunch of processors have different delays in computing. A job (here finding a solution of a system of linear equations) is distributed among the processors. After a certain deadline the output of the processors are combined and the result is produced.
Previous literature mainly neglect the slower processors and work with the best k out of n processors' outputs. Usually an MDS code is used to distribute the task among the processors. I understand the main contribution is to include the stragglers computations and not treat them as mere erasures.
I have few critical comments regarding the work.
1. Correctness of results:  I think there are serious problems in the assumptions (see line 116-117). If x_i s are all iid, it is not necessary that r_i s are all iid as well. This only happens when M is square invertible matrix. For example, if a row of M is zero then the corresponding entry of r_i is going to be always zero. The main results all depend on this assumption, which makes me doubtful about the veracity of results.
2. Assumptions without justification: From the introduction it seems like the main result is contained in Theorem 4.4. This theorem is true under two very specific assumptions.  One, the solutions of a systems of linear equations are all iid variables, and the number of iterations of each processor within a deadline are all iid random variables. These two seem like very stringent condition and I see no reason for these to be true in real systems. The authors also do not provide any motivation for these assumptions to be true either. Given this, I am unable to judge whether the theoretical results are of any value here.
Another key condition for the results to be true is n-k = o(\sqrt(n). Again it was not explained why this should be the case even asymptotically.
Also, I do not understand the implication of the random variable C(l) satisfying the ratio of variance and mean-square to be large. Why would this assumption make any sense?
Overall lack of motivations for these assumptions seem concerning.
3. Unfair comparison:  The replication scheme, that the paper compare their coding scheme with, has several problems. And I do not think that it is a good comparison. The assumption is that n -k is very small, i.e., o(\sqrt{n}). So computation of only o(\sqrt{n}) elements are computed twice. The rest of the processors are uncoded. This is a very specific coding scheme, and it is obvious that the Fourier matrix based scheme (which is an MDS code) will beat this scheme.
In the introduction it was claimed that: “To compare with the existing coding technique in [6], ..” But the theoretical comparison to the method of [6] is missing.
4. Novelty: I do not think the presented scheme is new. It is the same MDS code based scheme that was proposed by Lee et al (in ref [6]).
Overall the same encoding method has been used in both [6] and in this paper. The only difference is that the results of the stragglers are also used in this paper while decoding - so it is expected to get a better error rate. But in general, the claim of introduction, the prevention of ill-conditioning of MDS matrices in the erasure channel, remains unmitigated. It is just that a particular problem is considered in the paper.
5. Experiments: 9. The figures are not properly labeled. In the third plot of fig 2, ref [12] is mentioned, which should be [6] (I think). This is quite confusing.
I am further skeptical about this plot. At certain value of the T_deadline, the method of [6], should result in absolute 0 error. But this is not observed in the plot. It is also not clear why in the extension of the coded method the mean square error would increase.
6. More minor comments:
 The authors say “iterative methods for linear inverse problem” as if it is a very standard problem: however it is not to me. A more understandable term would be `finding solutions of a linear system'. So for quite long time I could not figure out what the exact problem they are trying to solve. This create difficulty reading the entirety of the introduction/abstract.
\rho has been used as spectral radius, and also in the heavy tail distributions.
=============
After the rebuttal:
I am able to clarify my confusion about independence of the random variables. So I do not have about the concern with correctness anymore. The point that I am still worried about is the unfair comparison with repetition codes! I do not think n-k =o(n) is the correct regime to use repetition code.
It seems to me from the rebuttal is, if T_deadline is large enough, then coding does not help (which makes sense). Is it clear what is the crossover point of T_deadline?
I am satisfied with the rebuttal in other points.


Testing and Learning on Distributions with Symmetric Noise Invariance
Kernel embeddings of distributions and the Maximum Mean Discrepancy (MMD), the resulting distance between distributions, are useful tools for fully nonparametric two-sample testing and learning on distributions. However, it is rare that all possible differences between samples are of interest -discovered differences can be due to different types of measurement noise, data collection artefacts or other irrelevant sources of variability. We propose distances between distributions which encode invariance to additive symmetric noise, aimed at testing whether the assumed true underlying processes differ. Moreover, we construct invariant features of distributions, leading to learning algorithms robust to the impairment of the input distributions with symmetric additive noise.

Overview: In this paper the authors present a method for making nonparametric methods invariant to noise from positive definite distributions by utilizing estimated characteristic functions while disregarding magnitude and only considering the phase. The authors apply the technique to two sample testing and nonparametric regression on measures, ie each "sample" is a collection of samples. The authors demonstrate that the techniques introduced behave as expected by performing tests on synthetic data and demonstrate reasonable performance of the new two sample tests and new feature on real and semi-synthetic datasets.
Theory: The fundamental idea behind this paper (disregarding the magnitude of a characteristic function) is not terribly revolutionary, but it is elegant, interesting, and likely to be useful so I think worthy of publication. Though the underlying concept is somewhat technical, the authors manage to keep the exposition clear. The authors explain the concept using population version of estimators and give what seem to be very reasonable finite sample estimators, but it would be nice to see some sort of consistency type result to round things off.
Experiments: The synthetic experiment clearly demonstrates that the new features behave as one might expect in for a two sample test, but it is disappointing that the technique seems to greatly reduce the test power compared to current nonparametric techniques. This difference is pretty striking in the >=2000 sample regime. While the technique is not effective for the Higgs Boson two sample test its ineffectiveness is in itself very intriguing, the difference between the two classes seems to be some sort of psd noise phenomena. For the last two experiments, the proposed technique works well on the semi-synthetic dataset (real data with synthetic noise added), but not particularly better, on the final real world data experiment. Ultimately the proposed technique doesn't demonstrate decidedly superior testing or regression performance on any real world dataset, although it does slightly outperform in certain sample regimes.
Exposition: The paper is a pleasure to read but there are some errors and problems which I will list at the end of this review.
Final Eval.: Overall I think the idea is interesting enough to be worthy of publication.
--Corrections By Line--
3: "rarely" should be "rare"
14: "recently" should be "which have recently been" to sound better
53: To be totally mathematically precise perhaps you should state the E is a psd rv again
115: add "the" after "However"
116: I'm assuming that "characteristic function" was meant to be included after "supported everywhere"
119: Should probably mention a bit more or justify why these "cases appear contrived"
123: The notation P_X and P_Y hasn't been introduced, but perhaps it is obvious enough
123: Here K is a function on a pair of probability measures on line 88 K is a function of two collections of data
134: We need some justification that this is actually an unbiased estimator. A lot of the obvious estimators in the MMD and related techniques literature are not unbiased
208: \chi^2(4)/4 is not technically correct, its the sample that is divided by 4 not the measure
268: I'm not familiar with this dataset. Please mention the space which the labels lie in. I'm assuming it is R.
Update: I've bumped it up a point after reading the other reviews and rebuttal. Perhaps I was too harsh to begin with.


On Optimal Generalizability in Parametric Learning
We consider the parametric learning problem, where the objective of the learner is determined by a parametric loss function. Employing empirical risk minimization with possibly regularization, the inferred parameter vector will be biased toward the training samples. Such bias is measured by the cross validation procedure in practice where the data set is partitioned into a training set used for training and a validation set, which is not used in training and is left to measure the outof-sample performance. A classical cross validation strategy is the leave-one-out cross validation (LOOCV) where one sample is left out for validation and training is done on the rest of the samples that are presented to the learner, and this process is repeated on all of the samples. LOOCV is rarely used in practice due to the high computational complexity. In this paper, we first develop a computationally efficient approximate LOOCV (ALOOCV) and provide theoretical guarantees for its performance. Then we use ALOOCV to provide an optimization algorithm for finding the regularizer in the empirical risk minimization framework. In our numerical experiments, we illustrate the accuracy and efficiency of ALOOCV as well as our proposed framework for the optimization of the regularizer.

This paper proposes an efficiently computable approximation of leave-one-out cross validation for parametric learning problems, as well as an algorithm for jointly learning the regularization parameters and model parameters. These techniques seem novel and widely applicable.
The paper starts out clearly written, though maybe some less space could have been spent on laying the groundwork, leaving more room for the later sections where the notation is quite dense.
Can you say anything about the comparsion between ALOOCV and LOOCV evaluated on only a subset of the data points (as you mention in l137-140), both in terms of computation cost and approximation accuracy?
Other comments:
l75: are you referring to PRESS? Please name it then.
l90: "no assumptions on the distribution" -- does that mean, no prior distribution?
Definition 7: the displayed equation seems to belong to the "such that" in the preceding sentence; please pull it into the same sentence. Also, I find it odd that an analytic function doesn't satisfy this definition (due to the "there exists one and only one"). What about a two-dimensional function that has non-differentiabilities in its uppper-right quadrant, so that along some cross sections, it is analytic?
l186-187: (relating to the remarks about Def7) This sounds a bit odd; it might be better to say something like "We remark that the theory could be extended to ...".
l250: Are you saying that you are not jointly learning the regularization parameter in this second example? If the material in section 4 doesn't apply here, I missed why; please clarify in that section.
Typographic:
l200: few -> a few
References: ensure capitalization of e.g. Bayes by using {}


Adversarial Ranking for Language Generation
Generative adversarial networks (GANs) have great successes on synthesizing data. However, the existing GANs restrict the discriminator to be a binary classifier, and thus limit their learning capacity for tasks that need to synthesize output with rich structures such as natural language descriptions. In this paper, we propose a novel generative adversarial network, RankGAN, for generating high-quality language descriptions. Rather than training the discriminator to learn and assign absolute binary predicate for individual data sample, the proposed RankGAN is able to analyze and rank a collection of human-written and machine-written sentences by giving a reference group. By viewing a set of data samples collectively and evaluating their quality through relative ranking scores, the discriminator is able to make better assessment which in turn helps to learn a better generator. The proposed RankGAN is optimized through the policy gradient technique. Experimental results on multiple public datasets clearly demonstrate the effectiveness of the proposed approach.

The paper proposes an alternative parameterization of the GAN discriminator based on relative ranks among a comparison set. The method shows better quantitative results for language generation on three (relatively small) language datasets.
Mathematically, there are some potential problems of the proposed method.
- Firstly, the original GAN identified the equilibrium of Eqn. (1) only for R(s|U,C) being a proper probability over a Bernoulli variable. However, the definition in Eqn. (4) does not guarantee this condition. Thus, it is difficult to see whether the proposed criterion can ensure the generator distribution to match the true data distribution.
- However, if one regards the expected log probability in Eqn. (4) as a definition of energy, the practically used criterion Eqn. (7) is more similar to the energy-based GAN. From this perspective, if one completes the two expectations on U and C omitted in Eqn. (7), (i.e., the full formula for the a term should be E_{s ~ P_h} [ E_{U ~ bar{U}} E_{C ~ bar{C}} log R(s|U,C) ] where bar{U} and bar{C} represent all possible reference and comparison sets respectively), and plugs Eqn. (4) into the full formula, it becomes E_{s ~ P_h} [ E_{U ~ bar{U}} E_{C ~ bar{C}} ( E_{u ~ U} log P(s|u,C) ) ] <= E_{s ~ P_h} [ log P(s) ], which is a lower bound of the ranker-defined likelihood. In other words, the energy function is defined as a lower bound of the likelihood, which is a little bit strange and not conventional.
- Putting things together, I don’t really understand what the proposed criterion is doing, and why it leads to better empirical performances.
In terms of the experiment result, despite the good quantitative results, it would be nice if some qualitative analysis were conducted to understand why and how the generated sentences could achieve higher scores. Also, is the result sensitive to the size of the comparison set |C| or the size of the reference set |U|?
In summary, the proposed method achieves nice empirical results in improving language generation under the GAN setting. However, given the content of the paper, it is still not clear why this method should lead to such improvement.


Universal Style Transfer via Feature Transforms
Universal style transfer aims to transfer arbitrary visual styles to content images. Existing feed-forward based methods, while enjoying the inference efficiency, are mainly limited by inability of generalizing to unseen styles or compromised visual quality. In this paper, we present a simple yet effective method that tackles these limitations without training on any pre-defined styles. The key ingredient of our method is a pair of feature transforms, whitening and coloring, that are embedded to an image reconstruction network. The whitening and coloring transforms reflect a direct matching of feature covariance of the content image to a given style image, which shares similar spirits with the optimization of Gram matrix based cost in neural style transfer. We demonstrate the effectiveness of our algorithm by generating high-quality stylized images with comparisons to a number of recent methods. We also analyze our method by visualizing the whitened features and synthesizing textures via simple feature coloring.

This submission describes a model for general and efficient style transfer. It is general in the sense that it does not need training for each style image that one would want to apply (unlike [26]), and it is efficient because it does not require solving an optimization problem as the initial work of Gatsby et al. [9].
The main idea is to train a deep decoder network, so that from any given layer in a VGG, one can generate the most likely image. This decoder is not trained as a classical autoencoder. Instead, given a fixed supervised VGG, an decoder counterpart of a VGG is trained so as to reconstruct the image.
Given this trained decoder, the authors propose to transfer style by taking a given layer, make the covariance of the activations the identity and in a second step, make them follow the covariance of the style image at the same layer (called coloring). Given these new activations the desired style-transfered image is obtained by applying the corresponding decoder.
Pros:
- The proposed approach is very simple and intuitive. It does not require fancy losses, and uses simple baseline techniques to approximately match the distributions of activations.
- This paper exhibits the very large expressive power of CNNs. Given a well trained decoder, simple linear operations in the feature space give meaningful outputs.
- The proposed Coarse to Fine approach leads to very good visual performance.
- The proposed algorithm is fairly efficient as for each new stye, it only requires computing several singular value decompositions (two for each layer, one for the image, one for the style).
- The proposed model only has one parameter, namely the tradeoff between style and content. This is a great advantage to say [26].
- Overall the paper reads well and is easy to follow.
Cons:
- The description of the Decoder training could be made more clear by introducing the fixed encoder, let's say \phi. Then the training of the decoder could be written as the minimization w.r.t. a decoder \psi of the loss. That would avoid using strange notations including I_\input I_\output (which actually depends on the function that is optimized over...).
- The formal description of the whitening and coloring could be made more practical by introducing the SVD of the activations (instead of using the eigenvalue decomposition of the covariance matrix).
Questions:
- To a large extent, this approach relies on the training of a good decoder, they quality of which strongly depends on the choice of data used for training. It would be interesting to see more results with worse decoders, trained on less data (or images with different statistics).
- The model assumes that the "support" of the decoder (subset of the input space that was covered by the inputs) contains the activations of the style images. Provided that this is true and that the decoder is well regularized, the output should be correct. Would it be possible to assess to what degree does a dataset such as COCO cover well this space? Otherwise, the decoder might have ill-defined behaviors for parts of the input space that it has never seen during training.
Overall, I think this submission in interesting and shows good results with a simple approach. I lean towards acceptance.


GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Generative Adversarial Networks (GANs) excel at creating realistic images with complex models for which maximum likelihood is infeasible. However, the convergence of GAN training has still not been proved. We propose a two time-scale update rule (TTUR) for training GANs with stochastic gradient descent on arbitrary GAN loss functions. TTUR has an individual learning rate for both the discriminator and the generator. Using the theory of stochastic approximation, we prove that the TTUR converges under mild assumptions to a stationary local Nash equilibrium. The convergence carries over to the popular Adam optimization, for which we prove that it follows the dynamics of a heavy ball with friction and thus prefers flat minima in the objective landscape. For the evaluation of the performance of GANs at image generation, we introduce the 'Fréchet Inception Distance" (FID) which captures the similarity of generated images to real ones better than the Inception Score. In experiments, TTUR improves learning for DCGANs and Improved Wasserstein GANs (WGAN-GP) outperforming conventional GAN training on CelebA, CIFAR-10, SVHN, LSUN Bedrooms, and the One Billion Word Benchmark.

Generative adversarial networks (GANs) are turning out to be a very important advance in machine learning. Algorithms for training GANs have difficulties with convergence. The paper proposes a two time-scale update rule (TTUR) which is shown (proven) to converge under certain assumptions. Specifically, it shows that GAN Adam updates with TTUR can be expressed as ordinary differential equations, and therefore can be proved to converge using a similar approach as in Borkar' 1997 work. The recommendation is to use two different update rules for generator and discriminator, with the latter being faster, in order to have convergence guarantees.
***Concerns:

1-Assumptions A1 to A6 and the following Theorem 1 are symmetric wrt h and g, thus one can assume it makes no difference which one of the networks (discriminator or generator) has the faster updates. A discussion on why this symmetry does not hold in practice, as pointed out by the authors in introduction, is necessary.

2-The assumptions used in the proof should be investigated in each of the provided experiments to make sure the proof is in fact useful, for example one of the key assumptions that a = o(b) is clearly not true in experiments (they use a = O(a) instead).

3-The less variation pointed out in Figure 2 are probably due to lower learning rate of the generator in TTUR compared to SGD, however since the learning rate values are not reported for Figure 2, we can not be sure.

4-In Figure 6, variance and/or confidence intervals should be reported.

5-The reported learning rate for TTUR appears to be only linearly faster for the discriminator (3 times faster in Figure 6, 1.25 times faster in Figure 3). This is not consistent with the assumption A2. Moreover, it is well known that a much faster discriminator for regular GANs undermines the generator training. However, according to Theorem 1, the system should converge. This point needs to be addressed.

6-There is insufficient previous/related work discussion, specifically about the extensive use of two time updates in dynamic systems, as well as the research on GAN convergence.

7-In general, theorems, specifically Theorem 2, are not well explained and elaborated on. The exposited ideas can become more clear and the paper more readable with a line or two of explanation in each case.

In summary, while the proofs for convergence are noteworthy, a more detailed investigation and explanation of the effect of assumptions in each of the experiments, and how deviating from them can hurt the convergence in practice, will help clarify the limitations of the proof.


Deep Subspace Clustering Networks
We present a novel deep neural network architecture for unsupervised subspace clustering. This architecture is built upon deep auto-encoders, which non-linearly map the input data into a latent space. Our key idea is to introduce a novel self-expressive layer between the encoder and the decoder to mimic the "selfexpressiveness" property that has proven effective in traditional subspace clustering. Being differentiable, our new self-expressive layer provides a simple but effective way to learn pairwise affinities between all data points through a standard backpropagation procedure. Being nonlinear, our neural-network based method is able to cluster data points having complex (often nonlinear) structures. We further propose pre-training and fine-tuning strategies that let us effectively learn the parameters of our subspace clustering networks. Our experiments show that our method significantly outperforms the state-of-the-art unsupervised subspace clustering techniques.

The paper addresses the problem of subspace clustering, i.e., separating a collection of data points lying in a union of subspaces according to underlying subspaces, using deep neural networks. To do so, the paper builds on the sparse subspace clustering idea: among all possible representations of a data point as a combination of other points in the dataset, the representation that uses the minimum number of points, corresponds to points from the same subspace. In other words, SSC uses the idea that for a data matrix $X$, a sparse solution of $X = X C$ (subject to $diag(C) = 0$) represents each point as a combination of a few other points from the same subspace. The paper proposes a deep neural network to transform the data into a new representation $Z = f_W(X)$ for which one searches for a sparse representation of $Z = Z C$, with the hope to learn more effective representations of data for clustering. To achieve this, the paper uses an auto-encoder scheme, where the middle hidden layer outputs are used as $Z$. To enforce $Z - Z C = 0$, the paper repeats the middle layer, with the hope that these two consecutive layers obtain same activations. If so, the weights  between the two layers would correspond to coefficient matrix $C$ which will be used then, similar to existing methods, to build an affinity matrix on data and to obtain clustering. The paper demonstrates experiments on Extended YaleB  and ORL face datasets as well as COIL20/100.
The reviewer believes that the paper is generally well-written and easy to read. The paper also takes a sufficiently interesting approach to connect subspace clustering with deep learning. On the other hand, there are several issues with the technical approach and the experiments that limit the novelty and correctness of the paper.
1) The reviewer believes that the idea of the paper is similar to the one in [36], which uses an auto-encoder scheme, taking the middle layer as $Z$. While [36] fixes the $C$ and solves for $Z$, the proposed method repeats the middle layer to also solve for $C$. Despite this similarity, these is only a sentence in the paper (line 82) referring to this work and the experiments do not compare against [36]. Thus there is a need to clearly discuss advantages and disadvantages with respect to [36] and compare against it in the experiments.
2) It is not clear to the reviewer how the paper enforces that the activations of the middle and the middle plus one layer must be the same. While, clearly one can initialize $C$ so that the consecutive activations will be the same, one the back propagation is done and the weights of the auto-encoder change, the activations of middle layer and the one after it will not be the same (unless the $\lambda_2$ is very large and the network achieves it global minima, which is generally not the case). As a result, in the next iteration, the network will minimize $Z - Y C$ for different $Y$ and $Z$, which is not desired.
3) One of the drawbacks of the proposed approach is the fact that the number of weights between the middle layers of the network is $N^2$, given $N$ data points. Thus, it seems that the method will not be scalable to very large datasets.
4) In the experiments (line 243), the paper discusses training using mini-batches. Notice that while using mini-batches makes sense in the context of recognition, where each sample can be applied separately to the network, using mini-batches does not make sense in the context of clustering discussed in the paper. Using a mini-batch of size $M < N$, how does the paper train a neural network between two layers, each with $N$ nodes? In other words, it is not clear how the mini-batch instances and in what order will be input the the network.


Saliency-based Sequential Image Attention with Multiset Prediction
Humans process visual scenes selectively and sequentially using attention. Central to models of human visual attention is the saliency map. We propose a hierarchical visual architecture that operates on a saliency map and uses a novel attention mechanism to sequentially focus on salient regions and take additional glimpses within those regions. The architecture is motivated by human visual attention, and is used for multi-label image classification on a novel multiset task, demonstrating that it achieves high precision and recall while localizing objects with its attention. Unlike conventional multi-label image classification models, the model supports multiset prediction due to a reinforcement-learning based training process that allows for arbitrary label permutation and multiple instances per label.

This submission proposes a recurrent attention architecture (composed of multiple steps) that operates on a saliency map that is computed with a feed forward network. The proposed architecture is based on the biological model of covert attention and overt attention. This is translated into a meta-controller (covert attention) that decides which regions need overt attention (which are then treated by lower-level controllers that glimpse in the chosen regions). This is implemented as a form of hierarchical reinforcement learning, where the reward function for the meta-controller is explicitly based on the idea of  multiset labels. The (lower-level) controllers receive reward for directing attention to the area with high priority, as output by a previously computed priority map. The whole architecture is trained using REINFORCE, a standard RL algorithm. The proposed architecture is used for multi-label, specifically multi-set setting where each image can have multiple labels, and labels may appear multiple times. The experiments are conducted on MNIST and MNIST multiset datasets, where images contain one single digit or multiple digits.
	The paper is well written and the model is well explained. The detailed visualizations in the supplementary material are required to understand the architecture with many components. The details of the architecture when considered with the supplementary material, are well motivated and explained; the paper is comprehensive enough to allow the reproduction of results. The topic is interesting and relevant for the community.
	Some questions are as follows:
	- The “meta controller” unit takes a saliency map and the class label prediction from the previous step as inputs and learns a covert attention mask. How is the saliency map initialized in the first step? Which saliency map does the “interface” unit use, the saliency map learned in the previous step or the initial one?
	- The controller takes the priority map and the glimpse vector as inputs, runs them through a recurrent net (GRU) and each GRU unit outputs a label. How are the number of GRU units determined here? Using the number of gaussians learned by the gaussian attention mechanism? Is this the same as the number of peaks in the priority map?
	A missing related work would be Hendricks et.al., Generating Visual Explanations, ECCV 2016 that also trains a recurrent net (LSTM) using REINFORCE algorithm to generate textual explanations of the visual classification decision. However it does not contain an integrated visual attention mechanism.
	The main weakness of the paper is the experimental evaluation. The only experimental results are reported on synthetic datasets, i.e. MNIST and MNIST multi-set. As the objects are quite salient on the black background, it is difficult to judge the advantage of the proposed attention mechanism. In natural images, as discussed in the limitations, detecting saliency with high confidence is an issue. However, this work having been motivated partially as a framework for an improvement to existing saliency models should have been evaluated in more realistic scenarios. Furthermore, the proposed attention model is not compared with any existing attention models. Also, a comparison with human gaze based as attention (as discussed in the introduction) would be interesting. A candidate dataset is CUB annotated with human gaze data in Karessli et.al, Gaze Embeddings for Zero-Shot Image Classification, CVPR17 (another un-cited related work) which showed that human gaze based visual attention is class-specific.
	Minor comment:
	- The references list contains duplicates and the publication venues and/or the publication years of many of the papers are missing.


Learning the Morphology of Brain Signals Using Alpha-Stable Convolutional Sparse Coding
Neural time-series data contain a wide variety of prototypical signal waveforms (atoms) that are of significant importance in clinical and cognitive research. One of the goals for analyzing such data is hence to extract such 'shift-invariant' atoms. Even though some success has been reported with existing algorithms, they are limited in applicability due to their heuristic nature. Moreover, they are often vulnerable to artifacts and impulsive noise, which are typically present in raw neural recordings. In this study, we address these issues and propose a novel probabilistic convolutional sparse coding (CSC) model for learning shift-invariant atoms from raw neural signals containing potentially severe artifacts. In the core of our model, which we call αCSC, lies a family of heavy-tailed distributions called α-stable distributions. We develop a novel, computationally efficient Monte Carlo expectation-maximization algorithm for inference. The maximization step boils down to a weighted CSC problem, for which we develop a computationally efficient optimization algorithm. Our results show that the proposed algorithm achieves state-of-the-art convergence speeds. Besides, αCSC is significantly more robust to artifacts when compared to three competing algorithms: it can extract spike bursts, oscillations, and even reveal more subtle phenomena such as cross-frequency coupling when applied to noisy neural time series.

In this paper, the authors propose a novel probabilistic convolutional sparse coding model (alpha-stable CSC) for learning shift-invariant atoms from raw neural signals. They propose an inference strategy based on Monte Carlo EM to deal efficiently with heavy tailed noise and take into account the polarity of neural activations with a positivity constraint. The formulation allows the use of fast quasi-Newton methods for the M-step which outperform previously proposed state-of-the-art ADMM based algorithms. The experiment results on LFP data demonstrate that such algorithms can be robust to the presence of transient artifacts in data and reveal insights on neural time-series without supervision.
The main contribution of this study is extending a general (alpha-stable distribution) framework, comparing to the Gaussian distribution. From experiment point of view, the proposed approach is very useful. However, there are a few comments and suggestions for the authors to consider:
(1)	Figure 1(a) only demonstrates the PDFs of alpha-stable distributions characterized by alpha and beta. Please try to describe the characters of the scale parameter and location parameter.
(2)	As written in the paper, “As an important special case of the alpha-stable distributions, we obtain the Gaussian distribution when alpha= 2 and beta = 0”, the authors emphasize that the current CSC models might be limited as they consider an L2 reconstruction error. While the extending model is more robust to corrupted data than the Gaussian models. However, I wonder if the experiments should include Gaussian Mixture Model as a fair comparison.
(3)	How to determine the parameters (i.e., lambda, N, T, L, K), respectively, in synthetic and LEP data experiments?


Learning from Complementary Labels
Collecting labeled data is costly and thus a critical bottleneck in real-world classification tasks. To mitigate this problem, we propose a novel setting, namely learning from complementary labels for multi-class classification. A complementary label specifies a class that a pattern does not belong to. Collecting complementary labels would be less laborious than collecting ordinary labels, since users do not have to carefully choose the correct class from a long list of candidate classes. However, complementary labels are less informative than ordinary labels and thus a suitable approach is needed to better learn from them. In this paper, we show that an unbiased estimator to the classification risk can be obtained only from complementarily labeled data, if a loss function satisfies a particular symmetric condition. We derive estimation error bounds for the proposed method and prove that the optimal parametric convergence rate is achieved. We further show that learning from complementary labels can be easily combined with learning from ordinary labels (i.e., ordinary supervised learning), providing a highly practical implementation of the proposed method. Finally, we experimentally demonstrate the usefulness of the proposed methods.

This paper proposes a novel problem setting and algorithm for learning from complementary labels. It shows an unbiased estimator of the classification risk can be obtained only from complementary labels, if a loss function satisfies a particular symmetric condition. The paper also provides estimation error bounds for the proposed method. Experiments show the approach can produce results similar to standard learning with ordinary labels on some datasets.
The idea and the theoretical results are interesting. But I question the usefulness of the proposed learning problem.  Though using complementary labels can be less laborious than ordinary labels, complementary labels are less informative than ordinary labels. If using N training instances with complementary labels cannot perform better (this is almost certain) than learning with N/(K-1) training instances with ordinary labels,  why should one use complementary labels? Moreover, based on the theoretical results, learning with complementary labels are restricted to a limited number of loss functions, while there are many state-of-the-art effective learning methods with ordinary labels.
The paper only suggests in the conclusion section that such complementary label learning can be useful in the context of privacy-aware machine learning. Then why not put this study within the privacy-aware learning setting and compare to the privacy-aware learning techniques?
For section 3: (1) On line 51, it says “ that incurs a large loss for large z”.  Shouldn’t be “that incurs a large loss for small z”?  (2) It is not clear how the derivation on Line 79-80 was conducted.  Why should \sum_{y\not= \bar{y}}\ell(g_y(x))/(K-1) =  \ell(g_{\bar{y}}(x)) ?
The experiments are very basic. In the benchmark experiments, it only compares to a self-constructed ML formulation and a one-hidden-layer neural network with ordinary labels. The way of translating the complementary labels to multi-labels is not proper since it actually brings much wrong information into the labels. The one-hidden-layer neural network is far from being a state-of-the-art multi-label classification model.  It is better to just compare with the standard learning with ordinary labels using the same PC with sigmoid loss.
Another issue is: why not use the whole datasets instead of part of them? It would also be interesting to see the comparison between complementary label learning with ordinary label learning with a range of different number of classes on the same datasets.


Variational Inference for Gaussian Process Models with Linear Complexity
Large-scale Gaussian process inference has long faced practical challenges due to time and space complexity that is superlinear in dataset size. While sparse variational Gaussian process models are capable of learning from large-scale data, standard strategies for sparsifying the model can prevent the approximation of complex functions. In this work, we propose a novel variational Gaussian process model that decouples the representation of mean and covariance functions in reproducing kernel Hilbert space. We show that this new parametrization generalizes previous models. Furthermore, it yields a variational inference problem that can be solved by stochastic gradient ascent with time and space complexity that is only linear in the number of mean function parameters, regardless of the choice of kernels, likelihoods, and inducing points. This strategy makes the adoption of largescale expressive Gaussian process models possible. We run several experiments on regression tasks and show that this decoupled approach greatly outperforms previous sparse variational Gaussian process inference procedures.

Summary:  The paper employs an alternative view of Titsias's sparse variational inference technique for Gaussian process regression, treating the mean function and covariance function as linear operators in RKHS. By decoupling the basis functions that each of these functions uses, the variational lower bound can now be computed in a O(NMa + NMb^2) complexity, linear in N, and linear in Ma (number of mean parameters) and quadratic in Mb (number of covariance parameters). Previous approaches have O(NM^2) complexity. This means one now can use much bigger Ma compared to Mb without significantly increasing the complexity. The catch is, however, the posterior prediction is no longer **calibrated** and as good, i.e. smaller Mb results in larger predictive variances. The method is compared against existing approaches on several regression datasets, showing lower predictive errors and achieving higher variational lower bounds.
Details:
The catch is that the predictive variance is large if Mb is small resulting in a poor predictive performance if measured using the log-likelihood. This can be seen in figure 1, and is mentioned in the text but has not been fully experimented (as the paper only uses the squared error metric which does not use the predictive variances).
Unless I have missed it, the basis locations are not optimised but selected using several initial mini-batches. Would there be any problems if we optimise locations of the inducing points/bases as in Titsias's approach?
The inducing points/inputs are grounded on some physical locations and can be plotted. Can we visualise the subspace parameterisation? For example, grounding some bases and plotting the samples generated from the corresponding GPs?
And why is there a kink/sudden change in the traces for all methods in the middle of Figure 1? The learning curves for SVI are still going down after 2000 iterations -- does this use SNGD for the q(u)'s parameters? I think recent works on SVI for GP models (Hensman et al 2014, Bonilla et al 2014, 2015) all use SGD for the mean and Cholesky parameters. This seems to work well in practice, and avoid the need to deal with natural gradients or select a separate learning rate. It would be better to compare to these variants of SVI.
Extensions to non-Gaussian likelihoods seem non-trivial as computing the expected log likelihood would decouple the variational parameters.


Bregman Divergence for Stochastic Variance Reduction: Saddle-Point and Adversarial Prediction
Adversarial machines, where a learner competes against an adversary, have regained much recent interest in machine learning. They are naturally in the form of saddle-point optimization, often with separable structure but sometimes also with unmanageably large dimension. In this work we show that adversarial prediction under multivariate losses can be solved much faster than they used to be. We first reduce the problem size exponentially by using appropriate sufficient statistics, and then we adapt the new stochastic variance-reduced algorithm of Balamurugan & Bach (2016) to allow any Bregman divergence. We prove that the same linear rate of convergence is retained and we show that for adversarial prediction using KL-divergence we can further achieve a speedup of #example times compared with the Euclidean alternative. We verify the theoretical findings through extensive experiments on two example applications: adversarial prediction and LPboosting.

This paper shows that "certain" adversarial prediction problems under multivariate losses can be solved "much faster than they used to be". The paper stands on two main ideas: (1) that the general saddle function optimization problem stated in eq. (9) can be simplified from an exponential to a quadratic complexity (on the sample size), and (2) that the simplified optimization problem, with some regularization, can be solved using some extension of the SVRG (stochastic variance reduction gradient) method to Bregman divergences.
The paper is quite focused on the idea of obtaining a faster solution of the adversarial problem. However, the key simplification is applied to a specific loss, the F-score, so one may wonder if the benefits of the proposed method could be extended to other losses. The extension of the SVRG is a more general result, it seems that the paper could have been focused on proposing Breg-SVRG, showing the adversarial optimization with the F-score as a particular application.
In any case, I think the paper is technically correct and interesting enough to be accepted.