Multi-armed Bandits with Compensation
We propose and study the known-compensation multi-armed bandit (KCMAB) problem, where a system controller offers a set of arms to many short-term players for T steps. In each step, one short-term player arrives at the system. Upon arrival, the player aims to select an arm with the current best average reward and receives a stochastic reward associated with the arm. In order to incentivize players to explore other arms, the controller provide proper payment compensations to players. The objective of the controller is to maximize the total reward collected by players while minimizing the total compensation. We first provide a compensation lower bound ⇥( P i i log T KLi ), where i and KL i are the expected reward gap and the Kullback-Leibler (KL) divergence between distributions of arm i and the best arm, respectively. We then analyze three algorithms for solving the KCMAB problem, and obtain their regrets and compensations. We show that the algorithms all achieve O(log T ) regret and O(log T ) compensation that match the theoretical lower bounds. Finally, we present experimental results to demonstrate the performance of the algorithms.

This paper is about multi-armed bandits. The twist is that the learner does not control directly the actions. Instead, in each round the learner chooses an amount of 'compensation' associated to each arm and the user chooses the arm for which the sum of the compensation and empirical mean is largest. The objective is to design algorithms for which the cumulative regret (with the normal definition) and cumulative compensation both have logarithmic growth.  The claimed results are as follows: (a) A lower bound showing that algorithms with subpolynomial regret necessarily must necessarily pay logarithmic cumulative compensation with the usual constants on *some* bandit problems.  (b) Upper bounds on the cumulative compensation for UCB, epsilon-greedy and a variant of Thompson sampling with appropriately chosen compensation levels.  To illustrate, the most obvious idea of offering the difference between the UCB and empirical mean as compensation leads to a sqrt(n) growth rate of the compensation, so something a little clever is needed. The authors point out the compensation is not required if the empirical mean of the arm chosen by UCB is the largest and then prove this happens with sufficiently few exceptions.  By the way, one might wonder if the DMED-style algorithm (Honda and Takamura, 2010) might not be adapted even more straightforwardly.  I have two reservations about this paper. First, the proofs are much more complicated than necessary. This is especially true of the lower bound, the proof of which is three pages. I include a suggestion for how to considerably shorten this proof below. Note that this sketch does not depend on the first arm having a mean of at least 0.9 and applies to all bandit problems, like Lai & Robbins result.  My second reservation is that the paper is not very cleanly written. There are many typos and the sketches are hard to follow. The proofs in the appendix even harder. And yet the claimed results do not seem especially surprising. I think a paper like this could be accepted at NIPS, but the analysis should be crisp and insightful.   Regarding the originality and significance. The setting is new to my knowledge. The techniques are not especially startling. The significance is hard to judge. There is some motivation given, but this is obviously a first step. In practical situations things would almost always be contextualized and so-on. Still, I think this aspect is Ok.   A few other comments:  1. I was wondering if perhaps you can prove a more generic result. Given /any/ algorithm can I design a black-box modification for which the compensation bound has the same order of the regret for that algorithm?  2. Perhaps you can get the constants right in the lower bound and upper bound. See below for the former. For the latter you could use the techniques from the KL-UCB algorithm.  3. All of the upper bounds involve simple modification of existing algorithms. Essentially what is required is choosing the compensation so that the behavior is the same as what you would expect in the standard setting. The lower bounds show you cannot do much better than this, but one still might wonder if the algorithms ought to change a little in the new setting. Mathematically it seems a pity the setting is not especially rich.  Overall the paper is below the bar for NIPS.   ********************************************** * LOWER BOUND SKETCH **********************************************  By Lai & Robbins result it holds that  P(N_i(T) >= (1-e) / KL log(n)) >= 1 - delta(T)  with delta(T) tending to 0.  Let epsilon > 0. I claim there exists a constant m that depends on the bandit, but not T such that the following four events hold jointly with probability at least 1/2 for all sufficiently large T: (a) N_i(T) >= (1 - e) / KL log(T) for all i (b) N_1(2m) >= m (c) Whenever N_1(t) >= m, then the empirical mean of arm 1 is at least mu_1 - epsilon (d) Whenever N_i(t) >= m, then the empirical mean of arm i is a most mu_i + epsilon for all i  Then since m does not depend on T we may take the limit as T tends to infinity when on the four events above it the algorithm must pay at least Delta_i - 2 epsilon for at least (1 - e) / KL log(T) - m rounds, which yields the desired form of the lower bound.  The claim is proven easily via a union bound. (a) using Lai & Robbins result, (b) by Markov's inequality and the assumption on subpolynomial regret. (c) and (d) by Hoeffding's bound and a union bound (Chernoff's is not required).    *********************** * MINORS ***********************  * The inequality in the sketch of Theorem 1 is the wrong way. * \Theta(f(n)) means that f(n) = O(f(n)) and f(n) = Omega(f(n)), so the claim of Theorem 1 is not accurate. In general I prefer not to see this notation in theorem statements.


Implicit Probabilistic Integrators for ODEs
We introduce a family of implicit probabilistic integrators for initial value problems (IVPs), taking as a starting point the multistep Adams-Moulton method. The implicit construction allows for dynamic feedback from the forthcoming timestep, in contrast to previous probabilistic integrators, all of which are based on explicit methods. We begin with a concise survey of the rapidly-expanding field of probabilistic ODE solvers. We then introduce our method, which builds on and adapts the work of Conrad et al. (2016) and Teymur et al. (2016), and provide a rigorous proof of its well-definedness and convergence. We discuss the problem of the calibration of such integrators and suggest one approach. We give an illustrative example highlighting the effect of the use of probabilistic integrators-including our new method-in the setting of parameter inference within an inverse problem.

## Summary  The paper proposes a novel algorithm for solving initial value problems in ODEs. The first part of the paper aims to review current literature on probabilistic ODE solvers arising in the field of probabilistic numerics (PN). Then, based on this review, the authors propose a novel probabilistic implicit solver for IVPs which retains some of the non-linearities of the vector field, but locally works with Gaussian approximations to the solution and/or the velocities for computational speedups. The paper contains a proof of convergence in mean square for diminishing step sizes. As a proof-of-concept, the authors demonstrate their solver as well as its application to system identification on a 2D-toy-problem.  ## Novelty & originality & Significance The paper is an interesting read since it is a combination of a review paper and a new proposed method. Apart from the latter, personally I think it has a high values since it summarizes the state-of-the-art of the field well, also for non-ODE experts. Additionally the proposed method (*implicit* rather than explicit PN solver) seems novel to me. It is an interesting approach to handling the cost-precision tradeoff for ODEs which more generally is inherent to any numerical solver. I am not super convinced about the practical relevance of the method in its current state and I’d encourage the authors to comment on the possible applications, especially in contrast to running a faster solver with a smaller step h for the same computational budget.  ## Clarity The paper is clearly written and well developed. It is however very dense. I’d actually like to see a longer journal version of this, too.  ## Major points - Eq. 3 and paragraph below. I do not see the point why Z, Z-tilde and x are named differently since, as I understand, all three represent the same random variable (the unknown solution).    - Eq. 4: the authors mention that their method could in principle use different transformations g but only the Gaussian is used. Did the authors experiment with other transformations? If yes, which? If not, do the authors think that for all practical purposes one might always be restricted to use Gaussian forms of g?   - a general point on Gaussians: the authors seem to point out repeatedly that some distributions are non-Gaussian. This is fair enough to distinguish their method from methods that do use Gaussians in all distributions. But for a reader it is sometimes harder to understand the novel method by having negative statements only (what it is not, rather than what it is). I’d encourage the authors to also point out more clearly which of the latent variables (F for instance? Or r?) are Gaussian locally, especially since Gaussians seem to appear later during the approximations and estimation of H. It would make for an easier read.  - l. 242 “… nowhere assuming that the distributions are Gaussian.” Related to the above point, this comment is a bit misleading and prone to misunderstandings since it is not clear what “the distributions” means. I’d encourage the authors to specify which distribution they were referring to, especially since Gaussian distributions do occur in various places.  - l. 270. “Then non-linearities from f are retained (to second order)” I am unsure, but isn’t the approximation linear in f but non-linear in z? At least l. 265 seems to indicate this.  - l. 271 “jointly Gaussian”. What is meant by “jointly” here? Is it jointly across iterations i? Or jointly given one specific i? The former is true, the latter is not I guess. Could the authors please specify.   - Eq. 10: H seems to be fitted in a kind of empirical Bayes. Could the authors comment on this, please. Especially in connection to the idea of “numerical data F”.  - A general point on applicability: As a potential user of the algorithms, what would be the application where the proposed algorithms is better than any existing ODE solver. And also better in which sense? I am asking because the authors motivate their algorithms also by the accuracy-cost tradeoff, and it left me wondering what I would actually use it for. I could, for example also just run a classic ODE solver with a finer step size h and, in the same time, have such a good approximation to my solution that I do not care very much about potential non-symmetric error bars.  - Experiments: please explain which of the approximations are used to generate the results. For instance I think the authors did not mention if the approximations of section 2.4 are used. Also how does the sampling exactly work. Could the authors please add a pseudo-code to the supplements.  - Figure 1 & experiments: Once given the empirical distribution (red dots), is it possible to generate an estimator for the solution? If yes, how? What would be the most favorable choice and why?  ## Minor points - l. 101 and l. 135 “This approach…” Both times a new paragraph is started with a reference to something mentioned before. Possibly the paragraphs can be divided somewhere else such that paragraphs contain one thought process completely. Then tracing back and referencing would become a lot easier.  - Figure 2. Could the authors please add an indication of the true value to the x-axis? Like this one has to trace-back to find the true value of theta in the text.  ## Post rebuttal Thank you for clarifying and addressing my concerns, especially also all the detailed comments [3].  [1] purpose of work: thank you for clarifying. That makes sense to me, although my question was more targeted towards specific examples or applications where, as you say, the richer UQ is preferred over tighter worst-case errors. I understand the high-level argument but would be interested in knowing an application where this trade-off is indeed in favor of the PN-ODE solver. If the authors are aware of one it might be worth mentioning it in the final version. In my mind, a less well-understood but small error is usually preferred over a well-understood, but larger error.  [2] notation Schober et al.: I see your point here. I’d still argue that it is worth mentioning that Z and \tilde{Z} represent the same RV and the distinction in notation only emphasizes the difference in their use. I am kind of picking on this since distinguishing the two is a bit counter-intuitive from a PN point of view and possibly misleading.  [3] all other questions: thanks for clarifying and addressing all the points. It is clearer now.


Analysis of Krylov Subspace Solutions of Regularized Nonconvex Quadratic Problems
We provide convergence rates for Krylov subspace solutions to the trust-region and cubic-regularized (nonconvex) quadratic problems. Such solutions may be efficiently computed by the Lanczos method and have long been used in practice. We prove error bounds of the form 1/t 2 and e 4t/ p  , where  is a condition number for the problem, and t is the Krylov subspace order (number of Lanczos iterations). We also provide lower bounds showing that our analysis is sharp.

Summary The authors consider optimization problems where one tries to minimize a (possibly non-convex) quadratic function in two different settings: - when the optimization domain is a ball with fixed radius, - when a cubic regularizer is added to the quadratic function. These two settings are motivated by their importance for second-order optimization. The goal of the article is to describe the convergence rate, in these settings, of Krylov subspace solutions (the Krylov subspace solution, at order t, is the solution when the problem is restricted to a t-dimensional subspace naturally associated with the quadratic function). The authors establish two convergence rates: one that is exponential in t, with a rate that depends of the (properly defined) condition number of the problem, and another one that scales like t^(-2), with a constant that depends logarithmically on the scalar product between two specific vectors (and that scales at worst like log^2(d) when a random perturbation is added). They show that the two bounds are essentially tight.   Main remarks I am unfortunately not very familiar with Krylov subspace methods and second-order optimization, so it is difficult for me to evaluate the position of this article in the literature, and its importance to the domain. Nevertheless, as a non-expert, I can say that I sincerely enjoyed reading this article, and found it very interesting; I hope it will be published. In more detail,  - Krylov subspace methods are an extremely important family of optimization algorithms. I therefore think that it is very valuable to precisely understand which convergence guarantees they can offer in all important use cases, and to develop new tools for proving such guarantees (as well as to improve the previous tools). The motivations from trust-regions and cubic-regularized Newton methods are also important.  - The article is very well-written, especially the technical parts. The proofs rely on polynomial approximation results that are relatively classical; the technical novelty lies in how these results are used, and combined with previous literature, to obtain the two convergence rates. The classical results are presented in full detail in the appendix; with this respect, I appreciated the effort that has been put into making the article self-contained. The rest is very clearly explained in the main part of the article, with all the necessary details, but also enough discussion so that the reader can get a good intuition of the principles behind the proofs.  - The two convergence rates, and the fact that they are optimal, are essentially the same as the optimal convergence rates for solving convex problems with first-order methods in two situations : (i) when the convex function is assumed to have a Lipschitz gradient (in which case the rate is O(t^(-2))) (ii) when the convex function is strongly convex, and has a Lipschitz gradient (in which case the rate is exponential, with the same dependency in the condition number as in Equation (4) of this article). Maybe the link could be discussed and clarified in the article?   Minor remarks and typos - Line 38: I did not find the expression "the sub-optimality ... is bounded by" very clear. - Line 120: "is choose" -> "is to choose"? - Line 220: "and os it is" -> "and it is" - Section 4, first paragraph: This is a minor point but, since the gap is smaller in the case of the cubic-regularized problem, is it immediate how to deduce the trust-region result from the cubic-regularized one, including the replacement of the cubic gap by the trust-region one in the right-hand side of Equation (18)? - Line 238: tau has to be larger than 1. - Bibliography: Reference [14] contains some weird characters, and it seems to me that the link given for reference [32] is broken. - Line 402: "for all y>=0" and "for all z>=1" could be added. - Line 415: in the left-hand side of the double inequality, I think that sqrt(kappa) should be sqrt(kappa)-1. - Line 417, second line of the block: "n-1" should be "n+1". - Line 440: g must be convex. - Line 485: I think that the last lambda_max should be a lambda_min. - The equation after line 489 could be split into two lines. - Line 491: The definition of n is missing. - Line 496: "(t+1)-dimensional" -> "d-dimensional"? - Equation after line 496: The definition of rho is actually the definition of 1/rho (we want lambda_star = rho ||s_star||). - Equation after line 501: There is a Delta missing. - Equaiton after line 525, left-hand side: "q^2(0)(u^Tv)" is probably missing in the denominator. - Equation after line 554: I do not get why the first R is not a \tilde R. - Equation after line 562: I think there is a minus sign missing in the right-hand side. - Line 578: "guarantee" -> "guarantees"  Added after author feedback  Thanks to the authors for taking my minor remarks into account. I am naturally still in favor of accepting the article.


Deep Attentive Tracking via Reciprocative Learning
Visual attention, derived from cognitive neuroscience, facilitates human perception on the most pertinent subset of the sensory data. Recently, significant efforts have been made to exploit attention schemes to advance computer vision systems. For visual tracking, it is often challenging to track target objects undergoing large appearance changes. Attention maps facilitate visual tracking by selectively paying attention to temporal robust features. Existing tracking-by-detection approaches mainly use additional attention modules to generate feature weights as the classifiers are not equipped with such mechanisms. In this paper, we propose a reciprocative learning algorithm to exploit visual attention for training deep classifiers. The proposed algorithm consists of feed-forward and backward operations to generate attention maps, which serve as regularization terms coupled with the original classification loss function for training. The deep classifier learns to attend to the regions of target objects robust to appearance changes. Extensive experiments on large-scale benchmark datasets show that the proposed attentive tracking method performs favorably against the state-of-the-art approaches.

This paper considers single-camera, single-target visual tracking where the target object can undergo large appearance changes. Attention maps helps visual tracking by selectively attending to temporally invariant motion patterns. The paper argues that existing tracking-by-detection approaches mainly use additional attention modules to generate feature weights. This paper proposes an algorithm consisting of feed-forward and backward operations to generate attention maps. The attention map is used as regularization terms coupled with the original classification loss function for training. The paper claims this is better than existing attention mechanisms used in visual tracking.   The paper lacks novelty and significant contributions due to the following. 1. Visualization techniques have computed the sensitivity of input pixels to loss function. So the technique does not seem to be new.   2. Using the attention map as regularization terms is somewhat interesting. However, the results are not much better than existing attention mechanism. As stated in the paper, the baseline tracker with reciprocative learning gain over the baseline vs the baseline with attentive feature scheme gain over the baseline are: 3.2% vs. 2.1% in distance precision metric, and 3.3% vs. 1.5% in overlap success rates.  3. The extensive experiments comparing [33] with proposed attention map with respect to many other methods in the literature does not seem to be very relevant.   There have been many papers on multi-target tracking, e.g.  https://arxiv.org/pdf/1604.03635.pdf, Online Multi-Target Tracking Using Recurrent Neural Networks https://arxiv.org/pdf/1710.03958.pdf, Detect to Track and Track to Detect  Multi-target tracking leverages state-of-the-art object detector. For two-stage object detectors, the proposal network acts like attention maps. For single-state object detectors, there is no need for attention. The authors might want to investigate whether attention map can reduce computation complexity in the context of multi-target tracking.   In summary, there are two key issues. One is whether attention regulation is novel enough or not. The other is whether performance improvement is significant or not. Unfortunately, the rebuttal does not address either of them. The rebuttal claims performance gain of +1.1% in distance precision and +1.8% in overlap success over the state-of-the-art methods is significant. The rebuttal just reiterates that they are the first to integrates attention maps as a regularization term into the loss function for end-to-end learning CNNs.


Learning Attentional Communication for Multi-Agent Cooperation
Communication could potentially be an effective way for multi-agent cooperation. However, information sharing among all agents or in predefined communication architectures that existing methods adopt can be problematic. When there is a large number of agents, agents cannot differentiate valuable information that helps cooperative decision making from globally shared information. Therefore, communication barely helps, and could even impair the learning of multi-agent cooperation. Predefined communication architectures, on the other hand, restrict communication among agents and thus restrain potential cooperation. To tackle these difficulties, in this paper, we propose an attentional communication model that learns when communication is needed and how to integrate shared information for cooperative decision making. Our model leads to efficient and effective communication for large-scale multi-agent cooperation. Empirically, we show the strength of our model in a variety of cooperative scenarios, where agents are able to develop more coordinated and sophisticated strategies than existing methods.

Summary: This paper describes and evaluates a complex reinforcement learning algorithm for achieving cooperation within a team of agents.  As I understand it, in this design, the agents all share the same policy network, but they sense different things.  To help the agents coordinate their behavior, the system learns when to communicate and combine “thoughts” (outputs from a layer of a neural network) with each other.  The authors empirically analyze the system in three different domains (cooperative navigation, cooperative push ball, and Predator-Prey).  The results achieved appear to outperform existing algorithms designed for similar situations.  Quality: The authors appear to combine together state-of-the-art algorithmic mechanisms in a very clever way.  The results suggest that the algorithm is effective.  I very much like that the authors evaluated the system in three different domains.  These attributes make the paper quite appealing.  But no paper is perfect (at least from the eye of reviewers).  In my opinion, the quality of the paper could be improved in three ways:  1.  Provide more details in order to make the paper reproducible: To me, the algorithm and results do not appear to be reproducible given the descriptions provided in the paper.  While the paper does a good job of overviewing the system, important details (parameter values, network designs, etc.) that are needed to reproduce the results are not given (at least that I saw).  I believe that it is likely that someone else trying to implement similar mechanisms but without the same details would get very different results (particularly if there were not experts in the field), as I believe these details are likely critical to the success of the system.  That said, I realize that providing sufficient details to reproduce complex algorithms is very difficult in a short paper format.  2.  More thorough algorithmic analysis: From the analysis of the results, it is a bit difficult to determine what makes the system tick.  Comparisons to previously developed algorithms are nice to establish baselines, but do not thoroughly isolate in all instances what makes the algorithms tick.  The authors do add nice analysis (e.g., testing ATOC with and without communication).  But even without communication, ATOC outperforms the other algorithms substantially in cooperative pushball (Table 2).  Is this because the learning mechanisms used in ATOC are just better, or did the authors just take more time to refine and optimize ATOC for the given domains more than the other algorithms?  3.  Statistical comparisons: It is also somewhat difficult to interpret the differences between the various algorithms, as no statistical tests or error bars are provided to communicate to the reader significance.  Clarity: The organization of the paper is quite good, and it is definitely possible for a reader to get a good high-level understanding of what is being done.  I do have a few suggestions (nitpicks in some sense) that may be helpful in providing additional clarity.  1.  At least from my perspective, the abstract and introduction didn’t spell out the domain very well up front.  Agents that “cooperate” with each other can take on many forms.  The algorithmic architecture described herein addresses one of these forms, that of a team of computer agents that share many aspects of the same “brain.”.  As recommended by Shoham et. al, 2007 (If multi-learning is the answer, what is the question), I recommend better spelling this out from the onset.  The domain addressed in this work (and hence potentially the kinds of algorithms that will work) is very different from other recent and older work in which communication of AI systems is designed and evaluated, including systems that interact with people and systems in which cooperation can be beneficial but in which the agents do not share the same interests.    2.  I may have missed it, but a good description of how the rewards are computed for each domain appears to be missing.  For example, I’m not sure how good a reward of -0.04 is (or what it represents) in the “cooperative navigation” task (Table 1).  3.  Several times throughout the paper, the authors use the terms “barely help,” “barely adapt,” and “hardly adapts” to describe other algorithms.  Each time these terms are used, I got confused.  The language seemed to be imprecise and could be interpreted as just attempts to “beat up” previous work.  I believe the authors are trying to communicate that the previous algorithms can sometimes have issues “scaling up” because of various design decisions — I just recommend using different language to describe that.  Originality: The work builds on and appears to improve upon a line of work that is emerging in the recent literature:  communication in team settings in which a single designer controls all robots.  I believe that the algorithmic architecture is novel.  Significance: On one hand, the authors appear to be combining together advanced methods in a very clever way.  The results achieved are relatively good, and it does appear that the work advances the state-of-the-art.  On the other hand, the perspective is a little bit strange, neither using a true multi-agent system (wherein all agents are separate) nor a fully centralized system.  Since the designer appears to have full control of the algorithms used by all robots, is there a reason that a computationally adept, but fully centralized system could not be used for computation rather than a “multi-agent system.”  Overall, the assumptions of the paper are not fully spelled out, nor are the algorithmic mechanisms thoroughly evaluated in isolation.  Too me, this dampens the significance of the contributions, but certainly does not eliminate them.  I find this paper to be a nice work.


Training Neural Networks Using Features Replay
Training a neural network using backpropagation algorithm requires passing error gradients sequentially through the network. The backward locking prevents us from updating network layers in parallel and fully leveraging the computing resources. Recently, there are several works trying to decouple and parallelize the backpropagation algorithm. However, all of them suffer from severe accuracy loss or memory explosion when the neural network is deep. To address these challenging issues, we propose a novel parallel-objective formulation for the objective function of the neural network. After that, we introduce features replay algorithm and prove that it is guaranteed to converge to critical points for the non-convex problem under certain conditions. Finally, we apply our method to training deep convolutional neural networks, and the experimental results show that the proposed method achieves faster convergence, lower memory consumption, and better generalization error than compared methods.

## Summary  The paper proposes a novel algorithm for backpropagation in neural networks, called ‘Feature Replay’ (FR) which speeds up the gradient computation in the backwards pass at the expense of some accuracy. This is done by splitting the net into K sub-modules in which each sub-layer is updated with an outdated gradient of the (at most) K-shifted residual. The authors claim that FR handles the accuracy vs. speed/memory-tradeoff better than existing algorithms. The authors include a proof of convergence based on several assumptions which the authors then attempt to validate experimentally. The algorithm is tested against two competitors & vanilla backprop on two well worn image classification benchmarks (CIFAR-10/100). Additional insights are provided in the form of memory consumption plots as well as plots that show speed vs. accuracy (num. K).  The paper seems to tackled an important problem in neural network training which is speeding up the training time. While I found the theoretical development of the method convincing, I cannot comment all too much about the choice of competitors and their execution in the experiments. The only proper baseline which is considered is vanilla backprop and DDG, while the second competitor (DNI) oddly seems to never really work.   ## Novelty & originality & Significance I am not an expert in the field so I am unsure about novelty & originality, but I personally have not seen the same algorithm before. Also the general formulation of the problem as a minimization task, which is then solved by an ad-hoc approximation possibly opens up different variations of the proposed algorithm in the future.   ## Clarity The motivation of the paper is clearly written and easy to follow.   ## Major points  - Eq. 8 introduces the constant sigma which, if it exists, bounds the deviation of the approximate gradient to the true one. Figure 3 shows empirical results of sigma. Could the authors please comment on how this is computed? Did they also compute the true gradient at each step? Also, since Eq. 8 assume no mini-batches while in Figure 3 there seems to be one value per epoch only, did the authors average the estimated sigmas? In that case there could be sigmas which are not bounded away from zero, although the mean has a large value.  - The authors emphasize that their algorithm does not suffer from approximation errors as much as competitors if the network is deep. Still, the error gradients seem to be less accurate by construction the deeper the network. How deep can I make the network such that the algorithm is still feasible? Did the authors conduct experiments in that direction? I am asking since the networks considered do not seem to be particularly deep after all.  - l. 109 ff. states that w^t approx w^{t-1} et cetera. Do the authors assume convergence here? If yes, it should be mentioned. Also, in what way is the approximation sign meant? I think there is no unique way of saying that a vector is similar to another one. Could the authors please clarify this paragraph?  - Could the authors comment on how to set K in practice? As a user of the proposed algorithm I would preferably not like to tune it. The paper says ‘we always set K very small’, what does that mean?  - The authors used SGD+momentum in their experiments. Why? Does the method not work with SGD since it introduces more noise to the gradients? SGD+momentum seems to be a sensible choice since it can be seen as averaging out some of the gradient noise, but I’d like to understand if it is essential for the proposed algorithm.  - Did the authors consider to lower the learning rate for DNI a bit? If the gradients are more noisy their elements are also larger in expectation. DNI might have converged with some mild tuning. This might have enabled a fairer comparison.  - Algorithm 1 is helpful.  ## Minor points & typos - l.130 ff. Could you please not use x and y for inputs to f. It is slightly confusing since e.g., y is used for the training targets, too.   ## Post-rebuttal Thank you for the clarifications and the thorough rebuttal. I increased my score. The details: [1] OK.  [2] Thank you for clarifying. If there is a way to include this information in the paper this would be very helpful for interpreting the plot properly. [3] cool. Thank you. [4] Thank you for clarifying. It would be helpful to add a small remark in the text. [5] OK. If there is a way of rephrasing a sentence to say this that would help.  [6] OK. Thank you for including the plot in the rebuttal. That was very helpful.  [7] I see. Thank you [8] Thanks


Estimators for Multivariate Information Measures in General Probability Spaces
Information theoretic quantities play an important role in various settings in machine learning, including causality testing, structure inference in graphical models, time-series problems, feature selection as well as in providing privacy guarantees. A key quantity of interest is the mutual information and generalizations thereof, including conditional mutual information, multivariate mutual information, total correlation and directed information. While the aforementioned information quantities are well defined in arbitrary probability spaces, existing estimators add or subtract entropies (we term them ΣH methods). These methods work only in purely discrete space or purely continuous case since entropy (or differential entropy) is well defined only in that regime. In this paper, we define a general graph divergence measure (GDM),as a measure of incompatibility between the observed distribution and a given graphical model structure. This generalizes the aforementioned information measures and we construct a novel estimator via a coupling trick that directly estimates these multivariate information measures using the Radon-Nikodym derivative. These estimators are proven to be consistent in a general setting which includes several cases where the existing estimators fail, thus providing the only known estimators for the following settings: (1) the data has some discrete and some continuous valued components (2) some (or all) of the components themselves are discrete-continuous mixtures (3) the data is real-valued but does not have a joint density on the entire space, rather is supported on a low-dimensional manifold. We show that our proposed estimators significantly outperform known estimators on synthetic and real datasets.

This paper develops multivariate information measures in fairly general probability spaces based on Bayesian Networks. The references include a variety of references on multivariate and conditional information theory, although some related work has appeared at NIPS in recent years, such as [47] and other work by the authors of [47]. The reference list and prior art sections appear to be adequate, though explicit pointers to a set of papers prototypical of the \Sigma H paradigm would be useful. The extension to multivariate information measures is an original advance in the context of prior literature, and highlights some defects in prior literature (lines 180-183).   The paper is well written overall. I feel like the technical content and style of this paper is more suited to a statistics or information theory venue, like IEEE Transactions on information theory, in line with many of the references presented, but is still within the purview of NIPS.   Figure 3 is useful in the context of describing the proposed estimator.   The presentation of numerical results in Figure 2 should include some confidence intervals for the estimates for better comparison of methods. The selection of experiments seems sufficient in Section 5, though calling your estimator GDM or something rather than "mixture" may make it a bit clearer.   Update: Re-scored in light of author's responses and reviews.


Learning with SGD and Random Features
Sketching and stochastic gradient methods are arguably the most common techniques to derive efficient large scale learning algorithms. In this paper, we investigate their application in the context of nonparametric statistical learning. More precisely, we study the estimator defined by stochastic gradient with mini batches and random features. The latter can be seen as form of nonlinear sketching and used to define approximate kernel methods. The considered estimator is not explicitly penalized/constrained and regularization is implicit. Indeed, our study highlights how different parameters, such as number of features, iterations, step-size and mini-batch size control the learning properties of the solutions. We do this by deriving optimal finite sample bounds, under standard assumptions. The obtained results are corroborated and illustrated by numerical experiments.

After Author Response:  Thanks to the authors for their various clarifications.  I have updated my score to an 8 accordingly.  I think the planned updates to the empirical section will add a lot of value. ============= Summary: This paper analyzes the generalization performance of models trained using mini-batch stochastic gradient methods with random features (eg, for kernel approximation), for regression tasks using the least squares loss. Their main theorem (Theorem 1) bounds the gap in generalization performance between the lowest risk model in the RKHS with the SGD trained model after t mini-batch updates; the bound is in terms of the learning rate, the mini-batch size, the number of random features M, the training set size n, and t.  They show that under certain choices of these parameters, M = O(sqrt(n)) features are sufficient to guarantee that the gap is <= 1/sqrt(n). This is the learning rate achieved by the exact kernel model, which has been proven to be optimal. This shows that instead of paying O(n^3) time and O(n^2) space to solve the exact kernel problem, one can spend O(dn * sqrt(n)) time and O(sqrt(n)d) space and solve the problem using random features and stochastic gradient descent.  These are significant savings.  The theorem also suggests that the learning rate should increase as the mini-batch size increases; this is validated empirically.  The paper also presents experiments showing that classification error saturates for M << n.  The work most relevant to this paper is the paper from NIPS of last year by Rudi and Rosasco: "Generalization Properties of Learning with Random Features" (I will refer to this paper as [RR17]). That paper proves a result quite similar to the one in this paper, but specifically for the kernel ridge regression estimator learned with random features (closed form solution, Equation 7 in that paper). The present work extends that result to the model learned using mini-batch SGD with random features. This is significant because in the large scale setting SGD is typically the method of choice. Solving for the KRR estimator requires storing the full random features covariance matrix in memory in order to invert it; this could be prohibitive when the number of random features is very large. Stochastic methods are thus better suited for the large-scale setting.  I will now discuss the quality, clarity, originality, and significance of this work: 1) Quality: Quality of the work is high. The paper solves a challenging theoretical question using sophisticated analysis.  The experiments are a weaker point of the paper.  Figure 1b is ok, showing how larger mini-batches should be paired with larger learning rates; though perhaps each column should be converted into a line on a plot. Figure 1a in my opinion could be presented a lot more clearly; I would prefer seeing plots on various datasets showing how performance varies as you increase the number of random features (after training has finished, for example). Showing results using color-scale for all 10 passes of the data is not a good use of space in my opinion.  2) Clarity: The work is presented quite clearly. For example, I appreciated how the corollaries explained how the various parameters could be set to achieve the desired rates.  I also liked how in lines 155-162 describes clearly what happens as you increase the batch size/learning rate.  3) Originality: This work is relatively original.  It extends recent results [RR17] to a new training setting.  I'm not certain how novel the proof techniques are.  4) Significance: I believe this work is an important step toward understanding the performance of models learned using random features with mini-batch SGD in general (the results in this paper only apply to least squares loss).  It appears that in practice learning models using SGD and random features is very effective; this work helps explain why.  This work also provides a way of understand how different SGD hyperparameters should be chosen together in order to achieve small excess risk --- this insight is relevant in practice.  Questions: -- Couldn't a result similar to the one in this paper follow pretty directly from the [RR17] result?  In particular, SGD on the regularized least squares objective should converge to the optimum at a certain rate; the optimum of this objective is precisely given by the closed form ridge regression solution, for which the [RR17] results presents generalization bounds. -- Could these results be extended to fastfood features? That would reduce the time to O(log(d) n * sqrt(n)) and the space to O(sqrt(n)).  NIT:  - Section 2.1:  'D' is used to denote dimension of data space X.  In section 2 (lines 68-73) 'd' is used. - Section 3.2: 'L' is never defined.  And in the equation for Cf(x), there should be an "f" before the "(x')\rho_X(x')". - Lines 188-189: "The smaller is \alpha the more stringent...".  I believe it should be: "The larger \alpha is, the more stringent..." - I don't think the acronym SGM is ever defined (stochastic gradient methods?) - Line 244: SMG should be SGM. - Lines 35-36 should cite [RR17]  [RR17] Alessandro Rudi, Lorenzo Rosasco: Generalization Properties of Learning with Random Features. NIPS 2017


Bandit Learning in Concave N -Person Games
This paper examines the long-run behavior of learning with bandit feedback in non-cooperative concave games. The bandit framework accounts for extremely lowinformation environments where the agents may not even know they are playing a game; as such, the agents' most sensible choice in this setting would be to employ a no-regret learning algorithm. In general, this does not mean that the players' behavior stabilizes in the long run: no-regret learning may lead to cycles, even with perfect gradient information. However, if a standard monotonicity condition is satisfied, our analysis shows that no-regret learning based on mirror descent with bandit feedback converges to Nash equilibrium with probability 1. We also derive an upper bound for the convergence rate of the process that nearly matches the best attainable rate for single-agent bandit stochastic optimization.

This paper presents a learning algorithm for participants in a strongly monotone convex game, such that if all participants use this learning algorithm, they converge to the unique Nash equilibrium of this game at a rate of O(n^{-1/3}) (i.e. at round n, each player is within O(n^{-1/3}) of their Nash equilibrium strategy). This extends a long line of work of understanding what sorts of equilibria learning strategies can/will converge to when used to play games (e.g. if everyone uses a low-regret strategy, the time-averaged play converges to a coarse correlated equilibrium).   The learning algorithm they propose can be thought of as a modification of online mirror descent. The main technical difficulty here is that online mirror descent is a first-order method, requiring the gradient of the loss function at the queried points, but in this setting only zeroth-order feedback (the loss from performing some action) is provided. Moreover, unlike other settings where it is possible to approximate the gradient of some point by zeroth-order feedback of several points (even just two) in the neighborhood of this point, here each player only has control over their own action (and thus cannot guarantee they can query multiple closeby points in the overall action space).   The trick the authors use to overcome this difficulty is called "simultaneous perturbation stochastic approximation", which allows one to get an "estimate" of a gradient at a point via a single zeroth-order query. The main idea is if one would like to query the gradient at some point x in R^d, they instead choose a random direction z_i in the unit d-sphere, and evaluate the function at x + delta*z_i. It can then be shown that f(x+delta*z_i)*d*z_i/delta is an unbiased estimator of the gradient of f~ at x, where f~ is some smoothed approximation to f. If f and its gradients are Lipschitz continuous, f~ gets closer and closer to f as delta goes to 0 (at the cost of the variance of this estimator exploding). Choosing delta to be something like n^{-1/3} optimizes this bias-variance tradeoff and leads to a convergence rate of O(n^{-1/3}).   I think this is a strong result. Monotone concave games are a large class of interesting games, and a natural setting for this sort of question given their unique equilibrium structure. I really liked the application of the perturbation trick, which has appeared in the literature before, but which I had not seen and is applied nicely here. I have not checked the technical details in the supplemental material carefully, but the proof sketch makes sense to me.   From a game theory perspective, it would be nice to have some slightly broader characterization of learning algorithms such that, if all players are running some learning algorithm in this class, then they converge to the Nash equilibrium of this game (at some convergence rate). This is a good first step in this direction, showing that there is some simple learning algorithm all learners can run to converge to a Nash equilbirium, but for the purpose of understanding how equilibria are achieved, it is unlikely that all players are running exactly this variant of mirror descent. I think it would be a very strong result if you could somehow show that if they are all using any sort of learning algorithm with "low X regret" (for some type of X regret), this convergence follows (such guarantees exist for convergence to CCE in general repeated games).   The paper was written exceptionally clearly -- as someone who does not work closely with mirror descent, I really appreciated the detailed preliminaries in section 2 and 3. I recommend this paper for acceptance.


Phase Retrieval Under a Generative Prior
We introduce a novel deep learning inspired formulation of the phase retrieval problem, which asks to recover a signal y 0 2 R n from m quadratic observations, under structural assumptions on the underlying signal. As is common in many imaging problems, previous methodologies have considered natural signals as being sparse with respect to a known basis, resulting in the decision to enforce a generic sparsity prior. However, these methods for phase retrieval have encountered possibly fundamental limitations, as no computationally efficient algorithm for sparse phase retrieval has been proven to succeed with fewer than O(k 2 log n) generic measurements, which is larger than the theoretical optimum of O(k log n). In this paper, we propose a new framework for phase retrieval by modeling natural signals as being in the range of a deep generative neural network G : R k ! R n . We introduce an empirical risk formulation that has favorable global geometry for gradient methods, as soon as m = O(kd 2 log n), under the model of a d-layer fully-connected neural network with random weights. Specifically, when suitable deterministic conditions on the generator and measurement matrix are met, we construct a descent direction for any point outside of a small neighborhood around the true k-dimensional latent code and a negative multiple thereof. This formulation for structured phase retrieval thus benefits from two effects: generative priors can more tightly represent natural signals than sparsity priors, and this empirical risk formulation can exploit those generative priors at an information theoretically optimal sample complexity, unlike for a sparsity prior. We corroborate these results with experiments showing that exploiting generative models in phase retrieval tasks outperforms both sparse and general phase retrieval methods.

This paper considers the sparse phase retrieval (SPR) problem with a generative prior. Specifically, the signal of interest y_0 is assumed to be generated as y_0 = G(x_0), where G is a known neural network and x_0 is an underlying low-dimensional generating seed. Given phaseless observation of the form b = |AG(x_0)|, the task is to recover x_0.   This paper focuses on a nonsmooth least-squares formulation for the problem   min 1/2* || |AG(x)| - |AG(x_0)| ||^2.   The paper shows when the weights of the generating neural network are Gaussian random and the activation function is taken as ReLU, with enough number of measurement (linear in the dimension of the seed vector ignoring dependency on other factors), the least-squares objective has a ``nice'' global landscape. A gradient descent style algorithm is designed to exploit the particular geometry of the objective.   The problem considered is novel. Linear measurement models with deep generative priors have been considered in Refs [4][17]. There is no previous work on the miss-phase problem yet.    The geometric result is interesting. Particularly, it implies that no spurious local mins exist for the least-squares objective, except for the global min that is desired and a scaled version of the global min. The proposed numerical algorithm seems simple enough to be practical.   My major concern is about the technical message the paper is trying to send. To make the point precise, I quote their text below Theorem 1 (Cf. Theorem 2) as follows:   "Our main result asserts that the objective function does not have any spurious local minima or saddle points away from neighborhoods of the true solution and a negative multiple of it. Hence if one were to solve (1) via gradient descent and the algorithm converged, the final iterate would be close to the true solution or a negative multiple of it."   1) Theorem 2 shows outside the two small neighborhoods around the global min and the scaled min, there is always a descent direction v in the sense that the one-sided directional directional derivative defined as   D_v(f) = lim_{t \to 0^+} [f(x +tv) - f(x)]/t   is negative. This is sufficient for ruling out presence of other local min, but not saddle points. To see this, consider the simple one dimensional function f(x, y) = -|x| + y^2. The one-sided directional derivatives at (0, 0) along both (1, 0) and (-1, 0) direction are -1, but (0, 0) is a stationary point and in fact it is a saddle point. So without further justification, the comment that there are no saddle points seems to be misplaced. To examine presence/absence of saddle points, it seems Clarke subdifferential might be needed.   2) In Fig 1, it may be better to adjust the viewpoint so that more details in the valley can be revealed and also the contour plot can be easily seen. I suspect there are saddle points along the valley connecting the two mins. Please help to confirm/disprove this.   3) If there were saddle points, how the gradient descent method described in Algorithm 1, which is first-order in nature, effectively escapes from the saddle point regions?   4) In theory, it is possible the current iterate is a nondifferentiable point. Algorithm 1 does not specify how to handle such cases.    Some additional comments below:  * Theorems 1 & 2: It is curious to know what happens within the two small neighborhoods not covered by the current results. As discussed in the paper, this might not matter in practice as \eps can be set to be some small constants so that finding any point would be acceptable numerically. However, this is polynomial dependency of the sample complexity on 1/\eps. Are these fundamental reasons for leaving out the small neighborhoods, or is it just a matter of technicality? I suspect some of the ideas in    -- Soltanolkotabi, Mahdi. "Structured signal recovery from quadratic measurements: Breaking sample complexity barriers via nonconvex optimization." arXiv preprint arXiv:1702.06175 (2017).    might be useful.   * Line 88: Under the Gaussian design model, PhaseLift can recover n-d signals with O(n) measurements due to the refined analysis in    -- Candès, Emmanuel J., and Xiaodong Li. "Solving quadratic equations via PhaseLift when there are about as many equations as unknowns." Foundations of Computational Mathematics 14.5 (2014): 1017-1026.   * Around Eq (4): In view of the superficial similarity of the formulation in Eq (4) and the current problem in Eq (1), and also the similarity in terms of the geometric result. It might be better to comment on the major innovation on the technical side in proving the current results, compared to Ref [17].   * Theorem 2: I guess here d is treated as a universal constant. Then, the discussion of \rho_d when d tends to infinity seems a bit irrelevant. Also both C and c depend on \eps, which in return is constrained by d due to the restriction that K_1 d^8 \eps^(1/4) \le 1. In the sample complexity m, since the dependency on d is explicitly stated, I think it might be more appropriate to account for the hidden dependency of c on d also.   * Sec 3.1: It is stated that "Such a direction exists by the piecewise linearity of the generative model G". I think probably this needs to be elaborated and possibly be shown rigorously.   * Exp 4.2: What happens when A consists of Fourier measurements, i.e., corresponding to the classic phase retrieval problem that is of concern in practice? Does the generative modeling offer any conceptual and practical advantage in this more practical setting?


Adversarial vulnerability for any classifier
Despite achieving impressive performance, state-of-the-art classifiers remain highly vulnerable to small, imperceptible, adversarial perturbations. This vulnerability has proven empirically to be very intricate to address. In this paper, we study the phenomenon of adversarial perturbations under the assumption that the data is generated with a smooth generative model. We derive fundamental upper bounds on the robustness to perturbations of any classification function, and prove the existence of adversarial perturbations that transfer well across different classifiers with small risk. Our analysis of the robustness also provides insights onto key properties of generative models, such as their smoothness and dimensionality of latent space. We conclude with numerical experimental results showing that our bounds provide informative baselines to the maximal achievable robustness on several datasets.

Description  The work theoretically studies the question of robustness of classifiers (in particular NNs) to adversarial attacks. The main result is an upper bound on such robustness of classifiers derived under certain assumptions. It is very general in the sense that the upper bound applies to all classifiers not necessarily NNs. The bound depends on the classification boundaries (in particular the number of classes) and on how “smooth” the distribution of data (images to classify) is. The bound can be computed numerically when the generative model is known. Authors propose experiments with DCGAN generated images and several classification networks architectures. The bound appears rather tight / useful in these experiments. There is an interesting discussion of the consequences.  Evaluation I thank the authors for their work. It is of high quality, and mostly clear and technically accurate. Below I question some of the assumptions / conclusions, which determines the degree of relevance of the work to the practical problems and phenomena of (in)stability of NNs. Specifically, 1) classifying all points of a smooth generative model may  be an inappropriate task formulation and 2) conclusions about the dependence on the dimensionality d are unclear or possibly incorrect. My current impression is that the general bound and further results: the relation between in-distribution and unconstrained robustness and transferability are derived under the definition of robustness which is not fully practically relevant. I am not certain about utility of these results, but the paper is definitely a significant contribution and a worthy publication.  Questions  Q1. The work starts with the motivation that NNs are vulnerable to adversarial attacks. The work also arrives at the conclusion that robustness of any classifier is rather limited, which correlates with the above motivation but may be unrelated to it. The derived theory is not specific to NNs, and thus cannot help to find out what is wrong with NNs specifically. The fact that a significant portion of data points admit small norm perturbations is coming from the combination of assumptions on smoothness and that we wish to classify the whole distribution, which I think is not realistic. Let me give an example: consider a generative model of human faces. A convex combination of latent variables corresponding to faces of two existing persons would generate a likely face (even more probable due to the normal distribution prior). It can be easily misclassified as either of the two persons. The paper builds on the fact that the fraction of such points that are likely to be generated and are ambiguous to classify is relatively large. The assumptions are thus not fully realistic together: either we need to consider a “real” face distribution of existing people (not smooth) or request the classification of the relevant subset only. I think the obtained result is unrelated to vulnerability of NNs, in particular they are tested with real distributions.   Q2. The dependence on dimensionality d is unclear. In the related work section 2, d denotes the dimensionality o the data space (=images). In the formal part, it is the dimensionality of the latent space. It is specifically unclear at line 124: “||x|| scales as \sqrt(d)$. I do not see why this would be true for $d$ being latent dimension. On the other hand: line 124: “assuming L is constant” is unclear. For example, if one increases the data dimension by upsampling the generated images, $L$ grows proportionally. In another place the paper mentions that the input data is considered to be normalized (thus ||x|| would have to be constant). It seems that all these assumptions cannot be used simultaneously. Please clarify.  Discussion It seems to me that the work is related to classical results establishing upper bounds on the classification accuracy. The reason for a general bound would be the same: the confusion of the data distribution. The bound is relevant when this becomes a prevailing problem (I believe this is the case in the experiments). Otherwise, other sources of classification errors / instability prevail.   Minor:  The idea of stability “under generative model” was not clear until section 3 to me, maybe could be explained earlier.  Final remarks I still have the concern of relevance. The authors argue that existing generators are producing images “from the training distribution”. This does not convince me since the prior distribution is Gaussian and therefore all latent states in the convex hull of latent representations of the training data are likely. They will unavoidably generate images that are inbetween two or more classes, confusing even for human. For example a face between man and woman. Small perturbations easily move them over the imaginary classification boundary. Imaginary is because these intermediate faces are not in the real data distribution. As another example consider the classification by a feature with overlapping conditional Gaussian distributions. Even the best Bayesian classifier will have a non-zero error. The submission shows a bound on stability related to such data overlap. This is different from sensitivity of NNs to small input perturbations in the case when training set is disconnected, consisting of several clusters. Furthermore, the paper studies experimentally networks trains for generation only. A more appropriate case for study would be a generator-discriminator pair such as semi-supervised variational autoencoders


Ex ante coordination and collusion in zero-sum multi-player extensive-form games
Recent milestones in equilibrium computation, such as the success of Libratus, show that it is possible to compute strong solutions to two-player zero-sum games in theory and practice. This is not the case for games with more than two players, which remain one of the main open challenges in computational game theory. This paper focuses on zero-sum games where a team of players faces an opponent, as is the case, for example, in Bridge, collusion in poker, and many non-recreational applications such as war, where the colluders do not have time or means of communicating during battle, collusion in bidding, where communication during the auction is illegal, and coordinated swindling in public. The possibility for the team members to communicate before game play-that is, coordinate their strategies ex ante-makes the use of behavioral strategies unsatisfactory. The reasons for this are closely related to the fact that the team can be represented as a single player with imperfect recall. We propose a new game representation, the realization form, that generalizes the sequence form but can also be applied to imperfect-recall games. Then, we use it to derive an auxiliary game that is equivalent to the original one. It provides a sound way to map the problem of finding an optimal ex-antecoordinated strategy for the team to the well-understood Nash equilibrium-finding problem in a (larger) two-player zero-sum perfect-recall game. By reasoning over the auxiliary game, we devise an anytime algorithm, fictitious team-play, that is guaranteed to converge to an optimal coordinated strategy for the team against an optimal opponent, and that is dramatically faster than the prior state-of-the-art algorithm for this problem.

The paper studies the problem of multi-player extensive-form games where players form teams. Within a team, players share the common payoff and the paper focuses on the zero-sum setting in the sense that two teams of players are playing against each other and the sum of utilities of both teams equals zero for each terminal state. Solving such games is challenging since they can be either modeled as multi player games with perfect recall, or as a two player game but with imperfect recall.   The authors argue that within such setting, it is reasonable to consider solution concepts where the players of a team are able to correlate their strategies -- i.e., to agree in a advance on a probability distribution from which they sample their pure strategies. Under this assumption, the authors show that the range of expected utilities can be represented as a convex set and an auxiliary perfect recall game of an exponential size can be constructed, solving which leads to the solution of the original game.   The authors propose a novel algorithm based on Fictitious Play in extensive-form games and experimentally demonstrate the performance on Kuhn and Leduc poker games with varying number of cards.   Overall, the idea and the technical results seem interesting, however, I had quite some difficulties to understand why the realization form is not a straightforward consequence of the choice of the authors focus on normal-form strategies -- I would suggest to include a more complex example instead of Figure 1, namely one, where restricting to normal-form strategies does not directly lead to a perfect recall game.  There are two concerns about the paper. First, it is difficult to agree with the statement that correlated strategies are the right choice for team games, especially due to the necessity of correlating before each game (or sampling the correlated strategies several times in advance, once for each play of 1 game). On the other hand, if you assume correlation via some actions during the play, then it is assumed that players have observable actions which limits the applicability (and can actually simplify the technical results). Such a correlation can make sense in other multi-agent adversarial scenarios (a team is fighting adversarial team). However, even in these tactical scenarios, the correlation with respect to receiving signals in each information set (instead of receiving one signal) seems more natural as argued by von Stengel and Forges when introducing Extensive-Form Correlated Equilibrium. Could the authors comment of that? Is it truly reasonable to focus on computing correlation over pure strategies in these games? And if so, do authors actually compute a correlated equilibrium in these games? How does TMECor differ from standard correlated equilibrium (not EFCE) in these games?  Second, the authors omit the recent algorithms for computing behavioral strategies in imperfect recall games (namely works by Cermak et al. "Combining Incremental Strategy Generation and Branch and Bound Search for Computing Maxmin Strategies in Imperfect Recall Games" and "Approximating maxmin strategies in imperfect recall games using A-loss recall property"). These works are computing optimal strategies in behavioral strategies in imperfect recall games -- that, as the authors state, leads to a non-linear, non-convex optimization problem. At the same time, the scalability of the existing algorithms reported in these prior works outperforms the running time of the algorithm in this paper by several orders of magnitude (number of sequences is 10^6 and higher solvable within hours in the works by Cermak et al., compared to 10^2 in this submission). Since one of the positive results of this submission is to simplify the strategy space and offer better and more scalable algorithms, it seems that current experimental results do not strongly support this claim. A natural question arises -- can the techniques and algorithms for computing behavioral strategies in imperfect recall developed by Cermak et al. be adopted for computing the solution concept the authors propose (technically, it is a bilinear mathematical program, so it actually can be simplified)? If not, why?  After rebuttal:  I am not that satisfied with the response of the authors since several questions were left unanswered.   1. I am aware that ex-ante correlation does not lead to a perfect recall game. I was merely suggesting the authors to use a more complex example in the paper so that it is clearly stated in the paper.  2. I am not convinced that ex-ante correlation is the right choice, however, this my subjective opinion and I admit that ex-ante correlation has its place in the literature/research.  The authors did not respond to my question regarding TMECor -- it seems to me that TMECor is a combination of two standard correlated equilibria played within the members of a team, joined in a zero-sum competition. If this is true, I am wondering whether there is some connection to the property that the realization form is a convex polyhedron -- e.g., does this mean that any CE in an imperfect recall game can be represented using the same strategy space?  3. The problem with scalability is as follows -- the authors suggest that they use TME for conceptual simplicity that should be positively reflected in the scalability. However, this is not currently the case, since the cited papers are able to solve larger games in behavioral strategies (that have all the negative aspects the authors state -- non-existence of Nash equilibria, non-linearity, non-convexity, etc.) compared to the current experiments in the submission. My question was, whether the advanced techniques that lead to this better scalability in the more complex strategy space can be modified to the proposed solution concept (i.e., not to directly apply cited algorithms -- which is what the authors responded to).    Overall, there are some concerns over the significance of the paper (connections to the standard solution concepts, scalability), however, there is some contribution and I can imagine that the paper can be helpful for other researchers in a follow-up work.


Occam's razor is insufficient to infer the preferences of irrational agents
Inverse reinforcement learning (IRL) attempts to infer human rewards or preferences from observed behavior. Since human planning systematically deviates from rationality, several approaches have been tried to account for specific human shortcomings. However, the general problem of inferring the reward function of an agent of unknown rationality has received little attention. Unlike the well-known ambiguity problems in IRL, this one is practically relevant but cannot be resolved by observing the agent's policy in enough environments. This paper shows (1) that a No Free Lunch result implies it is impossible to uniquely decompose a policy into a planning algorithm and reward function, and (2) that even with a reasonable simplicity prior/Occam's razor on the set of decompositions, we cannot distinguish between the true decomposition and others that lead to high regret. To address this, we need simple 'normative' assumptions, which cannot be deduced exclusively from observations.

This paper presents a novel and important insight in value alignment: that the No Free Lunch theorem for inferring human preferences cannot be alleviated using simplicity priors. It also claims to introduce this theorem, which was however already proved in Everitt et al, 2017 (as Theorem 11). The paper states that "A No Free Lunch result analogous to theirs applies in our case" (section 4.1), but it's not clear to me in what sense this result is analogous rather than identical (since the theorem statement and assumptions are exactly the same). This paper presents a different proof for the No Free Lunch theorem - please make clear how this proof is an improvement on the proof in the Everitt et al paper.  The presentation of the main insight (that simplicity priors don't work) has some issues as well. The argument relies on a key assumption that B(p-dot, R-dot) is complex even given human policy pi-dot (in section 5.2 on informal complexity results), which is made precise as l_L(pi-dot, B(p-dot, R-dot)) > l_L(pi-dot) + 5C (in section 5.4 on formal complexity results). This is a strong assumption that is not justified in the paper, and the coefficient 5 for the constant C is chosen in a seemingly contrived way to make the proof of Theorem 8 work out. The larger the constant C, the stronger this assumption is, and the constant C could be large, since it needs to simultaneously satisfy all the inequalities in Theorem 7. For the same reason, the result of Theorem 7 is not very impressive: the three pairs of interest are among the 2^(-K) N > 2^(l_L(pi-dot) + 4C) simplest pairs in the language L, which is exponential in C and thus is very large if C is large. The paper should provide some upper bounds on the value of C to make these results more meaningful. I expect the central claim of the paper probably still holds, and the informal argument in section 5.2 makes sense, but it needs more rigorous support.  The paper is clearly written and well-motivated, though a bit unpolished: - There are some incorrect theorem labels: Corollary 4 refers to theorems 4.1 and 4.2, which should be theorems 1 and 3. - Some terms are not defined before they are used, e.g. "simple function". - The conclusion section ends abruptly and seems unfinished.  The paper includes a thorough literature review and makes clear how this work relates to previous works and how this approach is more general than previous approaches to the problem. It opens up new avenues for research into normative assumptions for deducing human preferences from behavior.   UPDATE after reading author feedback:  - The explanation of the assumption that B(p-dot, R-dot) is complex given the human policy seems convincing, so I'm happy to increase my score. - I still think there is not much novelty in the NFL theorem. The planning function in this setting is analogous to the reward corruption function in the Everitt et al (2017) paper. The Everitt paper's NFL theorem shows that if we don't assume anything about the reward corruption function, the worst-case regret of any policy is high. This paper's NFL theorem analogously shows that if we don't assume anything about the planning function, the worst-case regret of any policy is high. The statement in the author feedback that the setting in the Everitt paper is "unrelated to reward inference" does not seem right, since that paper is about the agent's ability to infer the true reward function. Thus, I don't think it took much work to transfer the NFL theorem to the preference inference setting.  Overall, I would recommend that the authors put less emphasis on the NFL theorem and more emphasis on the "Simplicity does not help" section, since that is the main novel and impactful contribution of the paper.


Generalisation in humans and deep neural networks
We compare the robustness of humans and current convolutional deep neural networks (DNNs) on object recognition under twelve different types of image degradations. First, using three well known DNNs  GoogLeNet)   we find the human visual system to be more robust to nearly all of the tested image manipulations, and we observe progressively diverging classification error-patterns between humans and DNNs when the signal gets weaker. Secondly, we show that DNNs trained directly on distorted images consistently surpass human performance on the exact distortion types they were trained on, yet they display extremely poor generalisation abilities when tested on other distortion types. For example, training on salt-and-pepper noise does not imply robustness on uniform white noise and vice versa. Thus, changes in the noise distribution between training and testing constitutes a crucial challenge to deep learning vision systems that can be systematically addressed in a lifelong machine learning approach. Our new dataset consisting of 83K carefully measured human psychophysical trials provide a useful reference for lifelong robustness against image degradations set by the human visual system.

Update after rebuttal:  This is a very interesting and well executed paper. A large number of studies have said that DNNs yield performance that resembles that of humans and animals, but this study provides a more fine-grained comparison of the behavior and finds real differences. In particular, the study probes sensitivity to perturbed inputs, something that current DNNs are much less capable at than humans. This provides a valuable dataset and benchmark to aim for, and I expect the ML community will have algorithms that perform far better in short order.   I would encourage the authors to make as fine-grained and specific comparisons of similarities and differences as possible; to suggest potential reasons why these discrepancies might exist (clearly marking these reasons as somewhat speculative if they aren't backed by direct evidence); and to expand on potential routes toward better algorithms (again marking this as speculative if it is).  _________  Summary:  This paper probes the generalization abilities of deep networks and human observers, by undertaking an extensive psychophysical investigation of human performance on ImageNet in the presence of strong distortions of the input images. The results point to a large gap between DNNs (which can perform well when trained on a specific distortion) and human observers (who can perform well across all distortions). The paper releases the psychophysical dataset and image distortions to aid future research.  Major comments:  This paper undertakes an impressively controlled comparison of human performance and DNNs at a behavioral level. While previous work has shown broad similarities between DNNs and human behavior, this evaluation undertakes a much more challenging test of generalization abilities by considering extreme distortions. Additionally, in contrast to prior work which relied on Amazon Mechanical Turk to generate data, the present submission uses carefully controlled laboratory viewing conditions that remove some prior concerns (such as variation in timing, viewing distance, and how attentive AMT subjects might be). Overall this paper contributes a rigorous psychophysical dataset that could spur a variety of follow up work in the field.  The most striking deviation between human subjects and the DNNs investigated here is in the entropy of the response distribution. The DNNs apparently become stuck, responding to all distorted images with a single class. On the one hand this is indicative of a clear failure of the DNNs in their vanilla form. On the other hand, one could imagine simple procedures to improve the situation. For instance, one could adjust the temperature parameter of the final softmax to attain equal entropy to human subjects, and assume responses are sampled from this. It is well known that the final softmax layers are not calibrated probabilities, so this may be a fairer comparison. Another route would be through explicitly calibrating the probability distribution in the final layer, eg through dropout [75].


Neighbourhood Consensus Networks
We address the problem of finding reliable dense correspondences between a pair of images. This is a challenging task due to strong appearance differences between the corresponding scene elements and ambiguities generated by repetitive patterns. The contributions of this work are threefold. First, inspired by the classic idea of disambiguating feature matches using semi-local constraints, we develop an end-to-end trainable convolutional neural network architecture that identifies sets of spatially consistent matches by analyzing neighbourhood consensus patterns in the 4D space of all possible correspondences between a pair of images without the need for a global geometric model. Second, we demonstrate that the model can be trained effectively from weak supervision in the form of matching and non-matching image pairs without the need for costly manual annotation of point to point correspondences. Third, we show the proposed neighbourhood consensus network can be applied to a range of matching tasks including both category-and instance-level matching, obtaining the state-of-the-art results on the PF Pascal dataset and the InLoc indoor visual localization benchmark.

Paper overview: The authors propose a novel, end-to-end trainable deep architecture for finding dense correspondences between pairs of images. First, dense features are extracted from both images using a 2D CNN. Then, an exhaustive cross-correlation is performed between the feature maps to construct a 4D matching hyper-volume, which is fed to a "neighbourhood consensus network". This is, in essence, a 4D convolutional architecture which evaluates spatially local subsets of correspondences in order to detect latent consensus patterns, in a way reminiscent of traditional semi-local matching approaches. The final output of the network is a probability distribution over the set of dense matches between the images. The network is trained in a weakly-supervised manner, by processing pairs of matching / non matching images, without any explicit annotation on correspondences.  Comments: The paper is very well written, clear and easy to follow. The authors manage to convincingly communicate the intuition motivating their approach, and the technical sections are easy to understand. The problem being tackled is a fundamental one, and the proposed approach is interesting and novel (but see question #2 below). The experimental evaluation is quite convincing, even if I would have liked to see some quantitative results on the accuracy of instance-level matching, in addition to the final localization results when the proposed method is used as a component of InLoc. Over all this is a very good submission, but there are two main points I would like the authors to clarify:  1) As far as I can understand, the weakly-supervised loss described in section 3.5 doesn't impose any kind of "physically meaningful" constraint on the matches. The network could learn to select _any_ set of correspondences, as long as the soft and hard assignments for the positive image pairs are consistent with each other, even if these correspondences do not make any sense given the content of the two images. In essence, what is being optimized here is close to an entropy loss over the matching probability distributions. Is this the case? Have you ever observed any "drifting" behavior during training that could be due to this?  2) The proposed approach shares some similarities with recent works on stereo matching such as [1], i.e. it works by processing a matching cost volume through a CNN. Can the authors comment on this?  [1] Kendall, Alex, et al. "End-to-end learning of geometry and context for deep stereo regression." CoRR, vol. abs/1703.04309 (2017)


Removing Hidden Confounding by Experimental Grounding
Observational data is increasingly used as a means for making individual-level causal predictions and intervention recommendations. The foremost challenge of causal inference from observational data is hidden confounding, whose presence cannot be tested in data and can invalidate any causal conclusion. Experimental data does not suffer from confounding but is usually limited in both scope and scale. We introduce a novel method of using limited experimental data to correct the hidden confounding in causal effect models trained on larger observational data, even if the observational data does not fully overlap with the experimental data. Our method makes strictly weaker assumptions than existing approaches, and we prove conditions under which it yields a consistent estimator. We demonstrate our method's efficacy using real-world data from a large educational experiment.

Summary: This paper proposes a new method for leveraging experimental data to remove the bias arising from potential confounders in a related observational study. I find the key idea of the paper to be simple but clever, interesting, and potentially useful. The simulation based on real data (Section 5.2) is in my opinion quite convincing. On the other hand, I find the general exposition to be imprecise and difficult to read. I think that this manuscript has the substance to be a good paper if major work is undertaken to tighten it up and clarify the exposition. Specific comments below:   Substance:  1) I believe that the method you propose is sensible, but I am not yet convinced by its practical significance. It would be helpful if you could point to a number of real world datasets on which such a method could be used. I am not asking you to actually apply your method to other datasets, nor am I asking you to point to datasets you have access to — what I am asking is whether there are any concrete datasets you know of that could benefit from your method?  2) There is a general sense in which when estimating causal effects in observational studies, bias is often more of an issue than variance. I agree with that view, overall. Still, it would be useful if you could briefly discuss the variance of your estimator, and propose a way to estimate that variance. Proving a CLT (or if you can’t prove it, at least argue heuristically for it) would be also helpful for obtaining confidence intervals, etc…  3) The precision of your estimator $\hat{\theta}$ is limited by the sample size of the UNCONF dataset. Now if you look at the mean squared error of the estimator $\hat{\tau}$ that you construct, it is quite possible that because of the potential large variance of $\hat{\theta}$, it exceeds that of a naive estimator that will be biased due to confounding but will have much lower variance thanks to the large sample size of CONF. In effect, I am pointing to the fact that there is a bias / variance tradeoff at play here, in the choice between your estimator, and a naive estimator that assumes unconfoundedness. It would be useful to discuss this point.  4) In the introduction you write: “We discuss below why this [parametric correction] assumption is possibly reasonable […]”. I think that the insight from your paper — pushing the burden of extrapolation to the confounding term — is a very good one. You discuss this briefly on lines 156-159, but I think that you should emphasize this more.   Clarity:  I would usually include these remarks as minor comments, but in this particular case they are not minor. I am taking the time to be specific because I really believe that the paper could be improved dramatically if the writing was tighter.  1) The language you use is very imprecise:  - l.68: what do you mean by “scale” and “scope”?  - l.69: you talk about unconfoundedness, but you only define it on l.91. The sentence on l. 90: “the key assumption we make about the unconfounded data is its unconfoundedness” should alert you to the fact that something is wrong.  - The example in l.110-119 is confusing. What you assume is that dutiful patients are more likely to seek treatment. You should be explicit about this. As an aside, it is a matter of personal preference but you could consider wrapping this into an “example” environment to set it apart from the main text.  - l.131: \omega is not an estimator, and unbiased is not the right word for it.   - l 132: “[…] which interpolates between \hat{omega} to the RCT outcomes” what does this sentence mean?  - l. 238-242 is confusing. Why do you superscript Y by CATE?  - l. 248: you mean that you are sampling a *fraction* q’ of the rural…  etc…. Please go over your paper carefully eliminating these kinds of things.  2) l.139-144 should be a lemma. The proof does not add to the comprehension.   3) Please consider not using superscripts like “Conf” and “Unc”. Single letters “C” and “U” would do. Also, using the letter E is usually not a good idea when you take expectations (even if you use a slightly different symbol for expectations).  4) Typos:  - l.40: “this observational sample which might suffer […]” -> remove the word “which”.  - l.131 and l.132: its \omega not omega  - l.138: “doing so would be difficuly” -> difficult  - l 139: “[…] relies on a simply identity” -> simple etc…


Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
Semi-supervised learning (SSL) provides a powerful framework for leveraging unlabeled data when labels are limited or expensive to obtain. SSL algorithms based on deep neural networks have recently proven successful on standard benchmark tasks. However, we argue that these benchmarks fail to address many issues that SSL algorithms would face in real-world applications. After creating a unified reimplementation of various widely-used SSL techniques, we test them in a suite of experiments designed to address these issues. We find that the performance of simple baselines which do not use unlabeled data is often underreported, SSL methods differ in sensitivity to the amount of labeled and unlabeled data, and performance can degrade substantially when the unlabeled dataset contains out-ofdistribution examples. To help guide SSL research towards real-world applicability, we make our unified reimplemention and evaluation platform publicly available.

This paper proposes a systematic evaluation of SSL methods, studies the pitfalls of current approaches to evaluation, and, conducts experiments to show the impact of rigorous validation on kinds of conclusions we can draw from these methods.  I really like the paper and read it when it appeared on arXiv back in April. In many places we are lacking these kind of systematic approaches to robust evaluations and it's refreshing to see more of these papers emerging that question the foundation of our validation methodologies and provide a coherent evaluation.  Suggestions for improvements:  - The paper mainly deals with two image categorisation datasets. While these methods have been studied in many recent SSL papers, they also have their own limitations, some of which is mentioned in the paper. But the main problem is that it restricts them to a single domain which is image categorisation. It would have been interesting if we could see how these methods behave on other domains (e.g. NLP tasks, Audio/Speech, ...). - In terms of SSL methods, it would have been interesting to study an instance of graph/manifold based methods (e.g. "Deep Learning via Semi-Supervised Embedding", Weston et al, ICML 2008) as that would've completed the spectrum of traditional assumptions/approaches to SSL (density/margin based models (EntMin/TSVM), self-training (pseudo-labelling), co-training(might not be as applicable as you don't have data with multiple views), manifold based models (LapSVM/SemiEmb)). - Following from this, I would actually suggest that self-training should be also one of the default baselines as it's so trivial to implement that it basically is a good baseline to have around for any task. While pseudo-labelling paper [34] seems like doing that I'm not sure if the implementation details from the paper were followed (e.g. only using pseudo-labelling as a fine-tuning step or using a denoising AE) or you really performed a more classical self-training. - In Sec 4.2 where you improve the performance of the supervised model by utilising [16, 21, 11] and obtain a performance that's comparable to those of SSL with the baseline method, while you acknowledge that it's unfair to make the comparison, but what is preventing you from actually applying SSL on this highly optimised model and see how far down the test error goes? In other words, do SSL methods provide model-invariant benefit or we will only see improvements if we just used the baseline model? - In Sec 4.3, would be interesting to see if you removed the CIFAR-10 classes from ImageNet and re-ran the experiments what you would get out? Does your SVHN conclusion in this section generalises to CIFAR-10? - In Sec 4.5, Although outside of the scope of the paper do you have any intuition why in Fig 4b the performance of VAT is actually very good across different number of labelled samples? - In Sec 4.6, I would suggest to actually do experiments by restricting the size of the validation set to as you say 10% or so of the training set (or perform cross-validation) and report performance on the test set based on hyper-parameter optimisation on the smaller dev set.


Nonparametric Density Estimation with Adversarial Losses
We study minimax convergence rates of nonparametric density estimation under a large class of loss functions called "adversarial losses", which, besides classical L p losses, includes maximum mean discrepancy (MMD), Wasserstein distance, and total variation distance. These losses are closely related to the losses encoded by discriminator networks in generative adversarial networks (GANs). In a general framework, we study how the choice of loss and the assumed smoothness of the underlying density together determine the minimax rate. We also discuss implications for training GANs based on deep ReLU networks, and more general connections to learning implicit generative models in a minimax statistical sense.
under traditional statistical smoothness assumptions, we identify (i.e., prove matching upper and lower bounds on) minimax convergence rates for density estimation under several losses of the form (1). We also discuss some consequences this has for particular neural network implementations of GANs based on these losses. Finally, we study connections between minimax rates for explicit and implicit generative modeling, under a plausible notion of risk for implicit generative models.

Overview: This paper looks at nonparametric density estimation under certain classes of metrics, which the authors call "adversarial losses". To define adversarial losses, assume we have two probability distributions P and Q, and suppose X~P and Y~Q. Consider the supremum of |E[f(X)] - E[f(Y)]|, when f ranges over a class F_d of functions. This is the "adversarial loss with respect to class F_d", or simply the "F_d-metric", and generalizes several metrics, for instance the L_1 metric. Now, assume P is an unkown distribution belonging to some known class F_G, and we have n i.i.d. samples from P, and want to output an estimate another distribution Q in F_G so that the F_d-distance between P and Q is minimized. How small can we make this distance? Theorem 1 in this paper gives an upper bound for this question, and Theorem 3 gives a lower bound. Both bounds make smoothness assumptions about functions in F_d and F_G, and the bounds are tight for certain Sobolev spaces (Example 4). The upper bound is proved via an orthogonal series  estimator.  The motivation for this metric comes from generative adversarial networks (GANs), where the class F_d corresponds to functions that can be realized by a neural network with a certain architecture (the discriminator). In Theorem 7, the authors prove a certain performance guarantee for GANs (assuming perfect optimization), which improves the recent paper [25].  Finally, the paper compares the sample complexity of "explicit density estimation" as defined above, where the goal is to output a description of a distribution Q that is close to P, versus implicit density estimation, where the goal is to output a "generator function" that can generate i.i.d. samples from a distribution Q close to P. The main result here is Theorem 9, which states that, under certain assumptions, the two tasks have the same sample complexity up to constant factors. This result is the first of this spirit as far as I know.  The paper is notationally quite heavy and I haven't understood all results completely. I have also not checked any of the proofs, which appear in the supplementary material.  Pros: + given the current interest on GANs, the sample complexity result of this paper can be of significant interest.  + the notion of density estimation with respect to adversarial losses is a novel notion introduced recently, and this paper makes significant progress towards understanding this problem. In some sense this paper connects the classical density estimation literature with the new developments on generative models.  + the connection between implicit density estimation and explicit density estimation, made in this paper (Theorem 9), is novel.  + the results are mathematically interesting and quite non-trivial.  Cons: - there are lots of typos, and many of the terms have not been defined properly.   Comments for the author: * Line 26: losses -> metrics * Line 45: linear projection -> mapping * Line 49-50: delete "(1-)" * Line 66: 3 -> Section 3 * Line 80: q -> p * Line 80: please define Lebesgue norm. Also what is mu? Is mu = lambda? Please clarify. * Line 81: please define the sth-order fractional derivative, rather than referring the reader to a book. It is a central definition in your paper, which must  be standalone. * Line 85: is well-defined "for any P." * Line 85: define real-valued net. * Equation (4): hat{P} -> hat{P}_Z * Line 165: delete the second "that" * Line 172: define "parametric convergence" * Line 198: zeta^d -> (2zeta+1)^d * In Theorem 7, what is d ? * In Section 7, define the GAN setup (don't assume the reader knows what is the generator and what is the discriminator) * Line 255: on -> of * Line 265: a latent -> the latent * Line 266: missing closed brackets * Line 280: delete "below" * Line 296: no -> not * Line 318 and 319: estimators -> samplers * Section 8 is rather different from the rest of the paper. I would suggest moving it to the appendix, and using the opened up space to define all the used terms properly and elaborate more on the definitions and explain the techniques and intuition behind proofs. * In Lines 49-51 you mention many of the common metrics can be written in terms of F_d-metrics. Elaborate please. * For Theorem 1, do you need to make any assumptions about the orthogonal basis B? == After reading other reviews and authors' responses: thanks for the answers.  Definition of f^{(s)}: please mention the page number of the book on which this is defined to ease the reader. Line 26: currently the sentence reads "loss is measured not with L^p distances but rather with weaker losses" which is awkward. Lines 49--51: please mention mu is Lebesgue measure.