The Unreasonable Effectiveness of Structured Random Orthogonal Embeddings
We examine a class of embeddings based on structured random matrices with orthogonal rows which can be applied in many machine learning applications including dimensionality reduction and kernel approximation. For both the JohnsonLindenstrauss transform and the angular kernel, we show that we can select matrices yielding guaranteed improved performance in accuracy and/or speed compared to earlier methods. We introduce matrices with complex entries which give significant further accuracy improvement. We provide geometric and Markov chain-based perspectives to help understand the benefits, and empirical results which suggest that the approach is helpful in a wider range of applications.

The paper analyses the theoretical properties of a family random projections approaches to approximate inner products in high dimensional spaces. In particular the authors focus on methods based on random structured matrices, namely Gaussian orthogonal matrices and SD-matrices. The latter are indeed appealing since they require significantly less computations to perform the projections thanks to their underlying structure. The authors show that the methods considered perform comparably well (or better) with respect to the Johnson-Lindenstrauss transform (baseline based on unstructured Gaussian matrices). Moreover they show that further improvements can be achieved by extending SD-matrices to the complex domain. The authors extend their analysis to the case random feature based approximation of angular kernels.
The paper is well written and clear to read, however the discussion of some of the results could be elaborated more. For instance after Thm 3.3, which characterizes the MSE of the orthogonal JL transform based on SD-matrices, it is not discussed in much detail how this compares to the standard JL baseline. Cor. 3.4 does not really help much since it simply states that Thm 3.3 yields to lower MSE, without clarifying the entity of such improvement. In particular it appears that the improvement in performance of the OJLT over JLT is only in terms of constants (w.r.t. the number of sub-sampled dimensions m).
I found it a bit misleading in Sec. 4, to introduce a general family of kernels, namely the pointwise nonlinear Gaussian kernels, but then immediately focus on a specific instance of such class. The reader expects the following results to apply to the whole family but this is not the case. Reversing the order, and discussing PNG kernels only at the end of the section would probably help the discussion.
I found the term 'Unreasonable' in the title not explained in the text. Is it not reasonable to expect that adding structure to an estimator could make it more effective?
The Mean Squared Error (MSE) is never defined. Although being a standard concept, it is also critical to the theoretical analysis presented in the paper, so it should be defined nevertheless.


From Parity to Preference-based Notions of Fairness in Classification
The adoption of automated, data-driven decision making in an ever expanding range of applications has raised concerns about its potential unfairness towards certain social groups. In this context, a number of recent studies have focused on defining, detecting, and removing unfairness from data-driven decision systems. However, the existing notions of fairness, based on parity (equality) in treatment or outcomes for different social groups, tend to be quite stringent, limiting the overall decision making accuracy. In this paper, we draw inspiration from the fairdivision and envy-freeness literature in economics and game theory and propose preference-based notions of fairness-given the choice between various sets of decision treatments or outcomes, any group of users would collectively prefer its treatment or outcomes, regardless of the (dis)parity as compared to the other groups. Then, we introduce tractable proxies to design margin-based classifiers that satisfy these preference-based notions of fairness. Finally, we experiment with a variety of synthetic and real-world datasets and show that preference-based fairness allows for greater decision accuracy than parity-based fairness.

ML models control many aspects of our life such as our bank decision on approving a loan or a medical centers that use ML to make treatment decision. Therefore, it is paramount that these models are fair, even in cases where the data contains biases. This study follows previous studies in the field but argues that the definition of fairness provided in previous studies might pose a too high bar which results in unnecessary cost in terms of accuracy. Instead they present an alternative fairness definition, presenting an algorithm to apply it for linear classifiers together with some empirical results.
I find this paper very interesting and relevant but lacking in several aspects. I think that it will benefit from some additional depth which will make it a significant contribution. Here are some issues that I find with this paper:
1.	I am not sure that the treatment of the bias is comprehensive enough. For example, if z is a gender feature. A classifier may ignore this feature altogether but still introduce bias by inferring gender from other features such that height, weight, facial image, or even the existence of male/female reproducing organs. I could not figure out from the text how this issue is treated. This is important that in the current definition, if a model never treats a person with male reproducing organs for cancer, this model may be gender-fair. The paragraph at lines 94-100 gives a short discussion that relates to this issue but it should be made more explicit and clear.
2.	There is a sense of over-selling in this paper. The abstract suggests that there is an algorithmic solution but only in section 3 an assumption is added that the classification model is linear. This is a non-trivial constraint but there is no discussion on scenarios in which linear models are to be used.
Some additional comments that the authors may wish to consider:
3.	The introduction can benefit from concrete examples. For example, you can take the case of treating cancer: while male and female should receive equally good treatment, it does not mean that female should be treated for prostate cancer or male should be treated for breast cancer in the same rates.
4.	Lines 66-74 are repetition of previous paragraphs
5.	Line 79: “may” should be “many”
6.	When making the approximations (lines 171-175) you should show that these are good approximation in a sense that a solution to the approximated problem will be a good solution to the original problem. It may be easier to state the original problem as (9) and avoid the approximation discussion.


Dynamic Revenue Sharing *
Many online platforms act as intermediaries between a seller and a set of buyers. Examples of such settings include online retailers (such as Ebay) selling items on behalf of sellers to buyers, or advertising exchanges (such as AdX) selling pageviews on behalf of publishers to advertisers. In such settings, revenue sharing is a central part of running such a marketplace for the intermediary, and fixedpercentage revenue sharing schemes are often used to split the revenue among the platform and the sellers. In particular, such revenue sharing schemes require the platform to (i) take at most a constant fraction α of the revenue from auctions and (ii) pay the seller at least the seller declared opportunity cost c for each item sold. A straightforward way to satisfy the constraints is to set a reserve price at c/(1 − α) for each item, but it is not the optimal solution on maximizing the profit of the intermediary. While previous studies (by Mirrokni and Gomes, and by Niazadeh et al) focused on revenue-sharing schemes in static double auctions, in this paper, we take advantage of the repeated nature of the auctions. In particular, we introduce dynamic revenue sharing schemes where we balance the two constraints over different auctions to achieve higher profit and seller revenue. This is directly motivated by the practice of advertising exchanges where the fixed-percentage revenue-share should be met across all auctions and not in each auction. In this paper, we characterize the optimal revenue sharing scheme that satisfies both constraints in expectation. Finally, we empirically evaluate our revenue sharing scheme on real data.

This paper studies marketplaces where there is a single seller, multiple buyers, and multiple units of a single good that are sold sequentially. Moreover, there is an intermediary that sells the item for the seller. The intermediary is allowed to keep some of the revenue, subject to various constraints. The underlying auction is the second-price auction with reserve prices. The intermediary must determine how to set the reserve price in order to maximize its profit. Since this auction format is strategy-proof for the buyers, they assume the buyers bid truthfully. They also make the assumption that the seller will report its costs truthfully.
First, the authors study “single period revenue sharing,” where at every time step, the intermediary can keep at most an alpha fraction of the revenue and it must cover the seller’s reported cost if the item is sold. Next, they study “multi period revenue sharing,” where the aggregate profit of the intermediary is at most an alpha fraction of the buyers’ aggregate payment and the costs must be covered in aggregate as well. For both of these settings, the authors determine the optimal reserve price. Theoretically, the authors show that multi period revenue sharing has the potential to outperform single period revenue sharing when there are many bidders (the second-highest bid is high), and the seller’s cost is “neither too high nor too low.” They analyze several other model variants as well.
The authors experimentally compare the optimal pricing policies they come up with for the various models with the “naïve policy” which sets the reserve price at c/(1-alpha). Their experiments are on real-world data from an ad exchange. They show that the single period and multi period policies improve over the naïve policy, and the multi period policy outperforms the single period policy.
This paper studies what I think is an interesting and seemingly very applicable auction format. It’s well-written and the results are not obvious. I’d like to better understand the authors’ assumption that the seller costs are reported truthfully. The authors write, “Note that the revenue sharing contract guarantees, at least partially, when the constraint binds (which always happens in practice), the goals of the seller and the platform are completely aligned: maximizing profit is the same as maximizing revenue. Thus, sellers have little incentive to misreport their costs.” I think this means that in practice, the cost is generally so low in contrast to the revenue that the revenue sharing constraint dominates the minimum cost constraint. This doesn’t really convince me that the seller has little incentive to misreport her cost as very high to extract more money from the exchange. Also, as far as I understand, this statement detracts from the overall goal of the paper since it says that in practice, it’s always sufficient to maximize revenue, so the intermediary can ignore the sellers and can just run the Myerson auction. The application to Uber’s platform is also a bit confusing to me because Uber sets the price for the passenger and the pay for the driver. So is Uber the intermediary? The drivers seem to have no say in the matter; they do not report their costs.
=====After author feedback=====
That makes more sense, thanks.


Stochastic Submodular Maximization: The Case of Coverage Functions
Stochastic optimization of continuous objectives is at the heart of modern machine learning. However, many important problems are of discrete nature and often involve submodular objectives. We seek to unleash the power of stochastic continuous optimization, namely stochastic gradient descent and its variants, to such discrete problems. We first introduce the problem of stochastic submodular optimization, where one needs to optimize a submodular objective which is given as an expectation. Our model captures situations where the discrete objective arises as an empirical risk (e.g., in the case of exemplar-based clustering), or is given as an explicit stochastic model (e.g., in the case of influence maximization in social networks). By exploiting that common extensions act linearly on the class of submodular functions, we employ projected stochastic gradient ascent and its variants in the continuous domain, and perform rounding to obtain discrete solutions. We focus on the rich and widely used family of weighted coverage functions. We show that our approach yields solutions that are guaranteed to match the optimal approximation guarantees, while reducing the computational cost by several orders of magnitude, as we demonstrate empirically.

The paper discusses the development of stochastic optimization for submodular coverage functions. The results and proofs are a mish-mash of existing results and techniques; nevertheless the results are novel and should spark interest for discussions and further research directions.
The central contribution of the paper is providing with a framework to exploit the SGD machinery. To this effect, coverage functions can be harnessed, because the concave upperbound is obtained easily. While this seems restrictive, the authors present applications with good empirical performance vs classic greedy.
I have a couple of points:-
(a)	Is there to use projected SGD over Stochastic Frank-Wolfe [A] ? The latter will get rid of messy projection issues for general matroids, albeit the convergence might be slower.
(b)	An interesting aspect of the paper is connection between discrete optimization and convex optimization. I would suggest including more related work that discuss these connections [B,C,D,E]. [B] also talks about approx. form of submodularity, [C,D] both follow the notion of “lifting” atomic optimization to continuous domain.
Minor:
The paper is well-written overall. There are minor typos for which I suggest the authors spend some time cleaning up the paper. E.g. Line 22,24 “trough”, line 373 “mosst”. An equation reference is missing in the appendix.
Numbering of Proof Proposition 2 is messed up.
The proof of Theorem 2 seems to be missing, I understand it follows from pipage rounding, but it might be a good idea to give a Lemma or something with reference from which approx. guarantee for rounding follows. In general, adding more information about pipage rounding in the appendix for completeness might be a good idea.
The main text is missing a page of references.
[A] Linear Convergence of Stochastic Frank Wolfe Variants. Donald Goldfarb, Garud Iyengar, Chaoxu Zhou
[B] Restricted Strong Convexity Implies Weak Submodularity. Ethan R. Elenberg , Rajiv Khanna , Alexandros G. Dimakis , and Sahand Negahban
[C] On Approximation Guarantees for Greedy Low Rank Optimization Rajiv Khanna , Ethan R. Elenberg1 , Alexandros G. Dimakis , and Sahand Negahban
[D] Lifted coordinate descent for learning with trace-norm regularization. Miro Dudik  Zaid Harchaoui , Jérôme Malick
[E] Guaranteed Non-convex Optimization: Submodular Maximization over Continuous Domains. Andrew An Bian, Baharan Mirzasoleiman, Joachim M. Buhmann, Andreas Krause


AdaGAN: Boosting Generative Models
Generative Adversarial Networks (GAN) are an effective method for training generative models of complex data such as natural images. However, they are notoriously hard to train and can suffer from the problem of missing modes where the model is not able to produce examples in certain regions of the space. We propose an iterative procedure, called AdaGAN, where at every step we add a new component into a mixture model by running a GAN algorithm on a re-weighted sample. This is inspired by boosting algorithms, where many potentially weak individual predictors are greedily aggregated to form a strong composite predictor. We prove analytically that such an incremental procedure leads to convergence to the true distribution in a finite number of steps if each step is optimal, and convergence at an exponential rate otherwise. We also illustrate experimentally that this procedure addresses the problem of missing modes.

AdaGAN is a meta-algorithm proposed for GAN. The key idea of AdaGAN is: at each step reweight the samples and fits a generative model on the reweighted samples. The final model is a weighted addition of the learned generative models. The main motivation is to reduce the mode-missing problem of GAN by reweighting samples at each step.
It is claimed in the Introduction that AdaGAN can also use WGAN or mode-regularized GAN as base generators (line 55). This claim seems to be an overstatement. The key assumption of AdaGAN is that the base generator aims at minimizing the f-divergence (as mentioned in Line 136-137). This assumption does not hold for WGAN or mode-regularized GAN: WGAN minimizes the Wasserstein distance, and the mode-regularized GAN has an additional mode regularizer in the objective.
WGAN and mode-regularized GAN are state-of-the-art generative models targeting the mode-missing problem. It is more convincing if the author can demonstrate that AdaGAN outperforms the two algorithms.
Corollary 1 and 2 assume that the support of learned distribution P_g and the true distribution P_d should overlap for at least 1-beta. This is usually not true in practice. A common situation when training GAN is that the data manifold and the generation manifold are disjoint (see e.g., the WGAN paper).


Decomposition-Invariant Conditional Gradient for General Polytopes with Line Search
Frank-Wolfe (FW) algorithms with linear convergence rates have recently achieved great efficiency in many applications. Garber and Meshi (2016) designed a new decomposition-invariant pairwise FW variant with favorable dependency on the domain geometry. Unfortunately it applies only to a restricted class of polytopes and cannot achieve theoretical and practical efficiency at the same time. In this paper, we show that by employing an away-step update, similar rates can be generalized to arbitrary polytopes with strong empirical performance. A new "condition number" of the domain is introduced which allows leveraging the sparsity of the solution. We applied the method to a reformulation of SVM, and the linear convergence rate depends, for the first time, on the number of support vectors.

The main result of the paper is an analysis of the away-step Frank-Wolfe showing that with line search, and over a polyhedral domain, the method has linear convergence. This is somewhat restrictive, but more general than several recent related works, and while they analyze some delicate fixed step-size methods, their main point seems to be that they can analyze the line search case which is important since no one uses the fixed step-size methods in practice (since they depend on some unknown parameters).
Some parts of the paper are written well, and it appears the authors are highly competent. A good amount of proofs are in the main paper, and the appendix supports the paper without being overly long.
However, for someone like myself who is aware of Frank-Wolfe but not familiar with the pairwise or away-step FW and their analysis, nor with the simplex-like polytopes (SLP), the paper is quite hard to follow. Section 3.1 and 3.2 discuss a metric to quantify the difficulty of working over a given polytope, and this is central to the paper and would benefit from a more relaxed description. Also, much of the paper is concerned with the SLP case so that they can compare with previous methods, and show that they are comparable, even if this case (with its special stepsize) is not of much practical interest, so this adds to the complexity. And overall, it is simply a highly technical paper, and (like many NIPS papers) feels rushed in places -- for example, equation (4) by itself is not any kind of valid definition of H_s, since we need something like defining H_s as the maximum possible number such as (4) is true for all points x_t, all optima x^*, and all possible FW and away-directions.
The numerical experiments are not convincing from a practitioners point-of-view, but for accompanying a theory paper, I think they are reasonable, since they are designed to illustrate key points in the analysis (such Fig 2 showing linear convergence) rather than demonstrate this is a state-of-the-art method in practice.
After reading the paper, and not being a FW expert, I come away somewhat confused about what we need to guarantee linear convergence for FW. We assume strong convexity and smoothness (Lipschitz continuity of the gradient), and a polyhedral constraint set. At this point, other papers need the optimal solution (unique, via strong convexity) to be away from the boundary or similar, but this paper doesn't seem to need that. Can you clarify this?
Overall, I'm impressed, and mildly enthusiastic, but I think some of the parts I didn't understand well are my fault and some could be improved by a better-expressed paper, and overall I'm somewhat reserving judgement for now.
Minor comments:
- Line 26, "Besides," is not often used this way in written material.
- Line 36, lots of commas and the "e.g." make this sentence hard to follow
- Line 39, "empirical competency" is not a usual phrase
- Line 63, "SMO" is not defined, and I had to look at the titles of refs [19,20] to figure it out
- Eq (1), it's not clear if you require the polytope to be bounded (some authors use the terminology "polytope" to always mean bounded; maybe this is not so in the literature you cite, but it's unclear to a general reader). Basic FW requires a compact set, but since your objective is assumed to be strongly convex, it is also coercive, so has bounded level sets, so the FW step is still well-defined, and I know FW has been analyzed in this case. So it does not seem to be necessary that the polytope is bounded, and I was very unsure if it was or not (had I been able to follow every step of the proofs, this would probably have answered itself).
- Line 80, "potytope" is a typo
- Eq (4) is not a definition. This part is sloppy
- Eq (5) is vague. What does "second_max" mean? The only definition that makes sense is that if we define I to be the set of all indices that achieve the max, then the second max is the max over all remaining indices, though this could be an empty set, so then the value is not defined. Another definition that doesn't seem to work in your case is that we exclude only one optimal index. It was hard to follow the sentence after (5), especially given the vagueness of (5).
- Line 145 defines e_1, but this should be earlier in line 126.
- Section 3.2 would benefit from more description on what these examples are trying to show. Line 148 says "n which is almost identical to n-1", which by itself is meaningless. What are you trying to express here? I think you are trying to show the lower bound is n-1 and the upper bound is n?
- Line 160, is there a downside to having a slack variable?
- Line 297, "not theoretical" is a typo.
- Line 311, "Besides," is not used this way.
== Update after reading authors' rebuttal ==
Thanks for the updates. I'm still overall positive about the paper.


Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration
Computing optimal transport distances such as the earth mover's distance is a fundamental problem in machine learning, statistics, and computer vision. Despite the recent introduction of several algorithms with good empirical performance, it is unknown whether general optimal transport distances can be approximated in near-linear time. This paper demonstrates that this ambitious goal is in fact achieved by Cuturi's Sinkhorn Distances. This result relies on a new analysis of Sinkhorn iterations, which also directly suggests a new greedy coordinate descent algorithm GREENKHORN with the same theoretical guarantees. Numerical simulations illustrate that GREENKHORN significantly outperforms the classical SINKHORN algorithm in practice.
Dedicated to the memory of Michael B. Cohen

In this paper the authors discuss entropic regularized optimal transport with a focus on the computational complexity of its resolution. Their main contribution is a result giving relation between accuracy (in term of error wrt the constraint) and number of iterations for Sinkhorn (and their) algorithm. It show that a given accuracy can be obtained with a complexity near linear to the size of the problem which is impressive.
Using the in-depth computational study the authors propose a new algorithm with the same theoretical property of Sinkhorn but more efficient in practice. very small numerical experiment suggest that the propose GreenKhorn method works in practice better than Sinkhorn.
This paper is very interesting and propose a theoretical computational study that can be of huge help to the community. It also explain why the recent entropic regularization works so well in practice and when it shouldn't be used.  The new Greedy Sinkhorn algorithm is very interesting and seem to work very well in practice. I still have a few comment that I would like to be addressed in the rebuttal before giving a definite rating.
Comments:
- The whole paper seem a bit of a mess with theorems given between definitions and proofs scattered everywhere except below the theorem. I understand the the authors wanted to tell a story but in practice it makes the paper more difficult to read. Beginning with theorem 1 is ok and finishing with its proof is also OK. But the remaining should be done in classical math stuff theorem 2 followed by proof then theorem 3 that uses theorem 2 and so on.
- One of the take home message from theorem 1 is actually not that good: if you want to decrease the l1 error on the constraint by 10, you will need to perform 10^3=1000 times more iterations. this also explain why sinkhorn is known to converge rather quickly to a good enough solution but might take forever to actually converge to numerical precision. It should have been discussed a bit more. Actually I would really like to see numerical experiments that illustrate the O(1/k^{1/3}) convergence of the error.
- The propose Greenkhron is very interesting but it has strong similarity with the stochastic sinkhorn of [GCPB16]. My opinion is that the proposed method is better because it basically update only the variable that violate the most the constraints whereas stochastic will spend time updating already valid constraints since all variables are updated. Still good scientific experiments require the authors to provide these comparison and the fact that the code for [GCPB16] is available online gives the authors no excuse.


Scalable Variational Inference for Dynamical Systems
Gradient matching is a promising tool for learning parameters and state dynamics of ordinary differential equations. It is a grid free inference approach, which, for fully observable systems is at times competitive with numerical integration. However, for many real-world applications, only sparse observations are available or even unobserved variables are included in the model description. In these cases most gradient matching methods are difficult to apply or simply do not provide satisfactory results. That is why, despite the high computational cost, numerical integration is still the gold standard in many applications. Using an existing gradient matching approach, we propose a scalable variational inference framework which can infer states and parameters simultaneously, offers computational speedups, improved accuracy and works well even under model misspecifications in a partially observable system.

This paper proposes a mean-field method for joint state and parameter inference of ODE systems. This is a problem that continues to receive attention because of its importance in many fields (since ODEs are used so widely) and the limitations of existing gradient-based methods. The solution described in the paper is said to outperform other approaches, be more applicable, and scale better.
The paper lays out its contribution clearly in the introduction. It does a good job of discussing the merits and drawbacks of previous approaches, and explaining how it fits in this context. The proposed methodology appears sound (although it might be useful to explain some of the derivations in a little more detail) and the results indicate that it performs much better than the state of the art. The main advantage of the new method is its execution time, which is shown to be orders of magnitude faster than methods with comparable (or worse) accuracy. In particular, the scalability analysis appears very promising for the applicability of this method to larger models.
Overall, I found this to be a strong paper. My comments mainly concern the presentation, and most are minor.
- The biggest comment has to do with the derivations of the evidence lower bounds and Equation (16). I believe it would be useful to have some more details there, such as explicitly pointing out which properties they rely on. It is also not fully clear to me how the E-M steps of Algorithm 1 make use of the previous derivations (although that may be because I have limited experience in this field). There appears to be some space available for expansion but, if necessary, I suppose that sections 2 and 3 could be combined.
- Besides that, there is a statement that appears contradictory. Lines 82-86 describe how parameters and state are alternately sampled in Calderhead et al. and Dondelinger et al. However, lines 29-30 make it seem like these approaches cannot infer both state and parameters. I would suggest either to rephrase (perhaps remove the "simultaneously"?) or explain why this is a problem.
- As far as the experiments are concerned, I think it would be interesting to run the Lotka-Volterra example of Section 5.1 for longer, to confirm that the inferred state displays periodicity (perhaps comparing with the other methods as well). However, I believe the results do a good job of supporting the authors' claims.
Minor comments:
--------------
l2: grid free -> grid-free (also later in the text)
l52: the phrasing makes it sound like the error has zero mean and zero variance; perhaps rephrase to clarify that the variance can be generally elliptical / different for each state?
Equations 3 - 8: There is a small change in the notation for normal distributions between Eqs 3-4 and 5-8. For the sake of clarity, I think it is worth making it consistent, e.g. by changing the rhs of (3) to N(x_k | 0, C) instead of N(0,C) (or the other way around).
l71: What is the difference between C' and 'C ?
Equation 9: no upper bound on the sum
l99: contains -> contain
l99: "arbitrary large" makes me think that a sum term can include repetitions of an x_j (to effectively have power terms, as you would have in mass-action kinetics with stoichiometry coefficients > 1), but it is not fully clear. Is this allowed? (in other words, is M_{ki} a subset of {1,...,K}, or a multiset of its elements?)
l139: what does "number of states" refer to here? the number of terms in the sum in Eq 9?
Figure 1: in the centre-right plot, it seems like the estimate of \theta_4 is missing for the spline method
l155: Predator is x_2, prey is x_1
l163: move the [ after the e.g. : "used in e.g. [Calderhead et al. ..."
l168: usining -> using
l171: I would add commas around "at roughly 100 times the runtime" to make the sentence read easier
l175: partial -> partially
l187: (3) -> (Figure 3)
l195-196: Figure 2 does not show the noise changing; I think having two references would be more helpful: "[...] if state S is unobserved (Figure 2) and [...] than our approach (Figure 3)."
l198: perhaps "error" is better than "bias" here? Unless it is known how exactly the misspecification biases the estimation?
l202: Lorentz -> Lorenz (and later)
l207: stochstic -> stochastic
l208: number states -> number of states
l209: Due the dimensionality -> Due to the dimensionality,
l212: equally space -> equally spaced
l214: correct inferred -> correctly inferred
Figure 4: In the last two lines of the caption: wheras -> whereas; blue -> black (at least that's how it looks to me); is corresponding to -> corresponds to
l219: goldstandard -> gold standard
l253: modemodel -> modelling


Gradient Methods for Submodular Maximization
In this paper, we study the problem of maximizing continuous submodular functions that naturally arise in many learning applications such as those involving utility functions in active learning and sensing, matrix approximations and network inference. Despite the apparent lack of convexity in such functions, we prove that stochastic projected gradient methods can provide strong approximation guarantees for maximizing continuous submodular functions with convex constraints. More specifically, we prove that for monotone continuous DR-submodular functions, all fixed points of projected gradient ascent provide a factor 1 2 approximation to the global maxima. We also study stochastic gradient methods and show that after O(1 2 ) iterations these methods reach solutions which achieve in expectation objective values exceeding ( OPT 2 − ). An immediate application of our results is to maximize submodular functions that are defined stochastically, i.e. the submodular function is defined as an expectation over a family of submodular functions with an unknown distribution. We will show how stochastic gradient methods are naturally well-suited for this setting, leading to a factor 1 2 approximation when the function is monotone. In particular, it allows us to approximately maximize discrete, monotone submodular optimization problems via projected gradient ascent on a continuous relaxation, directly connecting the discrete and continuous domains. Finally, experiments on real data demonstrate that our projected gradient methods consistently achieve the best utility compared to other continuous baselines while remaining competitive in terms of computational effort.

This paper study the problem of continuous submodular maximization subject to a convex constraint. The authors show that for monotone weakly DR-submodular functions, all the stationary points of this problem are actually good enough approximations (gamma^2 / (1 + gamma^2), where gamma is the DR factor) of the optimal value. They also show that stochastic projected gradient descent and mirror descent converge to a stationary point in O(1/eps^2) iterations.
The paper is clear and well presented, with few typos. The results presented can handle more general constraints than prior work and can address the stochastic setting, but at the cost of a worst approximation factor (1/2 for DR-submodular functions as opposed to 1 - 1/e).
Some comments/questions:
-It is worth mentioning that stochastic projected gradient descent was also used recently for submodular minimization in "Chakrabarty, D., Lee, Y. T., Sidford, A., & Wong, S. C. W. Subquadratic submodular function minimization. STOC 2017."
- For the deficiency example of FW in A.2, it would be clearer to present the example with a minibatch setting, with m_i,n = 1/(b+1) batch size, the same example should follow through, but it would make it clearer that a slightly larger minibatch size won't fix the problem.
- Would this deficiency example hold also for the algorithm "Measured continuous greedy" in "Feldman, Moran, Joseph Naor, and Roy Schwartz. "A unified continuous greedy algorithm for submodular maximization." FOCS, 2011, whose updates are slightly different from FW?
- You reference the work [20] on p.2, but I did not find this work online?
- Can you add what are other examples of L-smooth continuous submodular functions, other than the multilinear extension?
- For a more convincing numerical comparison, it would be interesting to see the time comparison (CPU time not just iterations) against FW.
- What is the y-axis in the figures?
Some typos:
-line 453: f_ij - f_i - f_j (flipped)
-lemma B.1: right term in the inner product should be proj(y) - x.
-lemma B.2: first term after equality should be D_phi(x,y)
- Eq after line 514 and line 515: grad phi(x+) (nabla missing)
- In proof of lemma B.5: general norm instead of ell_2
- Eq after line 527 and 528: x_t and x^* flipped in inner product.


SchNet: A continuous-filter convolutional neural network for modeling quantum interactions
Deep learning has the potential to revolutionize quantum chemistry as it is ideally suited to learn representations for structured data and speed up the exploration of chemical space. While convolutional neural networks have proven to be the first choice for images, audio and video data, the atoms in molecules are not restricted to a grid. Instead, their precise locations contain essential physical information, that would get lost if discretized. Thus, we propose to use continuousfilter convolutional layers to be able to model local correlations without requiring the data to lie on a grid. We apply those layers in SchNet: a novel deep learning architecture modeling quantum interactions in molecules. We obtain a joint model for the total energy and interatomic forces that follows fundamental quantumchemical principles. Our architecture achieves state-of-the-art performance for benchmarks of equilibrium molecules and molecular dynamics trajectories. Finally, we introduce a more challenging benchmark with chemical and structural variations that suggests the path for further work.

Summary:
In order to design new molecules, one needs to predict if the newly designed molecules could reach an equilibrium state. All possible configurations of atoms do not lead to such state. To evaluate that, the authors propose to learn predictors of such equilibrium from annotated datasets, using convolutional networks. Different to previous works, that might lead to inaccurate predictions from convolutions computed in a discretized space, the authors propose continuous filter convolution layers. They compare their approach with two previous ones on 2 datasets and also introduce a third dataset. Their model works better than the others especially when using more training data.
I don't have a very strong background in physics so I'm not the best person to assess the quality of this work, but I found that the proposed solution made sense, the paper is clearly written and seems technically correct.
Minor: don't forget to replace the XXX
Points 3,4 should not be listed as contributions in my opinion, they are results
l. 26 : the both the -> both the
Ref 14: remove first names


A Probabilistic Framework for Nonlinearities in Stochastic Neural Networks
We present a probabilistic framework for nonlinearities, based on doubly truncated Gaussian distributions. By setting the truncation points appropriately, we are able to generate various types of nonlinearities within a unified framework, including sigmoid, tanh and ReLU, the most commonly used nonlinearities in neural networks. The framework readily integrates into existing stochastic neural networks (with hidden units characterized as random variables), allowing one for the first time to learn the nonlinearities alongside model weights in these networks. Extensive experiments demonstrate the performance improvements brought about by the proposed framework when integrated with the restricted Boltzmann machine (RBM), temporal RBM and the truncated Gaussian graphical model (TGGM).

Summary: The paper uses doubly truncated Gaussian distributions to develop a family of stochastic non-linearities parameterized by the truncation points. In expectation, different choices of truncation points are shown to recover close approximations to popularly employed non-linearities. Unsurprisingly, learning the truncation points from data leads to improved performance.
Quality and Clarity —  The authors present a technically sound piece of work. The proposed family of non-linearities are sensible and the authors do a good job of demonstrating their utility across a variety of problems. The manuscript is sufficiently clear.
Originality and Significance - The connection between the convolution of a Gaussian with an appropriately truncated Gaussian and popular neural network nonlinearities are well known and have been previously explored by several papers in supervised contexts (see [1, 2, 3]). In this context, learning the truncation points is a novel but obvious and incremental next step. To me, the interesting contribution of this paper lies in its empirical demonstration that learning the non-linearities leads to tangible improvements in both density estimation and predictive performance. It is also interesting that the learned non-linearities appear to be sigmoid-relu hybrids.
Detailed Comments:
1) Figures 2 indicates that the learned upper truncations are all relatively small values, in the single digits. This could be a consequence of the initialization to 1 and the low learning rates, causing the recovered solutions to get stuck close to the initial values. Were experiments performed with larger initial values for the upper truncation points?
2) a) Why are the s-learn experiments missing for TruG-TGGM?
   b)  Table 4 seems to suggest that stochasticity alone doesn’t lead to significant improvements over the ReLU MLP. Only when learning of non-linearities is added does the performance improve. What is unclear from this is whether stochasticity helps in improving performance at all, if the non-linearities are being learned. I will really like to see comparisons for this case with parameterized non-linearities previously proposed in the literature (citations 6/7/8 in the paper) to tease apart the benefits of stochasticity vs having more flexible non-linearities.
Minor:
There exists related older work on learning non-linearities in sigmoidal belief networks [4], which should be cited.
[1] Hernández-Lobato, José Miguel, and Ryan Adams. "Probabilistic backpropagation for scalable learning of bayesian neural networks." International Conference on Machine Learning. 2015.
[2] Soudry, Daniel, Itay Hubara, and Ron Meir. "Expectation backpropagation: Parameter-free training of multilayer neural networks with continuous or discrete weights." Advances in Neural Information Processing Systems. 2014.
[3] Ghosh, Soumya, Francesco Maria Delle Fave, and Jonathan S. Yedidia. "Assumed Density Filtering Methods for Learning Bayesian Neural Networks." AAAI. 2016.
[4] Frey, Brendan J., and Geoffrey E. Hinton. "Variational learning in nonlinear Gaussian belief networks." Neural Computation 11.1 (1999): 193-213.


Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols
Learning to communicate through interaction, rather than relying on explicit supervision, is often considered a prerequisite for developing a general AI. We study a setting where two agents engage in playing a referential game and, from scratch, develop a communication protocol necessary to succeed in this game. Unlike previous work, we require that messages they exchange, both at train and test time, are in the form of a language (i.e. sequences of discrete symbols). We compare a reinforcement learning approach and one using a differentiable relaxation (straightthrough Gumbel-softmax estimator (Jang et al., 2017)) and observe that the latter is much faster to converge and it results in more effective protocols. Interestingly, we also observe that the protocol we induce by optimizing the communication success exhibits a degree of compositionality and variability (i.e. the same information can be phrased in different ways), both properties characteristic of natural languages. As the ultimate goal is to ensure that communication is accomplished in natural language, we also perform experiments where we inject prior information about natural language into our model and study properties of the resulting protocol.

Increasing my score based on the authors rebuttal. The argument that the proposed method can complement human-bot training makes sense. Also, it seems RL baseline experiments were exhaustive. But the argument about the learnt language being compositional should be toned down since there is not enough evidence to support it.
Old reviews:
The paper proposes to use Gumbel-softmax for training sender and receiver agents in a referential game like Lazaridou (2016). Unlike the previous work, large number of distraction images and variable length messages are allowed. The results suggest the proposed method outperforms Reinforce baseline. In addition, the paper proposes two way to force the communication to be more like natural language.
My first concern is that the paper disregards continuous communication for being unlike natural language, and insists on using discrete symbols. But during training, continuous communication is allowed transmit gradients between the agents. This use of back-propagation makes it impossible to learn from interactions with human, which is probably vital for learning natural language.
Secondly, I think baseline experiments are not exhaustive and can be improved. Here are my points:
- L109: Learnt baseline (network itself predicts baseline values) is pretty standard in deep RL works for it is efficiency over moving average. Why it hasn't tried? Also it is common to add entropy regularization term to the loss to encourage exploration in deep RL. Has it tried?
- Appropriate baseline here continuous communication during training and discretization during testing as done in (Foerster, 2016)
- L135: a simple solution would be to decrease the temperature to zero during training. Has it tried?
- L163: why not hyper-parameters are not tuned? At least the learning rate should be tuned, especially in Reinforce baseline because the default values are tuned for supervised training.
At last, the claim of compositionally (L11) of the learnt communication is not supported by any evidence. Also, L196 suggests that word 5747 represents animals. Then, it should still represent the same thing even at different position as in natural language word "animal". The fact that it is not suggest that the learnt communication is actually unlike natural language. Why not similar analysis on reinforce baseline?
Other comments:
- L59: It is not a non-stationary environment. Since the sender and the receiver have the same reward, they can be considered as a single RL agent. Then, this agent writes words into the environment as "sender", then observes them at the next step as "receiver".
- How KL is optimized in sec 3.3? It can't be computed directly because there are exponentially many "m"s.
- Why there are two reinforce versions in figure 2 left. What is greedy argmax reinforce?
- It would be good to include a histogram of message lengths
- Why such large vocabulary size? What is the effect of smaller vocabulary?


Fair Clustering Through Fairlets
We study the question of fair clustering under the disparate impact doctrine, where each protected class must have approximately equal representation in every cluster. We formulate the fair clustering problem under both the k-center and the k-median objectives, and show that even with two protected classes the problem is challenging, as the optimum solution can violate common conventions-for instance a point may no longer be assigned to its nearest cluster center! En route we introduce the concept of fairlets, which are minimal sets that satisfy fair representation while approximately preserving the clustering objective. We show that any fair clustering problem can be decomposed into first finding good fairlets, and then using existing machinery for traditional clustering algorithms. While finding good fairlets can be NP-hard, we proceed to obtain efficient approximation algorithms based on minimum cost flow. We empirically demonstrate the price of fairness by quantifying the value of fair clustering on real-world datasets with sensitive attributes.

While there is a growing body of work on fair supervised learning, here the authors present a first exploration of fair unsupervised learning by considering fair clustering. Each data point is labeled either red or blue, then an optimal k-clustering is sought which respects a given fairness criterion specified by a minimum threshold level of balance in each cluster. Balance lies in [0,1] with higher values corresponding to more equal numbers of red and blue points in every cluster. This is an interesting and natural problem formulation - though a more explicit example of exactly how this might be useful in practice would be helpful.
For both k-center and k-median versions of the problem, it is neatly shown that fair clustering may be reduced to first finding a good 'fairlet' decomposition and then solving the usual clustering problem on the centers of each fairlet. While this is NP-hard, efficient approximation algorithms are provided based on combining earlier methods with an elegant min cost flow construction.
On 3 example datasets, it is shown that standard clustering yields low balance (perhaps not surprising). A preliminary investigation of the cost of fairness is provided where balance of at least 0.5 is required and the number of clusters is varied. It would be interesting to see also the sensitivity to balance: keep the number of clusters fixed, and vary the required balance.
Minor points:
l. 16 machine learning are -> machine learning is
In a few places, e.g. l. 19, \citep{} would improve the readability, so Kleinberg et al. (2017a) -> (Kleinberg et al. 2017a).
l. 53 define -> defines. In fact, I think you just look at the costs of regular clusterings compared to fair so don't really use this ratio?
Fig 1 caption, would give -> might give. x is assigned to a -> x is assigned to the same cluster as a
l. 102 other measure -> other measures, please could you elaborate?
l. 132, 136 Y -> Y_j as in l. 161
Near Definition 5, an image showing an example of how fairlets together make up the fair clustering would be helpful, though I appreciate it's hard given space constraints - in the Supplement would still be good.


Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Nets
Imitation learning has traditionally been applied to learn a single task from demonstrations thereof. The requirement of structured and isolated demonstrations limits the scalability of imitation learning approaches as they are difficult to apply to real-world scenarios, where robots have to be able to execute a multitude of tasks. In this paper, we propose a multi-modal imitation learning framework that is able to segment and imitate skills from unlabelled and unstructured demonstrations by learning skill segmentation and imitation learning jointly. The extensive simulation results indicate that our method can efficiently separate the demonstrations into individual skills and learn to imitate them using a single multi-modal policy. The video of our experiments is available at

The paper describes a new learning model able to discover 'intentions' from expert policies by using an imitation learning framework. The idea is mainly based on the GAIL  model which aims at learning by imitation a policy using a GAN approach. The main difference in the article is that the learned policy is, in fact, a mixture of sub-policies, each sub-policy aiming at automatically matching a particular intention in the expert behavior. The GAIL algorithm is thus derived with this mixture, resulting in an effective learning technique. Another approach is also proposed where the intention will be captured through a latent vector by derivating the InfoGAN algorithm for this particular case. Experiments are made on 4 different settings and show that the model is able to discover the underlying intentions contained in the demonstration trajectories.
Comments:
First of all, the task focused in the paper is interesting. Discovering 'intentions' and learning sub-policies is a key RL problem that has been the source of many old and new papers. The proposed models are simple but effective extensions of existing models.
 Actually, I do not really understand what is the difference made by the authors between 'intentions' and 'options' in the literature. It seems to me that intentions are more restricted than options since there are no natural switching mechanisms in the proposed approach while options models are able to choose which option to use at any time. Moreover, w.r.t options literature, the main originality of the paper is both in how the intentions are discovered (using GAIL), and also in the fact that this is made by imitation while many options learning models are based on a classical RL setting. A discussion on this point is important, both in the 'related work' part, but also in the experimental part (I will go back to this point later in the review).  At the end, it seems that the model discovered sub-policies, but is still unable to know how to assemble these sub-policies in order to solve complex problems. So the proposed model is more a first step to learn an efficient agent than a complete solution.  (This is for example illustrated in Section 6 "[...]the categorical intention variable is manually changed [...]"). The question is thus what can be done with the learned policy ? How the result of the algorithm will be used ?
Concerning the presentation of the model, the notations are not clear. Mainly, \pi^i gives the impression that the policy is indexed by the intention i (which is not the case, since, as far as I understand, the indexation by the intention is in fact contained in the notation $\pi(a|s,i)$) . Moreover, the section concerning the definition of the reward is unclear: as far as I understand, in your setting, trajectories are augmented with the intention value at each timestep. But this intention value i_t is not used during learning, but will be used for evaluating the reward value. I think that this has to be made more clear in the paper if accepted.
Concerning the experimental sections:
* First, the environments are quite simple, and only focused on fully-observable MDP. The extension of the proposed model to PO-MDP could be discussed in the paper (since I suppose it is not trivial).
* Second, there is no comparison of the proposed approach with options discovery/hierarchical models techniques like "Imitation Learning with Hierarchical Actions -- Abram L. Friesen and Rajesh P. N. Rao", or even "Active Imitation Learning of Hierarchical Policies
Mandana Hamidi, Prasad Tadepalli, Robby Goetschalckx, Alan Fern". This is a clear weak point of the paper.
*  I do not really understand how the quantitative evaluation is made (reward). In order to evaluate the quality of each sub-policy w.r.t each intention, I suppose that a manual matching is made between the value of $i$ and the 'real' intention. Could you please explain better explain that point ?
* The difference between the categorical intentions and continuous one is not well discussed and evaluated. Particularly, on the continuous case, the paper would gain if the interpolation between sub-policies is evaluated as it is suggested in the article.
Conclusion:
 An interesting extension of existing models, but with some unclear aspects, and with an experimental section that could be strenghtened


Clone MCMC: Parallel High-Dimensional Gaussian Gibbs Sampling
We propose a generalized Gibbs sampler algorithm for obtaining samples approximately distributed from a high-dimensional Gaussian distribution. Similarly to Hogwild methods, our approach does not target the original Gaussian distribution of interest, but an approximation to it. Contrary to Hogwild methods, a single parameter allows us to trade bias for variance. We show empirically that our method is very flexible and performs well compared to Hogwild-type algorithms.

This paper proposes a new parallel approximate sampler for high-dimensional Gaussian distributions. The algorithm is a special case of a larger class of iterative samplers based on a transition equation (2) and matrix splitting that is analysed in [9]. The algorithm is similar to the Hogwild sampler in term of the update formula and the way of bias analysing, but it is more flexible in the sense that there is a scalar parameter to trade-off the bias and variance of the proposed sampler.
I appreciate the detailed introduction about the mathematical background of the family of sampling algorithms and related works. It is also easy to follow the paper and understand the merit of the proposed algorithm. The illustration of the decomposition of the variance and bias in Figure 1 gives a clear explanation about the role of \eta.
Experiments on two covariance matrix shows that the proposed algorithm achieves lower error than Hogwild for some range of the \eta value. However, the range changes with the computation budget as well as the value of the covariance matrix. It is not clear to me if the proposed algorithm is better than Hogwild in practice if we do not have a good method/heuristic to choose a proper value for \eta.
Another problem with the experiment evaluation is that there is no comparison in the only real applications, image pinpointing-deconvolution. Does the clone MCMC give a better image recovery than other competing methods?


Sobolev Training for Neural Networks
At the heart of deep learning we aim to use neural networks as function approximators -training them to produce outputs from inputs in emulation of a ground truth function or data creation process. In many cases we only have access to input-output pairs from the ground truth, however it is becoming more common to have access to derivatives of the target output with respect to the input -for example when the ground truth function is itself a neural network such as in network compression or distillation. Generally these target derivatives are not computed, or are ignored. This paper introduces Sobolev Training for neural networks, which is a method for incorporating these target derivatives in addition the to target values while training. By optimising neural networks to not only approximate the function's outputs but also the function's derivatives we encode additional information about the target function within the parameters of the neural network. Thereby we can improve the quality of our predictors, as well as the data-efficiency and generalization capabilities of our learned function approximation. We provide theoretical justifications for such an approach as well as examples of empirical evidence on three distinct domains: regression on classical optimisation datasets, distilling policies of an agent playing Atari, and on large-scale applications of synthetic gradients. In all three domains the use of Sobolev Training, employing target derivatives in addition to target values, results in models with higher accuracy and stronger generalisation.

I have enjoyed reading this paper. The idea is clear and very intuitive. Figure 1 is very well conceived.
Now, I must say that, throughout my reading, I was looking for the authors to "confess" that the first-order derivatives might be entirely useless in certain situations. But in a situation where both f(x) and f'(x) are given, taken as ground truth, then it is certainly a wise idea to use the information about f'(x) in order to train a new network that approximates f(x) to solve a task.
However the problem is that, when a certain model M1 is trained to minimize a loss on a training set, then there is nothing that guarantees the sanity/usefulness of the gradients of M1. The derivatives depend a lot on the activation functions being used, for example. Then I don't see at all why there would be any general principle that would encourage us to imitate those gradients in order to learn another model M2 based on M1.
It is very possible that this doesn't happen in practice, on common datasets and reasonable models, and that's interesting. I think that it's partly what the paper studies : does this thing work in practice ? Okay, great ! Worth discussing in a paper with good experiments. Accept !
In the past, I have had a conversation with one of the great ML authorities in which he/she casually claimed that when two functions (f1, f2) are close, then their derivatives are also close. This doesn't sound unreasonable as a mental shortcut, but it's ABSOLUTELY INCORRECT in the mathematical sense. Yet people use that shortcut.
The authors refer to this indirectly, but I would really like to see the authors be more explicit about that fact, and I think it would make the paper slightly better. Just to be sure that nobody makes that mental shortcut when they read this stimulating paper.
I had a bit more difficulty understanding the subtleties of the Synthetic Gradient portion of the paper. I trust that it's sound.
Also, in Figure 4 I'm looking at certain plots and it seems like "regular distillation" very often gets worse with more iterations. (The x-axis are iterations, right?) Sobolev training looks better, but it's hard to say if it's just because the hyperparameters were not selected correctly. They kinda make regular distillation look bad. Moreover, the y-axis really plays with the scales and makes the differences look way bigger than they are. It's hard to know how meaningful those differences are.
To sum it up, I'm glad that I got to read this paper. I would like to see better, and more meaningful experiments, in a later paper, but I'm happy about what I see now.


A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning
This paper takes a step towards temporal reasoning in a dynamically changing video, not in the pixel space that constitutes its frames, but in a latent space that describes the non-linear dynamics of the objects in its world. We introduce the Kalman variational auto-encoder, a framework for unsupervised learning of sequential data that disentangles two latent representations: an object's representation, coming from a recognition model, and a latent state describing its dynamics. As a result, the evolution of the world can be imagined and missing data imputed, both without the need to generate high dimensional frames at each time step. The model is trained end-to-end on videos of a variety of simulated physical systems, and outperforms competing methods in generative and missing data imputation tasks.

The paper presents a time-series model for high dimensional data by combining variational auto-encoder (VAE) with linear Gaussian state space model (LGSSM). The proposed model takes the latent repressentation from VAE as the output of LGSSM. The exact inference of linear Gaussian state space model via Kalman smoothing enables efficient and accurate variational inference for the overall model. To extend the temporal dynamics beyond linear dependency, the authors use a LSTM to parameterize the matrices in LGSSM. The performance of the proposed model is evaluated through bouncing ball and Pendulum experiments.
LGSSM with dynamic parameterization is a nice trick to learn complex temporal dynamics through a simple probabilistic model, but has been exploited in the literature, e.g., [M. Seeger, D. Salinas, V. Flunkert NIPS 2016]. The proposed method does not really disentangle the object representation from the temporal dynamic, as the model is unlikely to infer the correct dynamics of an unseen object. The proposed method only models the temporal dynamic on top of the low dimensional latent representation from VAE.
The proposed method is evaluted on some simple examples, where the object representation is simple (mostly single object with low resolution image) and the dynamic is also simple (bouncing of a ball or Pendulum). It would be more convencing to show experiments with either complex data representation or complex dynamics (e.g., human motion). In Pendulum experiment, the proposed method is compared with other two methods in terms of ELBO. However, it is questionable how to compare ELBO from different models (e.g., the numbers of parameters are probably different.).


Hierarchical Methods of Moments
Spectral methods of moments provide a powerful tool for learning the parameters of latent variable models. Despite their theoretical appeal, the applicability of these methods to real data is still limited due to a lack of robustness to model misspecification. In this paper we present a hierarchical approach to methods of moments to circumvent such limitations. Our method is based on replacing the tensor decomposition step used in previous algorithms with approximate joint diagonalization. Experiments on topic modeling show that our method outperforms previous tensor decomposition methods in terms of speed and model quality.
obtained by standalone EM in this case is reasonable and desirable: when asked for a small number of latent variables EM yields a model which is easy to interpret and can be useful for data visualization and exploration. For example, an important application of low-dimensional learning can be found in mixture models, where latent class assignments provided by a simple model can be used to split the training data into disjoint datasets to which EM is applied recursively to produce a hierarchical clustering [12,13]. The tree produced by such clusterings procedure provides a useful aid in data exploration and visualization even if the models learned at each branching point do not accurately represent the training data.
In this paper we develop a hierarchical method of moments that produces meaningful results even in misspecified settings. Our approach is different from previous attemps to design MoM algorithms for misspecified models. Instead of looking for convex relaxations of existing MoM algorithms like in [14][15][16] or analyzing the behavior of a MoM algorithm with a misspecified number of latent states like in [17,18], we generalize well-known simultaneous diagonalization approaches to tensor decomposition by phrasing the problem as a non-convex optimization problem. Despite its non-convexity, the hierarchical nature of our method allows for a fast accurate solution based on low-dimensional grid search. We test our method on synthetic and real-world datasets on the topic modeling task, showcasing the advantages of our approach and obtaining meaningful results.

Spectral methods for learning latent variable models reduce the learning problem to the problem of performing some low-rank factorization over matrices collecting some moments of the target distribution. There are many variants of this in the literature, most approaches use SVD or other forms of tensor decompositions. This paper highlights an important limitation of many of these approaches. The main limitation is that most theoretical guarantees apply when the goal is to recover a model of k states assuming that the mixture distribution from which the moment matrix is computed also has k components. However, in many cases one is interested in learning models for l lower than k, what the authors call the misspecified setting.
This paper proposes a method that performs joint diagonalization by combining a
truncated SVD step with an optimisation. The theoretical analysis shows that this
algorithm can work in the misspecified setting. The paper presents an application
of the proposed algorithm to topic modelling.
I believe this is a nice contribution, it is true that there are already many
variants of the spectral method, each with its own claimed advantages. This been
said, the one proposed in this paper is technically sound and addresses a true
limitation of previous work.
In terms of references the paper is quite complete, there are however a couple of
relevant missing references:
- B.Balle et al. "Local Loss Optimisation in operator models: A new insight into spectral learning"; since it is one of the first papers to frame spectral learning directly as an optimization.
- More closely related
to this submission, B. Balle et al "A Spectral Learning of Finite State
Transducers" is to the best of my knowledge the first paper that proposes a
spectral method based on joint diagonalization.


Multi-Information Source Optimization
We consider Bayesian methods for multi-information source optimization (MISO), in which we seek to optimize an expensive-to-evaluate black-box objective function while also accessing cheaper but biased and noisy approximations ("information sources"). We present a novel algorithm that outperforms the state of the art for this problem by using a Gaussian process covariance kernel better suited to MISO than those used by previous approaches, and an acquisition function based on a one-step optimality analysis supported by efficient parallelization. We also provide a novel technique to guarantee the asymptotic quality of the solution provided by this algorithm. Experimental evaluations demonstrate that this algorithm consistently finds designs of higher value at less cost than previous approaches.

This paper deals with the important topic of optimization in cases where in addition to costly evaluations of the objective function, it is possible to evaluate cheaper approximations of it. This framework is referred to as MISO (multi-information source optimization) in the paper, where Bayesian optimization strategies relying on Gaussian process models are considered. An original MISO approach is presented, misoKG, that relies on an adaption of the Knowledge Gradient algorithm in multi-information source optimization settings. The method is shown to achieve
very good results and to outperform considered competitors on three test cases.
Overall, I am fond of the ideas and the research directions tackled by the paper, and I found the contributions principled and practically relevant. Given the potential benefits for the machine learning community and beyond, these contributions should definitely be published in a high-level venue such as NIPS. Yet, the paper could still be improved in several respects. Le me start by general remarks and then add minor comments.
A lot of emphasis is put throughout the paper, and starting in the abstract, on the use of "a novel Gaussian process covariance kernel". In the introduction, "two innovations" are listed as 1) the kernel and 2) the acquisition function. I think that this is misleading, and to me the main contributions concern by far the acquisition function in the first place, and the model is more a modality of application. By the way, the proposed model is a quite simple particular case
of the Linear Model of Corregionalization (which the authors mention), corresponding to an M*(M+1) matrix [1,I], i.e. a concatenation of a vector of ones with an identity matrix. Besides this, it also has quite some similarity with the Kennedy and O'Hagan model that was used by Forrester et al. in optimization, without neither the hierarchical aspect nor regression coefficients. In all, I have the feeling that using the model presented by the authors is indeed a clever thing to do, but to me it doesn't constitute a major contribution in itself. Not also that if misoKG is compared to several competitors, I did not see comparisons of the proposed model (in terms of fit) versus others. Also, what would prevent from using other criteria with this model or other models with the misoKG aquisition function?
Other than that, let me formulate a few comments/questions regarding the main contributions presented in the third section. First, a finite set of points is
introduced at each iteration for the calculation and the optimization of the acquisition function. I am wondering to what extent such a set is needed on the one
hand for the calculation, on the other hand for the optimization, and finally if there are good reasons (beyond the argument of simplicity) why the same
set should be used for both tasks. I conceive that optimizing over l and x at the same time is uneasy, but couldn't one optimize over D for every l and then
compare the solutions obtained for the different l's? By the way, an approach was proposed in "Bayesian Optimization with Gradients" (arXiv:1703.04389) to
avoid discretization in the calculation of a similar acquisition function; could such approach be used here? If yes but a discretization approach is preferred
here, maybe a discussion on when considering discrete subsets anyway would make sense? On a different note, I was wondering to what extent the calculation tricks
used around lines 158-172 differ from the ones presented in the PhD thesis of P. Frazier and related publications. Maybe a note about that would be a nice addition.
Finally, I found the formulation of Theorem 1 slightly confusing. One speaks of a recommendation x_N^star in A and then take limit over N. So we have a sequence of
points in A, right? Besides, it all relies on Corollary 1 from the appendix, but there the reader is referred to "the full version of the paper"...I do not have
that in my possession, and as of now I found the proof of Corollary 1 somehow sketchy. I can understand that there are good reasons for that, but it is bit strange
to comment on a result which proof is in appendix, which itself refers to a full version one doesn't have at hand!
A few minor comments follow:
* About the term "biased" (appearing in several places): I understand well that a zero-mean prior on the bias doesn't mean at all that there is no bias, but
maybe manipulating means of biases without further discussion on the distinction between the bias and its model could induce confusion. Also, around lines 54-55,
it is written that "...IS may be noisy, but are assumed to be unbiased" followed by "IS are truly unbiased in some applications" and "Thus, multi-fidelity methods
can suffer limited applicability if we are only willing to use them when IS are truly unbiased...". As far as I understand, several methods including the ones
based on the Kennedy and O'Hagan settings allow accounting for discrepances in the expectations of the different IS considered. So what is so different between
multi-fidelity optimization and what is presented here regarding the account of possible "biases"? From a different perspective, other approaches like the one of
"Quantile-Based Optimization of Noisy Computer Experiments with Tunable Precision" are indeed relying on responses with common expectation (at any given x).
* In line 43, the "entropy" is the maximizer's entropy, right?
* Still in line 43: why would the one-step optimality mean that the proposed approach should outperform entropy-based ones over a longer horizon?
* About "From this literature, we leverage...", lines 70-72: as discussed above, it would be wothwhile to point out more explicitly what is new versus what is
inherited from previous work regarding these computational methods.
* In line 79 and in the bibliography: the correct last name is "Le Gratiet" (not "Gratiet").
* In line 84: without further precision, argmax is a set. So, the mentioned "best design" should be one element of this set?
* In line 87: the "independence" is a conditional independence given g, I guess...?
* In page 3 (several instances): not clear if the bias is g(.)-f(l,.) or delta_l(.)=f(l,.)-g(.)?
* In line 159, the first max in the expectations should have braces or so around a_i + b_i Z.
* In line 165: using c is a bit slippery as this letter is already used for the evaluation costs...
* In "The Parallel Algorithm": strange ordering between 1) and 2) as 2) seems to describe a step within 1)...?


AIDE: An algorithm for measuring the accuracy of probabilistic inference algorithms
Approximate probabilistic inference algorithms are central to many fields. Examples include sequential Monte Carlo inference in robotics, variational inference in machine learning, and Markov chain Monte Carlo inference in statistics. A key problem faced by practitioners is measuring the accuracy of an approximate inference algorithm on a specific data set. This paper introduces the auxiliary inference divergence estimator (AIDE), an algorithm for measuring the accuracy of approximate inference algorithms. AIDE is based on the observation that inference algorithms can be treated as probabilistic models and the random variables used within the inference algorithm can be viewed as auxiliary variables. This view leads to a new estimator for the symmetric KL divergence between the approximating distributions of two inference algorithms. The paper illustrates application of AIDE to algorithms for inference in regression, hidden Markov, and Dirichlet process mixture models. The experiments show that AIDE captures the qualitative behavior of a broad class of inference algorithms and can detect failure modes of inference algorithms that are missed by standard heuristics.

[after feedback]
Thank you for the clarifications. I've updated my score.
I do still have some concerns regarding applicability, since a gold standard "ground truth" of inference is required. I do appreciate the situation described in the feedback, where one is trying to decide on some approximate algorithm to "deploy" in the wild, is actually fairly common. That is, the sort of setting where very slow MCMC can be run for a long time on training data, but on new data, where there is e.g. a real-time requirement, a faster approximate inference algorithm will be used instead.
[original review]
This paper introduces a new method for benchmarking the performance of different approximate inference algorithms. The approach is based on constructing an estimator of the “symmetric” KL divergence (i.e., the sum of the forward and reverse KL) between an approximation to the target distribution and a representation of the “true” exact target distribution.
The overall approach considered is interesting, and for the most part clearly presented. I would agree that there is lots of potential in approaches which directly consider the auxiliary random variables which occur within an approximate inference algorithm. The meta-inference performed here (as described in definition 3.2) relates estimation of the probability density assigned to a particular algorithm output to a marginal likelihood computation, marginalizing out the latent variables in the algorithm; SMC and AIS are then natural choices for meta-inference algorithms.
The primary drawback of this paper is that the proposed divergence estimate D-hat provided by AIDE does not seem to lend itself to use as a practical diagnostic tool. This symmetric KL does not allow us to (say) decide whether one particular inference algorithm is “better” than another, it only allows us to characterize the discrepancy between two algorithms. In particular, if we lack “gold standard” inference for a particular problem, then it is not clear what we can infer from the D-hat — plots such as those in figure 3 can only be produced contingent on knowing which (of possibly several) inference algorithms is unambiguously the “best”.
Do we really need a diagnostic such as this for situations where “gold standard” inference is computationally feasible to perform? I would appreciate it greatly if the authors could clarify how they expect to see AIDE applied. Figure 5 clearly demonstrates that the AIDE estimate is a strong diagnostic for the DPMM, but unless I am mistaken this cannot be computed online — only relative to separate gold-standard executions. The primary use case for this then seems to be for providing a tool for designers of approximate inference algorithms to accurately benchmark their own algorithm performance (which is certainly very useful! but, I don’t see e.g. how figure 5 fits in that context).
I would additionally be concerned about the relative run-time cost of inference and meta-inference. For example, in the example shown in figure 4, we take SMC with 1000 particles and the optimal proposal as the gold standard — how does the computational cost of the gold standard compare to that of e.g. the dashed red line for SMC with optimal proposal and 100 meta-inference runs, at 10 or 100 particles?
Minor questions:
• Why are some of the inner for-loops in algorithm 2 beginning from m = 2, …, M, not from m = 1?


Consistent Robust Regression
We present the first efficient and provably consistent estimator for the robust regression problem. The area of robust learning and optimization has generated a significant amount of interest in the learning and statistics communities in recent years owing to its applicability in scenarios with corrupted data, as well as in handling model mis-specifications. In particular, special interest has been devoted to the fundamental problem of robust linear regression where estimators that can tolerate corruption in up to a constant fraction of the response variables are widely studied. Surprisingly however, to this date, we are not aware of a polynomial time estimator that offers a consistent estimate in the presence of dense, unbounded corruptions. In this work we present such an estimator, called CRR. This solves an open problem put forward in the work of [3]. Our consistency analysis requires a novel two-stage proof technique involving a careful analysis of the stability of ordered lists which may be of independent interest. We show that CRR not only offers consistent estimates, but is empirically far superior to several other recently proposed algorithms for the robust regression problem, including extended Lasso and the TORRENT algorithm. In comparison, CRR offers comparable or better model recovery but with runtimes that are faster by an order of magnitude.

Summary. This submission studies robust linear regression under the settings where there are additive white observation noises, and where \alpha=\Omega(1) fraction of the responses are corrupted by an oblivious adversary. For this problem, it proposes a consistent estimator, called CRR, which is computationally easy because it is based on iterative hard thresholding. It is shown to be consistent (i.e., converging to the true solution in the limit of the number of responses tending to infinity) under certain conditions (Theorem 4).
Quality. In Theorem 4, I did not understand how \epsilon is defined, and consequently, the statement of the theorem itself. Is it defined on the basis of the problem or algorithm setting? Or does the statement of the theorem hold "for any" \epsilon>0, or does "there exist" \epsilon for which the statement hold? The same also applies to \epsilon and e_0 in Lemma 5.
In Figure 1, the results with d=500, n=2000, \sigma=2, and k=600 appear in the subfigures (a), (b), and (c), but those in the subfigure (a) seem different from those in (b) and (c), which might question the validity of the results presented here.
Clarity. I think that this submission is basically clearly written.
Originality. Although I have not done an extensive survey in the literature, proposing a computationally efficient algorithm, with consistency guarantee in the limit of the number n of responses tending to infinity, for the robust linear regression problem with Gaussian noise as well as with oblivious adversary, when the fraction \alpha of adversarial corruptions is \Omega(1).
Significance. Even though the fraction \alpha of adversarial corruptions for which CRR is shown to be consistent is quite small (\alpha \le 1/20000), it is still nontrivial to give a consistency guarantee even if a finite fraction of responses are corrupted. It would stimulate further studies regarding finer theoretical analysis as well as algorithmic development.
Minor points:
Line 64: incl(u)ding more expensive methods
Line 76: Among those cite(d)
Line 82 and yet guarantee(s) recovery.
Line 83: while allowing (allow)
Lines 61, 102, and 115: The notation E[...]_2 would require explicit definition.
Equation (5): The equation should be terminated by a period rather than a comma.
Line 164: (more fine -> finer) analysis
Line 173: the (Subset) Strong Smoothness Property
Line 246: consequently ensure(s)
Lines 254-264: I guess that all the norms appearing in this part should be the 2-norm, but on some occasions the subscript 2 is missing.
Lines 285-286: Explicitly mention that the Torrent-FC algorithm was used for comparison. Simply writing "the Torrent algorithm" might be ambiguous, and also in the figures the authors mentioned it as "TORRENT-FC".
Line 293: faster runtimes tha(n) Torrent
Line 295: both Torrent (and) ex-Lasso become(s) intractable with a(n) order
Line 296: Figure(s) 2 (c) and 2(d) show(s)
Line 303: Figure 2 (a -> b) shows
Line 310: we found CRR to (to) be


Efficient Second-Order Online Kernel Learning with Adaptive Embedding
Online kernel learning (OKL) is a flexible framework for prediction problems, since the large approximation space provided by reproducing kernel Hilbert spaces often contains an accurate function for the problem. Nonetheless, optimizing over this space is computationally expensive. Not only first order methods accumulate O( √ T ) more loss than the optimal function, but the curse of kernelization results in a O(t) per-step complexity. Second-order methods get closer to the optimum much faster, suffering only O(log T ) regret, but second-order updates are even more expensive with their O(t 2 ) per-step cost. Existing approximate OKL methods reduce this complexity either by limiting the support vectors (SV) used by the predictor, or by avoiding the kernelization process altogether using embedding. Nonetheless, as long as the size of the approximation space or the number of SV does not grow over time, an adversarial environment can always exploit the approximation process. In this paper, we propose PROS-N-KONS, a method that combines Nyström sketching to project the input point to a small and accurate embedded space; and to perform efficient second-order updates in this space. The embedded space is continuously updated to guarantee that the embedding remains accurate. We show that the per-step cost only grows with the effective dimension of the problem and not with T . Moreover, the second-order updated allows us to achieve the logarithmic regret. We empirically compare our algorithm on recent large-scales benchmarks and show it performs favorably.

The paper proposes an efficient second-order online kernel learning mainly by combining KONS and Nystrom method.
      NOVELTY
      The novelty is limited on both the methodological and theoretical contributions. The achieved results do not have profound implication for the advancement of theory and practice.
      WRITING QUALITY
      The English writing and organization of this paper are relatively good.  The reviewer strongly suggests the authors arrange Table 2 in the main paper rather than in Appendix because the experimental results in Table 2 are the core material.
      COMMENTS
      This paper works on accelerating second-order online kernel learning and matches the desired state-of-the-art regret bound. The proposed method improves KNOS by using some efficient Nystrom methods instead of the true kernel matrix. However, it seems that this manuscript has not been completed, and many time and accuracy comparisons in Table 1 have not been reported. Although this paper claims that the effective dimension is too large to complete the results in line 341, this explanation for the incomplete experiments is not convincing. If the effective dimension is the concern, this paper should provide more theoretical and practical analysis to show how it affects the PROS-N-KONS. In addition, the reviewer does not find the complexity comparisons among the compared methods. Thus, it is not convincing at all t that the proposed method is efficient.
      Also, the reviewer has three concerns on this paper:
      1) In the experimental setup, how to determine the values of beta, epsilon, and bandwidth in the kernel matrix? For beta and epsilon, the related parameter selection will take much time. If the learning performance is very sensitive to these two parameters, the efficiency of the proposed method decreases. Regarding the bandwidth, it has an effect on both the effective dimension and the learning accuracy.
      2) The experimental comparison is not convincing. The efficiency is the key contribution of the proposed methods, but the run time comparison with baselines is missing. Moreover, the Dual-SGD is conducted in only one selected dataset, which makes the model comparison less reasonable.

      3) For the ridge leverage scores, it is surprising to find that the paper does not cite paper [Alaoui], which originally proposes the ridge leverage scores in Nystrom approximation with the application to regression analysis. Actually, to my understanding,  [Alaoui] only shows the size in Nystrom approximation could be reduced while does not claim their method would be faster than previous methods, and therein the leverage score approximation in  Theorem 4 could cost much time even more than the time spent in Theorem 3 (just check how to satisfy the condition of his Theorem 3 and 4). Hope this concern could be helpful to tackle your negative claim and results in line 341.
      [Alaoui] El Alaoui, A., Mahoney, M. W. (2014). Fast randomized kernel methods with statistical guarantees. Advances in Neural Information Processing Systems.


Triangle Generative Adversarial Networks
A Triangle Generative Adversarial Network (∆-GAN) is developed for semisupervised cross-domain joint distribution matching, where the training data consists of samples from each domain, and supervision of domain correspondence is provided by only a few paired samples. ∆-GAN consists of four neural networks, two generators and two discriminators. The generators are designed to learn the two-way conditional distributions between the two domains, while the discriminators implicitly define a ternary discriminative function, which is trained to distinguish real data pairs and two kinds of fake data pairs. The generators and discriminators are trained together using adversarial learning. Under mild assumptions, in theory the joint distributions characterized by the two generators concentrate to the data distribution. In experiments, three different kinds of domain pairs are considered, image-label, image-image and image-attribute pairs. Experiments on semi-supervised image classification, image-to-image translation and attribute-based image generation demonstrate the superiority of the proposed approach.

Triangle GAN is an elegant method to perform translation across domains with limited numbers of paired training samples. Empirical results seem good, particularly the disentanglement of known attributes and noise vectors. I'm worried, though, that the comparison to the closest similar method is somewhat unfair, and in general that the relationship to it is somewhat unclear.
Relation to Triple GAN:
Your statements about the Triple GAN model's requirements on p_y(y | x) are somewhat misleading. Both in lines 53-56 and section 2.5, it sounds like you're saying that p_y(y | x) must be known a priori, which would be a severe limitation to their model. This, though, is not the case: rather, the Triple GAN model learns a classifier which estimates p_y(y | x), and show (their Theorem 3.3) that this still leads to the unique desired equilibrium. In cases where p_y(y | x) does not have a density, it is true that their loss function would be undefined. But their algorithm will still simply estimate a classifier which gives high probability densities, and I think their Theorem 3.3 still applies; the equilibrium will simply be unreachable. Your Propositions 1 and 2 also break when the distributions lack a joint density, and so this is not a real advantage of your method.
The asymmetry in the Triple GAN loss function (8), however, still seems potentially undesirable, and your approach seems preferable in that respect. But depending on the complexity of the domains, it may be easier or harder to define and learn a classifier than a conditional generator, and so this relationship is not totally clear.
In fact, it seems like your image-label mappings (lines 164-173) use essentially this approach of a classifier rather than a conditional generator as you described earlier. This model appears sufficiently different from your main Triangle GAN approach that burying its description in a subsection titled Applications is strange. More discussion about its relationship to Triangle GAN, and why the default approach didn't work for you here,
Similarly, the claim that Triple GAN cannot be implemented on your toy data experiment (section 4.1) seems completely wrong: it perhaps cannot with exactly the modeling choice you make of using G_x and G_y, but p_x(x | y) and p_y(y | x) are simple Gaussians and can certainly be modeled with e.g. a network outputting a predictive mean and variance, which is presumably what the Triple GAN approach would be in this case.
Theory:
The claim that "p(x, y) = p_x(x, y) = p_y(x, y) can be achieved in theory" (line 120) is iffy at best. See the seminal paper
Arjovsky and Bottou. Towards Principled Methods for Training Generative Adversarial Networks. ICLR 2017. arXiv: 1701.04862.
The basic arguments of this paper should apply to your model as well, rendering your theoretical analysis kind of moot. This subarea of GANs is fast-moving, but really I think any GAN work at this point, especially papers attempting to show any kind of theoretical results like your section 2.3, needs to grapple with its relationship to the results of that paper and the followup Wasserstein GAN (your [22]) line of work.
As you mention, it seems that many of these issues could be addressed in your framework without too much work, but it would be useful to discuss this in the theory section rather than simply stating results that are known to be of very limited utility.
Section 4.4:
In Figure 6, the edited images are clearly not random samples from the training set: for example, the images where you add eyeglasses only show women. This adds a natural question of, since the images are not random, were they cherry-picked to look for good results? I would strongly recommend ensuring that all figures are based on random samples, or if that is impractical for some reason, clarifying what subset you used (and why) and being random within that subset.
Smaller comments:
Lines 30-33: ALI and BiGAN don't seem to really fit into the paradigm of "unsupervised joint distribution matching" very well, since in their case one of the domains is a simple reference distribution over latent codes, and *any* pairwise mapping is fine as long as it's coherent. You discuss this a little more in section 3, but it's kind of strange at first read; I would maybe put your citations of [9] to [11] first and then mention that ALI/BiGAN can be easily adapted to this case as well.
Section 2.2: It would be helpful to clarify here that Triangle GANs are exclusively a "translation" method: you cannot produce an (x, y) sample de novo. This is fine for the settings considered here, but differs from typical GAN structures enough that it is probably worth re-emphasizing e.g. that when you refer to drawing a sample from p(x), you mean taking a training sample rather than some form of generative model.