Equality of Opportunity in Classification: A Causal Approach
The Equalized Odds (for short, EO) is one of the most popular measures of discrimination used in the supervised learning setting. It ascertains fairness through the balance of the misclassification rates (false positive and negative) across the protected groups -e.g., in the context of law enforcement, an African-American defendant who would not commit a future crime will have an equal opportunity of being released, compared to a non-recidivating Caucasian defendant. Despite this noble goal, it has been acknowledged in the literature that statistical tests based on the EO are oblivious to the underlying causal mechanisms that generated the disparity in the first place (Hardt et al. 2016). This leads to a critical disconnect between statistical measures readable from the data and the meaning of discrimination in the legal system, where compelling evidence that the observed disparity is tied to a specific causal process deemed unfair by society is required to characterize discrimination. The goal of this paper is to develop a principled approach to connect the statistical disparities characterized by the EO and the underlying, elusive, and frequently unobserved, causal mechanisms that generated such inequality. We start by introducing a new family of counterfactual measures that allows one to explain the misclassification disparities in terms of the underlying mechanisms in an arbitrary, non-parametric structural causal model. This will, in turn, allow legal and data analysts to interpret currently deployed classifiers through causal lens, linking the statistical disparities found in the data to the corresponding causal processes. Leveraging the new family of counterfactual measures, we develop a learning procedure to construct a classifier that is statistically efficient, interpretable, and compatible with the basic human intuition of fairness. We demonstrate our results through experiments in both real (COMPAS) and synthetic datasets.

This is a well-written, detailed paper on an important topic. The paper seeks to decompose sources of discrimination into spurious, direct and indirect effects. It strengthens connections between the causal literature on direct effects to the literature on discrimination in classification by connecting to the notion of Equalized Odds.  Specific comments:  (1) It is not clear what is the overall goal here. Other papers e.g. Nabi and Shpitser talk about trying to remove the effects of "unfair" pathways. Are there specific pathways that the authors think are "unfair"? Why does it make sense to condition on y?  This is done in the EO measure, but is that a good idea? Some contrast relating conditional vs. marginal measure would be useful.  (2) There is a close connection between what are called in this paper "conditional" counterfactual effects and papers on the effect of treatment on the treated. Also effects in strata defined by post-treatment outcomes. For example: Marshall M. Joffe (2002) Using information on realized effects to determine prospective causal effects. https://doi.org/10.1111/1467-9868.00311  (3) The authors in some places suggest that the model in Figure 1(b) represents a canonical graph for this problem. This is inaccurate because the SCM here presumes that there are no unobserved variables (so that the U terms are independent). In general there could be unobserved latent variables giving rise to confounding between any pair of variables in X,Z,Y,M. (hat{Y} is not confounded since it is a function of predictors that are, by construction, observed.)  (4) Lines 122-124: The authors say that additional evidence (Y=y) mean that "the original causal diagram no longer captures in an explicit fashion all the underlying mechanisms that relate X and hat{Y}" This seems to be a category mistake: the presence of additional evidence can induce (probabilistic) independence or dependence but it does not change mechanisms!  To address this, the authors suggest modifying the causal diagram (line 126). It is not clear why one couldn't just condition on y and use d-separation/connection as usual. (As noted, it seems a conceptual error to think of information changing a causal process.)  (5) In the definition of conditional causal model why is M_e required. Note that in the subsequent definitions interventions are on x_0 and x_1, while the "evidence" is just x (which could be some other value that is not x_0 or x_1).  Related: why are the potential outcomes in (1), (2), (3) subscripted by "e" (or "y"). Why does it make sense to consider intervening on e or y (since this is a response (that doesn't directly affect hat{Y}. Could this be left out as a subscript / intervention here? (Obviously e or y need to be conditioned on.) I guess the text at lines 144-145 makes the point that these could be omitted - though it seems there is an error here (see below) - but this doesn't explain why they were included in the first place!  (6) Lines 233-236: The authors suggest that direct and indirect effects could be learned from a randomized experiment. Though true of "controlled direct effects", this is not true of "natural" direct and indirect effects  such as (1) and (2) that involve nested counterfactuals y_{x1,.. W_{x0}}.   J. Robins and T.S. Richardson.  (2011).  Alternative Graphical Causal Models and the Identification of Direct Effects.  Chapter 6.  Causality and Psychopathology: Finding the Determinants of Disorders and their Cures, pp. 1-52.   http://www.csss.washington.edu/Papers/wp100.pdf  As noted by Robins & Richardson, the assumption of independent errors in an SCM is not empirically testable via randomized experiments (since it implies independence of cross-world counterfactuals such as W(x0) and Y(x1,w).  Minor comments: line 3 Hartz et al is not in the Bibliography. [Hardt et al??] line 43 functions => function lines 144-145. "Since Y is a non-descendant node of W, hat{Y}, it has no effect in the counterfactuals before the conditioning bar in the factors of Eq.1"  this should be: Since W, hat_{Y} are non-descendants of Y... no effect *on*...  it is also a bit hard to follow because Y doesn't show up in these terms, it seems implicitly e=Y here (?)  line 155: defender => defendant (?) [Unless this is about football players!]  line 170 Equation (3) SE_{x0,x1}. This notation is a bit inconsistent with that for DE and IE because x0 and x1 are not counterfactuals.  line 184 featbure  line 199 may not been  line 208 remove "as"  line 240 and 241. End of clause (1)  and of clause (3). "blocked by by C" in which graph?  line 249. where P(hat{y}_{x,w}). I think this should be: P(hat{y}_{x,w,hat{z}})  line 262 the the  line 274: Theorem 3. The assumption of an SCM with independent errors  is required here. (This is not empirically testable with any randomized experiment.)  line 299 (before) Figure 4 (b) Experiment 2. What do the bars going from 0.1 to -0.1 mean? What is the difference between these and no bar at all?  lines 304-318. Is it the case that the counterfactual conditional contrasts (DE, IE etc.,) are known here because the generating mechanism / SCM is known? (In other words, no estimation is being carried out.)  line 323. smaller equalized *odds*  line 330 line 328 to 330. In the context of this real data example why would one believe that the conditions in Theorem 2 apply (i.e. ctf explainability). Is it reasonable to assume there are no unmeasured confounders?  ==================================================  The authors reply:  "3. Conditional Causal Models (R3) The conditional causal model does not modify the existing causal mechanisms, but only highlights the spurious and causal pathways that are not distinguishable by d-separation in the (standard) original causal model."  This sounds reasonable, but it is *not* in keeping with the text in the original paper. Specifically, lines 122-127 in the original paper say the following:  "This phenomenon exposes that, whenever an additional evidence is provided (Y = y), the original causal diagram no longer captures in an explicit fashion all the underlying mechanisms that relate X and hat{Y} We propose a simple modification of the causal diagram to account for the aforementioned extra mechanisms arising due to conditioning."   My guess is that the word "mechanism" here is the source of the problems. This suggests changing the "underlying" SCM. I think that if the authors replaced this with "active pathways" it would have caused much less confusion.


Learning convex bounds for linear quadratic control policy synthesis
Learning to make decisions from observed data in dynamic environments remains a problem of fundamental importance in a number of fields, from artificial intelligence and robotics, to medicine and finance. This paper concerns the problem of learning control policies for unknown linear dynamical systems so as to maximize a quadratic reward function. We present a method to optimize the expected value of the reward over the posterior distribution of the unknown system parameters, given data. The algorithm involves sequential convex programing, and enjoys reliable local convergence and robust stability guarantees. Numerical simulations and stabilization of a real-world inverted pendulum are used to demonstrate the approach, with strong performance and robustness properties observed in both.

Summary: The paper presents a model-based approach for controlling linear systems subject to quadratic costs with stability guarantees. Robust LQ control, yielding a controller K that stabilizes all dynamics (A, B) within an uncertainty set, has previously been studied by Dean et al.  This paper attempts to improve robust solutions (make them less conservative) by optimizing over an approximate dynamics posterior, rather than considering the worst case dynamics. The approach is as follows: - Obtain a set of dynamics samples from estimation posterior and restrict it to stabilizable samples. - Formulate the problem as an SDP in the inverse value matrix X^{-1} and controller K, where the objective is a conservative upper bound on the average control cost of all dynamics samples (A_i, B_i, Pi_i). This is the common Lyapunov (CL) solution. - Propose a less conservative convex upper bound as follows. Given an initial controller \bar{K}, find the dynamics sample with the corresponding lowest cost \bar{X}. Then solve a modified problem where each sample gets its own value matrix X_i but K is still shared, and each X_i^{-1} is approximated using a Taylor series around \bar{X}. This procedure can be applied iteratively to improve CL or other solutions. - The final K stabilizes all dynamics in the convex hull of the samples for the CL approach, and also for the relaxed (proposed) approach if an additional stability constraint is introduced (but I don’t think it is). - Experiments are run on small-scale synthetic dynamics similar to previous work. Data is obtained by exciting the system with Gaussian inputs and force-resetting to 0 every 6 time steps. The proposed algorithm (and variations) do seem to find a good tradeoff between stability and cost; while they occasionally yield unstable controllers, their cost is lower than that of the fully robust controllers. Experiments are also run on inverted pendulum with similar conclusions.  Quality & clarity. This is a high-quality paper with clear writing, explanations, and derivations.  It proposes an interesting approach to robust control of LQ systems with unknown dynamics which trades occasional instability for lower control costs.   Originality: The contributions are mostly extensions of previous work, though the new upper bound is original.   Wrt the SDP formulation: at the beginning of Section 4, authors state that the principal contribution (btw principle-->principal lines 138, 145) of their paper is an SDP formulation of the problem. However, SDP formulations of LQ control have been known for a while, and the paper should include at least one reference.  e.g.  - V. Balakrishnan and L. Vandenberghe. Semidefinite programming duality and linear time-invariant systems. IEEE Transactions on Automatic Control, 2003.  - A Cohen et al. Online linear quadratic control with adversarially changing costs, ICML 2018. (most recently) Standard SDP formulations just solve for X and subsequently obtain K= −(R + B’XB)^{−1}B’XA. The formulation in the paper is only different in that it also optimizes over K in order to tie multiple X_i using the same K.   Significance: While the results are nice (improve overly conservative solutions while mostly maintaining stability), this approach may not be too useful for practical problems due to increased computation reqiurements. Solving LQ control using SDP relaxations is practical only for small-scale systems. In this paper, the computation cost is amplified by the number of dynamics samples and the use of an iterative procedure. This is perhaps not an issue in this paper since the controllers are computed offline from force-reset episodes. But it limits the use of the developed technique in large-scale problems, as well as safety- and stability-critical adaptive settings.   A few additional questions about experiments: - What happens if you make your approach fully stable - is the cost substantially higher? - The nominal controller seems to find lowest-cost solutions (when stable). Why is it not included in the pendulum experiments?


Transfer Learning with Neural AutoML
We reduce the computational cost of Neural AutoML with transfer learning. AutoML relieves human effort by automating the design of ML algorithms. Neural AutoML has become popular for the design of deep learning architectures, however, this method has a high computation cost. To address this we propose Transfer Neural AutoML that uses knowledge from prior tasks to speed up network design. We extend RL-based architecture search methods to support parallel training on multiple tasks and then transfer the search strategy to new tasks. On language and image classification tasks, Transfer Neural AutoML reduces convergence time over single-task training by over an order of magnitude on many tasks.

This paper applies both multi-task training and transfer learning to AutoML. The paper extends the ideas presented in the Neural Architectura Search (NAS) technique (Barret Zoph and Quoc V. Le. Neural architecture search with reinforcement learning. In ICLR, 2017). The authors maintain the two-layer solution, with one network "the controller" choosing the architectural parameters for the "child" network which is used to solve the targeted task. The performance of the child network is fed back to the controller network to influence its results.   The novelty of this paper is in the way this two-layer solution is used. Instead of using the controller to decide the network for one task, multiple tasks are considered at once during the search phase (multi-task training). In addition, once the network is trained for a series of tasks, transfer learning is used to train the controller network to guide the design of the child network. With this technique in place, the accuracy performance of the child network achieves close to state-of-the-art results in a lot fewer iterations/trails in the search space, considerably reducing the time it takes to find the design of a performant network.  The results of the paper discuss two different types of tasks: NLP text classification and image classification. The NLP side of the analysis is a lot richer than the image classification. There are 21 tasks that are split randomly into two groups: 8 tasks used for the multi-task pre-training, while the rest of the tasks are used in the transfer learning setting. The 8-task pre-trained model is transferred to each of the remaining tasks individually.  The results show that the number of iterations for finding the top-10 best models is drastically reduced even when doing multi-task transfer learning for the neural architecture search. The performance of the top-10 models is on-par with state of the art numbers achieved by manually-designed network, which is a really nice result.  There are several way in which the paper could improve. I will list them in the order of importance:  1. My main concern with the paper is in the discussion of the time cost and savings achieved by this technique. I would have liked to see a discussion on how long it takes to pre-train the network on the 8 tasks before actually transfering it to a new task. I understand that this time is amortized across training networks for a lot of tasks, but I think it is an important detail of the technique that shouldn't be ignored.  2. The image classification part of the paper, while it shows that the technique works for a different domain, hence hopefully it generalizes, is rather weak. I'm guessing the results are not as interesting also due to the smaller dataset and simpler task.  3. There are few parts in the paper that could be explained a bit better. In particular, it's not entirely clear to me what the NAML approach really is. I'm guessing in the NAML case, instead of a random search, the two-layer controller-child network is used with feedback for only one task being fed from the child to the controller. Is NAML really the approach in NAS? If not, what are the differences?  Nit-picks: The arrangement of the figures is a bit strange. In particular, there doesn't seem to be a relationship among the graphs in Figure 4. I would try to rearrange them without prioritizing on esthetics, but rather on flow and understanding. Same goes for Figure 1. which is discussed towards the end of the paper.  Overall, I find the main idea of the paper simple and neat and I enjoyed reading the paper.  -------------------------- AFTER REBUTTAL: I really like the idea of applying Transfer Learning to AutoML. I think both areas are hot right now and having one example of transfer learning applied to AutoML is valuable to the community. That being said, for this paper to be referred to in the future, it needs to have clear writing for the Experimental Section. Most of the questions that were posed in the reviews stem from not understanding the experimental section and that's a pity. I hope the authors will take the feedback from these reviews (including the clarifications provided in the author response) and include them in the paper.


Stochastic Primal-Dual Method for Empirical Risk Minimization with O(1) Per-Iteration Complexity
Regularized empirical risk minimization problem with linear predictor appears frequently in machine learning. In this paper, we propose a new stochastic primaldual method to solve this class of problems. Different from existing methods, our proposed methods only require O(1) operations in each iteration. We also develop a variance-reduction variant of the algorithm that converges linearly. Numerical experiments suggest that our methods are faster than existing ones such as proximal SGD, SVRG and SAGA on high-dimensional problems.

******* After Rebuttal ***** The authors have addressed almost all my questions. I have decided not to increase my grade since my criticism on intuition & readability and time plots really depends on the quality of the re-write and the plots. In the rebuttal the authors have suggested that their method can be deduced using a saddle point formulation and they will include time plots, which I look forward to seeing. But still, it depends on the re-write. Furthermore on why I feel I cannot raise my grade, as pointed out by Reviewer 1, an application where data access is truly the bottleneck (such as a distributed setting) would be very welcome and would result in an excellent and well rounded paper i.e. good theoretical results, and a good practical result.  *****************************   The papers introduces a method with O(1) data access and O(1) cost per iteration (under the assumption that the regularizer is separable) for solving the primal-dual formulation of empirical risk minimization for generalized linear models. This is the first time I’ve seen a O(1) per iteration cost, and thus I find the paper quite interesting in the regime where loading data is truly the bottleneck. On the downside, very little intuition or any form of a derivation of the algorithms is offered, making the paper only suitable for experts in convex optimization and proximal methods. Under the assumption that the loss functions are Lipschitz and convex, the authors establish the O(1/sqrt(t)) and O(1/t) convergence for their algorithm when the regularizer is convex and strongly convex, respectively. They also extend their algorithm to a variance reduced version with linear convergence (under the additional assumption that the loss functions are smooth), with a rate that is comparable to SVRG. These theoretical results are as strong as could be hoped for and, together with some limited experiments, is a substantial contribution, and for this reason I recommend accepting the paper. The reason I do not recommend a strong accept is due to a minor issues that I detail next.  Extended reals and a bounded domain: For your proofs and calculations you assume g_j is a functional over a bounded domain which is why the solution to   x’ = prox_{g_j} (x)  depends on the indicator function as such  x’ \in partial g_j(x’) +(x’-x) + \partial 1_{X_j}(x’)  Thus g_j should be defined as g_j: X_j \mapsto \R \cup \{ \infty \}. This is only a minor detail, but important insofar that it occults the dependency on the bounded sets X_j and guarantees the feasibility of the iterates x^t. Without this clarification an inexperienced reader may not understand how feasibility is maintained. This raises the further question, what are these bounded domains in your experiments on logistic regression? And is the projection onto these bounded domains computed in practice, or is this only a technical requirement for the proofs? This should be clarified.  Clarify assumptions earlier on:  I believe the assumptions need to be brought forward. Since the major selling points of the algorithm is that each iteration costs only O(1), as stated in the abstract. But this only holds for separably regularizers where both the loss and regularizer are prox friendly. This would even fit and be appropriate in the abstract.    Intuition & readability: Very little intuition of explanations are offered for Algorithm 1 and 2 aside from stating that Algorithm 1 is, in some sense, a randomized coordinate descent type method. But this is not clear, and without such clarifications I feel the paper is only accessible to experts. The paper would benefit from a derivation or better explanation of either algorithm. In particular, relating the development of your methods to the Primal Dual Hybrid Gradient (PDHG) of Chambolle and Pock [1] would help ground the reader. Furthermore it seems this paper [2] might be quite related since they do formally take a coordinate ascent step in a PDHG type method (though they do not use and stochastic gradient step). Note you still have 1/3 of a page to fill before reaching the 8 page maximum.  Experiments: The experiments only report the efficiency of the algorithms in terms of data passes. I understand that in certain settings this may indeed be the dominant cost, and the experiments nicely illustrate how the new algorithms perform well in this setting since they only require O(1) data access per iteration. But including a metric of either time or approximate flops would be very informative. It is understandable that these new O(1) algorithms will perhaps not perform well in clock time on a single cpu, but it would enrich the understanding of the empirical performance of these new algorithms to have additional metrics. This can also be partly remedied by simulating a distributed setting, or implementing a parallel implementation. Another minor issue with the experiments is that the logistic loss is not truly prox friendly. Though, as pointed out by the authors, a prox step over a logistic loss can be efficiently computed with a Newton type method in one dimension, this is still not the same as an exact prox step, indeed you cannot perform an exact prox step using an iterative algorithm.   Finally, I found the following minor mistakes:  line 16: … independent of n line 20: … at a sublinear rate … line 193: To allow for greater reproducibility, the authors should explain how they tuned the stepsizes. A grid search over a certain grid?   Supplementary material: line 20: There is a missing ``\partial’’ from ``s \in \partial g_j(x_j^{‘t}) + \partial ...  Sentences ending with a comma: such as on line 28 and 32.   [1] A first-order primal-dual algorithm for convex problems with applications to imaging, Antonin Chambolle, Thomas Pock  [2] STOCHASTIC PRIMAL-DUAL HYBRID GRADIENT ALGORITHM WITH ARBITRARY SAMPLING AND IMAGING APPLICATIONS, Antonin Chambolle, Matthias Ehrhardt, Peter Richtarik and Carola-Bibiane Schönlieb.  Justifying confidence score: I am an expert in stochastic algorithms for solving the convex empirical risk minimization problem. I am also very familiar with randomized coordinate type algorithms in this and other settings. This is why I am confident in my assessment.


Fast Greedy MAP Inference for Determinantal Point Process to Improve Recommendation Diversity
The determinantal point process (DPP) is an elegant probabilistic model of repulsion with applications in various machine learning tasks including summarization and search. However, the maximum a posteriori (MAP) inference for DPP which plays an important role in many applications is NP-hard, and even the popular greedy algorithm can still be too computationally expensive to be used in largescale real-time scenarios. To overcome the computational challenge, in this paper, we propose a novel algorithm to greatly accelerate the greedy MAP inference for DPP. In addition, our algorithm also adapts to scenarios where the repulsion is only required among nearby few items in the result sequence. We apply the proposed algorithm to generate relevant and diverse recommendations. Experimental results show that our proposed algorithm is significantly faster than state-of-the-art competitors, and provides a better relevance-diversity trade-off on several public datasets, which is also confirmed in an online A/B test.

Summary: This paper introduces an exact algorithm for greedy mode finding for DPPs which is faster by a factor of M (ground set size) than previous work on greedy MAP algorithms for DPPs; the authors also show that this algorithm can be further sped up when diversity is required over only a sliding window within long recommendations. As an additional contribution, the authors show that modeling recommendation problems with DPPs and generating recommendations via their algorithm outperforms other standard (non-DPP) recommender algorithms along various metrics.    -------------------  This paper builds upon the recent growing interest in leveraging Determinantal Point Processes for various machine learning problems. As the authors mention, a key advantage of DPPs is their ability to tractably balance quality and diversity requirements for most operations, with mode estimation being one of the only operations that remains NP-hard. Indeed, sampling from a DPP has been used in previous literature, presumably as a more scalable alternative to greedy MAP finding (e.g. for network compression). Although the usefulness of DPPs for recommender systems is now an accepted fact, the analysis provided in section 5 and 6.2 remains interesting, in particular thanks to the discussion of the tunable scaling of diversity and quality preferences and how it can easily be incorporated into the new formulation of the greedy algorithm. However, I would like to see a detailed theoretical comparison of the proposed greedy algorithm to the derandomized version of the lazy DPP sampling algorithm proposed in the thesis (Gillenwater, 2014).   --------------------  Quality: This paper is of good quality. Clarity: This paper is very clear. Originality: This paper is marginally novel - see below regarding its potential similarities to prior work. To my knowledge, however, there has not been an explicit mention of a fast greedy algorithm for DPP sampling prior to this submission. Significance: Fast greedy algorithms for DPPs are significant, as well as more broadly within recommendation systems (although the M^3 complexity might be a more general impediment to the use of the greedy MAP algorithm for DPPs within recommendations over very large set sizes).   --------------------  Detailed comments:  - Could you please detail the difference between your proposed algorithm and a derandomized greedy version of Alg. 2 of "Approximate Inference for DPPs" (Gillenwater, 2014) that uses all eigenvalues for the variable set V? In particular, the initialization of your variables d_i^2 is the same as that of the z_i of Gillenwater's Alg. 2, and the update rules also seem similar (although they are derived using a different logic).   - You might want to explicitly mention that higher MRR/ILAD/... are desirable for readers who aren't familiar with recommender systems. - Could you provide some intuition as to how the greedy algorithm performs when compared to simple DPP sampling for recommendations? It would be interesting to see how much of a gain the greedy approach provides over just sampling multiple times (O(M^3) if you need to compute the kernel eigendecomposition) from the DPP and keeping the subset with the highest probability.


Non-monotone Submodular Maximization in Exponentially Fewer Iterations
In this paper we consider parallelization for applications whose objective can be expressed as maximizing a non-monotone submodular function under a cardinality constraint. Our main result is an algorithm whose approximation is arbitrarily close to 1/2e in O(log 2 n) adaptive rounds, where n is the size of the ground set. This is an exponential speedup in parallel running time over any previously studied algorithm for constrained non-monotone submodular maximization. Beyond its provable guarantees, the algorithm performs well in practice. Specifically, experiments on traffic monitoring and personalized data summarization applications show that the algorithm finds solutions whose values are competitive with state-of-the-art algorithms while running in exponentially fewer parallel iterations.

The authors consider the parallel complexity of non-monotone submodular maximization subject to a cardinality constraint. They develop an algorithm for this problem and they are able to show it has a (1 / 2e)-factor optimization for this problem in polylog(n) parallel rounds. They experimentally observe that their algorithm is able to achieve a good approximation in far fewer rounds.  The theoretical results of this paper are very strong. It is typically difficult to obtain good approximation guarantees for non-monotone functions.  The paper is well written.  The author's algorithm is a tweak of the algorithmic framework proposed in [BS18a, BS18b, BRS18]. Some of the proof ideas are similar to BRS18. The authors should better differentiate the proof for this problem by discussing the difficulties unique to this problem.  One important experiment missing is running a real parallel or distributed implementation of the algorithm and observing how well the algorithm parallelizes in practice. How does the performance improve with the number of processors and how does this compare to parallel implementations of other algorithms such as the greedy algorithm?  The guess-and-check idea for the algorithm is related to the algorithm in the following work: Badanidiyuru, Ashwinkumar, et al. "Streaming submodular maximization: Massive data summarization on the fly." Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2014. The authors should cite this work.  One improvement to the psedocode:  if E_{R ~ U(X)}[...] <= ..., then sample R ~ U(X) and return R \cap X^+  As written, it was somewhat confusing at first with R as both the variable under the expectation and the random sample.  The authors should clearly write the value of k used in the experiments, as this defines the number of iterations for the competing algorithms.  *** Post-Rebuttal ***  Thank you for your response. While I agree that the experiments provided bring insights that a real parallel implementation would not, I still think a real parallel implementation would be useful. I think simply seeing how the running time decreases as the number of CPUs increases would be insightful. Also it seems to me that the RandomGreedy algorithm has a natural Map/Reduce style parallel algorithm that could be used for comparison.


A Neural Compositional Paradigm for Image Captioning
Mainstream captioning models often follow a sequential structure to generate captions, leading to issues such as introduction of irrelevant semantics, lack of diversity in the generated captions, and inadequate generalization performance. In this paper, we present an alternative paradigm for image captioning, which factorizes the captioning procedure into two stages: (1) extracting an explicit semantic representation from the given image; and (2) constructing the caption based on a recursive compositional procedure in a bottom-up manner. Compared to conventional ones, our paradigm better preserves the semantic content through an explicit factorization of semantics and syntax. By using the compositional generation procedure, caption construction follows a recursive structure, which naturally fits the properties of human language. Moreover, the proposed compositional procedure requires less data to train, generalizes better, and yields more diverse captions.

*Summary* The paper proposes a novel approach to image captioning, which adopts a recursive approach to assembling concepts detected from a visual pipeline, based on picking noun phrases and connecting them with relation phrases, which disentangles the content from the syntacic information (to a first order). The paper then shows that using such an explicit factorization in the generation mecahnism can help achieve better diversity in captions, and better generalization with lesser amount of data, and really good results on the SPICE metric which focusses more on the syntactic structure of image cpations.  *Strengths* + The paper has a pretty comprehensive list of references + The experimental results seem really thorough and consistently done, compared to most image captioning papers. The same CNN is used for all the methods and hyperparameters are all set very systematically. + It is very interesting that the proposed approach does really well on SPICE, but significantly poorly on other metrics, and it makes a lot of sense given the SPICE metric that such a method should do well on it, as well. + The experiment on using the ground truth noun phrases and using the compositional part of the model is also really nice, and gives an insight into the what the model has learnt and how performance with this paradigm might improve with better visual detectors.  *Weakness*  1. Related work which would be good to cite is [A], which also first predicts the noun phrases and then uses that to influence the structure of the captionr, and It would be good to compare and contrast both conceptually what the differences are and otherwise. 2. L133-134: Not clear what we mean by not only the complete captions but also parts thereof, and generally argument around why filling in the blanks is hard is unconvincing here. Note that there are ways to do bi-directional beam search for fill in the blanks kind of applications [C]. 3. How time consuming is the forward pass to create the caption? It seems like a pretty computationally intensive approach. How sensitive is performance to the number of noun phrases which get selected? (L202). 4. How do the statistics of the relation phrases look like? How often are the rarer relation phrases actually predicted in a sentence? Chunking things into relation phrases could have the disadvantage that one might see individual words but not the phrase at once and now we are stuck with the problem of heavy tail in relations (which is more severe than heavy tail in words).  5. Fig. 4: Why is there a bump between 0.0 and 0.1 for CompCap, and why is performance so high even at 0.0? It is realy confusing.  6. What is the training objective used for training the whole model? How are modules and losses set up? Are all the operations performed differentiable? It seems like the model makes hard decisions when picking which noun phrase to use or which relation phrase to use, does this mean there are only sequence level rewards and the whole model is trained via REINFORCE/ policy gradient? It seems like the paper is incomplete in this regard. (*) 7. L227: in a lot of ways the experiment in this paragraph is not fair, since the baseline approaches are learning to caption from scratch while the proposed approach is using noun phrase detectors trained on the full data. It would be good to report what happens if we also train the noun phrase detectors only on a part of the data. (*)  *Preliminary Evaluation* There is a major detail about the training objective and algorithms missing from the paper, however, the crux of the approach that is presented is very interesting, and the results as well as experimental analysis look great and are very thorough and creative. This would be an interesting contribution to the conference, but important details about the paper are missing, which should be fixed, marked with (*) above.  References: [A]: Gan, Zhe, Chuang Gan, Xiaodong He, Yunchen Pu, Kenneth Tran, Jianfeng Gao, Lawrence Carin, and Li Deng. 2016. “Semantic Compositional Networks for Visual Captioning.” arXiv [cs.CV]. arXiv. http://arxiv.org/abs/1611.08002. [B]: Huang, Qiuyuan, Paul Smolensky, Xiaodong He, Li Deng, and Dapeng Wu. 2017. “Tensor Product Generation Networks.” arXiv [cs.CV]. arXiv. http://arxiv.org/abs/1709.09118. [C]: Sun, Qing, Stefan Lee, and Dhruv Batra. 2017. “Bidirectional Beam Search: Forward-Backward Inference in Neural Sequence Models for Fill-in-the-Blank Image Captioning.” arXiv [cs.CV]. arXiv. http://arxiv.org/abs/1705.08759.


Multi-domain Causal Structure Learning in Linear Systems
We study the problem of causal structure learning in linear systems from observational data given in multiple domains, across which the causal coefficients and/or the distribution of the exogenous noises may vary. The main tool used in our approach is the principle that in a causally sufficient system, the causal modules, as well as their included parameters, change independently across domains. We first introduce our approach for finding causal direction in a system comprising two variables and propose efficient methods for identifying causal direction. Then we generalize our methods to causal structure learning in networks of variables. Most of previous work in structure learning from multi-domain data assume that certain types of invariance are held in causal modules across domains. Our approach unifies the idea in those works and generalizes to the case that there is no such invariance across the domains. Our proposed methods are generally capable of identifying causal direction from fewer than ten domains. When the invariance property holds, two domains are generally sufficient.
leveraged for the purpose of causal discovery, which is our focus in the paper. Moreover, it has been shown that causal modeling can be exploited to facilitate transfer learning [SJP + 12,ZSMW13]. This is because of the coupling relationship between causal modeling and distribution change: Causal models constrain how the data distribution may change, while distribution change exhibits such changes.
In this paper, we focus on linear systems, with observational data for the variables in the system given in multiple domains, across which the causal coefficients and/or the distribution of the exogenous noises may vary. We will present efficient approaches to exploiting such changes for causal structure learning. The proposed methods are based on the principle that although the cause and the effect variables are dependent, the mechanism that generates the cause variable changes independently from the mechanism that generates the effect variable across domains. The same principle was utilized in [ZHZ + 17] for exploiting non-stationary or heterogeneous data for causal structure learning. However, since that work considers a non-parametric approach, it is restricted to general independence tests among distributions, which may not have high efficiency. The remaining previous works on causal discovery from multiple domains usually assume that a certain type of invariance holds across domains [PBM16,GSKZ17]. Our approach unifies the idea in those works, and generalizes to the case that there is no such invariance across the domains. We present structure learning methods, which are generally capable of identifying causal direction from fewer than ten domains (as a special case, when the invariance property holds, two domains generally suffices).
The rest of the paper is organized as follows. We first introduce our approach for finding causal relations from multi-domain observational data in a system comprising two variables in Section 2. We propose our methods for identifying causal direction in Subsections 2.1 and 2.2. These methods are evaluated in Subsection 2.3. We generalize our method to causal structure learning in networks of variables in Section 3, where the generalized version of our methods are presented in Subsections 3.1 and 3.2 and evaluated in Subsection 3.4. We also present an approach for combining our proposed methods with standard conditional independence-based structure learning algorithms in Subsection 3.3. All the proofs are provided in the supplementary materials.

This paper proposes an approach to learn causal directionality -- first in the two-variable case, then generalized to DAGs, in the settings where multiple datasets of the same system are recorded across different "domains" (could be different time points, or different measurement instruments for example). The approach is based on an assumption called the "independent change condition" (ICC): for cause C and effect E, P(C|E) and P(E) should change independently across different domains. Further, linearity is assumed.  Under these assumptions, the authors develop interesting algorithms that first identify causal directions in the two-variable case, which is then generalized to structure learning. Importantly these algorithms could be used with a partial result from another structure learning method, such as PC, as input to identify further possible orientations.  Strong aspects of the paper are:  - The idea is innovative (the ICC concept itself is not new, but the algorithmic ideas appear to be) and leads to an interesting class of methods, which might be extensible further.  - The paper is well-written and clear (up to a few issues noted below).   - Due attention is given to the efficiency of the statistical testing procedures that underpin the algorithm. Two variants are presented in Section 3 that substantially increase power and reduce computation efforts compared to "naive" approaches. I emphasize this because structure learning papers sometimes tend to neglect the issues surrounding statistical testing.  Points of criticism are:   - Significance: I frankly speaking do not understand why we the ICC would be a good assumption. In the article, only two literature references are given (which by the way are hard to locate due to errors in the bibliography: DJM+10 appears to have a wrong author list and ZHZ+17 has a wrong title). I would have liked more motivation or perhaps a real-world example explaining why this would be a good assumption. If we allow all  parameters to change across domains, why would specific parameters change only independently?  - Clarity: I had trouble with Definition 3, which is central for understanding Theorem 4. What does "ordering" mean here precisely? It cannot be just a permutation of the variables, as one might suspect when looking at Theorem 4, because a permutation has no edges. From the examples given it seems that "ordering"  means instead "DAG" but that cannot be true either since otherwise all DAGs with no edges would be consistent with all other DAGs according to the definition. My working assumption was that order here means in fact a "connected partial order", in which case Theorem 4 appears to hold, but I might be misunderstanding this. It would be very useful if the authors could clarify this.  - Quality: The paper would benefit from a real-world example to demonstrate the applicability of the proposed techniques. Only simulated data is being used at the moment.  Minor points: - The paper should have a conclusion, even if it's only brief. - Assumption 1: I was wondering here why this is called "Faithfulness". You do explain this later after Assumption 2 (for the network case) and I'd suggest moving up this explanation. - Algorithm 1: I was wondering if it is really so important in which way you compute these regression coefficients and residual variances . Wouldn't this provide the same results as, for instance, simply running the regression of each variable on each other variable with higher order? I understand that you need to explain this somewhere to get to precise algorithm runtimes but lines 215 to 224 seemed to interrupt the flow of exposition a little and seemed to get in the way of your explanation of the actual structure learning approach. Perhaps it could just be mentioned that there's a way to compute all required parameters in time O(p^3) and put the details into supplement.  = After author feedback =  Thank you for your responses. A clarification of the "ordering" concept in the final version would indeed be greatly appreciated.


GradiVeQ: Vector Quantization for Bandwidth-Efficient Gradient Aggregation in Distributed CNN Training
Data parallelism can boost the training speed of convolutional neural networks (CNN), but could suffer from significant communication costs caused by gradient aggregation. To alleviate this problem, several scalar quantization techniques have been developed to compress the gradients. But these techniques could perform poorly when used together with decentralized aggregation protocols like ring all-reduce (RAR), mainly due to their inability to directly aggregate compressed gradients. In this paper, we empirically demonstrate the strong linear correlations between CNN gradients, and propose a gradient vector quantization technique, named GradiVeQ, to exploit these correlations through principal component analysis (PCA) for substantial gradient dimension reduction. GradiVeQ enables direct aggregation of compressed gradients, hence allows us to build a distributed learning system that parallelizes GradiVeQ gradient compression and RAR communications. Extensive experiments on popular CNNs demonstrate that applying GradiVeQ slashes the wall-clock gradient aggregation time of the original RAR by more than 5X without noticeable accuracy loss, and reduces the end-to-end training time by almost 50%. The results also show that GradiVeQ is compatible with scalar quantization techniques such as QSGD (Quantized SGD), and achieves a much higher speed-up gain under the same compression ratio.

This describes how a PCA during full-vector can be used to predict a good compression method. They do a good job arguing that PCA among gradient vectors should be a common case in machine learning. The results look good.  On the down side, it requires a fair amount of coding work to include it in what you do, because you still have a periodic "full gradient" phase. The PCA and how it is approximated are practical heuristics, so I don't expect a proof to be possible without a bit of fine print. I don't think there was a discussion of what happens with rare classes.  In this last case, some gradients might be really badly represented.  In such cases, I think you can detect the badly compressed gradient representations and "do something different".  But I think the paper is OK as is, because it proposes a clean approach that can hopefully be compared with the other "do something different" compression schemes that target more the communication than the aggregation cost of distributed SGD methods.  #### After authors' rebuttal and other reviews #### I thought the authors responded well to the reviewers' questions and propose some good changes to the paper.  The rebuttal did not really address my class imbalance concern, which I guess also bothered Reviewer #3, who commented about the time-invariance assumption.  In both cases a bothersome feature is that the applicability of a PCA assumption for gradient compression might break down, and you might not notice it.  So if "GradiVeQ is able to detect compression performance by measuring the training compression loss" [rebuttal] refers to a locally measured compression fidelity, this should be in the paper --- if you can formulate a loss that degrades somewhat gracefully over the time-invariance "L" period, then both Reviewer-3 and  I might be satisfied that a feature implementation has a decent chance of automagically detecting when the compressor's assumptions are beginning to malfunction.  I'm keeping my overall score the same.


SLAYER: Spike Layer Error Reassignment in Time
Configuring deep Spiking Neural Networks (SNNs) is an exciting research avenue for low power spike event based computation. However, the spike generation function is non-differentiable and therefore not directly compatible with the standard error backpropagation algorithm. In this paper, we introduce a new general backpropagation mechanism for learning synaptic weights and axonal delays which overcomes the problem of non-differentiability of the spike function and uses a temporal credit assignment policy for backpropagating error to preceding layers. We describe and release a GPU accelerated software implementation of our method which allows training both fully connected and convolutional neural network (CNN) architectures. Using our software, we compare our method against existing SNN based learning approaches and standard ANN to SNN conversion techniques and show that our method achieves state of the art performance for an SNN on the MNIST, NMNIST, DVS Gesture, and TIDIGITS datasets.
Event based methods such as SpikeProp [10] and EvSpikeProp [11] have the derivative term defined only around the firing time, whereas [12][13][14] ignore the temporal effect of spike signal. In Section 3.1 we describe the strengths and weaknesses of these approaches in more detail.
The main contribution of this paper is a general method of error backpropagation for SNNs (Section 3) which we call Spike LAYer Error Reassignment (SLAYER). SLAYER distributes the credit of error back through the SNN layers, much like the traditional backprop algorithm distributes error back through an ANN's layers. However, unlike backprop, SLAYER also distributes the credit of error back in time because a spiking neuron's current state depends on its previous states (and therefore, on the previous states of its input neurons). SLAYER can simultaneously learn both synaptic weights and axonal delays, which only a few previous works have attempted [15,16].
We have developed and released 3 a CUDA accelerated framework to train SNNs using SLAYER. We demonstrate SLAYER achieving state of the art accuracy for an SNN on neuromorphic datasets (Section 4) for visual digit recognition, action recognition, and spoken digit recognition.
The rest of the paper is organized as follows. We start by introducing notation for a general model of a spiking neuron and extending it to a multi-layer SNN in Section 2. Then, in Section 3 we discuss previously published methods for learning SNN parameters before deriving the SLAYER backpropagation formulae. In Section 4, we demonstrate the effectiveness of SLAYER on different benchmark datasets before concluding in Section 5.

This work contributes a new error backpropagation method for spiking neural networks which is capable of learning both weight and axonal delay parameters in a spiking neural network, and establishes a new SOTA on multiple tasks.  This work helps address the long-held challenge in the neuromorphic community of designing a learning algorithm capable of learning (approximately) as well as standard deep neural networks for fixed inputs, but is also capable of learning from time-varying signals which more closely match the native representation of spiking neural networks.  This submission was prepared carefully and clearly, with a thorough set of comparison benchmarks to other standard spiking benchmarks.  The benchmarks further span both image and audio challenges.  Recent work is compared against and referenced, and CNNs and fully-connected architectures similar to those used in the state-of-the-art are used.  The derivation of the modified backpropagation rule is well-described.  Further, a GPU-optimized toolbox will also be released, all bolstering the work's quality, clarity, and significance.  Discretizing temporal history to enable backpropagation and approximating a Dirac function with a more easily-differentiable function is not an entirely surprising approach, but it is used effectively here.  It would be useful to see the rule tested on more difficult temporal data than TIDIGITS, in which the SOM on top of MFCCs provides significant precomputation; similarly, a more thorough examination of learning axonal delays would improve the motivation for adding the additional complexity.  For example, do learned delays in NMNIST correspond to saccade timing, or just play a role in expanding the parameter space for learning?  Finally, some of the choices, e.g. spike rates for true vs. negative classes seem arbitrary.  Why were 180 spikes/sec used in DVS gesture classification for true and 30 spikes/sec used for false classes? The work could be improved by comparing against ANNs on spiking data, and referencing works such as [1] that featurize on top of neuromorphic data to suggest whether the limited performance is now more from the SNN models or the input streams themselves.  However these are small points.  Overall, this work has the potential to strongly impact researchers working in neuromorphic models.  [1] Anumula, Jithendar, et al. “Feature Representations for Neuromorphic Audio Spike Streams.” Frontiers in Neuroscience 12 (2018): 23. https://doi.org/10.3389/fnins.2018.00023.


Low-shot Learning via Covariance-Preserving Adversarial Augmentation Networks
Deep neural networks suffer from over-fitting and catastrophic forgetting when trained with small data. One natural remedy for this problem is data augmentation, which has been recently shown to be effective. However, previous works either assume that intra-class variances can always be generalized to new classes, or employ naive generation methods to hallucinate finite examples without modeling their latent distributions. In this work, we propose Covariance-Preserving Adversarial Augmentation Networks to overcome existing limits of low-shot learning. Specifically, a novel Generative Adversarial Network is designed to model the latent distribution of each novel class given its related base counterparts. Since direct estimation of novel classes can be inductively biased, we explicitly preserve covariance information as the "variability" of base examples during the generation process. Empirical results show that our model can generate realistic yet diverse examples, leading to substantial improvements on the ImageNet benchmark over the state of the art.

*** After rebuttal: Thank you for the additional experiments and clarifications, they were very helpful. I feel that both the precise training algorithm as well as the distinction of the two types of prototypes are important points to add to the revised version. I agree with the other reviewers that Section 3 can be written in a more clear way, and it would also be helpful to double-check the text for grammar for the final version.  *** Original review: This paper put forward a novel approach to low-shot learning by feature augmentation. The problem of low-shot learning is to learn classifiers between a set of previously-unseen classes given only a few examples of each. It is assumed that a potentially large set of ‘training’ or ‘base’ classes is given, but this set is disjoint from the set of classes that will be presented at test time. Given the insufficiency of the training data for learning a low-shot classifier, generating additional data is a reasonable strategy and there has been increasingly more research on this recently (the authors correctly cite the relevant works).   The approach presented here works by ‘translating’ examples of the base classes into examples of novel classes, inspired by unsupervised set-to-set translation methods. This method is designed to overcome a weakness of previous work with regards to the variance of these generated examples. In particular, since the true distribution of the novel class is not known, it’s difficult to estimate its intra-class variance. Previous works have implicitly assumed that all classes have the same variance, which is an unrealistic assumption. This work tackles this problem by instead assuming that a novel class will have similar covariance to the covariance of its most related base classes.   In every low-shot learning episode, comprised of a set of N novel classes, they sample a set of base classes that are relevant to those novel classes. The relevance between a novel and base classes is determined via a metric inspired by the prototypical network approach: a prototype is created for each base class, which is the average representation of its examples, and a prototype is created for each novel class similarly. The base class prototypes that are the nearest-neighbors of a novel class prototype correspond to its most related classes. In every such batch consisting of N novel classes and the set of most relevant base classes, an adversarial network is employed to ‘transform’ each base class example into a novel class example. The GAN is trained as usual with the difference that the generator’s classification task has (N+1) targets: each example will either belong to one of the N novel classes of the batch, or is a generated example (ie an example that is transformed from one of the related base classes into something that should look like an example of a novel class). Successful training of this GAN will make the generated examples of the novel classes indistinguishable from the real examples of the novel classes. They then introduce additional components to the loss function to match the covariance of the generated examples for a class to the covariance of the real class from which they were transformed.  Pros - Using the intra-class variance of classes that are related to the novel class seems like a reasonable proxy for the intra-class variance of the novel class. Generating examples this way seems indeed like a promising way of sampling more diverse examples that are still plausible representatives of the novel class. - They progressively add layers of complication to their method. By evaluating each of these variants they justify that their final approach is well-motivated and all its components are necessary for achieving good performance. The simplest version is a standard conditional GAN whose purpose is to translate examples of base classes into examples of novel classes such that these generated examples are indistinguishable from the real examples of novel classes. Then, they take extra steps to ensure that the distribution of the base class that is most related to a novel class is faithfully maintained during translation. They do this using a cyclized GAN objective. Their final variant adds an additional component to the loss that explicitly penalizes differences in the covariance between the related base class and the translated examples for a given novel class. - They verify both quantitatively and qualitatively that their generated images are more diverse than competing approaches - Their method achieves state-of-the-art results on ImageNet  Cons - An important component of the algorithm was left unclear: how the training progresses. One potential option would be to train episodically, through a series of training episodes each of which mimic a low-shot test episode. In that case, N base classes would be sampled for a training episode, and their most related other base classes would be included (as determined by class prototypes) would contribute the examples that would be ‘translated’ into the N chosen base classes. Then a classifier for this episode would be learned using the k real examples of each of the N classes, together with the fake examples. Finally, based on the performance of this newly learned classifier on a disjoint set of examples originating from these N classes, the model’s weights are updated (e.g. embedding network, generator parameters, etc). Though this seems natural, it’s not stated whether this is indeed how training proceeds. It’s important to clarify this point. - Clarity is sometimes compromised due to using certain words incorrectly. The word ‘incur’ is used multiple times in a confusing way for example. - Experimentally it would be nice to compare to other low-shot learning methods that do not necessarily work by feature augmentation. For example metric learning and meta-learning methods present a different solution which is shown to be effective as well. It would be nice to compare against some representatives of these families, e.g. Prototypical Network, Matching Network, Siamese Network, MAML (Finn et al, 2017).   Other comments: - It should be noted that this approach utilizes an additional piece of information than many of the competitors: it utilizes the data of the base classes not only at meta-training time but also during meta-testing. In particular, even at test time the real data of the base classes constitutes the input to the generator for translating those images into fake data of the novel classes. - A prototypical network is an episodic model: it averages the representations of the points *of a given episode* that belong to the same class to form the prototype of that class. Here, as I understand it, to form the prototype of a class, all examples in the entire base set that belong to that class are averaged. Therefore I wouldn’t call this a Prototypical Network. This process can just be explained as representing each class as a cluster whose mean is the average of the features of the corresponding class points. - Prototypical Matching Network (PMN) is an entry in the table, without a reference. Also, no description is provided for that method. Please cite the appropriate work.  - Similarly for P-FH: there is a short description provided but unclear which previous work this method comes from. Is this a new proposed baseline? If so it would be good to give more details. - Line 248: “its embedding technique is more complex”, referring to the embedding technique of FI compared to this proposed model. It wasn’t clear why it’s more complex.  In a nutshell, the authors put forth a new model for low-shot learning by feature augmentation whose purpose is to generate as diverse as possible examples that still plausibly belong to each novel class. They experimentally demonstrate that their approach achieves state-of-the-art on ImageNet and generated more diverse examples than previous works. The down sides of the paper are mostly in terms of clarity, caused both by the inappropriate or confusing use of words as well as omitting details of the algorithm that I feel are important. Finally, it would be useful to compare experimentally against other approaches that tackle low-shot learning in ways other than feature augmentation.


MULAN: A Blind and Off-Grid Method for Multichannel Echo Retrieval
This paper addresses the general problem of blind echo retrieval, i.e., given M sensors measuring in the discrete-time domain M mixtures of K delayed and attenuated copies of an unknown source signal, can the echo locations and weights be recovered? This problem has broad applications in fields such as sonars, seismology, ultrasounds or room acoustics. It belongs to the broader class of blind channel identification problems, which have been intensively studied in signal processing. Existing methods in the literature proceed in two steps: (i) blind estimation of sparse discrete-time filters and (ii) echo information retrieval by peak-picking on filters. The precision of these methods is fundamentally limited by the rate at which the signals are sampled: estimated echo locations are necessary on-grid, and since true locations never match the sampling grid, the weight estimation precision is impacted. This is the so-called basis-mismatch problem in compressed sensing. We propose a radically different approach to the problem, building on the framework of finite-rate-of-innovation sampling. The approach operates directly in the parameter-space of echo locations and weights, and enables near-exact blind and off-grid echo retrieval from discrete-time measurements. It is shown to outperform conventional methods by several orders of magnitude in precision.

This paper presents a method for estimating continuous-time delays for echos in synthetic multichannel impulse responses from underdetermined observations by doing so in the frequency domain.  The fundamental trick enabling this technique to work is the "annihilating filter technique", by which a convolution in the frequency domain with a second-order polynomial is able to cancel out signals that are exponential in frequency, such as delays.  This technique also applies for weighted sums of such signals, such as impulse responses made up of echoes.  It is able to work on irregularly sampled frequencies (e.g., a subset of the frequencies computed by a discrete Fourier transform).  Experiments are performed on synthetic impulse responses made up of pure delays with no additional noise and show that the proposed approach is significantly more accurate in identifying the locations and amplitudes of a known number of echoes than a LASSO-based approach and a standard cross-relation method.  Other experiments show that the method is still able to identify the time of 46% of 7 impulses within 1us from 2-microphone recordings and that this ability improves with the number of microphones.  Quality: The paper is technically sound.  Its logic is relatively easy to follow, well presented, and seems to be correct.  It discloses (but perhaps doesn't highlight) the assumptions that it makes in terms of the compositions of the impulse responses being analyzed (sums of pure impulses) and the necessity of providing the correct number of impulses in the impulse response. Experimental evaluation shows that it works very well within these constraints.  Under-determined impulse response estimation is a hard problem and this approach is able to perform quite well for a constrained, but meaningful, subset of conditions.  Clarity: The paper is very clearly written. It provides a thorough background to impulse responses and their estimation using clear notation and diagrams.  The descriptions of "four important bottlenecks" of existing methods seems a little over-emphasized, in that the first two (non-sparse and on-grid) seem to be quite similar, and the third (strongly rely on knowledge of filter length) is similar to the proposed method's strong reliance on knowledge of the number of echoes.  I believe that I would be able to reproduce the proposed approach from its description in this paper.  Originality: The paper does a very good job of summarizing existing approaches to the problem.  This approach of finite rate of innovation solutions to impulse estimation has recently been applied to other problems, but as far as I can tell has not been applied to impulse response estimation before.  Significance: While the subset of noise-free recordings of a known number of perfectly specular and frequency-independent echoes is somewhat limited, it also provides a solid model of more realistic signals, thus paving the way for further development.  It seems like this model should be robust to some amount of added noise, although it is not clear how robust it is to model mismatch or imperfect reflections.  The proposed approach addresses a difficult task in a better way than previous work, thus advancing the state-of-the-art in a demonstrable way.  The derived algorithm is a simple alternating minimization, where each step is a closed-form eigenvalue decomposition.  The fact that it can be used on an arbitrary subset of frequencies from a DFT is also interesting and facilitates follow-up work.   Minor comments: * Line 112: "the later" should be "the latter" * Equation (22): should Q be equal to the sum of the Toep_0(a_m)Diag(x_m) terms instead of their concatenation?   I have read the authors' response.


MacNet: Transferring Knowledge from Machine Comprehension to Sequence-to-Sequence Models
Machine Comprehension (MC) is one of the core problems in natural language processing, requiring both understanding of the natural language and knowledge about the world. Rapid progress has been made since the release of several benchmark datasets, and recently the state-of-the-art models even surpass human performance on the well-known SQuAD evaluation. In this paper, we transfer knowledge learned from machine comprehension to the sequence-to-sequence tasks to deepen the understanding of the text. We propose MacNet: a novel encoder-decoder supplementary architecture to the widely used attention-based sequence-to-sequence models. Experiments on neural machine translation (NMT) and abstractive text summarization show that our proposed framework can significantly improve the performance of the baseline models, and our method for the abstractive text summarization achieves the state-of-the-art results on the Gigaword dataset.

Update after Author Feedback:  After reading all the reviews and the author feedback, I have two overall comments. The paper is branded as a transfer learning paper, but I'm left disappointed in this respect. I find it very surprising that the attention can be transferred at all, but it is such a small contribution to the MacNet Architecture's overall improvements, that it seems a hard sell. Focal losses have been used before and encoders have been transferred before, but they also contribute to performance improvements... Second comment: the ablations on summarization are necessary for a camera-ready version -- that seems like a hole right now, so I hope they are included in future versions. Overall, I'm still a 6 because you find a combination of things (with some surprising novelty) that improve performance, and it has shown that I should experiment with those things in the future.  Original Review:  Recent work (McCann et al 2017) had shown that the context vectors  (CoVe) of LSTMs trained as encoders for machine translation contained useful information for other tasks. The authors of this work show that CoVe from question answering encoders can be similarly used to augment performance of other tasks (Eq. 5). More surprisingly, and perhaps more relevant to the community, they also show that very specific higher layers of NLP models trained on one task can be transferred successfully to perform similar functions for other tasks (Eq. 12). This transfer is from question answering (machine comprehension) to both machine translation and summarization, substantial tasks, and experiments show that this transfer improves performance on both. They also demonstrate the increased effectiveness of using a focal loss inspired loss function (Eq. 15).   Pros:   While reading Section 4 MacNet Architecture, I found myself taking notes on the kinds of ablations I would want to see, and I was glad to find most of these ablations in Section 5.  The information conveyed by the writing was clear. MacNet as a package seems to clearly be superior to not using MacNet.  Not really a pro or a con:  I spent a while puzzling over the transfer of G_{enc}. It just isn’t clear to me why the authors would not use the same kinds of inputs G_{enc} was trained with, and the authors don’t justify why transferring by concatenation where the authors chose to is better than using those representations as additional embeddings.    Cons:  I worry a bit that the paper focuses mostly on discussion of transfer learning, but then nearly as much of the gain in performance for NMT comes from the alternative loss function in Eq. 15. In summarization, the performance gains are about 1 point across the board, but again, is this really just because of the alternative loss function? I would prefer to see the NMT ablations for summarization as well. The second biggest contributor to gains seems to be the Encoding layer for NMT, but this is the transfer that we shouldn’t be surprised would work given the prior work in this direction. That the Modeling layer helps at all is surprising, but on its own its at most it gives a gain of .4 on NMT; it is not clear how many runs it took to get that .4, so it seems possible that the transfer of the modeling layer doesn’t help much at all for most training runs.   Overall:  I’m left somewhat skeptical that any of the three contributions on their own are really very significant, but all together as MacNet, they work, there is one surprise, and I learned that I should experiment with a new loss for sequence generation tasks. I tend to think this makes for a good submission, but I’m skeptical about the significance of the transfer learning of the modeling layer. Given that seems to be the biggest novelty to me, this means I’m a bit hesitant about the overall contribution as well, which leads me to give it a 6. I’ve included some more detailed comments below with some additional suggestions and clarification questions. Given that the empirical results aren’t there yet to clear up some of my concerns, my confidence is 4, but I look forward to hearing from the authors during the rebuttal phase.  Combinations:  “We use a simple MLP classifier on the combination of {mj}nj=1 and G to locate the start and end positions of the answer.” (line 13) What is the combination here? Concatenation? Summation? Something else?  Suggested citation:  Question Answering through Transfer Learning from Large Fine-grained Supervision Data (Min et al. 2017) In that work, they show that pretraining on question answering datasets like SQuAD can benefit downstream tasks like SNLI. Its clearly very different from your work in key ways, but it is related in that it shows how QA models trained on SQuAD can transfer well to other tasks.   Citation spacing:  It looks like all of your citations would be better off with a space between them and the preceding text. Try using ~\cite instead of \cite (or corresponding ~\citep and ~\citet)  Pointer-Generator Comparison:  Are the examples in Table 5 randomly chosen? This would be worth mentioning. Otherwise, many will assume you’ve cherry-picked examples that make your model look better. If that is what you did, well… I would suggest randomly choosing them to be more fair.  Transparency in Equations:  In Eq. 2, you use f_{att} and cite Set et al. 2017, but it would be nice to include those equations in an appendix so that readers don’t have to then go lookup another paper just to see how your model is actually implemented.  Embeddings, tokenization, etc.:  Character-level embeddings are obtained by training a CNN over the characters of each word (lines 111-112). G_{enc} is then trained on GloVe and character embeddings from that CNN, which are summarized as f_{rep}. (Eq. 1) Then in Eq. 5, G_{enc} is used to take in Emb(x_s). The authors say that a fully connected layer transforms Emb(x_s) to the right size for G_{enc}, but this is left out of Eq. 5. It should be included there to avoid confusion.   This fully connected transformation is confusing because for NMT, unlike for MC, the data is split into subword units, so it is not clear that G_{enc} would have even seen many of the newly tokenized forms of the data. Why not use f_{rep} again for the inputs? How many subword units were used?  Enough to make sure that G_{enc} still sees mostly words that it saw during MC training? Similar questions arise again for the Pointer-Generator + MacNet. It seems like that fully connected layer has to learn the transformation from Emb(x_s) to f_{rep}, in which case, why have f_{rep} at all if Emb + fully connected can do that, and to reiterate why not just use f_{rep} instead of learning it?  Point of transfer for G_{enc}:  What if you just used \tilde e_s (Eq. 5) as additional embeddings the way that CoVe suggested? Instead of the concatenation with the other encoder?  Contribution of alternative loss:  It seems like Baseline + MacNet differs from Baseline + Encoding Layer + Modeling Layer only in the use of Eq. 15 in the former. Is this correct? If so, then this seems to be almost as important as the actual transfer learning in your final performance gains over the Baseline in Table 2. How much gain comes from just switching to that kind of loss? And how does this kind of ablation work with summarization? Is it all about the loss in that case?  I'm reserving very confident on reproducibility for code that is going to be open sourced. Anything with transfer learning and big tasks like NMT and summarization can be finnicky, so I'm sure people are bound to run into problems. The details seem to be in the paper so that wouldn't be the headache.


Reinforcement Learning of Theorem Proving
We introduce a theorem proving algorithm that uses practically no domain heuristics for guiding its connection-style proof search. Instead, it runs many MonteCarlo simulations guided by reinforcement learning from previous proof attempts. We produce several versions of the prover, parameterized by different learning and guiding algorithms. The strongest version of the system is trained on a large corpus of mathematical problems and evaluated on previously unseen problems. The trained system solves within the same number of inferences over 40% more problems than a baseline prover, which is an unusually high improvement in this hard AI domain. To our knowledge this is the first time reinforcement learning has been convincingly applied to solving general mathematical problems on a large scale.

This paper presents a theorem proving algorithm that leverages Monte-Carlo simulations guided by reinforcement learning  from previous proof searches. In particular, to guid the search, the authors use the UCT formula augmented by (1) learning prior probabilities of actions for certain proof states (policy learning), and (2) learning the values corresponding to the same proof states (policy evaluation). The proposed algorithm is using no domain engineering. The authors evaluate their proposal on two datasets: Miz40 and M2k. The evaluation shows the proposed solution outperforms state-of-the are by solving ~40% more problems.  This is a solid paper: it addresses a hard problem, and provide solutions that show significant improvements over the prior work. I have a few questions however, which mainly seek clarifications of the evaluation results: - You mention "We have also experimented with PUCT as in AlphaZero, however the results are practically the same." Can you provide some more color on this? How close were the results? Any tradeoffs? - Does rlCoP with policy and value guidance solve all the problems mlCoP solves on Miz40 dataset? If not, which problems does mlCoP solve and rlCoP doesn't? Same questions for rlCop with policy guidance vs mlCoP for the M2k dataset. - Is there anything special about the solutions shown in Sec 4.6 besides the fact they are more involving? It would be nice to provide some more intuition of why your algorithm perform well in those cases. - When you are training rlCoP on the Miz40 dataset how do you pick the problems in the training set? Randomly? How robust are the results to picking a different training set?  Minor: - "Published improvements in the theorem proving field are typically between 3 and 10 %." Can you please give some references to support your claim. - Please user either Mizar40 or Miz40 for consistency.


Rest-Katyusha: Exploiting the Solution's Structure via Scheduled Restart Schemes
We propose a structure-adaptive variant of a state-of-the-art stochastic variancereduced gradient algorithm Katyusha for regularized empirical risk minimization. The proposed method is able to exploit the intrinsic low-dimensional structure of the solution, such as sparsity or low rank which is enforced by a non-smooth regularization, to achieve even faster convergence rate. This provable algorithmic improvement is done by restarting the Katyusha algorithm according to restricted strong-convexity (RSC) constants. We also propose an adaptive-restart variant which is able to estimate the RSC on the fly and adjust the restart period automatically. We demonstrate the effectiveness of our approach via numerical experiments.

I have read the response. The authors have addressed my question to satisfaction. ============================================== This paper presents an optimization algorithm for regularized ERM when the solution has low-dimensional structure (in the current paper, the specific structure is sparsity). The algorithm periodically applies Katyusha to the regularized ERM, while initializing each round of Katyusha from the approximate solution of previous round. When the strong convexity parameter is not known, the authors propose a line-search type of method for estimating it. The algorithm is demonstrated to have good performance on several datasets for solving the Lasso problem.     While I could parse the results fine, I am not very familiar with the detailed techniques in this area, so my confidence is not high.  My understanding is that Proposition 3.2 characterizes how close is the regularized ERM solution from the population optimum, thus optimizing the regularized ERM well means being close to the population solution. On the other hand, if one can establish the effective RSC for the regularized ERM, the objective becomes effectively strongly convex, even if the original objective is not globally strongly convex. It appears these two results were established by [3] and [36] previously. And the authors can then apply state-of-the-art algorithms for smooth and strongly-convex objectives, with Katyusha being a particular choice.  From this perspective (if it is correct), the theoretical result presented here may not be very technically challenging to obtain. On the other hand, the method for locating the strong convexity parameter can be a practical contribution.  Another question: In fact, if the RSC parameter mu_c is known and fixed, do we gain anything from restarting Katyusha, than simply running it once?


Scalable Coordinated Exploration in Concurrent Reinforcement Learning
We consider a team of reinforcement learning agents that concurrently operate in a common environment, and we develop an approach to efficient coordinated exploration that is suitable for problems of practical scale. Our approach builds on seed sampling [1] and randomized value function learning [11]. We demonstrate that, for simple tabular contexts, the approach is competitive with previously proposed tabular model learning methods [1]. With a higher-dimensional problem and a neural network value function representation, the approach learns quickly with far fewer agents than alternative exploration schemes.

Main ideas of the submission The authors investigate the problem of efficient coordinated concurrent exploration in environments too large to be addressed by tabular, model-based methods. This is a continuation of [1], where the principles of seed sampling were developed for efficient coordinated concurrent exploration, using a tabular model based algorithm. Since the algorithm was only tested on trivial tasks in [1], the authors first demonstrate the effectiveness of this tabular method on a more challenging problem (swinging up and balancing a pole), compared to trivial extensions of known methods (UCB, Posterior sampling) to the concurrent setting. Following that, they suggest a model-free extension to seeding that is based on function approximation with randomized value functions [9] – a concept that facilitates the combination of the seeding principle with generalization. The authors also suggest some concrete algorithms (SLSVI, STD) that support this concept, show that its performance on the trivial examples of [1] is comparable to that of tabular seed sampling, and show its effectiveness on another pole-balancing problem, which is too difficult to be addressed by tabular methods.  Strong points • Showing that the tabular seed sampling method suggested in [1] is effective compared to other concurrent exploration methods, even on the more difficult problem of balancing up and stabilizing a pole. • Extending the seeding idea in [1] to complex continuous environments. • Although the authors suggest some concrete algorithm implementations (SLSVI, STD) – the concept itself is very general and can be employed in various versions. • The authors also discuss attempts to lower the computational cost – such as limiting the number of operating models (section 3.2.3). • The suggested method is shown to achieve a performance, which is almost as good as the one achieved by the tabular seeding method from [1], although the current method has a much less informative prior. They also use a high dimensional task to demonstrate that the exploration becomes more efficient as the number of agents increases, and is more efficient than \epsilon-greedy high dimensional exploration.  Weak points and suggestions • Although seeding is a promising principle for efficient concurrent exploration, this idea is not backed up by any theoretical analysis and the benefits are only exemplified computationally. Although [14] has developed provable upper exploration cost bounds for the concurrent exploration case, [12] and [9] have only developed such results only for the single-agent case. In a wider perspective, while the current submission and [1] repeatedly claim that they have found three necessary properties for efficient concurrent exploration, there is no supporting theory for that. • The concrete suggested algorithms essentially rely on existing methods (LSVI and TD) for which the computational cost has been well tested in previous work. The only additional cost per agent is the drawing of a random seed, and they also suggest a way to limit the number of active models to save computational costs. This means that essentially – the computational cost of the algorithm itself is the number of active models times the exploration cost for a single agent. I think that this should be noted in the submission, so that the readers are able to compare the computational requirements with previous works. • The seeding method suggested in this paper is quite general, and lacks some guidelines on how to choose the distribution of the seed Z. The authors of [1], make the effort to show that the distribution of the seed Z at the beginning of the episode, is chosen in a specific manner so that at each time - \hat\theta(z,H_t)\sim \theta|H_t  (Where \theta|H_t is the posterior estimation of the model parameter \theta, based on the history up to that time H_t, and \hat\theta(z,H_t) is the parameter chosen by an agent based on a seed z and history H_t. Note that the randomness stems from the seed z).  However, there is no indication at all on a how to effectively choose the seed Z in the current submission. Although the concept of a randomizing seed is intuitive, a poor choice can ruin the exploration (as the seed acts to modify sampled rewards), which makes this work incomplete. • The agents all explore the same MDP, but a more interesting situation is concurrent exploration on a set of different environments with some shared properties (as investigated in [14]). Since the environments are not identical, there are some properties that agents should share and some that they should not, and that is another difficulty that the agents must overcome. This also closely relates to the problem of transfer learning. • The two example tasks discussed in the submission (balancing up a pole) are not complicated enough in the sense that the environment is deterministic. It would be interesting to see how the algorithms operate in such a system where, for example, a noisy wind is introduced. Furthermore, the task of steadying a pole can surely benefit from concurrent exploration: When effective, the concurrent exploration can make the learning converge faster. However, there are surely more interesting tasks where concurrent exploration and cooperation between agents is necessary for the achievement of the overall target, and it would be interesting to see if the seeding-method operates well in these tasks too.  Technical details: • In the example task of balancing up a pole – the differential equations for the cart and mass are taken from [18] in their references, which seems to direct to the following paper: Cohen, Michael A., and Stephen Grossberg. "Absolute stability of global pattern formation and parallel memory storage by competitive neural networks." IEEE transactions on systems, man, and cybernetics 5 (1983): 815-826.  Even when taking the friction \mu to be zero, the equations does not seems to match the ones presented in the submission. • Line 79: There is no detailed explanation regarding the state-actions that receive a reward of 1. For example, it is unclear whether the pole must be balanced with a zero velocity, or must just be in an upright position. In fact, the authors explain it more clearly in the second pole example (Note 1 on page 7 of the submission), but it should be explained here as well. • Line 138: The paraphrasing should be changed, as it is unclear whether the samples perturbed by an agent are also visible to other agents. • The formula for computing \tilde\theta_h used in the SLSVI of section 3.2.1 is said to run for h=H-1,...,0. Since we consider a finite horizon problem in which each agent only takes H actions at each episode – wouldn’t it make more sense for this equation, before the m-th action, to run for only h=H-1,...,m?  Since LSVI aims to find the parameters for which the Q-function is the closest to satisfying the finite horizon Bellman equations, restricting this equation to h=H-1,...,m  means that at the m-th action, this would be the next choice considering that there are only H-m transitions left. • The minimization operation present in the formula for computing L(\theta) following eq. 209 seems to be a typo, since a minimization operation is the purpose of the SGD. Furthermore, there seems to be a discount factor added to this formula – which means that this SGD is used to solve the discounted MDP problem, and not the finite horizon one as stated before. However, this is never specified within the text. • The example tasks of section 4.1 employ the generalized seeding algorithms, but it is not specified how the random seed is set (meaning, the distribution of the random noise terms Z that are added to the rewards is not mentioned) – and thus, these results cannot be reproduced by readers. • Example 4.2 – it is unclear what the "instantaneous reward" axis in figure 3 is. Is it the reward received per agent divided by one second, or is it the reward per time ste


COLA: Decentralized Linear Learning Lie He
Decentralized machine learning is a promising emerging paradigm in view of global challenges of data ownership and privacy. We consider learning of linear classification and regression models, in the setting where the training data is decentralized over many user devices, and the learning algorithm must run ondevice, on an arbitrary communication network, without a central coordinator. We propose COLA, a new decentralized training algorithm with strong theoretical guarantees and superior practical performance. Our framework overcomes many limitations of existing methods, and achieves communication efficiency, scalability, elasticity as well as resilience to changes in data and allows for unreliable and heterogeneous participating devices.

The authors consider the problem learning a (regularized) linear model over a graph of agents. At each iteration, the agents exchange with their neighbors and perform an (approximate) proximal gradient step. They provide linear and sublinear rates depending on the strong convexity of the functions.  Despite interesting ideas, including the rate derivations, this papers lack connections with the literature and some clarifications in its present form.  [Distributed Setup] The distributed setup is rather well explained although it is not exactly standard as the data matrix is split by the *columns* (i.e. the features) among the agents; usually, the lines/examples are split. This difference should be more clearly mentioned and motivated in a practical context (near l64 page 2). As the simulations are performed on a single machine, one would expect a justification for considering this kind of data splitting and network topology. * Then, the communications are done using a doubly-stochastic mixing matrix as commonly accepted in Gossip theory. The paragraph before Assumption 1 is unclear: it is well known that as soon as an (undirected) graph is connected, the spectral gap is positive, this would make assumption 1 more readable.   [Optimization algorithm] As the authors mention (Sec 1.2, e.g. 108), the proposed algorithm falls into the primal-dual methods.  * At first glance, it seems close (as the matrix A cannot be inverted) to Condat-Vu's algorithm (Condat, L. (2013). A primal–dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. Journal of Optimization Theory and Applications, 158(2), 460-479.) Decentralized algorithms based on this algorithm were proposed, see e.g. Bianchi, P., Hachem, W., & Iutzeler, F. (2016). A coordinate descent primal-dual algorithm and application to distributed asynchronous optimization. IEEE Transactions on Automatic Control, 61(10), 2947-2957. The synchronous version (DADMM+ in their paper) looks indeed close to what you propose and should definitively be compared to the proposed method.  * Line 5 in the algorithm you approximatively solve problem (2), which leads to two concerns: (i) why: minimizing (2) seems equivalent to a proximity operation on the corresponding g_i on a point that is a gradient step on the coordinates of f with stepsize tau/sigma'; this seems to be computable exactly quite cheaply especially for the Lasso and Ridge problems considered in the experiments; (ii) how: I don't get how to obtain approximate solutions of this problem apart from not computing some coordinates but then theta can be arbitrarily bad and so are the rates. * I do not see a discussion on how to choose gamma. Furthermore, it seems rather explicit that the gradient descent is performed with the inverse of the associated Lipschitz constant.  [Convergence rates] The derivations seem valid but their significance is kind of obfuscated. * In Theorem 1, the assumption that g has a bounded support (or rather that the iterates stay in a bounded subspace) is rather strong but ok, what bothers me is that (i) it is hard to decipher the rate, a simple case with theta = 0, gamma =1 might help. In addition, the assumption on rho also would require some explanations. * In Theorem 2, it look that we actually have the usual conditioning/T rate but clarifications are also welcome * In Theorem 3, two conditions bother me: (i) mu tau >= D_1 seem like a minimal conditioning of the problem to have a linear rate; (ii) the condition of beta seems to look like a condition on how well connected the graph has to be to get the linear rate. Again, this is not discussed.   Minor comments: * l91 "bridgin"G * l232: the authors say "CoLa is free of tuning parameters" but the gradient descent is actually natively performed with the inverse Lipschitz constant which is a possibility in most algorithms of the literature (and can be local as extended in Apx E1). Furthermore, there is also the parameter gamma to tune, theta to adapt...   ============ After Rebuttal ============  After reading the author feedback and the other reviews, I upgraded my score to 6. The main reasons for that upgrade are: * the improvement of their main theorem to any connected graph * a misconception on my side for the difficulty of pb. (2)  However, it still seems important to me to clarify the non-triviality of (2), mention the relations between CoCoA and ADMM, and clarify that gamma and Theta are not hyper-parameter (cf l. 12-14 of the feedback).