Recycling Privileged Learning and Distribution Matching for Fairness
Equipping machine learning models with ethical and legal constraints is a serious issue; without this, the future of machine learning is at risk. This paper takes a step forward in this direction and focuses on ensuring machine learning models deliver fair decisions. In legal scholarships, the notion of fairness itself is evolving and multi-faceted. We set an overarching goal to develop a unified machine learning framework that is able to handle any definitions of fairness, their combinations, and also new definitions that might be stipulated in the future. To achieve our goal, we recycle two well-established machine learning techniques, privileged learning and distribution matching, and harmonize them for satisfying multi-faceted fairness definitions. We consider protected characteristics such as race and gender as privileged information that is available at training but not at test time; this accelerates model training and delivers fairness through unawareness. Further, we cast demographic parity, equalized odds, and equality of opportunity as a classical two-sample problem of conditional distributions, which can be solved in a general form by using distance measures in Hilbert Space. We show several existing models are special cases of ours. Finally, we advocate returning the Pareto frontier of multi-objective minimization of error and unfairness in predictions. This will facilitate decision makers to select an operating point and to be accountable for it.

This paper proposes a framework which can learn classifiers that satisfy multiple notions of fairness such as fairness through unawareness, demographic parity, equalized odds etc. The proposed framework leverages ideas from two different lines of existing research namely, distribution matching and privileged learning, in order to accommodate multiple notions of fairness.  This work builds on two prior papers on fairness - Hardt et. al. and Zafar et. al. in order to create a more generalized framework for learning fair classifiers. The proposed method seems interesting and novel, and the ideas from privileged learning and distribution matching have not been employed in designing fair classifiers so far. The idea of proposing a generalized framework which can handle multiple notions of fairness is quite appealing.
The paper, however, has the following weaknesses:
1) the evaluation is weak; the baselines used in the paper are not even designed for fair classification
2) the optimization procedure used to solve the multi-objective optimization problem is not discussed in adequate detail
Detailed comments below:
Methods and Evaluation: The proposed objective is interesting and utilizes ideas from two well studied lines of research, namely, privileged learning and distribution matching to build classifiers that can incorporate multiple notions of fairness. The authors also demonstrate how some of the existing methods for learning fair classifiers are special cases of their framework. It would have been good to discuss the goal of each of the terms in the objective in more detail in Section 3.3. The part that is probably the most weakest in the entire discussion of the approach is the discussion of the optimization procedure. The authors state that there are different ways to optimize the multi-objective optimization problem they formulate without mentioning clearly which is the procedure they employ and why (in Section 3). There seems to be some discussion about the same in experiments section (first paragraph) and I think what was done is that the objective was first converted into unconstrained optimization problem and then an optimal solution from the pareto set was found using BFGS. This discussion is still quite rudimentary and it would be good to explain the pros and cons of this procedure w.r.t. other possible optimization procedures that could have been employed to optimize the objective.
The baselines used to compare the proposed approach and the evaluation in general seems a bit weak to me. Ideally, it would be good to employ baselines that learn fair classifiers based on different notions (E.g., Hardt et. al. and Zafar et. al.) and compare how well the proposed approach performs on each notion of fairness in comparison with the corresponding baseline that is designed to optimize for that notion. Furthermore, I am curious as to why k-fold cross validation was not used in generating the results. Also, was the split between train and test set done randomly? And, why are the proportions of train and test different for different datasets?
Clarity of Presentation:
The presentation is clear in general and the paper is readable. However, there are certain cases where the writing gets a bit choppy. Comments:
1. Lines 145-147 provide the reason behind x*_n being the concatenation of x_n and z_n. This is not very clear.
2. In Section 3.3, it would be good to discuss the goal of including each of the terms in the objective in the text clearly.
3. In Section 4, more details about the choice of train/test splits need to be provided (see above).
While this paper proposes a useful framework that can handle multiple notions of fairness, there is scope for improving it quite a bit in terms of its experimental evaluation and discussion of some of the technical details.


On Separability of Loss Functions, and Revisiting Discriminative Vs Generative Models
We revisit the classical analysis of generative vs discriminative models for general exponential families, and high-dimensional settings. Towards this, we develop novel technical machinery, including a notion of separability of general loss functions, which allow us to provide a general framework to obtain`1 convergence rates for general M -estimators. We use this machinery to analyze`1 and`2 convergence rates of generative and discriminative models, and provide insights into their nuanced behaviors in high-dimensions. Our results are also applicable to differential parameter estimation, where the quantity of interest is the difference between generative model parameters.

The paper compares generative and discriminative models for general exponential families, also in the high-dimensional setting. It develops a notion of separability of general loss functions, and provides a framework for obtaining l_{infty} convergence rates for M-estimators. This machinery is used to analyze convergence rates (both in l_{infty} and l_2) of generative and discriminative models.
I found the paper quite hard to follow, but this might potentially be also due to my lack of expertise in high dimensional estimation. While the introduction, as well as background and setup (Section 2) are accessible and quite easy to follow, in the following sections the paper becomes quite dense with equations and technical results, sometimes given without much guidance to the reader. Sometimes authors' comments made me quite confused; for instance, in lines 215-218, Corollary 4 and 3 are compared (for discriminative and generative models) and it is stated that "both the discriminative and generative approach achieve the same convergence rate, but the generative approach does so with only a logarithmic dependence on the dimension", whereas from what I can see, the dependence on the dimension in both Corollary 3 and 4 is the same (sqrt(log p))?
The core idea in the paper is the notion of separability of the loss. The motivating example in the introduction is that in the generative model with conditional independence assumption, the corresponding log-likelihood loss is fully separable into multiple components, each of which can be estimated separately. In case of the corresponding discriminative model (e.g. logistic regression), the loss is much less separable. This motivates the need to introduce a more flexible notion of separability in Section 3. It turns out that more separable losses require fewer samples to converge.
Unfortunately, I am unable see this motivation behind the definition of separability (Definition 1 and 2), which states that the error in the first order approximation of the gradient is upper bounded by the error in the parameters. I think it would improve the readability of the paper if the authors could motivate why is this particular definition used here and how is it related to the intuitive notion of separability of the loss into multiple components.
Starting from Corollary 3, the authors use notation "\succsim" without explanation. What does it mean?
It is hard for me to judge the significance of the paper, as I not familiar with previous work in this setup (apart from the work by Ng & Jordan), but I found the results quite interesting, if they were only presented more clearly. Nevertheless, I evaluate the theoretical contribution to be quite strong. What I found missing is a clear message about when to use generative vs. discriminative methods in high dimension (similar to the message given by Ng & Jordan in their paper).
Small remarks:
36: "could be shown to separable" -> "could be shown to be separable"
65: n -> in
87: : and j should be in the subscript
259: bayes -> Bayes
254-265: the "Notation and Assumptions" paragraph -- the Bayes classifier is defined only after its error was already mentioned.


Flexible statistical inference for mechanistic models of neural dynamics
Mechanistic models of single-neuron dynamics have been extensively studied in computational neuroscience. However, identifying which models can quantitatively reproduce empirically measured data has been challenging. We propose to overcome this limitation by using likelihood-free inference approaches (also known as Approximate Bayesian Computation, ABC) to perform full Bayesian inference on single-neuron models. Our approach builds on recent advances in ABC by learning a neural network which maps features of the observed data to the posterior distribution over parameters. We learn a Bayesian mixture-density network approximating the posterior over multiple rounds of adaptively chosen simulations. Furthermore, we propose an efficient approach for handling missing features and parameter settings for which the simulator fails, as well as a strategy for automatically learning relevant features using recurrent neural networks. On synthetic data, our approach efficiently estimates posterior distributions and recovers ground-truth parameters. On in-vitro recordings of membrane voltages, we recover multivariate posteriors over biophysical parameters, which yield model-predicted voltage traces that accurately match empirical data. Our approach will enable neuroscientists to perform Bayesian inference on complex neuron models without having to design model-specific algorithms, closing the gap between mechanistic and statistical approaches to single-neuron modelling.

The authors present an Approximate Bayesian Computation approach to fitting dynamical systems models of single neuron dynamics. This is an interesting paper that aims to bring ABC to the neuroscience community. The authors introduce some tricks to handle the specifics of neuron models (handling bad simulations and bad features), and since fitting nonlinear single neuron models is typically quite painful, this could be a valuable tool. Overall, this paper is a real gem.
My biggest issue is that, in its current form, many of the proposed improvements over [19] are a bit unclear in practice. Sections 2.2-2.3 were difficult to parse for someone less familiar with ABC/MDNs. Since these are the main improvements, it would help to add/clarify here. Additionally, the classifier for bad simulations, the imputation for bad features, and the recurrent network for learning features need to be included in the algorithm in the appendix.
Additionally, the value of imputation for missing features in neural models is plausible, but completely theoretical in the paper’s current form. For the HH model, for instance, latency is not a feature and it seems there aren’t good simulations with bad features. Is there a clear example where imputation would be necessary for inference?
Minor issues:
In section 2. “could” should be replaced with “here we do.” One challenge in reading the paper is that many of the improvements seem to be purely hypothetical, until you get to the results.
It would be helpful to have a more complete description of the features used in the main text, and also to clarify at the beginning of section 2.1 that x=f(s).
Using a recurrent neural network for learning features seems like an interesting direction, but the results here are more of a teaser than anything else. Completely unpacking this could likely be a paper by itself.
For the GLM simulation -- what features are used? Also, the data augmentation scheme could be more clear in the appendix. If I understand correctly it isn’t necessary for the generative model, but just for getting the true posteriors and in PG-MCMC.


Maxing and Ranking with Few Assumptions
PAC maximum selection (maxing) and ranking of n elements via random pairwise comparisons have diverse applications and have been studied under many models and assumptions. With just one simple natural assumption: strong stochastic transitivity, we show that maxing can be performed with linearly many comparisons yet ranking requires quadratically many. With no assumptions at all, we show that for the Borda-score metric, maximum selection can be performed with linearly many comparisons and ranking can be performed with O(n log n) comparisons.

Thanks for the rebuttal, which was quite helpful and clarified most of the issues raised in the review.
The authors study the problem of PAC maximum selection (maxing) and ranking of n elements via random pairwise comparisons. Problems of that kind have recently attracted attention in the ML literature, especially in the setting of dueling bandidts. In this paper, the only assumption made is strong stochastic transitivity of the pairwise winning probabilisties. Under this assumption, the authors show that maxing can be performed with linearly many comparisons while ranking has a quadratic sample complexity. Further, for the so-called Borda ranking, maximum selection and ranking can be performed with O(n) and O(n log n) comparisons, respectively.
The paper addresses an interesting topic and is well written. That being said, I'm not convinced that is offers significant new results. It's true that I haven't seen the first two results in exactly this form before, but there are very similar ones, and besides, the results are not very surprising. Further, the argument for the result regarding the non-existence of a PAC ranking algorithm with O(n log n) comparisons in the SST setting is not totally clear.
More importantly, the last two results are already known. Theorem 8 this known as Borda reduction, see e.g., Sparse Dueling Bandits, Generic exploration and k-armed voting bandits, Relative upper confidence bound for the k-armed duelling bandit problem. Likewise, Theorem 9 is already known, see e.g., Crowdsourcing for Participatory Democracies, Efficient elicitation of social choice functions.
Consequently, the only result that remains, being both novel and relevant, is the PAC maxing algorithm and its analysis. Nevertheless, a study of its performance on real data is missing, as well as a comparison with other algorithms.
Indeed, the experiments are not very convincing, especially because only a simple synthetic setting is considered, and no experiments for (Borda) ranking are included. SEQ-ELIMINATE is a subroutine of OPT-MAXIMIZE, so does it really makes sense to compare the two? It would be rather more meaningful to compare one of the two with other maxing algorithms in the main paper and move their comparison, if necessary, to the appendix. Lines 189-299: models with other parameters should be considered. In all maxing experiments, it should be tested whether the found element is the actual maxima.

Minor issues:
- 51: an \epsilon-ranking (and not \epsilon-rankings)
- 100: comparisons (and not comparison)
- 117-118: Please reconsider the statement: "the number of comparisons used by SEQ-ELIMINATE is optimal up to a constant factor when \delta <= 1/n".  \delta <= 1/n => n <= 1/\delta => O(n/\epsilon^2 \log(n/\delta)) = O(n/\epsilon^2 \log(1/\delta^2)) != \Omega(n/\epsilon^2 \log(1/\delta))
- 117-118: Please reconsider the statement: "but can be \log n times the lower bound for constant \delta", since n/\epsilon^2 \log(1/\delta) \times \log(n) != n/\epsilon^2 \log(n/\delta).
- 120-122: It is not clear how should such a reduction together with the usage of SEQ-ELIMINATE lead to an optimal sample complexity up to constants. PLease elaborate.
- 130-131: "replacing or retaining r doesn’t affect the performance of SEQ-ELIMINATE". The performance in terms of what? And why there is no effect?
- 165: same comment as for 117-118
- 167: in 120 it was stated that S' should have a size of O(n/\log n), and here It's O(\sqrt(n\log n))
- 165-169: The procedure described here (the reduction + usage of SEQ-ELIMINATE) is different from the one described in 120-122. And again, it's not clear how should such a procedure results in an order-wise optimal algorithm.
- 204: Lemma 12 does not exist.
- 207: Lemma 11 does not exist.
- 210: at least (and not atleast)
- 215: \delta is missing in the arguments list of PRUNE
- 221-222: same comment as for 165 and 117-118
- 238-239: The sample complexity is not apparent in both cases (distinguished in algorithm 2)


Multi-way Interacting Regression via Factorization Machines
We propose a Bayesian regression method that accounts for multi-way interactions of arbitrary orders among the predictor variables. Our model makes use of a factorization mechanism for representing the regression coefficients of interactions among the predictors, while the interaction selection is guided by a prior distribution on random hypergraphs, a construction which generalizes the Finite Feature Model. We present a posterior inference algorithm based on Gibbs sampling, and establish posterior consistency of our regression model. Our method is evaluated with extensive experiments on simulated data and demonstrated to be able to identify meaningful interactions in applications in genetics and retail demand forecasting.
interactions are observed only infrequently, subject to a constraint that the interaction order (a.k.a. interaction depth) is given. Neither SVM nor FM can perform any selection of predictor interactions, but several authors have extended the SVM by combining it with 1 penalty for the purpose of feature selection (Zhu et al., 2004) and gradient boosting for FM (Cheng et al., 2014) to select interacting features. It is also an option to perform linear regression on as many interactions as we can and combine it with regularization procedures for selection (e.g. LASSO (Tibshirani, 1996) or Elastic net (Zou & Hastie, 2005)). It is noted that such methods are still not computationally feasible for accounting for interactions that involve a large number of predictor variables.
In this work we propose a regression method capable of adaptive selection of multi-way interactions of arbitrary order (MiFM for short), while avoiding the combinatorial complexity growth encountered by the methods described above. MiFM extends the basic factorization mechanism for representing the regression coefficients of interactions among the predictors, while the interaction selection is guided by a prior distribution on random hypergraphs. The prior, which does not insist on the upper bound on the order of interactions among the predictor variables, is motivated from but also generalizes Finite Feature Model, a parametric form of the well-known Indian Buffet process (IBP) (Ghahramani & Griffiths, 2005). We introduce a notion of the hypergraph of interactions and show how a parametric distribution over binary matrices can be utilized to express interactions of unbounded order. In addition, our generalized construction allows us to exert extra control on the tail behavior of the interaction order. IBP was initially used for infinite latent feature modeling and later utilized in the modeling of a variety of domains (see a review paper by Griffiths & Ghahramani (2011)).
In developing MiFM, our contributions are the following: (i) we introduce a Bayesian multi-linear regression model, which aims to account for the multi-way interactions among predictor variables; part of our model construction includes a prior specification on the hypergraph of interactions -in particular we show how our prior can be used to model the incidence matrix of interactions in several ways; (ii) we propose a procedure to estimate coefficients of arbitrary interactions structure; (iii) we establish posterior consistency of the resulting MiFM model, i.e., the property that the posterior distribution on the true regression function represented by the MiFM model contracts toward the truth under some conditions, without requiring an upper bound on the order of the predictor interactions; and (iv) we present a comprehensive simulation study of our model and analyze its performance for retail demand forecasting and case-control genetics datasets with epistasis. The unique strength of the MiFM method is the ability to recover meaningful interactions among the predictors while maintaining a competitive prediction quality compared to existing methods that target prediction only.
The paper proceeds as follows. Section 2 introduces the problem of modeling interactions in regression, and gives a brief background on the Factorization Machines. Sections 3 and 4 carry out the contributions outlined above. Section 5 presents results of the experiments. We conclude with a discussion in Section 6.

The paper presents a Bayesian method for regression problems where the response variables can depend on multi-way combinations of the predictors. Via a hyper graph representation of the covariates interactions, the model is obtained from a refinement of the Finite Feature Model.
The idea of modelling the interactions as the hyper edges of a hyper graph is interesting but the proposed model seems to be technically equivalent to the original Finite Mixture Model. Moreover, from the experimental results, it is hard to assess if the new parametrised extension is an improvement respect to the baseline.
Here are few questions and comments:
- is the idea of modelling interactions via an unbounded membership model new? Finite Mixture Models have been already used for predicting proteins interactions in [1], what are the key differences between that work and the proposed method?
- the main motivation for modelling nonlinear interactions via multi-way regression is the interpretability of the identified model. Given the data, can one expect the nonlinear realisation to be unique? I think that a brief discussion about the identifiability of the model would improve the quality of the paper.
- can the model handle higher powers of the same predictor (something like y = z x^2)?
- the number of true interactions is usually expected to be very small respect to the number of all possible interactions. Does the model include a sparsity constraint on the coefficients/binary matrices? Is the low-rank assumption of the matrix V related to this?
- is the factorised form of the multi-way interactions (matrix V in equation 3) expected to be flexible enough?
- In the experiments:
i) the true coefficients were chosen at random? Was there a threshold on their magnitude?
ii) how is the `quality’ of the recovered coefficients defined? What is the difference between this and the `exact recovery’ of the interactions?
iii) the standard FMM model (alpha=1) performs quite well independently of the number of multi-way interactions. Was that expected?
iv) why does the random forest algorithm perform so well in the gene dataset and badly on the synthetic dataset? Why was the baseline (Elastic net algorithm) not used on the real-world experiments?
————————————————
[1] Krause et al. 2006 Identifying protein complexes in high-throughput protein interaction screens using an infinite latent feature model


Learning with Feature Evolvable Streams
Learning with streaming data has attracted much attention during the past few years. Though most studies consider data stream with fixed features, in real practice the features may be evolvable. For example, features of data gathered by limitedlifespan sensors will change when these sensors are substituted by new ones. In this paper, we propose a novel learning paradigm: Feature Evolvable Streaming Learning where old features would vanish and new features would occur. Rather than relying on only the current features, we attempt to recover the vanished features and exploit it to improve performance. Specifically, we learn two models from the recovered features and the current features, respectively. To benefit from the recovered features, we develop two ensemble methods. In the first method, we combine the predictions from two models and theoretically show that with the assistance of old features, the performance on new features can be improved. In the second approach, we dynamically select the best single prediction and establish a better performance guarantee when the best model switches. Experiments on both synthetic and real data validate the effectiveness of our proposal.

This paper presents two new methods for learning with streaming data under the assumption that feature spaces may evolve so old features vanish and new features occur. Examples of this scenario are environmental monitoring sensors or RFIDs for moving goods detection. Two different approaches are proposed, both based on the combination of two models: one trained with the old features and the other one based on the new features. Depending on how these models are combined, the authors propose two different methods for Feature Evolvable Streaming Learning (FESL): FESL-c that combines the predictions of both models, and FESL-s that selects the best model each time. Theoretical bounds on the cumulative loss of the both FESL methods as a function of the cumulative loss of the two models trained over different feature spaces are also provided. Experiments on datasets from the UCI Machine Learning Repository, Reuter’s data, and RFID real data have been conducted, and results show that the proposed models are able to provide competitive results in all cases when compared to three baseline methods.
Overall, I think this is a very interesting paper that addresses a real-world problem that is quite common, for example, in environmental sensoring applications. I think that the paper is well-written and well-organized, and the experiments show the significance of the proposed methodology. Additionally, I positively value the use of RFID real data.
* Contribution with respect to transfer learning. My main concern is about the quantification of the degree of contribution of this work, as I cannot see clearly the difference what is the difference between the proposed approach and the use of transfer learning in two different feature sets (of course, taking into account that transfer learning cannot be applied from t=1,…, T1-B). I would like the authors to clarify this point.
* Theorems 1 and 2. I would also like the authors to discuss about the practical implications of Theorem 1 and Theorem 2, as in both cases convergence to 0 is obtained when T2 is large enough but that is not the case in Figures 1 and 2. Additionally, it seems to me that although the upper bound for FESL-s algorithm is tighter than for the FESL-c method, the convergence is slower. Please, clarify this point.
* Optimal value for s. I do not understand why there is an inequality in Equation 9 instead of an equality according to the assumptions made (the first model is better than the second one for t smaller than s, and the second model is better than the first one for t larger than s). In this regard, I can see in Figure 3 that this assumption holds in practice, but the value for s depends on the problem you are working on. Providing some intuition about the range of values for s would be valuable.
* Datasets and experimental setup. I have some questions:
  - I would like the authors to clarify why the Gaussian noise was added to the UCI datasets instead of considering two different subsets of features. I guess that this is because it is assumed that the features spaces are not so different, but I would like the authors to discuss this point.
  - Please, provide more details about how batch datasets was transformed into streaming data. I suppose it was done randomly, but then I am wondering whether results on 10 independent rounds are enough to have reliable results and I would like to see the variability of the results provided in terms of the cumulative loss.
  - It is not clear to me how the hyperparameters of the methods (delta and eta) were adjusted. Were they set according to Theorems 1 and 2? What about the hyperparameters in the baseline methods?
  - I do not understand the sentence (page 7): “So for all baseline methods and our methods, we use the same parameters and the comparison is fair”.
* Results. It would be desirable to provide similar tables to Table 1 for the Reuter’s and RFID data.


Finite Sample Analysis of the GTD Policy Evaluation Algorithms in Markov Setting
In reinforcement learning (RL) , one of the key components is policy evaluation, which aims to estimate the value function (i.e., expected long-term accumulated reward) of a policy. With a good policy evaluation method, the RL algorithms will estimate the value function more accurately and find a better policy. When the state space is large or continuous Gradient-based Temporal Difference(GTD) policy evaluation algorithms with linear function approximation are widely used. Considering that the collection of the evaluation data is both time and reward consuming, a clear understanding of the finite sample performance of the policy evaluation algorithms is very important to reinforcement learning. Under the assumption that data are i.i.d. generated, previous work provided the finite sample analysis of the GTD algorithms with constant step size by converting them into convex-concave saddle point problems. However, it is well-known that, the data are generated from Markov processes rather than i.i.d. in RL problems.. In this paper, in the realistic Markov setting, we derive the finite sample bounds for the general convex-concave saddle point problems, and hence for the GTD algorithms. We have the following discussions based on our bounds. (1) With variants of step size, GTD algorithms converge. (2) The convergence rate is determined by the step size, with the mixing time of the Markov process as the coefficient. The faster the Markov processes mix, the faster the convergence. (3) We explain that the experience replay trick is effective by improving the mixing property of the Markov process. To the best of our knowledge, our analysis is the first to provide finite sample bounds for the GTD algorithms in Markov setting.

It is well known that the standard TD algorithm widely used in reinforcement learning does not correspond to the gradient of any objective function, and consequently is unstable when combined with any type of function approximation. Despite the success of methods like deep RL, which combines vanilla TD with deep learning, theoretically TD with nonlinear function approximation is demonstrably unstable. Much work on fixing this fundamental flaw in RL has been in vain, till the work on gradient TD methods by Sutton et al. Unfortunately, these methods work, but their analysis was flawed, based on a heuristic derivation of the method.
A recent breakthrough by Liu et al. (UAI 2015) showed that gradient TD methods are essentially saddle point methods that are pure gradient methods that optimize not the original gradient TD loss function (which they do not), but rather the saddle point loss function that arises when converting the original loss function into the dual space. This neat trick opened the door to a rigorous finite sample analysis of TD methods, and has finally surmounted the obstacles in the last three decades of work on this problem. Liu et al. developed several variants of gradient TD, including proximal methods that have improved theoretical and experimental performance.
This paper is an incremental improvement to Liu et al., in that it undertakes a similar analysis of saddle point TD methods under a more realistic assumption that the samples are derived from a Markov chain (unlike Liu et al., which assumed IID samples). This paper provides an important next step in our understanding of how to build a more rigorous foundation for the currently shaky world of gradient TD methods. The analysis is done thoroughly and it is supplemented with some simple experiments.
The work does not address the nonlinear function approximation case at all. All the current work on Atari video games and deep RL is based on nonlinear function approximations. Is there any hope that gradient TD methods will be shown someday to be robust (at least locally) with nonlinear function approximation?


Reducing Reparameterization Gradient Variance
Optimization with noisy gradients has become ubiquitous in statistics and machine learning. Reparameterization gradients, or gradient estimates computed via the "reparameterization trick," represent a class of noisy gradients often used in Monte Carlo variational inference (MCVI). However, when these gradient estimators are too noisy, the optimization procedure can be slow or fail to converge. One way to reduce noise is to generate more samples for the gradient estimate, but this can be computationally expensive. Instead, we view the noisy gradient as a random variable, and form an inexpensive approximation of the generating procedure for the gradient sample. This approximation has high correlation with the noisy gradient by construction, making it a useful control variate for variance reduction. We demonstrate our approach on a non-conjugate hierarchical model and a Bayesian neural net where our method attained orders of magnitude (20-2,000×) reduction in gradient variance resulting in faster and more stable optimization.

Summary
This paper proposes a control variate (CV) for the reparametrization gradient by exploiting a linearization of the data model score.  For Gaussian random variables, such a linearization has a distribution with a known mean, allowing its use as a CV.  Experiments show using the CV results in faster (according to wall clock time) ELBO optimization for a GLM and Bayesian NN.  Furthermore, the paper reports 100 fold (+) variance decreases during optimization of the GLM.
Evaluation
Method:  The CV proposed is clever; the observation that the linearization of the data score has a known distribution is non-obvious and interesting.  This is a contribution that can easily be incorporated when using the reparametrization trick.  The only deficiencies of the method are (1) it requires the model’s Hessian and (2) application is limited to Gaussian random variables.  It also does not extend to amortized inference (i.e. variational autoencoders), but this limitation is less important.  I think the draft could better detail why it’s limited to Gaussian variables (which is a point on which I elaborate below under ‘Presentation’).  Some discussion of the hurdles blocking extension to other distributions would improve the draft and be appreciated by readers.  For instance, it looks like the linearized form might have a known distribution for other members of the location-scale family (since they would all have the form f(m) + H(m)(s*epsilon))?  Thoughts on this?
Experiments:  Experiments are limited to small models (by modern standards)---GLM and one-layer / 50 hidden unit Bayesian NN---but are adequate to demonstrate the correctness and utility of the method.  I would have liked to see what held-out performance gains can be had by using the method---you must have looked at held-out LL?---but I realize this is not crucial for assessing the method as a CV.  Still I think the paper is strong enough that even a negative or neutral result would not hurt the paper’s quality.
Presentation:  The paper’s presentation of the method’s derivation is clear and detailed.  I had no trouble seeing the logical flow from equation to equation.  However, I think the draft’s description is somewhat ‘shallow’---by that I mean, while the algebra is clear, more discussion and intuition behind the equations could be included.  Specifically, after the method is derived, I think a section discussing why it's limited to Gaussian rv’s and why it can’t be extended to amortized inference would be beneficial to readers.  It took me two-to-three reads through the draft before I fully understood these limitations.  To make room for this discussion, I would cut from the introduction and VI background since, if a person is interested in implementing your CV, they likely already understand VI, the ELBO, etc.
Minutiae: Table 1 is misreferenced as ‘Table 3’ in text.
Conclusion:  This paper was a pleasure to read, and I recommend its acceptance.  The methodology is novel and interesting, and the experiments adequately support the theory.  The draft could be improved with more exposition (on limitations, for instance), but this is not a crucial flaw.


Wider and Deeper, Cheaper and Faster: Tensorized LSTMs for Sequence Learning
Long Short-Term Memory (LSTM) is a popular approach to boosting the ability of Recurrent Neural Networks to store longer term temporal information. The capacity of an LSTM network can be increased by widening and adding layers. However, usually the former introduces additional parameters, while the latter increases the runtime. As an alternative we propose the Tensorized LSTM in which the hidden states are represented by tensors and updated via a cross-layer convolution. By increasing the tensor size, the network can be widened efficiently without additional parameters since the parameters are shared across different locations in the tensor; by delaying the output, the network can be deepened implicitly with little additional runtime since deep computations for each timestep are merged into temporal computations of the sequence. Experiments conducted on five challenging sequence learning tasks show the potential of the proposed model.

This paper proposes Tensorized LSTMs for efficient sequence learning. It represents hidden layers as tensors, and employs cross-layer memory cell convolution for efficiency and effectiveness. The model is clearly formulated. Experimental results show the utility of the proposed method.
Although the paper is well written, I still have some questions/confusion as follows. I would re-consider my final decision if the authors address these points in rebuttal.
1. My biggest confusion comes from Sec 2.1, when the authors describe how to widen the network with convolution (lines 65-73). As mentioned in text, "P is akin to the number of stacked hidden layers", and the model "locally-connects" along the P direction to share parameters. I think it is a strategy to deepen the network instead of widening it, since increasing P (the number of hidden layers) won't incur more parameters in the convolution. Similarly, as mentioned in lines 103-104, tRNN can be "widened without additional parameters by increasing the tensor size P". It does not make sense, as increasing P is conceptually equivalent to increasing the number of hidden layers in sRNN. This is to deepen the network, not to widen it.
2. The authors claim to deepen the network with delayed outputs (Sec 2.2). They use the parameter L to set up the network depth. However, as shown in Eq. 9, L is determined by P and K, meaning that we can not really set up the network depth as a free parameter. I guess in practice, we would always pre-set P and K before experiments, and then derive L from Eq. 9. It seems over-claimed in lines 6-10, which reads like "we could freely set up the width and depth of the network".
3. The authors claims that the proposed memory cell convolution is able to prevent gradient vanishing/exploding (line 36). This is not verified theoretically or empirically. The words "gradient vanishing/exploding" are even not mentioned in the later text.
4. In the experiments, the authors compared tLSTM variants in the following dimentions: tensor shape (2D or 3D), normalization (no normalization, LN, CN), memory cell convolution (yes or no), and feedback connections (yes or no). There are 2x3x2x2=24 combinations in total. Why just pick up the six combinations in lines 166-171? I understand it become messy when comparing too many methods, but there are some interesting variants like 2D tLSTM+CN. Also, it might help to split the experiments in groups, like one for normalization strategy, one for memory cell convolution, one for feedback connections, etc.


Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization
Overfitting is one of the most critical challenges in deep neural networks, and there are various types of regularization methods to improve generalization performance. Injecting noises to hidden units during training, e.g., dropout, is known as a successful regularizer, but it is still not clear enough why such training techniques work well in practice and how we can maximize their benefit in the presence of two conflicting objectives-optimizing to true data distribution and preventing overfitting by regularization. This paper addresses the above issues by 1) interpreting that the conventional training methods with regularization by noise injection optimize the lower bound of the true objective and 2) proposing a technique to achieve a tighter lower bound using multiple noise samples per training example in a stochastic gradient descent iteration. We demonstrate the effectiveness of our idea in several computer vision applications.

This paper introduces a method for regularizing deep neural network by noise. The core of the approach is to draw connections between applying a random perturbation to layer activations and the optimization of a lower-bound objective function. Experiments for four visual tasks are carried out, and show a slight improvement of the proposed method compared to dropout.
On the positive side:
- The problem of regularization for training deep neural network is a crucial issue, which has a huge potential practical and theoretical impact.
- The connection between regularization by noise and the derivation of a lower bound of the training objective is appealing to provide a formal understanding of the impact of noise regularization techniques, e.g. dropout.
On the negative side:
- The aforementioned connection between regularization by noise and training objective lower bounding seems to be a straightforward adaptation of [9] in the case of deep neural networks. For the most important result given in Eq (6), i.e. the fact that using several noise sampling operations gives a tighter bound on the objective function than using a single random sampling (as done in dropout), the authors refer to the derivation in [9]. Therefore, the improvement and positioning with respect to ref [9] is not clear to me.
- Beyond this connection, the implementation of the model given in Section 4 for the case of dropout is also straightforward: several (random) forward passes are performed and a weighted average of the resulting gradients is computed using Eq (8), before applying the backward pass.
- Experiments are carried out in various vision tasks, which choice could be better justified. It is not clear why these tasks are highly regularization demanding: the noise seems to be applied in a few (fully connected layers). In addition, the improvement brought out by the proposed method is contrasted across tasks (e.g. small improvement in UCF-101 and below the published baseline for image captioning).
As a conclusion, although the approach is interesting, its novelty level seems to be low and the experiments should be more targeted to better illustrate the benefit of the multiple sampling operations. In their rebuttal, I would like the authors to answer my concern about positioning with respect to [9].


Deep Mean-Shift Priors for Image Restoration
In this paper we introduce a natural image prior that directly represents a Gaussiansmoothed version of the natural image distribution. We include our prior in a formulation of image restoration as a Bayes estimator that also allows us to solve noise-blind image restoration problems. We show that the gradient of our prior corresponds to the mean-shift vector on the natural image distribution. In addition, we learn the mean-shift vector field using denoising autoencoders, and use it in a gradient descent approach to perform Bayes risk minimization. We demonstrate competitive results for noise-blind deblurring, super-resolution, and demosaicing.

Summary
-------
The paper proposes a way to do Bayesian estimation for image restoration problems by using gradient descent to minimize a bound of the Bayes risk. In particular, the paper uses Gaussian smoothing in the utility function and the image prior, which allows to establish a connection between the gradient of the log-prior and denoising auto-encoders (DAEs), which are used to learn the prior from training data. Several image restoration experiments demonstrate results on par with the state-of-the-art.
Main comments
-------------
The positive aspects of the paper are its good results (on par with state-of-the-art methods) for several image restoration tasks and the ability of the same learned prior to be used for several tasks (mostly) without re-training. Another strong aspect is the ability to jointly do noise- and kernel-blind deconvolution, which other methods typically cannot do.
I found it somewhat difficult to judge the novelty of the paper, since it seems to be heavily influenced/based on the papers by Jin et al. [14] (noise-adaptive approach, general derivation approach) and Bigdeli and Zwicker [4] (using DAEs as priors, mean-shift connection). Please properly attribute where things have been taken from previous work and make the differences to the proposed approach more clear. In particular:
- The mean-shift connection is not really discussed, just written that it is "well-known". Please explain this in more detail, or just remove it from the paper (especially from the title).
- Using a Gaussian function as/in the utility function seems to be taken from [14] (?), but not acknowledged. The entire (?) noise-adaptive approach seems to be adopted from [14], please acknowledge/distinguish more thoroughly (e.g. in lines 168-171).
Although the experiments demonstrate that the proposed method can lead to good results, many approximations and bounds are used to make it work. Can you give an explanation why the method works so well despite this?
Related to this, if I understand correctly, the whole noise sampling approach in section 3.3 is only used to make up for the fact that the learned DAE is overfitted to noisy images? I don't understand how this fixes the problem.
Regarding generalization, why is it necessary to train a different DAE for the demosaicing experiments? Shouldn't the image prior be generic?
Related to this, the noise- and kernel-blind experiments (section 5.2) use different gradient descent parameters. How sensitive is the method towards these hyper-parameters?
The paper doesn't mention at all if the method has reasonable runtimes for typical image sizes. Please provide some ballpark test runtimes for a single gradient descent step (for the various restoration tasks).
Miscellaneous
-------------
- I found the abstract hard to understand without having read the rest of the paper.
- It does seems somewhat artificial to include p'(x)/p(\tilde{x}) in Eq. 5 instead of just removing this term from the utility function and simply replacing p(x) with the smoothed prior p'(x).
- Typo in line 182: "form" -> "from"
- Please define the Gaussian function g_\sigma in Eq. 6 and later.
- Table 2:
	- EPLL/TV-L2 + NA not explained, how can they use the noise-adaptive approach?
	- Misleading results: e.g. RTF-6 [28] only trained for \sigma=2.55, doesn't make sense to evaluate for other noise levels. Please just leave the table entry blank.


Overcoming Catastrophic Forgetting by Incremental Moment Matching
Catastrophic forgetting is a problem of neural networks that loses the information of the first task after training the second task. Here, we propose a method, i.e. incremental moment matching (IMM), to resolve this problem. IMM incrementally matches the moment of the posterior distribution of the neural network which is trained on the first and the second task, respectively. To make the search space of posterior parameter smooth, the IMM procedure is complemented by various transfer learning techniques including weight transfer, L2-norm of the old and the new parameter, and a variant of dropout with the old parameter. We analyze our approach on a variety of datasets including the MNIST, CIFAR-10, Caltech-UCSDBirds, and Lifelog datasets. The experimental results show that IMM achieves state-of-the-art performance by balancing the information between an old and a new network.

[UPDATE]
After digging through the supplementary material I have managed to find at least a hint towards possible limitations of the method being properly acknowledged. Not including an objective evaluation of limitations is a flaw of this otherwise well written paper, especially when the method relies crucially on weight transfer (as the authors point out outside the main paper, i.e. supplementary text and rebuttal). However, weight transfer is known to be an inadequate initialization technique between different problem classes and the authors don't clearly address this issue, nor do they properly qualify the applicability of the method.
In balance, this paper does give sufficient evidence that weight transfer and some form of parameter averaging are promising directions of future investigation, at least in a subset of interesting cases. Perhaps with publication chances of expanding this subset increase.
[REVIEW TEXT]
The paper evaluates the idea of incremental moment matching (IMM) as a solution for catastrophic forgetting in neural networks. The method is thoroughly benchmarked, in several incarnations, against state-of-the-art baselines on standard ‘toy’ problems defined on top of MNIST, as well as more challenging ImagNet2CUB and the Lifelog dataset. A new parameterization, dubbed ‘drop-transfer’ is proposed as an alternative to standard weight initialization of model parameters on new tasks.
Pros:
- Comprehensive experiments are provided which show superior performance on standard MNIST classification variants, in comparison to previous methods.
- Encouraging results are shown on natural image datasets, although here many earlier methods were not applied, so there were few literature results to compare against.
Cons:
- Although differences exist between the pairs or triplets of problems considered to illustrate ‘catastrophic forgetting’, technically speaking, these problems are extremely related, sharing either the input support or the class distribution to some degree; this is in contrast to current challenges in the field, which consider sequences of arbitrarily different reinforcement learning problems, such as different Atari video games (Kirkpatrick et al. 2016, Rusu et al., 2016). Therefore, I find it difficult to tell how proficient the method is at reducing catastrophic forgetting in the general case. I recommend considering more standard challenges from the literature.
- Another important consideration for such works is evaluating how much learning is possible with the given model when following the proposed solution. Since catastrophic forgetting can be alleviated by no further learning throughout most of the model, it follows that proficient learning of subsequent tasks, preferably superior to starting from scratch, is a requirement for any ‘catastrophic forgetting’ solution. As far as I can tell the proposed method does not offer a plausible answer if tasks are too different, or simply unrelated; yet these are the cases most likely to induce catastrophic forgetting. In fact, it is well understood that similar problems which share the input domain can easily alleviate catastrophic forgetting by distilling the source task network predictions in a joint model using target dataset inputs (Li et al., 2016, Learning without Forgetting).
In summary, I believe variations of the proposed method may indeed help solve catastrophic forgetting in a more general setting, but I would like to kindly suggest evaluating it against current standing challenges in the literature.


Associative Embedding: End-to-End Learning for Joint Detection and Grouping
We introduce associative embedding, a novel method for supervising convolutional neural networks for the task of detection and grouping. A number of computer vision problems can be framed in this manner including multi-person pose estimation, instance segmentation, and multi-object tracking. Usually the grouping of detections is achieved with multi-stage pipelines, instead we propose an approach that teaches a network to simultaneously output detections and group assignments. This technique can be easily integrated into any state-of-the-art network architecture that produces pixel-wise predictions. We show how to apply this method to multi-person pose estimation and report state-of-the-art performance on the MPII and MS-COCO datasets.

The paper proposes a method for the task of multi-person pose estimation. It is based on the stacked hour glasses architecture for single person pose estimation which is combined with an associative embedding. Body joint heatmaps are associated with tag heatmaps indicating which individual the joint belongs to. A new grouping loss is defined to assess how well predicted tags agree with ground truth joint associations. The approach is evaluated on the MPII multi-person and on MS-COCO, achieving state of the art performance for both of them.
Strengths:
- State-of-the-art performance is achieved for the multi-person pose estimation problem on MPII Multi-person and MS-COCO.
Weakness:
- The paper is rather incremental with respect to [31]. The authors adapt the existing architecture for the multi-person case producing identity/tag heatmaps with the joint heatmaps.
- Some explanations are unclear and rather vague. Especially, the solution for the multi-scale case (end of Section 3) and the pose refinement used in section 4.4 / table 4. This is important as most of the improvement with respect to state of the art methods seems to come from these 2 elements of the pipeline as indicated in Table 4.
Comments:
The state-of-the-art performance in multi-person pose estimation is a strong point. However, I find that there is too little novelty in the paper with respect to the stacked hour glasses paper and that explanations are not always clear. What seems to be the key elements to outperform other competing methods, namely the scale-invariance aspect and the pose refinement stage, are not well explained.


Houdini: Fooling Deep Structured Visual and Speech Recognition Models with Adversarial Examples
Generating adversarial examples is a critical step for evaluating and improving the robustness of learning machines. So far, most existing methods only work for classification and are not designed to alter the true performance measure of the problem at hand. We introduce a novel flexible approach named Houdini for generating adversarial examples specifically tailored for the final performance measure of the task considered, be it combinatorial and non-decomposable. We successfully apply Houdini to a range of applications such as speech recognition, pose estimation and semantic segmentation. In all cases, the attacks based on Houdini achieve higher success rate than those based on the traditional surrogates used to train the models while using a less perceptible adversarial perturbation.

This paper presents a way to create adversarial examples based on a task loss (e.g. word error rate) rather than the loss being optimized for training.  The approach is tested on a few different domains (pose estimation, semantic segmentation, speech recognition).
Overall the approach is nice and the results are impressive.  My main issues with the paper (prompting my "marginal accept" decision) are:
- The math and notation is confusing and contradictory in places, e.g. involving many re-definitions of terms.  It needs to be cleaned up.
- The paper does discuss enough whether optimizing the task loss with Houdini is advantageous against optimizing the surrogate loss used for training.  It seems in some cases it is clearly better (e.g. in speech recognition) but the scores given in the image tasks do not really show a huge difference.  There also aren't any qualitative examples which demonstrate this.
Overall I would be happy to see this paper accepted if it resolves these issues.
Specific comments:
- There are various typos/grammatical errors (incl. a broken reference at line 172); the paper needs a proofread before publication.
- "we make the network hallucinate a minion"  What is a minion?
- "While this algorithm performs well in practice, it is extremely sensitive to its hyper-parameter"  This is a strong statement to make - can you qualify this with a citation to work that shows this is true?
- "\eps \in R^d is a d-dimensional..."  I don't see you defining that \theta \in R^d, but I assume that's what you mean; you should say this somewhere.
- Some of your terminology involves redefinitions and ambiguity which is confusing to me.  For example, you define g_\theta(x) as a network (which I presume outputs a prediction of y given x), but then you later define g_\theta(x, y) as the "score output by the network for an example".  What do you mean by the "score output by the network" here?  Typically the network outputs a prediction which, when compared to the target, gives a score; it is not a function of both the input and the target output (you have defined y as the target output previous).  You seem to be defining the network to include the target output and some "score".  And what score do you mean exactly?  The task loss? Surrogate loss? Some other score?  It seems like maybe you mean that for each possible output y (not a target output) given the intput x, the network outputs a score, but this involves a redefinition of y from "target output" to "possible output".  You also define y as the target, but y_\theta(x) as a function which produces the predicted target.
- "To compute the RHS of the above quantity..." Don't you mean the LHS?  The RHS does not involve the derivative of l_H.
- The arguments of l_H have changed, from \theta, x, y to \hat{y}, y in equation (9).  Why?
- "Convergence dynamics illustrated in Fig. 4 " - Figure 4 does not show convergence dynamics, it shows "most challenging examples for the pose re-targeting class..."
- Suggestion: In image examples you show zoomed-in regions of the image to emphasize the perturbations applied.  Can you also show the same zoomed-in region from the original image?  This will make it easier to compare.
- "Notice that Houdini causes a bigger decrease in terms of both CER and WER than CTC for all tested distortions values."  I think you mean bigger increase.
- In Table 5 it's unclear if "manual transcription" is of the original speech or of the adversarially-modified speech.
- In listening to your attached audio files, it sounds like you are not using a great spectrogram inversion routine (there are frame artefacts, sounds like a buzzing sound).  Maybe this is a phase estimation issue?  Maybe use griffin-lim?
- Very high-level: I don't see how you are "democratizing adversarial examples".  Democratize means "make (something) accessible to everyone".  How is Houdini relevant to making adversarial examples accessible to everyone?  Houdini seems more like a way to optimize task losses for adversarial example generation, not a way of making examples avaiable to people.


Federated Multi-Task Learning
Federated learning poses new statistical and systems challenges in training machine learning models over distributed networks of devices. In this work, we show that multi-task learning is naturally suited to handle the statistical challenges of this setting, and propose a novel systems-aware optimization method, MOCHA, that is robust to practical systems issues. Our method and theory for the first time consider issues of high communication cost, stragglers, and fault tolerance for distributed multi-task learning. The resulting method achieves significant speedups compared to alternatives in the federated setting, as we demonstrate through simulations on real-world federated datasets.

The paper generalizes an existing framework for distributed multitask learning (COCOA) and extends it to handle practical systems challenges such as communications cost, stragglers, etc. The proposed technique (MOCHA) makes the framework robust and faster.
Pros:
- Rigorous theoretical analysis
- Sound experiment methodology
Cons:
- Not very novel (compared to [47]) and just a mild variation over [47] including the analysis.
Major Comments:
---------------
1. The Equation 5 depends on \Delta \alpha_t* (minimizer of sub-problem). My understanding is that this minimizer is not known and cannot be computed accurately at every node, and therefore, we need the approximation. If so, then how do the nodes set \theta^h_t? Appendix E.3 states that this parameter is directly tuned via H_i. However, the quality should be determined by the environment, and not supposed to be 'tuned'. If tuned, then it might act as a 'learning rate' -- instead of taking the full step in the direction of the gradient, the algorithm takes a partial step.
2. The main difference between this paper and [47] seems to be that instead of the same \theta in [47], this paper has defined separate \theta^h_t for each node t.
Minor comments / typos:
-----------------------
39: ...that MTL is -as- a natural ....
108: ...of this (12) in our... -- should be (2)


k-Support and Ordered Weighted Sparsity for Overlapping Groups: Hardness and Algorithms
The k-support and OWL norms generalize the 1 norm, providing better prediction accuracy and better handling of correlated variables. We study the norms obtained from extending the k-support norm and OWL norms to the setting in which there are overlapping groups. The resulting norms are in general NP-hard to compute, but they are tractable for certain collections of groups. To demonstrate this fact, we develop a dynamic program for the problem of projecting onto the set of vectors supported by a fixed number of groups. Our dynamic program utilizes tree decompositions and its complexity scales with the treewidth. This program can be converted to an extended formulation which, for the associated group structure, models the k-group support norms and an overlapping group variant of the ordered weighted 1 norm. Numerical results demonstrate the efficacy of the new penalties.

Summary:
This paper designs new norms for group sparse estimation. The authors extend the k-support norm and the ordered weighted norm to the group case (with overlaps). The resulting (latent) norms are unfortunately NP-hard to compute, though. The main contribution is an algorithm based on tree decomposition and dynamic programming for computing the best approximation (under the Euclidean norm) under group cardinality constraints. This algorithm improves the previous work by a factor of m (# of groups). A second contribution is an extended formulation for the proposed latent group norms. Such formulation can be plugged into standard convex programming toolboxes, as long as the underlying tree width is small. Some synthetic experiments are provided to justify the proposed norms (and their extended formulations).
Overall this paper appears hard to follow, largely due to its technical nature. The organization is also a bit off: the utility of the dynamic programming algorithm in section 3 is largely unclear, and the experiment section did not verify this part at all. While being mostly a computational paper, strangely, the experiment section only compared the statistical performances of different norms, without conducting any computational comparison. While the technical results seem to be solid and improve previous work, it will be more convincing if more can be said on what kind of problems can benefit from the authors' results. With these much more sophisticated norms, one becomes easily overwhelmed by the fine mathematical details but somehow also gets more suspicious about their practical relevance (which unfortunately is not well addressed by the current experiments).
Other comments:
Lines 93-95: the parameter nu disappears in the convex envelope?
Line 97: the weights need to be nonnegative.
Line 343: this proof of Theorem 2.1 works only for 1 < p < infty? For instance, if p = 1, then Omega = |.|_1 always. Please fully justify the claim "Omega is the ell_p norm if and only if ...", as the experiments did use p=infty.
Line 346: please justify "if lambda < 1, the solution is (1-lambda)y if and only if ..."
Line 141:  I do not see why the claim N <= |V| is correct. do you mean max_j |X_j| <= |V|?
Experiments: should also include a set of experiments that favors the latent group norm, to see that if the proposed norms perform (much?) worse under an unfavorable condition. Also, please compare with the elastic net regularizer (in the correlated design case).


Local Aggregative Games
Aggregative games provide a rich abstraction to model strategic multi-agent interactions. We introduce local aggregative games, where the payoff of each player is a function of its own action and the aggregate behavior of its neighbors in a connected digraph. We show the existence of a pure strategy -Nash equilibrium in such games when the payoff functions are convex or sub-modular. We prove an information theoretic lower bound, in a value oracle model, on approximating the structure of the digraph with non-negative monotone sub-modular cost functions on the edge set cardinality. We also define a new notion of structural stability, and introduce γ-aggregative games that generalize local aggregative games and admit -Nash equilibrium that is stable with respect to small changes in some specified graph property. Moreover, we provide algorithms for our models that can meaningfully estimate the game structure and the parameters of the aggregator function from real voting data.

Summary:
The authors consider games among n players over a digraph G=(V,E), where each player corresponds to a node in V, the payoff function of each player depends on the choices of its own and those of neighbors. A further assumption is that the payoff function is Lipschitz and convex (or submodular). For games with sufficiently smooth payoff functions, the authors show that an approximate PSNE exists. Also, the authors prove that there exists a digraph for which any randomized algorithms approximating PSNE needs exponentially many value-queries for a submodular payoff function.
Then the authors consider a variant of the game called gamma-local aggregative game, for which the payoff function depends on not only neighbors but all other players under the condition that other players choices affect the payoff in the magnitude of gamma^(distance from the player). For such games, the authors show a sufficient condition that an approximate PSNE exists. They further consider a learning problem for the games where, given approximate PSNEs as examples, the learner’s goal is to predict the underlying digraph. An optimization problem for the learning problem is proposed. Finally, the authors show experimental results over real data sets.
Comments:
The proposed game is a generalization of the aggregative game to a graph setting. The technical contribution of the paper is to derive sufficient/necessary conditions for the existence of approximate PSNEs for the games. On the other hand, the theoretical results for learning graphs from approximate PSNEs is yet to be investigated in the sense that no sample complexity is shown. However, the proposed formulation provides interesting results over real data sets.
I read the authors' rebuttal comments and I would appreciate if the authors reflect your comments to me in the final version.


Learning to Compose Domain-Specific Transformations for Data Augmentation
Data augmentation is a ubiquitous technique for increasing the size of labeled training sets by leveraging task-specific data transformations that preserve class labels. While it is often easy for domain experts to specify individual transformations, constructing and tuning the more sophisticated compositions typically needed to achieve state-of-the-art results is a time-consuming manual task in practice. We propose a method for automating this process by learning a generative sequence model over user-specified transformation functions using a generative adversarial approach. Our method can make use of arbitrary, non-deterministic transformation functions, is robust to misspecified user input, and is trained on unlabeled data. The learned transformation model can then be used to perform data augmentation for any end discriminative model. In our experiments, we show the efficacy of our approach on both image and text datasets, achieving improvements of 4.0 accuracy points on CIFAR-10, 1.4 F1 points on the ACE relation extraction task, and 3.4 accuracy points when using domain-specific transformation operations on a medical imaging dataset as compared to standard heuristic augmentation approaches.

This paper addresses an interesting and new problem to augment training data in a learnable and principled manner. Modern machine learning systems are known for their ‘hunger for data’ and until now state-of-the-art approaches have relied mainly on heuristics to augment labeled training data. This paper tries to reduce the tedious task of finding a good combination of data augmentation strategies with best parameters by learning a sequence of best data augmentation strategies in a generative adversarial framework while working with unsupervised data.
      Plus points:
      1. The motivation behind the problem is to reduce human labor without compromising the final discriminative classification performance. The problem formulation is pretty clear from the text.
      2. The solution strategy is intuitive and based on realistic assumptions. The weak assumption of class invariance under data transformations enable the use of unsupervised training data and a generative adversarial network to learn the optimum composition of transformation functions from a set of predefined data transformations. To address the challenge of transformation functions being possibly non-differentiable and non-deterministic, the authors propose a reinforcement learning based solution to the problem.
      3. The experiments are well designed, well executed and the results are well analyzed. The two variants (mean-field and LSTM) of the proposed method almost always beat the basic and heuristics based baselines. The proposed method shows its merit on 3 different domains with very different application scenarios. The method experimentally proves its adaptability and usefulness of using domain specific transformation functions in the experiment with Mammography Tumor Classification experiment. The authors did a good job by showing the robustness of the method in the TF Misspecification experimentation (L279-286).
      Minus points:
      1. I have confusion regarding the role of the discriminator which I’m writing in detail below. I can understand that the generator's task is to produce less number of out-of-distribution null class (eqn. between L113 and 114). With the weak assumption (eq.(2)), this means that the generator's task is to produce mostly an image of the same class as that of the input. Then if I'm understanding correctly, the discriminator's task is to have low value of D(.) when a transformed example is passed through the discriminator (first term of eqn between L117-L118). At the same time, the discriminator's task is to have high value of D(.) when a non-transformed example is passed through the discriminator (second term of eqn between L117-L118). Thus the 'game' being played between the generator and the discriminator is that the generator is trying to (to propose a sequence of transformations to) generate in-class (i.e., non null-class) examples while the discriminator is trying to tell whether the input coming to it is a null-class (coming from a bad generator which produces null-class example) or not (coming from the real distribution of examples or from a good generator). If my understanding (above) is true, I'd request to put some explanatory texts along this line (the game analogy and what is the role of the discriminator i.e., when will the discriminator give a high value and when low value). And if my understanding is not in the right direction, then I'd like to see a better explanation of what the authors tried to achieve by the generative adversarial objective.
      Side comment: Please provide eqn. numbers to all equations even if it is not referred anywhere in the text. It helps the readers to refer to them.
      In light of the clear motivation of a new problem, succinct solution strategy and well designed and executed experiments I’m positive about the work. However, I’d like to see the authors’ rebuttal about the confusion I wrote above.
=======
After reading the fellow reviewers' comments and the the authors' response in detail, I'm more positive about this work. The authors' did a good job in clarifying my question about the role of the discriminator and have told that they would add a clarification along those lines. I agree with the authors that the focus of the work was on conducting a study of getting optimum sequence of transformation functions for data augmentation and data augmentation makes more sense when the availability of data is limited. Notwithstanding, the authors did a good job by providing the performance of their method fin the fully supervised scenario for the sake of completeness and on the request of the fellow reviewers. This also adds to the robustness of the proposed method. Thus, considering the novel problem definition, neat solution strategy and supportive experiments I am going for an even stronger accept. To me, it is a candidate for an oral presentation.


An Inner-loop Free Solution to Inverse Problems using Deep Neural Networks
We propose a new method that uses deep learning techniques to accelerate the popular alternating direction method of multipliers (ADMM) solution for inverse problems. The ADMM updates consist of a proximity operator, a least squares regression that includes a big matrix inversion, and an explicit solution for updating the dual variables. Typically, inner loops are required to solve the first two subminimization problems due to the intractability of the prior and the matrix inversion. To avoid such drawbacks or limitations, we propose an inner-loop free update rule with two pre-trained deep convolutional architectures. More specifically, we learn a conditional denoising auto-encoder which imposes an implicit data-dependent prior/regularization on ground-truth in the first sub-minimization problem. This design follows an empirical Bayesian strategy, leading to so-called amortized inference. For matrix inversion in the second sub-problem, we learn a convolutional neural network to approximate the matrix inversion, i.e., the inverse mapping is learned by feeding the input through the learned forward network. Note that training this neural network does not require ground-truth or measurements, i.e., data-independent. Extensive experiments on both synthetic data and real datasets demonstrate the efficiency and accuracy of the proposed method compared with the conventional ADMM solution using inner loops for solving inverse problems.

An interesting paper that solves linear inverse problems using a combination of two networks: one that learns a proximal operator to the signal class of interest, and the other that serves as a proxy for a large scale matrix inversion. The proximal operator is reusable whenever the signal domain is unchanged. One would need to retrain only the matrix inversion network when the underlying problem is changed. This is a significant advantage towards reusability of the training procedure.
Strengths
+ Novelty
+ A very interesting problem
Weaknesses
- An important reference is missing
- Other less important references are missing
- Bare-bones evaluation
The paper provides an approach to solve linear inverse problems by reducing training requirements. While there is some prior work in this area (notably the reference below and reference [4] of the paper), the paper has some very interesting improvements over them. In particular, the paper combines the best parts of [4] (decoupling of signal prior from the specific inverse problem being solved) and Lista (fast implementation). That said, evaluation is rather skim — almost anecdotal at times — and this needs fixing. There are other concerns as well on the specific choices made for the matrix inversion that needs clarification and justifications.
1) One of the main parts of the paper is a network learnt to invert (I + A^T A). The paper used the Woodbury identity to change it to a different form and learns the inverse of (I+AA^T) since this is a smaller matrix to invert. At test time, we need to apply not just this network but also "A" and "A^T" operators.
A competitor to this is to  learn a deep network that inverts (I+A^T A). A key advantage of this is that we do not need apply A and A^T during test time. It is true that that this network learns to inverts a larger matrix ... But at test time we have increased rewards. Could we see a comparison against this ? both interms of accuracy as well as runtime (at training and testing) ?
2) Important reference missing. The paper is closely related to the idea of unrolling, first proposed in, “Lista”
http://yann.lecun.com/exdb/publis/pdf/gregor-icml-10.pdf
While there are important similarities and differences between the proposed work and Lista, it is important that the paper talks about them and places itself in appropriate context.
3) Evaluation is rather limited to a visual comparison to very basic competitors (bicubic and wiener filter). It would be good to have comparisons to
- Specialized DNN: this would provide the loss in performance due to avoiding the specialized network.
- Speed-ups over [4] given the similarity (not having this is ok given [4] is an arXiv paper)
- Quantitative numbers that capture actual improvements over vanilla priors like TV and wavelets and gap to specialized DNNs.
Typos
- Figure 2: Syntehtic


Random Projection Filter Bank for Time Series Data
We propose Random Projection Filter Bank (RPFB) as a generic and simple approach to extract features from time series data. RPFB is a set of randomly generated stable autoregressive filters that are convolved with the input time series to generate the features. These features can be used by any conventional machine learning algorithm for solving tasks such as time series prediction, classification with time series data, etc. Different filters in RPFB extract different aspects of the time series, and together they provide a reasonably good summary of the time series. RPFB is easy to implement, fast to compute, and parallelizable. We provide an error upper bound indicating that RPFB provides a reasonable approximation to a class of dynamical systems. The empirical results in a series of synthetic and real-world problems show that RPFB is an effective method to extract features from time series.

This paper proposes a new algorithm for learning on time series data. The basic idea is to introduce random projection filter bank for extracting features by convolution with the time series data. Learning on time series data can then be performed by applying conventional machine learning algorithms to these features. Theoretical analysis is performed by studying both the approximation power and statistical guarantees of the introduced model. Experimental results are also presented to show its effectiveness in extracting features from time series.
Major comments:
  (1) It seems that the algorithm requires the information on both $M$ and $\epsilon_0$, which depend on the unknown target function $h^*$ to be estimated. Does this requirement on an apriori information restrict its practical application? Since $Z_i'$ are complex numbers, the hypothesis function in (9) would take complex outputs. It is not clear to me how this complex output is used for prediction.
  (2) It is claimed that the discrepancy would be zero if the stochastic process is stationary. However, this seems not to be true as far as I can see. Indeed, the discrepancy in Definition 1 is a random variable even if the stochastic process is i.i.d.. So, it depends on the realization of $X$. If this is true, Theorem 1 is not clear to understand since it involves a discrepancy which would not be zero in the i.i.d. learning case.
  (3) There exists some inconsistences in Algorithm 1 and materials in Section 4. Algorithm 1 is a regularization scheme while ERM is considered in Section 4. There are $m$ time series in Algorithm 1, while $m=1$ is considered in Section 4. The notation is not consistent, i.e., $f$ is used in Algorithm 1 and $h$ is used in Section 4.
  (4) The motivation in defining features in Section 3 is not clear to me. It would be better if the authors can explain it more clearly so that readers without a background in complex analysis can understand it well.
Minor comments:
  (1) line 28: RKSH
  (2) line 65, 66: the range of $f\in\mathcal{F}$ should be the same as the first domain of $l$
  (3) eq (11), the function space in the brace seems not correct
  (4) the notation * in Algorithm 1 is not defined as far as I can see
  (5) $\delta_2$ should be $\delta$ in Theorem 1
  (6) line 254: perhaps a better notation for feature size is $n$?
  (7) Appendix, second line in equation below line 444, a factor of $2$ is missed there?
  (8) Appendix, eq (28), the identity should be an inequality? I suggest the authors to pay more attention to (28) since the expectation minus its empirical average is not the same as the empirical average minus expectation.


Stochastic Approximation for Canonical Correlation Analysis
We propose novel first-order stochastic approximation algorithms for canonical correlation analysis (CCA). Algorithms presented are instances of inexact matrix stochastic gradient (MSG) and inexact matrix exponentiated gradient (MEG), and achieve ✏-suboptimality in the population objective in poly( 1 ✏ ) iterations. We also consider practical variants of the proposed algorithms and compare them with other methods for CCA both theoretically and empirically.

I read the rebuttal and thank the authors for the clarifications about differences between the proposed method and previous work in [3,4]. I agreed with Reviewer 2's concerns about the limits of the evaluation section. Giving experiments on only one medium scale data set, it is insufficient to show the potential computational advantages of the proposed method (scale easily to very large datasets) compared to the alternative methods.
Summary
The paper proposes two novel first-order stochastic optimization algorithms to solve the online CCA problem. Inspired from previous work, the proposed algorithms are instances of noisy matrix stochastic / exponentiated gradient (MSG / MEG):  at each iteration, for a new sample (x_t , y_t), the algorithms work by first updating the empirical estimates of the whitening transformations which define the inexact gradient partial_t, followed by a projection step over the constrain set of the problem to get a rank-k projection matrix (M_t). The theoretical analysis and experimental results show the superiority of the algorithms compared to state-of-the-art methods for CCA.
Qualitative Assessment
Significance - Justification:
The main contributions of the paper are (1) the derivation of a new noisy stochastic mirror descent method to solve a convex relaxation for the online version of CCA and (2) the provision of
generalization bounds (or convergence guarantees) for the proposed algorithms. These contributions appear to be incremental advances considered that similar solutions have been proposed previously  for PCA [3] and PLS [4].
Detailed comments:
The paper on the whole is well written, the algorithms are explained clearly, the theorems for the algorithms are solid and sound, and the performance of the algorithms outperforms that of state-of-the-art methods for CCA.
Some questions/comments:
What differentiates your algorithm from that of the previous work in references [3, 4]? Except solving different problems (CCA vs PCA and PLS), is your algorithm fundamentally different in some ways (e.g. add whitening transformations to alleviate the gradient updates for CCA problems or provide convergence analysis)?
The reviewer read references [3, 4] and noticed that the algorithms in your papers are similar to those proposed in [3, 4] in terms of similar convex relaxation and stochastic optimization algorithms. The reviewer would have expected the authors to provide a comparison/summary of the differences / similarities between previous work [3, 4, 21] and current paper in the related work. This would also highlight explicitly the main contributions of the paper.
There are a few typos:
Line 90: build -> built; study -> studied
Line 91: in the batch setting ->  in a batch setting
Line 96: pose -> posed;  consider -> considered
Line 101-102: rewrite this sentence “none of the papers above give generalization
bounds and focus is entirely on analyzing the quality of the solution to that of the empirical minimizer”. I guess you intend to express the following: “all the papers mentioned above focus entirely / mainly on analyzing the quality of the solution to that of the empirical minimizer, but provide no generalization bounds”
remove line 150


Revenue Optimization with Approximate Bid Predictions
In the context of advertising auctions, finding good reserve prices is a notoriously challenging learning problem. This is due to the heterogeneity of ad opportunity types, and the non-convexity of the objective function. In this work, we show how to reduce reserve price optimization to the standard setting of prediction under squared loss, a well understood problem in the learning community. We further bound the gap between the expected bid and revenue in terms of the average loss of the predictor. This is the first result that formally relates the revenue gained to the quality of a standard machine learned model.

This paper studies revenue optimization from samples for one bidder. It is motivated by ad auction design where there are trillions of different items being sold. For many items, there is often little or no information available about the bidder’s value for that precise good. As a result, previous techniques in sample-based auction design do not apply because the auction designer has no samples from the bidder’s value distribution for many of the items. Meanwhile, ads (for example) are often easily parameterized by feature vectors, so the authors make the assumption that items with similar features have similar bid distributions. Under this assumption, the authors show how to set prices and bound the revenue loss.
At a high level, the algorithm uses the sample and the bid predictor h(x) to cluster the feature space X into sets C1, …, Ck. The algorithm finds a good price ri per cluster Ci. On a freshly sampled feature vector x, if the feature vector falls belongs to the cluster Ci, then the reserve price r(x) = ri. The clustering C1, …, Ck is defined to minimize the predicted bid variance of each group Cj. Specifically, the objective is to minimize sum_{t = 1}^k \sqrt{\sum_{i,j:xi,xj \in Ct} (h(xi) – h(xj))^2}. The authors show that this can be done efficiently using dynamic programming. The best price per cluster can also be found in polynomial time. For each cluster Ct, let St be the set of samples (xi, bi) such that xi falls in Ct. The algorithm sets the price rt to be the bid bi such that (xi,bi) falls in St and bi maximizes empirical revenue over all samples in St.

The authors bound the difference between the expected revenue achieved by their learning algorithm and an upper bound on optimal revenue. This upper bound is simply E_{(x,b)}[b]. They show that the bound depends on the average bid over the sample and the error of the bid predictor h. The bound is also inversely proportional to the number of clusters k and the number of samples. They perform experiments on synthetic data and show that their algorithm performs best on “clusterable” synthetic valuation distributions (e.g., bimodal). It also seems to perform well on real ad auction data.

Overall, I think the motivation of this work is compelling and it’s an interesting new model for the sample-based mechanism design literature. I’d like to see more motivation for the assumption that the learner knows the bid prediction function h. Is it realistic that an accurate prediction function can be learned? It seems to me as though the learner would probably have to have a pretty robust knowledge about the underlying bid function in order for this assumption to hold. The paper could potentially be strengthened by some discussion about this assumption.

In the experimental section, it would be interesting to see what the theoretical results guarantee for the synthetic data and how that compares to the experimental results.

I’m confused about equation (12) from the proof of lemma 1. We know that r(x) >= h(x) – eta^{2/3}. Therefore, Pr[b < r(x)] >= P[b < h(x) – eta^{2/3}] = P[h(x) – b > eta^{2/3}]. But the proof seems to require that Pr[b < r(x)] <= P[b < h(x) – eta^{2/3}] = P[h(x) – b > eta^{2/3}]. Am I confusing something?
=========After feedback =========
Thanks, I see what you mean in Lemma 1. Maybe this could be made a bit clearer since in line 125, you state that r(x) := max(h(x) - eta^{2/3}, 0).


A-NICE-MC: Adversarial Training for MCMC
Existing Markov Chain Monte Carlo (MCMC) methods are either based on generalpurpose and domain-agnostic schemes, which can lead to slow convergence, or problem-specific proposals hand-crafted by an expert. In this paper, we propose A-NICE-MC, a novel method to automatically design efficient Markov chain kernels tailored for a specific domain. First, we propose an efficient likelihood-free adversarial training method to train a Markov chain and mimic a given data distribution. Then, we leverage flexible volume preserving flows to obtain parametric kernels for MCMC. Using a bootstrap approach, we show how to train efficient Markov chains to sample from a prescribed posterior distribution by iteratively improving the quality of both the model and the samples. Empirical results demonstrate that A-NICE-MC combines the strong guarantees of MCMC with the expressiveness of deep neural networks, and is able to significantly outperform competing methods such as Hamiltonian Monte Carlo.

This paper describes a novel adversarial training procedure to fit a generative model described by a Markov chain to sampled data. The Markov chain transitions are based on NICE, and so are reversible and volume preserving. It is therefore straightforward to use these as proposals in a Metropolis MCMC method to sample from arbitrary distributions. By repeatedly fitting the Markov chain model to samples from preliminary runs, we can hope that we'll end up with an MCMC method that mixes well on an arbitrary target distribution. Like HMC, the Markov chain is actually on a joint distribution of the parameters of interest, and some auxiliary random draws used to make a deterministic proposal.
It's a promising idea. The MCMC examples in the paper aren't particularly compelling. They're ok, but limited. However, this is the sort of algorithm that could work and be useful in some realistic situations.
Quality
The paper appears to be technically sound. It describes a valid MCMC method. It's unclear when a bootstrapping procedure (a precise procedure isn't given in the paper) will converge to a working proposal mechanism. The paper ends alluding to this issue. Variational inference would find a feasible region of parameters, but (like all MCMC methods), the method could be trapped there.
There is a danger that parts of the distribution are ignored. The cost function doesn't ensure that the proposal mechanism will get everywhere. Future work could investigate that further. Mixing updates with a more standard MCMC mechanism may reveal blind spots.
The toy experiments indicate that the method can learn to hop modes with similar densities in 2D. But don't reflect statistical models where MCMC inference is used. Of course section 3 shows some interesting transitions can be learned in high dimensions. However, the interest is in application to MCMC, as there is plenty of work on generating synthetic images using a variety of procedures.
The experiments on Bayesian logistic regression are a starting point for looking at real statistical analyses. Personally, if forced to use HMC on logistic regression, I would estimate and factor the covariance of the posterior S = LL', using a preliminary run, or other approximate inference. Then reparameterize the space z = L^{-1}x, and run HMC on z. Equivalently set the mass matrix to be S^{-1}. Learning a linear transformation is simpler and more standard than fitting neural nets -- would the added complication still be worth it?
Projects like Stan come with demonstrations. Are there more complicated distributions from real problems where A-NICE-MC would readily out-perform current practice?
Clarity
The paper is fairly clearly written. The MCMC algorithm isn't fully spelled out, or made concrete (what's p(v)?). It would only be reproducible (to some extent) for those familiar with both HMC and NICE.
Originality
While the general idea of fitting models and adapting proposals has appeared periodically in the MCMC literature, this work is novel. The adversarial method proposed seems a good fit for learning proposals, rather than boldly trying to learn an explicit representation of the global distribution.
Significance
Statisticians are unlikely to adopt this work in its current form. It's a rather heavier dependency than "black-box variational inference", and I would argue that fitting neural nets is not generically black-box yet. The work is promising, but more compelling statistical examples would help.
Volume preserving proposals won't solve everything. One might naively think that the only challenge for MCMC is to come up with proposal operators that can make proposals to all the modes. However, typical samples from the posterior of a rich model can potentially vary in density a lot. There could be narrow tall modes, or short large basins of density, potentially with comparable posterior mass. Hierarchical models in particular can have explanations with quite different densities. A volume-preserving proposal can usually only change the log-density by ~1, or the Metropolis rule will reject too often.
Minor
The discussion of HMC and rules that return to (x,v) after two updates are missing the idea of negating the velocity. In HMC, the dynamics only reverse if we deterministically set
    v' <- -v'
after the first update.
Please don't use the term "Exact Sampling" in the header to section 4.1. In the MCMC literature that term usually means generating perfect independent samples from a target distribution, a much stronger requirement than creating updates that leave the target distribution invariant: http://dbwilson.com/exact/
The filesize of this paper is larger than it should be, making the paper slow to download and display. Figure 2 seems to be stored as a .jpeg, but should be a vector graphic -- or if you can't manage that, a .png that hasn't passed through being a .jpeg. Figure 3 doesn't seem to be stored as a .jpeg, and I think would be ~10x smaller if it were(?).
Several bibtex entries aren't capitalized properly. Protect capital letters with {.}.