Ranking Data with Continuous Labels through Oriented Recursive Partitions
We formulate a supervised learning problem, referred to as continuous ranking, where a continuous real-valued label Y is assigned to an observable r.v. X taking its values in a feature space X and the goal is to order all possible observations x in X by means of a scoring function s : X → R so that s(X) and Y tend to increase or decrease together with highest probability. This problem generalizes bi/multi-partite ranking to a certain extent and the task of finding optimal scoring functions s(x) can be naturally cast as optimization of a dedicated functional criterion, called the IROC curve here, or as maximization of the Kendall τ related to the pair (s(X), Y ). From the theoretical side, we describe the optimal elements of this problem and provide statistical guarantees for empirical Kendall τ maximization under appropriate conditions for the class of scoring function candidates. We also propose a recursive statistical learning algorithm tailored to empirical IROC curve optimization and producing a piecewise constant scoring function that is fully described by an oriented binary tree. Preliminary numerical experiments highlight the difference in nature between regression and continuous ranking and provide strong empirical evidence of the performance of empirical optimizers of the criteria proposed.

The paper studied the problem of continuous ranking, which is a generalization the bi-partite/multi-partite ranking problem in that the ranking label Y is a continuous real-valued random variable. Mathematical formulation of the problem is given. The problem of continuous ranking considered as multiple sub-problems, each corresponds to a bipartite ranking problem. Necessary conditions for the existence of optimal solutions are given. The IROC (and IAUC) measures are proposed for evaluating the performances of scoring functions. Finally, a continuous ranking algorithm called Crank is proposed. Experimental results on toy data showed the effectiveness of the proposed Crank algorithm.
The problem investigated in the paper is very interesting and important for learning to rank and related areas. The paper is well written. The proposed framework, measures, and algorithms are clearly presented. The empirical evaluation of the paper is weak, as it is based on toy data and weak baselines.
Pros.
1.	Generalizes the bipartite/multi-partite ranking problems to the continuous ranking problem. Rigorous formulation of the problem and strong theoretical results.
2.	Extending the conventional ROC to IROC for measuring the continuous ranking functions.
3.	Proposing Crank algorithm for conducting continuous ranking.
Cons.
1.	The authors claim that “theoretical analysis of this algorithm and its connection with approximation of IROC* are beyond the scope of this paper and will be the subject of a future work”. I am not convinced that “its connections with approximation of IROC* are beyond the scope of this paper”. As to my understanding of the paper, the goal of proposing IROC is to guide the proposal of new continuous ranking algorithms. Thus, it is very important to build the connections. Otherwise, the performances of the Crank algorithm cannot reflect the effectiveness of the proposed theoretical framework.
2.	The experiments in the paper are really weak. The proposed Crank algorithm is tested based on a toy data. The authors conclude in the last section “… finds many applications (e.g., quality control, chemistry)…”. It is necessary to test the proposed framework and solutions on real problems. The baselines are CART and Kendall, which are not designed for the ranking problem. It is better to compare the algorithm with state-of-the-art bipartite and multi-partite ranking models. The generation of the toy examples, the setting of parameters are not given, which makes it is hard to reproduce the results.
3.	The literature survey of the paper is not enough. Since the continuous ranking problem is a generalization of the bipartite ranking and multi-partite ranking problems, it is better if the authors could use a section to analyze the advantages and disadvantages of existing ranking models, and their real applications. Currently, only a few references are listed in Section 2.2.
4.	Minor issue: Line 52: Kendall tau  Kendall $\tau$; Line 66: F^{-1}(u) = inf{s\in R: F(s}>= t), the last t should be u;


Faster and Non-ergodic O(1/K) Stochastic Alternating Direction Method of Multipliers
We study stochastic convex optimization subjected to linear equality constraints. Traditional Stochastic Alternating Direction Method of Multipliers [1] and its Nesterov's acceleration scheme [2]  can only achieve ergodic O(1/ √ K) convergence rates, where K is the number of iteration. By introducing Variance Reduction (VR) techniques, the convergence rates improve to ergodic O(1/K) [3,4]. In this paper, we propose a new stochastic ADMM which elaborately integrates Nesterov's extrapolation and VR techniques. With Nesterov's extrapolation, our algorithm can achieve a non-ergodic O(1/K) convergence rate which is optimal for separable linearly constrained non-smooth convex problems, while the convergence rates of VR based ADMM methods are actually tight O(1/ √ K) in non-ergodic sense. To the best of our knowledge, this is the first work that achieves a truly accelerated, stochastic convergence rate for constrained convex problems. The experimental results demonstrate that our algorithm is faster than the existing state-of-the-art stochastic ADMM methods.

The paper considers the acceleration of stochastic ADMM. The paper basically combines existing techniques like Nesterov's acceleration technique, and increasing penalty parameter and variance reduction technique. The clearness and readability of the paper needs to be improved. The algorithm is designed for purpose of the theoretical bounds but not for a practically faster algorithm. Overall, the paper needs to be improved.
1. In the abstract and introduction, the paper does not clearly  point out the result is based on the average of the last m iterations when state non-ergodic convergence rate. The paper also does not point out the use of increasing penalty parameter which is an important trick to establish the theoretical result.
2. Algorithm 2 looks like a warm start algorithm by increasing the penalty parameter (viz. step size). For a warm start algorithm, the theoretical result is expected for the average of the last loop iterates and outer-loop iterates.
3.  If the theoretical result could be established without increasing penalty parameter, it is desirable. Trading penalty parameter (speed) for theoretical bound is not a significant contribution.


Counterfactual Fairness
Machine learning can impact people with legal or ethical consequences when it is used to automate decisions in areas such as insurance, lending, hiring, and predictive policing. In many of these scenarios, previous decisions have been made that are unfairly biased against certain subpopulations, for example those of a particular race, gender, or sexual orientation. Since this past data may be biased, machine learning predictors must account for this to avoid perpetuating or creating discriminatory practices. In this paper, we develop a framework for modeling fairness using tools from causal inference. Our definition of counterfactual fairness captures the intuition that a decision is fair towards an individual if it the same in (a) the actual world and (b) a counterfactual world where the individual belonged to a different demographic group. We demonstrate our framework on a real-world problem of fair prediction of success in law school.

This paper presents an interesting and valuable contribution to the small but growing literature on fairness in machine learning.  Specifically, it provides at least three contributions: (1) a definition of counter factual fairness; (2) an algorithm for learning a model under counter factual fairness; and (3) experiments with that algorithm.
The value and convincingness of each of these contributions declines steadily. The value of the contributions of the current paper is sufficient for acceptance, though significant improvements could be made in the clarity of exposition of the algorithm and the extent of the experimentation with the algorithm.
Section 4.2 outlines why is is likely that very strong assumptions will need to be made to effectively estimate a model of Y under counterfactual fairness.  The assumptions (and the implied analysis techniques) suggest conclusions that will not be particularly robust to violations of those assumptions.  This implies a need for significant empirical evaluation of what happens when those assumptions are violated or when other likely non-optimal conditions hold.  Realistic conditions that might cause particular problems include measurement error and non-homogeneity of causal effect.  Both of these are very likely to be present in the situations in which fairness is of concern (for example, crime, housing, loan approval, and admissions decisions).  The paper would be significantly improved by experiments examining model accuracy and fairness under these conditions (and under errors in assumptions).
The discussion of fairness and causal analysis omits at least one troubling aspect of real-world discrimination, which is the effect of continuing discrimination after fair decisions are made. For example, consider gender discrimination in business loans.  If discriminatory actions of customers will reduce the probability of success for a female-owned business, then such businesses will have a higher probability of default, even if the initial loan decisions are made in an unbiased way. It would be interesting to note whether a causal approach would allow consideration of such factors and (if so) how they would be considered.
Unmeasured selection bias would seem to be a serious threat to validity here, but is not considered in the discussion.  Specifically, data collection bias can affect the observed dependencies (and, thus, the estimation of potential unmeasured confounders U and the strength of causal effects).  Selection bias can occur due to selective data collection, retention, or reporting, when such selection is due to two or more variables.  It is relatively easy to imagine selection bias arising in cases where discrimination and other forms of bias are present.  The paper would be improved by discussing the potential effects of selection bias.
A few minor issues: The paper (lines 164-166 and Figure 1) references Pearl's twin network, but the explanation of twin networks in caption of Figure 1 provides insufficient background for readers to understand what twin networks are and how they can be interpreted.  Line 60 mentions "a wealth of work" two sentences after using "wealth" to refer to personal finances, and this may be confusing to readers.


Prototypical Networks for Few-shot Learning Vector Institute
We propose Prototypical Networks for the problem of few-shot classification, where a classifier must generalize to new classes not seen in the training set, given only a small number of examples of each new class. Prototypical Networks learn a metric space in which classification can be performed by computing distances to prototype representations of each class. Compared to recent approaches for few-shot learning, they reflect a simpler inductive bias that is beneficial in this limited-data regime, and achieve excellent results. We provide an analysis showing that some simple design decisions can yield substantial improvements over recent approaches involving complicated architectural choices and meta-learning. We further extend Prototypical Networks to zero-shot learning and achieve state-ofthe-art results on the CU-Birds dataset.

Summary: This paper addresses he recently re-popularised few-shot classification task. The idea is to represent each class as its mean/prototype within a learned embedding space, and then recognising new classes via softmax over distances to the prototypes. The model is trained by randomly randomly sampling classes and instances per episode. This is appealing due to its simplicity and speed compared to other influential few-shot methodologies [21,28]. Some insights are given about the connection to mixture density estimation in the case of Bregman divergence-based distances, linear models, and matching networks. The same framework extends relatively straightforwardly to zero-shot learning by making the class prototype be the result of a learned mapping from meta-data like attributes to the prototype vector. The model performs comparable or better than contemporary few/zero-shot methods on Omniglot, miniImageNet, and CUB.
Strengths/Weaknesses:
+ Intuitive and appealingly elegant method, that is simple and fast.
+ Authors provide several interpretations which draw connections drawn to other methods and help the reader understand well.
+ Some design choices are well explained , e.g. Euclidean distance outperforms cosine for good reason.
+ Good results
- Some other design decisions (normalisation; number of training classes per episode,  etc) less well explained. How much of good results is the proposed method per-se, and how much of it is tuning this stuff?
- Why the zero-shot part specifically works so well should be better explained.
Details:
- Recent work also has 54-56% on CUB. (Chanpinyo et al, CVPR’16, Zhang & Salgrama ECCV’16)
- This may not necessarily reduce the novelty of this work, but the use of mean of feature vectors from the same class has been proposed for the enhancement over training general (not few-shot specifically) classification model. [A Discriminative Feature Learning Approach for Deep Face Recognition, Wen et al, ECCV 2016]. The “center” in the above paper matches “prototype”. Probably this connection should be cited.
- For the results of zero-shot learning on CUB dataset, i.e., Table 3 page 7, the meta-data used here are “attribute”. This is good for fair comparison. However, from the perspective of getting better performance, better meta-data embeddings options are available. Refer to table 1 in “Learning Deep Representations of Fine-Grained Visual Descriptions, Reed et al, CVPR 2016”. It would be interesting to know the performance of the proposed method when it is equipped with better meta-data embeddings.
Update: Thanks to the authors for their response to the reviews. I think this paper is acceptable for NIPS.


Cortical microcircuits as gated-recurrent neural networks
Cortical circuits exhibit intricate recurrent architectures that are remarkably similar across different brain areas. Such stereotyped structure suggests the existence of common computational principles. However, such principles have remained largely elusive. Inspired by gated-memory networks, namely long short-term memory networks (LSTMs), we introduce a recurrent neural network in which information is gated through inhibitory cells that are subtractive (subLSTM). We propose a natural mapping of subLSTMs onto known canonical excitatory-inhibitory cortical microcircuits. Our empirical evaluation across sequential image classification and language modelling tasks shows that subLSTM units can achieve similar performance to LSTM units. These results suggest that cortical circuits can be optimised to solve complex contextual problems and proposes a novel view on their computational function. Overall our work provides a step towards unifying recurrent networks as used in machine learning with their biological counterparts.

A new recurrent neural network model is presented. It has similar functional features as LSTMs
			but additive gating instead of multiplicative gating for the input and output gates.
			With this additive gating mechanism there are some striking similarities to cortical circuits.
			In my opinion, this papers could be really interesting for both computational neuroscientists and machine learners.
			For small network sizes the proposed model performs as good or better than simple LSTMs on some non-trivial tasks.
			It is, however, somewhat disappointing that For the language modelling task it seems, however, that the multiplicative forget gates are still needed for good performance in larger networks.
			Further comments and questions:
			1. I guess the sigma-subRNN have also controlled forget gates (option i on line 104).
			   If yes, it would be fair to highlight this fact more clearly in the results/discussion section.
			   It is somewhat disappointing that this not so biologically plausible feature
			   seems to be needed to achieve good performances on the language modelling tasks.
			2. Would it be possible to implement the controlled forget gate also with a subtractive gate.
			   Maybe something like c_t = relu(c_{t-1} - f_t) + (z_t - i_t),
			   which could be interpreted as a strong inhibition (f_t) reseting the reverberating loop before new input (z_t - i_t) arrives.
			   But maybe the sign of c_t is important....
			3. line 111: Why is it important to have rectified linear transfer functions?
			   Doesn't sigma-subRNN perform better in some cases?
			   From the perspective of biological plausibility the transfer function should have the range [0, f_max]
			   where f_max is the maximal firing rate of a neuron (see e.g. the transfer function of a LIF neuron with absolute refractoriness).
			4. line 60 "inhibitory neurons act subtractively": I think one should acknowledge that
			   the picture is not so simple in biological systems (e.g. Chance et al. 2002 https://doi.org/10.1016/S0896-6273(02)00820-6 or
			   Müllner et al. 2015 https://doi.org/10.1016/j.neuron.2015.07.003).
			5. In my opinion it is not needed to explicitely state the well known Gershgorin circle theorem in the main text.
			6. Fig 3f: BGRNN? Should this be subRNN?
I read through the author’s rebuttal and my colleagues’ review, and decided to not change my assessment of the paper.


Deep Lattice Networks and Partial Monotonic Functions
We propose learning deep models that are monotonic with respect to a userspecified set of inputs by alternating layers of linear embeddings, ensembles of lattices, and calibrators (piecewise linear functions), with appropriate constraints for monotonicity, and jointly training the resulting network. We implement the layers and projections with new computational graph nodes in TensorFlow and use the Adam optimizer and batched stochastic gradients. Experiments on benchmark and real-world datasets show that six-layer monotonic deep lattice networks achieve state-of-the art performance for classification and regression with monotonicity guarantees.

This paper proposes a novel deep model that is monotonic to user specified inputs, called deep lattice networks (DLN's), which is composed of layers of linear embeddings,  ensembles of
 lattices and calibrators. The results about the function classes of DLN's are also provided.
Authors should put more effort to make the paper readable for the people that are not familiar with the technicality.  However the contributions are interesting and significant. As [3] proposed a 2 layer model,  which has a layer of calibrators followed by an ensemble of lattices, the advantages of proposed deeper model could be presented in a more convincing way by showing better results as the results seem closer to Crystals and it is not clear whether the produced results are stable.
The computational complexity should be also discussed clearly.  Each projection step in optimization should be explained in more detail. What is the computational complexity of the projection for the calibrator? What proportion of training time the projections take?  Are the projections accurate or suboptimal in the first iterations? And how the overall training time compares to baselines?
Does the training of the proposed model produces bad local minima for deeper models? At most how many layers could be trained with current optimization procedure?
It seems most of the results are based on the data that is not publicly available. To make the results reproducible, authors should add more publicly available data to experiments. Also artificial data  would help to demonstrate some sanity checks, e.g. a case that two layer [3] did not work well but deeper model work better, the intermediate layer visualizations etc .
Authors should also explain better why the model has interpretability and debuggability advantages.


Learning with Average Top-k Loss
In this work, we introduce the average top-k (AT k ) loss as a new aggregate loss for supervised learning, which is the average over the k largest individual losses over a training dataset. We show that the AT k loss is a natural generalization of the two widely used aggregate losses, namely the average loss and the maximum loss, but can combine their advantages and mitigate their drawbacks to better adapt to different data distributions. Furthermore, it remains a convex function over all individual losses, which can lead to convex optimization problems that can be solved effectively with conventional gradient-based methods. We provide an intuitive interpretation of the AT k loss based on its equivalent effect on the continuous individual loss functions, suggesting that it can reduce the penalty on correctly classified data. We further give a learning theory analysis of MAT k learning on the classification calibration of the AT k loss and the error bounds of AT k -SVM. We demonstrate the applicability of minimum average top-k learning for binary classification and regression using synthetic and real datasets.

This paper investigates a new learning setting: optimizing the average k largest (top-k) individual functions for supervised learning. This setting is different from the standard Empirical Risk minimization (ERM), which optimize the average loss function over datasets. The proposed setting is also different from maximum loss (Shalev-Shwartz and Wexler 2016), which optimize the maximum loss. This paper tries to optimize the average top-k loss functions. This can be viewed as a natural generalization of the ERM and the maximum loss. The authors summary it as a convex optimization problem, which can be solved with conventional gradient-based method. The authors give some learning theory analyses of setting on the classification calibration of the Top-k loss and the error bounds of ATk-SVM. Finally, the authors present some experiments to verify the effectiveness of the proposed algorithm.
This work is generally well-written with some advantages as follows:
1) The authors introduce a new direction for supervised learning, which is a natural generalization of ERM and the work of (Shalev-Shwartz and Wexler 2016).
2) Some theoretical analyses are presented for the proposed learning setting.
3) The author present a learning algorithm.
Cons:
1) Some statements are not clear, for example, top-k loss, which is similar to top-k ranking; more important, ensembles loss gives some impressions for ensemble learning whereas they are totally different.
2) When we used the average top-k loss? I do not think that the authors make clear explanations. Intuitively, the performance (w.r.t. accuracy) of average top-k loss is less than ERM without noise and outliers, while I guess the average too-k loss algorithm may have good performance when deal with noise data and outliers.
3) How to choose the parameter k? The authors use cross-validation in experiments, while there is no some analyses.
4) The authors should present some t-test on the performance on benchmark datasets. I doubt some experimental results. For example, the accuracy for German is about 0.79 for standard learning algorithms.
5) How about the efficiency in comparison with ERM and the work of (Shalev-Shwartz and Wexler 2016).


Estimating Accuracy from Unlabeled Data: A Probabilistic Logic Approach
We propose an efficient method to estimate the accuracy of classifiers using only unlabeled data. We consider a setting with multiple classification problems where the target classes may be tied together through logical constraints. For example, a set of classes may be mutually exclusive, meaning that a data instance can belong to at most one of them. The proposed method is based on the intuition that: (i) when classifiers agree, they are more likely to be correct, and (ii) when the classifiers make a prediction that violates the constraints, at least one classifier must be making an error. Experiments on four real-world data sets produce accuracy estimates within a few percent of the true accuracy, using solely unlabeled data. Our models also outperform existing state-of-the-art solutions in both estimating accuracies, and combining multiple classifier outputs. The results emphasize the utility of logical constraints in estimating accuracy, thus validating our intuition.

This paper builds on the work of Platanios et al. (2014, 2016) on estimating the accuracy of a set of classifiers for a given task using only unlabeled data, based on the agreement behavior of the classifiers. The current work uses a probabilistic soft logic (PSL) model to infer the error rates of the classifiers. The paper also extends this approach to the case where we have multiple related classification tasks: for instance, classifying noun phrases with regard to their membership in multiple categories, some of which subsume others and some of which are mutually exclusive. The paper shows how a PSL model can take into account these constraints among the categories, yielding better error rate estimates and higher joint classification accuracy.
[UPDATED] I see this paper as just above the acceptance threshold. It is well written and the methodology seems sound. It provides a compelling example of improving prediction accuracy by applying a non-trivial inference algorithm to multiple variables, not just running feed-forward classification.
The paper is less strong in terms of novelty. It's not surprising that a system can achieve better accuracy by using cross-category constraints. The paper does this with an existing PSL system (from Bach et al. 2015). The paper discusses a customized grounding algorithm in Section 3.3.2 (p. 6), but it seems to boil down to iterating over the given instances and classifier predictions and enumerating rule groundings that are not trivially satisfied. The only other algorithmic novelty seems to be the sampling scheme in the second paragraph of Section 3.3.3 (p. 7). [UPDATE: The authors say in their response that this sampling scheme was critical to speeding up inference so it could run at the necessary scale. If this is where a significant amount of the work's novelty lies, it deserves more space and a more thorough exposition.]
The paper could also benefit from deeper and more precise analysis. A central choice in the paper is the use of probabilistic soft logic, where variables can take values in [0, 1], rather than a framework such as Markov Logic Networks (MLNs) that deals with discrete variables. The paper says that this choice was a matter of scalability, and "one could just as easily perform inference for our model using MLNs" (lines 213-214, p. 5). I don't think this is true. The proposed model applies logical operators to continuous-valued variables (for the error rates, and the predictions of probabilistic classifiers), which is meaningful in PSL but not MLNs. If one were to model this problem with an MLN, I think one would replace the error rate variables with learnable weights on rules connecting the classifier outputs to the true category values. Such modeling decisions would change the scalability story: there would be no edges in the MLN between variables corresponding to different input instances (e.g., different noun phrases). So one could do exact inference on the sub-networks for the various instances in parallel, and accumulate statistics to use in an EM algorithm to learn the weights. This might be roughly as efficient as the optimization described in Section 3.3.3. [UPDATE: In the MLN I'm envisioning here, with one connected component per noun phrase, information is still shared across instances (noun phrases) during parameter estimation: the weights are shared across all the instances. But when the parameters are fixed, as in the E-step of each EM iteration, the inference algorithm can operate on each noun phrase independently.]
The experimental results could also use more analysis. It's not surprising that the proposed LEE algorithm is the best by all metrics in the first row of Table 1 (p. 8), where it takes cross-category constraints into account. I have a number of questions about the other two rows:
[UPDATE: Thanks for the answers to these questions.]
* What are the "10%" data sets? I didn't find those mentioned in the text.
* Does simple majority voting really yield an AUC of 0.999 on the uNELL-All data set? If that's the case, why are the AUCs for majority voting so much lower on NELL-7 and NELL-11? Which setting is most representative of the full problem faced by NELL?
* Why does the proposed method yield an AUC of only 0.965 on uNELL-All, performing worse than majority voting and several other methods? [UPDATE: The authors note that the classifier weights from the proposed method end up performing worse than equal weighting on this data set. This could use more analysis. Is there a shortcoming in the method for turning error rates into voting weights?]
* On the other three data sets in these rows, the proposed approach does better than the algorithms of Platanios et al. 2014 and 2016, which also use reasonable models. Why does the new approach do better? Is this because the classifiers are probabilistic, and the new approach uses those probabilities while the earlier approaches binarize the predictions? [UPDATE: This still seems to be an open question. The paper would be more satisfying if included some additional analysis on this issue.]
* For a given probabilistic classifier on a given labeled data set, the two methods for computing an "error rate" -- one thresholding the prediction at 0.5 and counting errors; the other averaging the probabilities given to incorrect classifications -- will yield different numbers in general. Which way was the "true" error rate computed for these evaluations? Is this fair to all the techniques being evaluated? [UPDATE: The authors clarified that they thresholded the predictions at 0.5 and counted errors, which I agree is fair to both probabilistic and non-probabilistic classifiers.]
Smaller questions and comments:
* p. 1, Figure 1: If there is a shortage of space, I think this figure could be removed. I skipped over it on my first reading of the paper and didn't feel like I was missing something.
* p. 2, line 51: "we define the error rate..." Wouldn't it be helpful to consider separate false positive and false negative rates?
* p. 2, line 55: The probabilistic equation for the error rate here doesn't seem syntactically valid. The error rate is not parameterized by a particular instance X, but the equation includes \hat{f}^d_j(X) and (1 - \hat{f}^d_j(X)) outside the P_D terms. Which X is used there? I think what's intended is the average over the data set of the probability that the classifier assigns to the incorrect classification.
* p .4, line 148: Were the techniques described in this Identifiability section used in the experiments? If so, how was the prior weight set?
* p. 5, Equation 3: Why does the model use \hat{f}^{d_1}_j here, rather than using the unobserved predicate f^{d_1} (which would be symmetrical with f^{d_2}) and relying on constraint propagation?
* p. 5, line 219: This line starts using the symbol f for a probability density, which is confusing given the extensive use of f symbols for classifiers and target functions.
* p. 6, line 223: What is p_j? Is it always 1 in the proposed model?
* p. 6, line 225: How were the lambda parameters set in the experiments?
* p. 6, line 257: The use of N^{d_1} in these counting expressions seems imprecise. The expressions include factors of D^2 or D, implying that they're summing over all the target functions in D. So why is the N specific to d_1?
* p. 7, line 300: "response every input NP". Missing "for" here.
* p. 8, Table 1: The use of different scaling factors (10^-2 and 10^-1) in different sections of the table is confusing. I'd recommend just presenting the numbers without scaling.


Collecting Telemetry Data Privately
The collection and analysis of telemetry data from user's devices is routinely performed by many software companies. Telemetry collection leads to improved user experience but poses significant risks to users' privacy. Locally differentially private (LDP) algorithms have recently emerged as the main tool that allows data collectors to estimate various population statistics, while preserving privacy. The guarantees provided by such algorithms are typically very strong for a single round of telemetry collection, but degrade rapidly when telemetry is collected regularly. In particular, existing LDP algorithms are not suitable for repeated collection of counter data such as daily app usage statistics. In this paper, we develop new LDP mechanisms geared towards repeated collection of counter data, with formal privacy guarantees even after being executed for an arbitrarily long period of time. For two basic analytical tasks, mean estimation and histogram estimation, our LDP mechanisms for repeated data collection provide estimates with comparable or even the same accuracy as existing single-round LDP collection mechanisms. We conduct empirical evaluation on real-world counter datasets to verify our theoretical results. Our mechanisms have been deployed by Microsoft to collect telemetry across millions of devices.

The authors propose a more efficient mechanism for local differential privacy. In particular, they propose a 1 bit local differential privacy method that enables mean estimation in a distributed and effective way.  Also a d-bit method is proposed for histogram estimation. It is argued that these two goals are of partical relevance in collection of telemetry data. The main advance is in the communication efficiency - which is key to large scale deployment. The authors also note that this mechanism is indeed deployed at large scale in industry and likely consistutes the largest deployment of differential privacy to date. While this on its own does not add to the value of the scientific contribution - it does reinforce the point of an efficient, effective and practical approach.
The authors submitted another version of their paper as supplementary material that contains the proofs. But it also contains more figures and extended main text. This is borderline. The supplementary material should only contain the additional material - while the main paper should be self-contained and not repeated. It is also weird, that the reviewer has to hunt for proofs in provided text. There should be a clear separation between submission and supplementary material.
Overall, the presentation is clear - partially due to outsourcing parts of the submission to the supplementary. However, alpha-rounding part remains partially unclear. Can the authors please elaborate on the claimed guarantees?
While the experimental study is interesting - there is a question about reproducibility of the results.As it looks this is real user and company data - how would follow up work compare here? It would be beneficial to provide an alternative solution -- potential simulated data -- so that comparison remain open source.
However, the experiments are considered convincing and support the main claim.


Online Learning for Multivariate Hawkes Processes
We develop a nonparametric and online learning algorithm that estimates the triggering functions of a multivariate Hawkes process (MHP). The approach we take approximates the triggering function f i,j (t) by functions in a reproducing kernel Hilbert space (RKHS), and maximizes a time-discretized version of the log-likelihood, with Tikhonov regularization. Theoretically, our algorithm achieves an O(log T ) regret bound. Numerical results show that our algorithm offers a competing performance to that of the nonparametric batch learning algorithm, with a run time comparable to parametric online learning algorithms.

This paper describes an algorithm for optizimization of Hawkes process parameters in on-line settings, where non-parametric form of a kernel is learnt. The paper reports a gradient approach to optimization, with theoretical analysis thereof. In particular, the authors provide: a regret bound, justification for simplification steps (discretization of time and truncation of time over which previous posts influence a new post), an approach to a tractable projection of the solution (a step in the algorithm), time complexity analysis.
The paper is very well written, which is very helpful given it is mathematically involved. I found it tackling an important problem (on-line learning is important for large scale datasets, and non-parametricity is a very reasonable setting when it is hard to specify a reasonable kernel form a priori). The authors provide theoretical grounding for their approach. I enjoyed this paper very much and I think it is going to be influencial for practical application of Hawkes processes.
Some comments:
- Can you comment on the influence of hyperparameter values (regularization coefficients, potentially also discretization step)? In experiments you set them a-priori to some fixed values, and it is not uncommon in the experiments that values of regularization hyperparameters have a huge influence on the experiments.
- Your experiments are quite limited, especially the synthetic experiments where many values are fixed and a single matrix of influence forms is considered. I think it would be much more compelling if some simulation was considered, where values would be repeatedly drawn. Also, why do you think this specific setting is interesting/realistic?
- I suggest making the font in Figures larger (especially for Figure 2 it is too small), it doesn't look good now. This is especially annoying in the supplementary (Figure 4).


Deep Hyperalignment
This paper proposes Deep Hyperalignment (DHA) as a regularized, deep extension, scalable Hyperalignment (HA) method, which is well-suited for applying functional alignment to fMRI datasets with nonlinearity, high-dimensionality (broad ROI), and a large number of subjects. Unlink previous methods, DHA is not limited by a restricted fixed kernel function. Further, it uses a parametric approach, rank-m Singular Value Decomposition (SVD), and stochastic gradient descent for optimization. Therefore, DHA has a suitable time complexity for large datasets, and DHA does not require the training data when it computes the functional alignment for a new subject. Experimental studies on multi-subject fMRI analysis confirm that the DHA method achieves superior performance to other state-of-the-art HA algorithms.

Deep Hyperalignment
[Due to time constraints, I did not read the entire paper in detail.]
The problem of functional alignment of multi-subject fMRI datasets is a fundamental and difficult problem in neuroimaging. Hyperalignment is a prominent class of anatomy-free functional alignment methods. The present manuscript presents a extension to this using a deep NN kernel. I believe this to be novel, though I don’t know the literature well enough to be confident of that. While not a surprising research direction, this is a natural and promising method to consider, and the experimental results are excellent.
The methods are explained thoroughly, with detailed proofs in supplementary material. I did however find this section too dense and without a good summary of the final design and parameter choices. A figure sketching the method would have been very helpful. Related work is surveyed well. The new method could be described as an amalgam of several existing approaches. However the various elements have been combined thoughtfully in a novel way, and carefully implemented.
The experiments are impressively thorough, involving several open datasets and several competing methods, with detailed reporting of methods, including the parameters used for each of the alignment methods. Some more detail on preprocessing would have been helpful. It seems there has been some validation of parameters on the test data (“the best performance for each dataset is reported” p6), which is not ideal, however the number of parameter values tested is small and the results are so good that I feel they would stand up to a better training/validation/test split. My main concern about this section is that all of the results reported were computed by the present authors; it would be helpful to also flag any comparable results in the literature.
The quality of the English is mostly good but with occasional problems. E.g “time complexity fairly scales with data size” is unclear, and the sentence in which it appears is repeated three times in the manuscript (including once in the abstract). Some proof-reading is in order.


Compatible Reward Inverse Reinforcement Learning
Inverse Reinforcement Learning (IRL) is an effective approach to recover a reward function that explains the behavior of an expert by observing a set of demonstrations. This paper is about a novel model-free IRL approach that, differently from most of the existing IRL algorithms, does not require to specify a function space where to search for the expert's reward function. Leveraging on the fact that the policy gradient needs to be zero for any optimal policy, the algorithm generates a set of basis functions that span the subspace of reward functions that make the policy gradient vanish. Within this subspace, using a second-order criterion, we search for the reward function that penalizes the most a deviation from the expert's policy. After introducing our approach for finite domains, we extend it to continuous ones. The proposed approach is empirically compared to other IRL methods both in the (finite) Taxi domain and in the (continuous) Linear Quadratic Gaussian (LQG) and Car on the Hill environments.

This paper proposes an approach for behavioral cloning that constructs a function space for a particular parametric policy model based on the null space of the policy gradient.
I think a running example (e.g., for discrete MDP) would help explain the approach. I found myself flipping back and forth from the Algorithm (page 6) to the description of each step. I have some lingering confusion about using Eq. (4) to estimate discounted vectors d(s,a) since (4) only computed d(s). I assume a similar estimator is employed for d(s,a).
My main concern with the paper’s approach is that of overfitting and lack of transfer. The main motivation of IRL is that the estimated reward function can be employed in other MDPs (without rewards) that have similar feature representations. Seeking to construct a sufficiently rich feature representation that can drive the policy gradient to zero suggests that there is overfitting to the training data at the expense of generalization abilities. I am skeptical then that reasonable generalization can be expected to modifications of the original MDP (e.g., if the dynamics changes or set of available actions changes) or to other MDPs. What am I missing that allows this approach to get beyond behavioral cloning?
My understanding of the Taxi experiment in particular is that data is generated according to the policy model under line 295 and that the proposed method uses this policy model, while comparison methods are based on different policy models (e.g., BC instead estimates a Boltzmann policy). This seems like an extremely unfair advantage to the proposed approach that could account for all of the advantages demonstrated over BC. What happens when likelihood is maximized for BC under this actual model? Experiments with data generated out of model and generalization to making predictions beyond the source MDP are needed to improve the paper.
In summary, though the method proposed is interesting, additional experiments that mimic “real-world” settings are needed before publication to investigate the benefits of the approach suitability to noisy data and for generalization purposes.


Improved Training of Wasserstein GANs
Generative Adversarial Networks (GANs) are powerful generative models, but suffer from training instability. The recently proposed Wasserstein GAN (WGAN) makes progress toward stable training of GANs, but sometimes can still generate only poor samples or fail to converge. We find that these problems are often due to the use of weight clipping in WGAN to enforce a Lipschitz constraint on the critic, which can lead to undesired behavior. We propose an alternative to clipping weights: penalize the norm of gradient of the critic with respect to its input. Our proposed method performs better than standard WGAN and enables stable training of a wide variety of GAN architectures with almost no hyperparameter tuning, including 101-layer ResNets and language models with continuous generators. We also achieve high quality generations on CIFAR-10 and LSUN bedrooms. †

+ good results
+ pretty pictures
- exposition could be improved
- claims are not backed up
- minor technical contribution
- insufficient evaluation
Let me first say that the visual results of this paper are great. The proposed algorithm produces pretty pictures of bedrooms and tiny cifar images.
Unfortunately the rest of the paper needs to be significantly improved for acceptance at NIPS. The exposition and structure of the paper is below NIPS standards. For example
L3: "..., but can still generate low-quality samples or fail to converge in some settings." I'm assuming the authors mean "can only..."?
l6: "pathological behavior". Pathological seems the wrong word here...
Sections 3 and 4 should be flipped. This way the exposition doesn't need to refer to a method that was not introduced yet.
Secondly, the paper is full of claims that are not backed up. The reviewer recommends the authors to either remove these claims or experimentally or theoretically back them up. Here are a few:
 L101: Proof that only simple functions have gradient norm almost everywhere, note that almost everywhere is important here. There can be exponentially many parts of the landscape that are disconnected by gradient norm!=1 parts.
If a proof is not possible, please remove this claim.
 L 144: Two sided penalty works better. Please show this in experiments.
The actual technical contribution seems like a small modification of WGAN. Box constraint is relaxed to a l2 norm on gradients.
Finally the evaluation is insufficient. The paper shows some pretty generation results for various hyper-parameter setting, but doesn't show any concrete numeric evaluation, apart from the easy to cheat and fairly meaningless inception score. The authors should perform a through user study to validate their approach, especially in light of a slim technical contribution.
Minor:
How is (3) efficiently evaluated? Did the authors have to modify specific deep learning packages to compute the gradient of the gradient? How is this second derivative defined for ReLUs?
Post rebuttal:
The authors answered some of my concerns in the rebuttal. I'm glad the authors promised to improve the writing and remove the misleading claims. However the rebuttal didn't fully convince me, see below.
I'm still a bit hesitant to accept the argument that prettier pictures are better, the authors rebuttal didn't change much in that regard. I'm sure some image filter, or denoising CNN could equally improve the image quality. I would strongly suggest the authors to find a way to quantitatively evaluate their method (other than the inception score).
I'm also fairly certain that the argument about the second derivative of ReLU's is wrong. The second derivative of a relu is not defined, as far as I know. The authors mention it's all zero, but that would make it a linear function (and a relu is not linear)... The reviewers had an extensive discussion on this and we agree that for ReLU networks the objective is not continuous, hence the derivative of the objective does not point down hill at all times. There are a few ways the authors should address this:
 1. Try twice differentiable non-linearities (elu, tanh or log(1+exp(x)).
 2. Show that treating the ReLU as a linear function (second derivative of zero) does give a good enough gradient estimate, for example through finite differences, or by verifying how often the "gradient" points down hill.


Tensor Biclustering
Consider a dataset where data is collected on multiple features of multiple individuals over multiple times. This type of data can be represented as a three dimensional individual/feature/time tensor and has become increasingly prominent in various areas of science. The tensor biclustering problem computes a subset of individuals and a subset of features whose signal trajectories over time lie in a low-dimensional subspace, modeling similarity among the signal trajectories while allowing different scalings across different individuals or different features. We study the information-theoretic limit of this problem under a generative model. Moreover, we propose an efficient spectral algorithm to solve the tensor biclustering problem and analyze its achievability bound in an asymptotic regime. Finally, we show the efficiency of our proposed method in several synthetic and real datasets.

This paper studies the tensor biclustering problem, which is
	defined as follows: given an order-3 tensor that is assumed to
	be of rank 1 with additive Gaussian noise, we want to find
	index sets along with mode 1 and mode 2 where the factor
	vectors have non-zero entries. The paper considers several
	methods and analyzes their recovery performance. The
	statistical lower bound (i.e. the best achievable performance)
	is also evaluated. The performance is also evaluated by experiments.
	The paper is clearly written, self-contained, and easy to follow.
	Overall, this paper aims to develop a relatively minor
	problem, tensor biclustering, theoretically in depth, and it
	seems to be succeeded. For computationally efficient methods
	(the proposed algorithms), the asymptotic order of variance
	where the non-zero indices are correctly recovered is
	identified. In addition, the optimal bound of performance is
	evaluated, which is helpful as a baseline.
	Still, I feel there is room for improvement.

	1. The model formulation (1) looks less general. The strongest
	restriction may be the setting q=1. Although the paper says
	the analysis can be generalized into the case q>1 (Line 83),
	it is not discussed in the rest of paper. For me, the analysis
	for q>1 is not trivial. It is very helpful if the paper
	contains discussion on, for q>1, how we can adapt the
	analysis, how the theoretical results change, etc.
	2. Synthetic experiments can be elaborate. I expected to see
	the experiments that support the theoretical results
	(e.g. Theorems 1 and 2). For example, for various n and
	\sigma_1^2, you can plot monochrome tiles where each tile
	represents the recovery rate at corresponding n and \sigma_1^2
	(e.g. the tile is black if the rate is 0 and is white if the
	rate is 1), which ideally shows the phase transition that the
	theorems foreshow.


Joint distribution optimal transportation for domain adaptation
This paper deals with the unsupervised domain adaptation problem, where one wants to estimate a prediction function f in a given target domain without any labeled sample by exploiting the knowledge available from a source domain where labels are known. Our work makes the following assumption: there exists a nonlinear transformation between the joint feature/label space distributions of the two domain P s and P t that can be estimated with optimal transport. We propose a solution of this problem that allows to recover an estimated target P f t = (X, f (X)) by optimizing simultaneously the optimal coupling and f . We show that our method corresponds to the minimization of a bound on the target error, and provide an efficient algorithmic solution, for which convergence is proved. The versatility of our approach, both in terms of class of hypothesis or loss functions is demonstrated with real world classification and regression problems, for which we reach or surpass state-of-the-art results.

The manuscript introduces a new technique for unsupervised domain adaptation (DA) based on optimal transport (OT). Previous related techniques (that are state-of-the-art) use optimal transport to find a mapping between the marginal distributions of source and target.
The manuscript proposes to transports the joint distributions instead of the marginals. As the target does not have labels, the learned prediction function on the target is used as proxy. The consequence is an iterative algorithm that alternates between 1) estimating the transport for fixed data points and labeling, and 2) estimating the labeling function for fixed transport. As such, the algorithm can be seen as a self-learning inspired extension of previous work, which executed 1) and 2) only once.
In addition to the algorithm, the manuscript contains a high probability generalization bound, similar to the discrepancy-based bounds of Ben-David et al [24]. It is used as justification that minimizing the joint OT distance should result in better predictions. Formally, this is not valid, however, because the bound is not uniform in the labeling function, so it might not hold for the result of the minimization.
There are also experiments on three standard datasets that show that the proposed method achieve state-of-the-art performance (though the differences to the previous OT based methods are small).
strength:
- the paper is well written, the idea makes sense
- the method seems practical, as both repeated steps resemble previous OT-based DA work
- the theory seems correct (but is not enough to truly justify the method, see below)
- the experiments are done well, including proper model selection (which is rare in DA)
- the experimental results are state-of-the-art
weaknesses:
- the manuscript is mainly a continuation of previous work on OT-based DA
- while the derivations are different, the conceptual difference is previous work is limited
- theoretical results and derivations are w.r.t. the loss function used for learning (e.g.
  hinge loss), which is typically just a surrogate, while the real performance measure would
  be 0/1 loss. This also makes it hard to compare the bounds to previous work that used 0-1 loss
- the theorem assumes a form of probabilistic Lipschitzness, which is not explored well.
  Previous discrepancy-based DA theory does not need Prob.Lipschitzness and is more flexible
  in this respect.
- the proved bound (Theorem 3.1) is not uniform w.r.t. the labeling function $f$. Therefore,
  it does not suffice as a justification for the proposed minimization procedure.
- the experimental results do not show much better results than previous OT-based DA methods
- as the proposed method is essentially a repeated application of the previous work, I would have
  hoped to see real-data experiments exploring this. Currently, performance after different number
  of alternating steps is reported only in the supplemental material on synthetic data.
- the supplemental material feels rushed in some places. E.g. in the proof of Theorem 3.1, the
  first inequality on page 4 seems incorrect (as the integral is w.r.t. a signed measure, not a
  prob.distr.). I believe the proof can be fixed, though, because the relation holds without
  absolute values, and it's not necessary to introduce these in (3) anyway.
- In the same proof, Equations (7)/(8) seem identical to (9)/(10)
questions to the authors:
- please comment if the effect of multiple BCD on real data is similar to the synthetic case
***************************
I read the author response and I am still in favor of accepting the work.


Log-normality and Skewness of Estimated State/Action Values in Reinforcement Learning
Under/overestimation of state/action values are harmful for reinforcement learning agents. In this paper, we show that a state/action value estimated using the Bellman equation can be decomposed to a weighted sum of path-wise values that follow log-normal distributions. Since log-normal distributions are skewed, the distribution of estimated state/action values can also be skewed, leading to an imbalanced likelihood of under/overestimation. The degree of such imbalance can vary greatly among actions and policies within a single problem instance, making the agent prone to select actions/policies that have inferior expected return and higher likelihood of overestimation. We present a comprehensive analysis to such skewness, examine its factors and impacts through both theoretical and empirical results, and discuss the possible ways to reduce its undesirable effects.

This paper focuses on the problem arising from skewness in the distribution of value estimates, which may result in over- or under-estimation. With careful analysis, the paper shows that a particular model-based value estimate is approximately log-normally distributed, which is skewed and thus leading to the possibility of over- or under-estimation. It is further shown that positive and negative rewards induce opposite sort of skewness. With simple experiments, the problem of over/underestimation is illustrated.
This is an interesting paper with some interesting insights on over/underestimation of values. I would like the reviewers to clarify the following two issues.
The results are implied for a particular kind of model-based value estimation method, more specifically, one which relies on definition 3.1. The implication of this result to other kinds of estimation methods, especially the model-free ones is not clear. A discussion is required on why a similar effect is expected with other kinds of values estimation method.
It is disappointing to have no experiments with the suggested remedies, such as the balancing the impact of positive and negative rewards and reward shaping. Some experiments are expected here to see whether these are indeed helpful. And an elaborate discussion of reward balancing and how to achieve it could also improve the quality of the paper.


Eigen-Distortions of Hierarchical Representations
We develop a method for comparing hierarchical image representations in terms of their ability to explain perceptual sensitivity in humans. Specifically, we utilize Fisher information to establish a model-derived prediction of sensitivity to local perturbations of an image. For a given image, we compute the eigenvectors of the Fisher information matrix with largest and smallest eigenvalues, corresponding to the model-predicted most-and least-noticeable image distortions, respectively. For human subjects, we then measure the amount of each distortion that can be reliably detected when added to the image. We use this method to test the ability of a variety of representations to mimic human perceptual sensitivity. We find that the early layers of VGG16, a deep neural network optimized for object recognition, provide a better match to human perception than later layers, and a better match than a 4-stage convolutional neural network (CNN) trained on a database of human ratings of distorted image quality. On the other hand, we find that simple models of early visual processing, incorporating one or more stages of local gain control, trained on the same database of distortion ratings, provide substantially better predictions of human sensitivity than either the CNN, or any combination of layers of VGG16.
We develop a method for comparing hierarchical image representations in terms of their ability to explain perceptual sensitivity in humans. Specifically, we utilize Fisher information to establish a model-derived prediction of sensitivity to local perturbations of an image. For a given image, we compute the eigenvectors of the Fisher information matrix with largest and smallest eigenvalues, corresponding to the model-predicted most-and least-noticeable image distortions, respectively. For human subjects, we then measure the amount of each distortion that can be reliably detected when added to the image. We use this method to test the ability of a variety of representations to mimic human perceptual sensitivity. We find that the early layers of VGG16, a deep neural network optimized for object recognition, provide a better match to human perception than later layers, and a better match than a 4-stage convolutional neural network (CNN) trained on a database of human ratings of distorted image quality. On the other hand, we find that simple models of early visual processing, incorporating one or more stages of local gain control, trained on the same database of distortion ratings, provide substantially better predictions of human sensitivity than either the CNN, or any combination of layers of VGG16.
Human capabilities for recognizing complex visual patterns are believed to arise through a cascade of transformations, implemented by neurons in successive stages in the visual system. Several recent studies have suggested that representations of deep convolutional neural networks trained for object recognition can predict activity in areas of the primate ventral visual stream better than models constructed explicitly for that purpose (Yamins et al. [2014], Khaligh-Razavi and Kriegeskorte [2014]). These results have inspired exploration of deep networks trained on object recognition as models of human perception, explicitly employing their representations as perceptual distortion metrics or loss functions (Hénaff and Simoncelli [2016], Johnson et al. [2016], Dosovitskiy and Brox [2016]).
On the other hand, several other studies have used synthesis techniques to generate images that indicate a profound mismatch between the sensitivity of these networks and that of human observers. Specifically, Szegedy et al. [2013] constructed image distortions, imperceptible to humans, that cause their networks to grossly misclassify objects. Similarly, Nguyen and Clune [2015] optimized randomly initialized images to achieve reliable recognition by a network, but found that the resulting 'fooling images' were uninterpretable by human viewers. Simpler networks, designed for texture classification and constrained to mimic the early visual system, do not exhibit such failures (Portilla and Simoncelli [2000]). These results have prompted efforts to understand why generalization failures of this type are so consistent across deep network architectures, and to develop more robust training methods to defend networks against attacks designed to exploit these weaknesses (Goodfellow et al. [2014]).
From the perspective of modeling human perception, these synthesis failures suggest that representational spaces within deep neural networks deviate significantly from those of humans, and that methods for comparing representational similarity, based on fixed object classes and discrete sampling of the representational space, are insufficient to expose these deviations. If we are going to use such networks as models for human perception, we need better methods of comparing model representations to human vision. Recent work has taken the first step in this direction, by analyzing deep networks' robustness to visual distortions on classification tasks, as well as the similarity of classification errors that humans and deep networks make in the presence of the same kind of distortion (Dodge and Karam [2017]).
Here, we aim to accomplish something in the same spirit, but rather than testing on a set of handselected examples, we develop a model-constrained synthesis method for generating targeted test stimuli that can be used to compare the layer-wise representational sensitivity of a model to human perceptual sensitivity. Utilizing Fisher information, we isolate the model-predicted most and least noticeable changes to an image. We test these predictions by determining how well human observers can discriminate these same changes. We apply this method to six layers of VGG16 (Simonyan and Zisserman [2015]), a deep convolutional neural network (CNN) trained to classify objects. We also apply the method to several models explicitly trained to predict human sensitivity to image distortions, including both a 4-stage generic CNN, an optimally-weighted version of VGG16, and a family of highly-structured models explicitly constructed to mimic the physiology of the early human visual system. Example images from the paper, as well as additional examples, are available at http://www.cns.nyu.edu/~lcv/eigendistortions/.

PAPER SUMMARY
The paper attempts to measure the degree to which learned image representations from convolutional networks can explain human perceptual sensitivity to visual stimuli. This question is well-motivated on the neuroscience side by recent studies that have used learned image representations to predict brain activity in primates, and on the computer vision side by recent methods for image synthesis which have used image representations learned for object recognition as a surrogate for human perception.
The paper develops a clever experimental paradigm to test the extent to which a (differentiable) image representation agrees with human perception by experimentally measuring the human discrimination threshold for image distortions along eigendirections of the Fisher information matrix corresponding to extremal eigenvalues.
Using this experimental technique, the paper demonstrates that for a VGG-16 model pretrained for ImageNet classification, image representations at earlier layers correlate with human perception more strongly than those at later layers.
The paper next explores image representations from models trained explicitly to match human judgements of perceived image distortion, training both a four-layer CNN and an On-Off model whose structure reflects the structure of the human LGN. Both models are able to fit the training data well, with the CNN slightly outperforming the On-Off model. However using the experimental paradigm from above, the paper shows that image representations from the On-Off model correlate much more strongly with human perception than either the shallow CNN model or representations from any layer of the VGG-16 model trained on ImageNet.
FEATURES FROM IQA CNN
For the four-layer IQA CNN from Section 3.1, it seems that only the image representation from the final layer of the network were compared with human perceptual judgements; however from Figure 3 we know that earlier layers of VGG-16 explain human perceptual judgements much better than later layers. I’m therefore curious about whether features from earlier layers of the IQA CNN would do a better job of explaining human perception than final layer features.
RANDOM NETWORKS AND GANS
Some very recent work has shown that features from shallow CNNs with no pooling and random filters can be used for texture synthesis nearly as well as features from deep networks pretrained on ImageNet (Ustyuzhaninov et al, ICLR 2017), suggesting that the network structure itself and not the task on which it was trained may be a key factor in determining the types of perceptual information its features capture. I wonder whether the experimental paradigm of this paper could be used to probe the perceptual effects of trained vs random networks?
In a similar vein, I wonder whether features from the discriminator of a GAN would better match human perception than ImageNet models, since such a discriminator is explicitly trained to detect synthetic images?
ON-OFF MODEL FOR IMAGE SYNTHESIS
Given the relative success of the On-Off model at explaining human perceptual judgements, I wonder whether it could be used in place of VGG features for image synthesis tasks such as super-resolution or texture generation?
OVERALL
Overall I believe this to be a high-quality, well-written paper with an interesting technical approach and exciting experimental results which open up many interesting questions to be explored in future work.
POST AUTHOR RESPONSE
I did not have any major issues with the paper as submitted, and I thank the authors for entertaining some of my suggestions about potential future work. To me this paper is a clear accept.


Subset Selection and Summarization in Sequential Data
Subset selection, which is the task of finding a small subset of representative items from a large ground set, finds numerous applications in different areas. Sequential data, including time-series and ordered data, contain important structural relationships among items, imposed by underlying dynamic models of data, that should play a vital role in the selection of representatives. However, nearly all existing subset selection techniques ignore underlying dynamics of data and treat items independently, leading to incompatible sets of representatives. In this paper, we develop a new framework for sequential subset selection that finds a set of representatives compatible with the dynamic models of data. To do so, we equip items with transition dynamic models and pose the problem as an integer binary optimization over assignments of sequential items to representatives, that leads to high encoding, diversity and transition potentials. Our formulation generalizes the well-known facility location objective to deal with sequential data, incorporating transition dynamics among facilities. As the proposed formulation is non-convex, we derive a max-sum message passing algorithm to solve the problem efficiently. Experiments on synthetic and real data, including instructional video summarization, show that our sequential subset selection framework not only achieves better encoding and diversity than the state of the art, but also successfully incorporates dynamics of data, leading to compatible representatives.

The paper consider a new problem of subset selection from set X that represents objects from sequence Y with additional constraint that the selected subset should comply with underlaying dynamic model of set X. The problem seems interesting to me and worth investigation.
The novel problem boils down to maximization problem (11). The authors suggest a new message-passing scheme for the problem.
The problem can be seen as a multilabel energy minimization problem (T nodes, M labels, chain structure with additional global potential -log(\Phi_{card}(x_{r_1}, \ldots, x_{r_T}))) written in overcomplete representation form [23]. The problem is well known in energy minimization community as energy minimization with label costs [2]. It is shown that global potential -log(\Phi_{card}(x_{r_1}, \ldots, x_{r_T})) can be represented as a set of pairwise submodular potentials with auxiliary variables [2, 3]. Therefore, the problem (11) can be seen as multilabel energy minimization with pairwise potentials only. There are tons of methods including message passing based to solve the problem approximately (TRW-S, LBP, alpha-beta swap, etc). While they may work worse than the proposed message passing, the thoughtful comparison should be performed.
I believe that the paper in its current form cannot be accepted. Although the considered problem is interesting, the related work section should include [1,2,3]. Also, different baselines for experimental evaluation should be used (other optimization schemes for (11)), as it is clear from the experiments that DPP based methods do not suit for the problem. I think that the improved version may have a great impact.
typos:
- (8) \beta was missed
- L229 more more -> more
- L236 jumping, jumping -> jumping
[1] Wainwright, M.J., Jordan, M.I.: Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning, 2008
[2] Delong, Andrew, et al. "Fast approximate energy minimization with label costs." International journal of computer vision 96.1 (2012): 1-27.
[3]  M. A. Fischler and R. C. Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381–395, 1981.


Deanonymization in the Bitcoin P2P Network
Recent attacks on Bitcoin's peer-to-peer (P2P) network demonstrated that its transaction-flooding protocols, which are used to ensure network consistency, may enable user deanonymization-the linkage of a user's IP address with her pseudonym in the Bitcoin network. In 2015, the Bitcoin community responded to these attacks by changing the network's flooding mechanism to a different protocol, known as diffusion. However, it is unclear if diffusion actually improves the system's anonymity. In this paper, we model the Bitcoin networking stack and analyze its anonymity properties, both pre-and post-2015. The core problem is one of epidemic source inference over graphs, where the observational model and spreading mechanisms are informed by Bitcoin's implementation; notably, these models have not been studied in the epidemic source detection literature before. We identify and analyze near-optimal source estimators. This analysis suggests that Bitcoin's networking protocols (both pre-and post-2015) offer poor anonymity properties on networks with a regular-tree topology. We confirm this claim in simulation on a 2015 snapshot of the real Bitcoin P2P network topology.

The paper explores the effect of modifying the transaction-flooding protocol of the bitcoin network as a response to deanonymisation attacks in the original pre-2015 protocol. The authors model the original protocol (trickle) and the revised variant (diffusion) and suggest that deanonymisation can equally be established with both protocols. The authors further support their claim by exploring both models in simulations.
Quality of Presentation:
The paper is well-written (minor aspects: some incomplete sentences, unresolved reference) and provides the reader with the necessary background information to follow.
The representation of the transaction-flooding protocols is comprehensive and introduced assumptions are highlighted.
Content-related questions:
- The authors refer to probability of detecting participants of 30%, which is relatively low (but indeed considerable). A fundamental assumption of performing the deanonymisation, however, is that the listening supernode would connect to all active bitcoin nodes. How realistic would it be in the future (assuming further growth of the network) that a supernode would be able to do so? -- Edit: The argument for the use of botnets made by the authors is a convincing counterpoint.
- While the overall work is involved, I find that the paper does not comprehensively answer the original claim that both trickle and diffusion perform equally good or bad at maintaining anonymity. Observing Figure 6, we can find that Trickle and Diffusion simulated show differences in probability of around 0.1 (with Diffusion performing worse than Trickle), which is not negligible. Figure 7 shows the operation on a snapshot of the real network, with an inverted observation: Diffusion performs better than Trickle. (Admittedly, they converge for larger number of eavesdropper connections.) The discrepancy and conflicting observations are worth discussing, and the statement that both protocols have `similar probabilities of detection' needs to be clarified (How do you establish 'similarity'?).


Predicting Organic Reaction Outcomes with Weisfeiler-Lehman Network
The prediction of organic reaction outcomes is a fundamental problem in computational chemistry. Since a reaction may involve hundreds of atoms, fully exploring the space of possible transformations is intractable. The current solution utilizes reaction templates to limit the space, but it suffers from coverage and efficiency issues. In this paper, we propose a template-free approach to efficiently explore the space of product molecules by first pinpointing the reaction center -the set of nodes and edges where graph edits occur. Since only a small number of atoms contribute to reaction center, we can directly enumerate candidate products. The generated candidates are scored by a Weisfeiler-Lehman Difference Network that models high-order interactions between changes occurring at nodes across the molecule. Our framework outperforms the top-performing template-based approach with a 10% margin, while running orders of magnitude faster. Finally, we demonstrate that the model accuracy rivals the performance of domain experts.

Summary:
This work provides a novel approach to predict the outcome of organic chemical reactions. A reaction can be computationally regarded as graph-prediction problem: given the input of several connected graphs (molecules), the model aims to predict a fully-connected graph (reaction product) that can be obtained by performing several graph edits (reaction) on some edges and nodes (reaction center) in the input graphs.
Past reaction predictions involving exhaustively enumeration of reaction centers and fitting them to a large number of existing reaction templates, which is very inefficient and hard to scale. In this work, the author proposed a template-free method to predict the outcome. It is a 3 step pipeline: 1) identify the reaction center given the input graphs using a Weisfeiler-Lehman Network. 2) generate candidate products based their reactivity score and chemical constraints. 3) rank the candidate products using a Weisfeiler-Lehman Difference Network.
The proposed method outperformed an existing state-of-art method on a benchmark chemistry dataset in both accuracy (10% rise) and efficiency (140 times faster), and also outperformed human chemist experts.
Qualitative Evaluation:
Quality:
The work is technically sound. The proposed method is well supported by experiments in both real world dataset and human expert comparisons.
Clarity:
This work describes their work and methods clearly. The experiments are introduced in details.
Originality:
This work provides a novel solution for reaction outcome prediction, which does not need prior knowledge of reaction templates.
The author may want to relate some past NIPS work on computational chemistry to their work:
Kayala, Matthew A., and Pierre F. Baldi. "A machine learning approach to predict chemical reactions." Advances in Neural Information Processing Systems. 2011.
Significance:
The work outperforms the state-of-art of reaction product prediction in both accuracy and efficiency. The user study experiment shows that it also outperforms human experts.


A simple model of recognition and recall memory
We show that several striking differences in memory performance between recognition and recall tasks are explained by an ecological bias endemic in classic memory experiments -that such experiments universally involve more stimuli than retrieval cues. We show that while it is sensible to think of recall as simply retrieving items when probed with a cue -typically the item list itself -it is better to think of recognition as retrieving cues when probed with items. To test this theory, by manipulating the number of items and cues in a memory experiment, we show a crossover effect in memory performance within subjects such that recognition performance is superior to recall performance when the number of items is greater than the number of cues and recall performance is better than recognition when the converse holds. We build a simple computational model around this theory, using sampling to approximate an ideal Bayesian observer encoding and retrieving situational co-occurrence frequencies of stimuli and retrieval cues. This model robustly reproduces a number of dissociations in recognition and recall previously used to argue for dual-process accounts of declarative memory.

This is a beautifully written paper on a central aspect of human memory research. The authors present a simple model along with a clever experimental test and simulations of several other benchmark phenomena.
My main concern is that this paper is not appropriate for NIPS. Most computer scientists at NIPS will not be familiar with, or care much about, the intricate details of recognition and recall paradigms. Moreover, the paper is necessarily compact and thus it's hard to understand even the basic descriptions of phenomena without appropriate background. I also think this paper falls outside the scope of the psych/neuro audience at NIPS, which traditionally has focused on issues at the interface with machine learning and AI. That being said, I don't think this entirely precludes publication at NIPS; I just think it belongs in a psychology journal. I think the authors make some compelling points addressing this in the response. If these points can be incorporated into the MS, then I would be satisfied.
Other comments:
The model doesn't include any encoding noise, which plays an important role in other Bayesian models like REM. The consequences of this departure are not clear to me, since the models are so different in many ways, but I thought it worth mentioning.
In Figure 2, I think the model predictions should be plotted the same way as the experimental results (d').
I didn't understand the point about associative recognition in section 5.3. I don't think a model of associative recognition was presented. And I don't see how reconsolidation is relevant here.
Minor:
- Eq. 1: M is a set, but in the summation it denotes the cardinality of the set. Correct notation would be m \in \mathcal{M} underneath the summation symbol. Same issue applies to Eq. 2.
- I'm confused about the difference between M' and M_k.
- What was k in the simulations?


Balancing information exposure in social networks
Social media has brought a revolution on how people are consuming news. Beyond the undoubtedly large number of advantages brought by social-media platforms, a point of criticism has been the creation of echo chambers and filter bubbles, caused by social homophily and algorithmic personalization. In this paper we address the problem of balancing the information exposure in a social network. We assume that two opposing campaigns (or viewpoints) are present in the network, and that network nodes have different preferences towards these campaigns. Our goal is to find two sets of nodes to employ in the respective campaigns, so that the overall information exposure for the two campaigns is balanced. We formally define the problem, characterize its hardness, develop approximation algorithms, and present experimental evaluation results. Our model is inspired by the literature on influence maximization, but there are significant differences from the standard model. First, balance of information exposure is modeled by a symmetric difference function, which is neither monotone nor submodular, and thus, not amenable to existing approaches. Second, while previous papers consider a setting with selfish agents and provide bounds on bestresponse strategies (i.e., move of the last player), we consider a setting with a centralized agent and provide bounds for a global objective function.

This paper proposes the problem of exposure balancing in social networks where the independent cascade is used as the diffusion model. The usage of symmetric difference is quite interesting and the approximation scheme to shovel the problem is plausible.
The experiments setup and the way the two cascades are defined and curated are interesting too.
It’s not clear if the two social network campaigns should be opposing or not? Is it a necessity for the model?
Furthermore, some hashtags are not really in opposition to each other.  For example, on obamacare issue, the tag @sentedcurze could represent many many topics related to Senator Ted Cruz including the anti-obamacare.
Any elaboration on the choice of \alpha=0 for the correlated setting.
Given the small cascades size, is the diffusion network learned reliable enough? It looks more like a heuristic algorithm for learning the diffusion network which might be of high variance and less robust given the small cascades size.
In Figure 3 why for some cases the difference between methods is increasing and for some cases it is decreasing?
Using the term “correlated” at the first though reminds the case that the two cascades are competing against each other to attract the audience. This looks to be a better definition of correlation than the one used in the paper. But in this paper, it’s not clear until the user reaches to page 3. I would suggest to clarify this earlier in the paper.
Showing standard deviations on the bar charts of figure 1 would have helped assess the significance of the difference visually. Furthermore, reporting significance levels and p-values could make the results stronger.
A few very related work are missing. Especially paper [1] has formulated the problem of exposure shaping in social networks. Furthermore, paper [2] considers two cascades who are competing against each other and optimizes one of them so they can hit the same number of people, i.e., trying to balance their exposure.  Given papers [1] and [2] I would say the current paper is not the first study of balancing g exposures in social networks. They are closely related.
Paper [3] has proposed a point-process based model for competing cascades in social networks that is related and finally paper [4]  proposes an algorithm to optimize the activities in social networks so that the network reaches to a specific goal.
[1] Farajtabar, Mehrdad, et al. "Multistage campaigning in social networks." Advances in Neural Information Processing Systems. 2016.
[2]  Farajtabar, Mehrdad, et al. "Fake News Mitigation via Point Process Based Intervention." arXiv preprint arXiv:1703.07823 (2017).
[3] Zarezade, Ali, et al. "Correlated Cascades: Compete or Cooperate." AAAI. 2017.
[4] Zarezade, Ali, et al. "Cheshire: An Online Algorithm for Activity Maximization in Social Networks." arXiv preprint arXiv:1703.02059 (2017).


Safe Adaptive Importance Sampling
Importance sampling has become an indispensable strategy to speed up optimization algorithms for large-scale applications. Improved adaptive variants-using importance values defined by the complete gradient information which changes during optimization-enjoy favorable theoretical properties, but are typically computationally infeasible. In this paper we propose an efficient approximation of gradient-based sampling, which is based on safe bounds on the gradient. The proposed sampling distribution is (i) provably the best sampling with respect to the given bounds, (ii) always better than uniform sampling and fixed importance sampling and (iii) can efficiently be computed-in many applications at negligible extra cost. The proposed sampling scheme is generic and can easily be integrated into existing algorithms. In particular, we show that coordinate-descent (CD) and stochastic gradient descent (SGD) can enjoy significant a speed-up under the novel scheme. The proven efficiency of the proposed sampling is verified by extensive numerical testing.

The authors present a "safe" adaptive importance sampling strategy for coordinate descent and stochastic gradient methods. Based on lower and upper bounds on the gradient values, an efficient approximation of gradient based sampling is proposed. The method is proven to be the best strategy with respect to the bounds, always better than uniform or fixed importance sampling and can be computed efficiently for negligible extra cost. Although adaptive importance sampling strategies have been previously proposed, the authors present a novel formulation of selecting the optimal sampling distribution as a convex optimization problem and present an efficient algorithm to solve it.
This paper is well written and a nice contribution to the study of importance sampling techniques.
Comments:
Proof of Lemma 2.1 -- seems to be missing a factor of 2 in alpha^*.
Example 3.1 - In (7) you want to maximize, so in example 3.1, it would appear that setting c to the upper or lower bound is better than using uniform sampling. Is that what this example is intending to show? It is confusing with the statement directly prior claiming the naive approach of setting c to the upper or lower bound can be suboptimal.
Line 4 of Algorithm 4 - given that m = max(l^{sort}), will this condition ever be satisfied?
Line 7 of Algorithm 4 - should be u^{sort} instead of c^{sort}?
I think the numerical results could benefit from comparisons to other adaptive sampling schemes out there (e.g., [2], [5],[21]), and against fixed importance sampling with say a non-uniform distribution.
Why are there no timing results for SGD?
Title of the paper in reference [14] is incorrect.
Add reference: Csiba and Richtarik. "Importance Sampling for Minibatches" 2016 on arXiv.
============
POST REBUTTAL
============
I have read the author rebuttal and I thank the authors for addressing my questions/comments. I think this paper is a clear accept.


Attention Is All You Need
The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include an encoder and a decoder. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 Englishto-German translation task, improving over the existing best results, including ensembles, by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.0 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature.

The paper presents a new architecture for encoder/decoder models for sequence-to-sequence modeling that is solely based on (multi-layered) attention networks combined with standard Feed-Forward networks as opposed to the common scheme of using recurrent or convolutional neural networks. The paper presents two main advantages of this new architecture: (1) Reduced training time due to reduced complexity of the architecture, and (2) new State-of-the-Art result on standard WMT data sets, outperforming previous work by about 1 BLEU point.
Strengths:
- The paper argues well that (1) can be achieved by avoiding recurrent or convolutional layers and the complexity analysis in Table 1 strengthens the argument.
- (2) is shown by comparing the model performance against strong baselines on two language pairs, English-German and English-French.
The main strengths of the paper are that it proposes an entirely novel architecture without recurrence or convolutions, and advances state of the art.
Weaknesses:
- While the general architecture of the model is described well and is illustrated by figures, architectural details lack mathematical definition, for example multi-head attention. Why is there a split arrow in Figure 2 right, bottom right? I assume these are the inputs for the attention layer, namely query, keys, and values. Are the same vectors used for keys and values here or different sections of them? A formal definition of this would greatly help readers understand this.
- The proposed model contains lots of hyperparameters, and the most important ones are evaluated in ablation studies in the experimental section. It would have been nice to see significance tests for the various configurations in Table 3.
- The complexity argument claims that self-attention models have a maximum path length of 1 which should help maintaining information flow between distant symbols (i.e. long-range dependencies). It would be good to see this empirically validated by evaluating performance on long sentences specifically.
Minor comments:
- Are you using dropout on the source/target embeddings?
- Line 146: There seems to be dangling "2"


Large-Scale Quadratically Constrained Quadratic Program via Low-Discrepancy Sequences
We consider the problem of solving a large-scale Quadratically Constrained Quadratic Program. Such problems occur naturally in many scientific and web applications. Although there are efficient methods which tackle this problem, they are mostly not scalable. In this paper, we develop a method that transforms the quadratic constraint into a linear form by sampling a set of low-discrepancy points [16]. The transformed problem can then be solved by applying any state-of-the-art large-scale quadratic programming solvers. We show the convergence of our approximate solution to the true solution as well as some finite sample error bounds. Experimental results are also shown to prove scalability as well as improved quality of approximation in practice.

Contributions:
The paper proposes a sampling based method to approximate QCQPs. The method starts from a sampling of the hypercube, the mapping it to the sphere, and finally to the ellipsoid of interest. Unfortunately the authors don't provide enough evidence why this particular sampling is more efficient than simpler ones. How is the complexity dependence on the dimension? (Net seems to be exponential in dimension). The theorems seem to hide most dimensionality constants. My main question: How does the approach compare experimentally and theoretically to naive sampling?
While the main advantage of the contribution is the scalability, the authors do unfortunately not in detail compare (theoretically and practically) the resulting method and its scalability to existing methods for QCQP.
The paper is considering convex QCQP only, and does not comment if this problem class is NP hard or not. For the list of applications following this statement, it should be mentioned if those are convex or not, that is if they fall into the scope of the paper or not. This is not the only part of this paper where it occurred to me that the writing lacked a certain amount of rigor and referenced wikipedia style content instead of rigorous definitions.
Experiments: No comparison to state-of-the-art second order cone solvers or commercial solvers (e.g. CPLEX and others) is included. Furthermore, as stated now, unfortunately the claimed comparison is not reproducible, as details on the SDP and RLT methods are not included. For example no stopping criteria and implementation details are included. No supplementary material is provided.
How was x^* computed in the experimental results? How about adding one real dataset as well from the applications mentioned at the beginning?
Riesz energy: "normalized surface measure" either formally define it in the paper, or precisely define it in the appendix, or at least provide a scientific reference for it (not wikipedia). The sentence "We omit details due to lack of space" probably takes as much space as to define it? (Current definition on the sphere is not related to the actual boundary of S).
Style: At several points in the paper you mention "For sake of compactness, we do not go deeper into this" and similar. This only makes sense if you at least more precisely state then why you discuss it, or at least make precise what elements you require for your results. In general, if such elements do add to the value of the paper, then they would be provided in an appendix, which the authors here did not.
line 105: missing 'the'
line 115: in base \eta: Definition makes no sense yet at this point because you didn't specify the roles of m and eta yet (only in the next sentence, for a special case).
line 149: space after 'approaches'. remove 'for sake of compactness'
== update after rebuttal ==
I thank the authors for partially clarifying the comparison to other sampling schemes, and on x^*.
However, I'm still disappointed by the answer "We are presently working on comparing with large scale commercial solvers such as CPLEX and ADMM based techniques for SDP in several different problems as a future applications paper.". This is not to be left for the future, this comparison to very standard competing methods would be essential to see the merits of the proposed algorithm now, and significantly weakens the current contribution.