Mapping distinct timescales of functional interactions among brain networks
Brain processes occur at various timescales, ranging from milliseconds (neurons) to minutes and hours (behavior). Characterizing functional coupling among brain regions at these diverse timescales is key to understanding how the brain produces behavior. Here, we apply instantaneous and lag-based measures of conditional linear dependence, based on Granger-Geweke causality (GC), to infer network connections at distinct timescales from functional magnetic resonance imaging (fMRI) data. Due to the slow sampling rate of fMRI, it is widely held that GC produces spurious and unreliable estimates of functional connectivity when applied to fMRI data. We challenge this claim with simulations and a novel machine learning approach. First, we show, with simulated fMRI data, that instantaneous and lag-based GC identify distinct timescales and complementary patterns of functional connectivity. Next, we analyze fMRI scans from 500 subjects and show that a linear classifier trained on either instantaneous or lag-based GC connectivity reliably distinguishes task versus rest brain states, with ∼80-85% cross-validation accuracy. Importantly, instantaneous and lag-based GC exploit markedly different spatial and temporal patterns of connectivity to achieve robust classification. Our approach enables identifying functionally connected networks that operate at distinct timescales in the brain.

The authors apply instantaneous and lag-based Granger-Geweke causality (iGC and dGC) to infer brain network connections at distinct timescales from functional MRI data. With simulated fMRI data and through analysis of real fMRI recordings, they presented the following novel insights. First, iGC and dGC provide robust measures of brain functional connectivities, resolving over long-term controversy in the field. Second, iGC and dGC can identify complementary functional connections, in particular, the latter can find excretory-inhibitory connections. Third, downsampling the time series to different extents provides an efficient method for recovering connections at different timescales.
Clarity:
The paper is well organised and clearly written with nice figures. The result of real data analysis may be difficult to understand without background knowledge in neuroscience.
Significance:
I think that the insights written in this paper have significant impact on researchers working on this topic. Recently analyses of resting-state fMRI data have led to many interesting results in neuroscience. However, most publications employed correlation matrices for measuring functional connectivities. So, if iGC and dGC would become alternative tools, we could extract/grasp multi timescale processing in the brain. In order to convince more researchers of practical usefulness of iGC and dGC, it would be necessary to compare with existing techniques further and to work on more real fMRI examples.
On the other hand, the scope of this paper is rather narrow and impacts on other research topics in the NIPS community are rather limited.
Correctness:
Since the authors applied existing connectivity measures, I don't expect any technical errors.
Originality:
I feel that a  certain amount of researchers are sceptical about applications of GC measures to fMRI data. I have not seen the messages of this paper which may resolve over long-term controversy in the field.
To sum up, the novel insights derived from analysis of simulated and real fMRI data with GC measures are interesting and could have potential to advance the research field. Since further real-world examples would be preferable to convince other researchers in the field, I would say that this paper is weak accept.
There are a few minor comments.
- p.3 Eq.(2): I know a MVAR model with variable time lag for each connection which is determined by DTI fibre length. The simulation model in Section 2 assumes variable timescale at each node. I wonder whether we can set timescale of each connections separately at a predetermined value in this formulation or we have to further extend the model in Section 2 to construct more realistic model incorporating knowledge from structural MRI data.
- Section 3: I guess that the language task could contain slow brain processing and might be a good example to show multi timescale network structures by fMRI downsampling. I wonder whether there exist other tasks in HCP which we can see similar results.


NeuralFDR: Learning Discovery Thresholds from Hypothesis Features
As datasets grow richer, an important challenge is to leverage the full features in the data to maximize the number of useful discoveries while controlling for false positives. We address this problem in the context of multiple hypotheses testing, where for each hypothesis, we observe a p-value along with a set of features specific to that hypothesis. For example, in genetic association studies, each hypothesis tests the correlation between a variant and the trait. We have a rich set of features for each variant (e.g. its location, conservation, epigenetics etc.) which could inform how likely the variant is to have a true association. However popular empirically-validated testing approaches, such as Benjamini-Hochberg's procedure (BH) and independent hypothesis weighting (IHW), either ignore these features or assume that the features are categorical or uni-variate. We propose a new algorithm, NeuralFDR, which automatically learns a discovery threshold as a function of all the hypothesis features. We parametrize the discovery threshold as a neural network, which enables flexible handling of multi-dimensional discrete and continuous features as well as efficient end-to-end optimization. We prove that NeuralFDR has strong false discovery rate (FDR) guarantees, and show that it makes substantially more discoveries in synthetic and real datasets. Moreover, we demonstrate that the learned discovery threshold is directly interpretable.

This paper proposes a procedure to learn a parametric function mapping a set of features describing a hypothesis to a (per-hypothesis) significance threshold. In principle, if this function is learnt appropriately, the approach would allow prioritising certain hypotheses based on their feature representation, leading to a gain in statistical power. The function generating the per-hypothesis rejection regions is trained to control the False Discovery Proportion (FDP) with high probability, provided some restrictive assumptions are satisfied.
While multiple hypothesis testing is perhaps not a mainstream machine learning topic nowadays, key application domains such as computational biology or healthcare still heavily rely on multiple comparisons, making this a topic worth studying. A clear trend in this domain is the design of methods able to incorporate prior knowledge to upweight or downweight some hypotheses; the method described in this paper is one more example of this line of work.
Despite the fact that the topic is relevant, I have some serious concerns about the practical applicability of the proposed approach, as well as other aspects of the paper.
***** Major Points *****
(1) In my opinion, the paper heavily relies on assumptions that are simply false for all motivating applications described in the paper and tackled in the experiments.
Firstly, the method assumes that triplets (P_{i}, \mathbf{X}_{i}, H_{i}) are i.i.d. If this assumption were not satisfied, the entire method would break down, as cross-validation would no longer be appropriate to validate the decision rules learnt on a training set.
Unfortunately, the independence assumption cannot be accepted in bioinformatics. For example, in genome-wide association studies, variants tend to be statistically dependent due to phenomena such as linkage disequilibrium (LD). In RNA-Seq analysis, gene expression levels are correlated according to complex co-expression networks, which are possibly tissue-specific. More generally, unobserved confounding variables such as population structure, batch effects or cryptic relatedness might introduce additional non-trivial statistical dependencies between the observed predictors. There exist straightforward extensions of the Benjamini-Hochberg (BH) procedure to guarantee FDR control under general dependence. Unless the authors extend their approach to allow for arbitrary dependencies between hypotheses, I do not believe the method is valid for the applications included in the experiments. Moreover, the assumption that all triplets (P_{i}, \mathbf{X}_{i}, H_{i}) are identically distributed is also hard to justify in practice.
Another questionable assumption, though perhaps of lesser practical importance when compared to the two previous issues, is the fact that the alternative distribution $f_{1}(p \mid \mathbf{x})$ must be non-increasing. The authors claim this is a standard assumption, which is not necessarily true (e.g. see [1] for a recent counter-example, where the alternative distribution is modelled as a beta distribution with learnable mean and precision).
(2) The paper seems to make a heavy emphasis on the use of neural networks, to the point this fact is present in the title and name of the method. However, neural networks play a small role in the entire approach, as essentially all that is needed is a parametric function to map the input feature space describing each hypothesis to a significance threshold in the unit interval. Most importantly, all motivating examples are extremely low-dimensional (1D to 5D) making it likely that virtually any parametric function with enough capacity will suffice.
I believe the authors should include exhaustive experiments to study how different function classes perform in this task. By doing so, the authors might prove to which extent neural networks are an integral part of their approach. Moreover, it might occur that simpler, easier to interpret functions perform equally well, in which case it might be beneficial to use those instead.
(3) While the paper contains virtually no grammar mistakes, being easy to read in that regard, the quality of the exposition could be significantly improved.
In particular, Section 3 lacks detail in the description of Algorithm 1. Throughout the paper, the notion of cross-validation is rather confusing, as in this particular case distinct folds correspond to subsets of features (i.e. hypotheses) rather than samples used to compute the corresponding P-values. I believe this should be clarified early-on. Also, the variables $\gamma_{i}$ appear out of nowhere and are not appropriately discussed in the text. In my opinion, the authors should additionally adopt the notation used in Equation (8) in Supplementary Section 2 for Equation (3), in order to make it clearer that what they are truly optimising is only a smooth approximation to the true objective function.
The description of the implementation and training procedure in Supplementary Section 2 is also insufficient, as it is lacking a proper justification of the hyperparameter choices (e.g. network architecture, $\lambda$ values, optimisation scheme) and experimental results describing the robustness of their approach to such hyperparameters. In practice, setting these hyperparameters will be a considerable hurdle for many practitioners in computational biology and statistical genetics, which might not necessarily be familiar with standard techniques in deep learning. Therefore, I believe these issues should be described in greater detail for this work to be of use for such communities.
***** Minor Points *****
The use of cross-validation makes the results random by construction, which is an undesirable property in statistical association testing. The authors should carry out experiments to demonstrate the stability of their results across different random partitions and to which extent a potential underlying stratification of features might be problematic for the cross-validation process.
Decision rules in multiple hypothesis testing ultimately strongly depend on the number n of hypothesis being tested. However, the proposed approach does not explicitly depend on n but, rather, relies on the cross-validation set and the test set containing exactly the same number of hypotheses and all hypotheses being i.i.d. It would be interesting if the authors were able to relax that requirement by accounting for the number of tests explicitly rather than implicitly.
The paper contains several wrong or imprecise claims that however do not strongly affect the overall contribution:
 - “The P-value is the probability of a hypothesis being observed under the null” -> This is not a correct definition of P-value. A P-value is the probability of observing a value of the test statistic that indicates an association at least as extreme under the assumption that the null hypothesis holds.
 - “The two most popular quantities that conceptualize the false positives are the false discovery proportion (FDP) and the false discovery rate (FDR)” -> Family-Wise Error Rate continues to be popular and has a considerably longer history than any of those quantities. Moreover, while it is true that FDR is extremely popular and de-facto replacing FWER in many domains, FDP is relatively less frequently used in comparison to FWER.
- Definition 1 is mathematically imprecise, as it is not handling the case $D(t)=0$ properly.
- “Moreover, $\mathbf{x}$ may be multidimensional, ruling out the possibility of non-parametric modelings, such as spline-based methods or the kernel based methods, whose number of parameters grows exponentially with the dimensionality” -> Kernel-based methods can be applied in this setting via inducing variables (e.g. [2]). Additionally, like mentioned earlier, many other approximators other than neural networks require less an exponentially-growing number of parameters on the dimension of the feature space.
- Lemma 1 does not show that the “mirroring estimator” bounds the true FD from above, as Line 155 seems to claim. It merely indicates that its expectation bounds the expected FD, which is a different statement than what is discussed between Lines 143-151.
- As mentioned earlier in this review, the statement in Lines 193-194 is not completely correct. Five dimensions is definitely not a high-dimensional setting by any means. At least thousands of features should have been used to justify that claim.
The paper would benefit from proof-reading, as it contains frequent typos. Some examples are:
- Line 150: $(t(\mathbf{x}), 1)$ -> $(1-t(\mathbf{x}), 1)$
- Line 157: “approaches $0$ very fast as $p \rightarrow 0$” -> “approaches $0$ very fast as $p \rightarrow 1$”
- Line 202: “Fig. 3 (b,c)” -> “Fig. 4 (b,c)”
- Lines 253-254: The alternative distribution $f_{1}$ is missing parenthesis
- Supplementary Material, proof of Theorem 1 and Lemma 2: All terms resulting from an application of Chernoff’s bound are missing parenthesis as well. While it might have been intentional, in my opinion it makes the proof a bit harder to read.
References:
[1] Li, Y., & Kellis, M. (2016). Joint Bayesian inference of risk variants and tissue-specific epigenomic enrichments across multiple complex human diseases. Nucleic acids research, 44(18), e144-e144.
[2] Dustin Tran, Rajesh Ranganath, David M. Blei - The variational Gaussian process. A powerful variational model that can universally approximate any posterior. International Conference on Learning Representations, 2016


Learning spatiotemporal piecewise-geodesic trajectories from longitudinal manifold-valued data
We introduce a hierarchical model which allows to estimate a group-average piecewise-geodesic trajectory in the Riemannian space of measurements and individual variability. This model falls into the well defined mixed-effect models. The subject-specific trajectories are defined through spatial and temporal transformations of the group-average piecewise-geodesic path, component by component. Thus we can apply our model to a wide variety of situations. Due to the non-linearity of the model, we use the Stochastic Approximation ExpectationMaximization algorithm to estimate the model parameters. Experiments on synthetic data validate this choice. The model is then applied to the metastatic renal cancer chemotherapy monitoring: we run estimations on RECIST scores of treated patients and estimate the time they escape from the treatment. Experiments highlight the role of the different parameters on the response to treatment.

The paper proposes a mixed effects dynamical model which describes individual trajectories as being generated from transformations of an average behaviour. The work is motivated by modelling of cancer treatment, but the proposed method is more generally applicable.
The paper is well-written and -structured, and it considers an interesting application. I have limited knowledge of Riemannian geometry, but the methodology was not hard to follow and appears sound (to the extent that I can judge); that said, I cannot comment on the novelty of the approach. I generally found the presentation to be clear (despite some inconsistencies in the notation). The evaluation is fairly comprehensive and explained well in the text, and the model seems to perform well, especially with regards to capturing a variety of behaviours. It is interesting that the error does not monotonically increase as the sample size increases, but I suppose its variation is not that great anyway.
Overall, I believe this is a strong paper that is worthy of acceptance. Most of my comments have to do with minor language or notation issues and are given below. I only have two "major" observations. The first concerns Section 2.2, where the notation changes from the previous section. Specifically, the time warping transformation (previously \psi) is now denoted with \phi, while the space warping (previously \phi) is now strangely \phi(1) (line 154). This is pretty confusing, although presumably an oversight. Secondly, in Figure 1 and its description in Section 4.1, there is no explanation of what the vertical axis represents - presumably the number of cancerous cells, or something similar?
Other than that, I think it would be useful to slightly expand the explanation in a couple of places. The first of these is the introduction of the transformations (Section 2.1), where I personally would have welcomed a more intuitive explanation of the time and space warping (although Figure 1b helps with that some). The second place is Section 2.2: here, various decisions are said to be appropriate for the target application (eg lines 140, 155, 166), but without explaining why that is; some details of how these quantities and decisions correspond to the application might be useful. Some of the terminology could also be better explained, especially relating to the treatment, e.g. "escaping", "rupture". Finally, to better place the work in context, the paper could perhaps include a more concrete description of what the proposed model allows (for example, in this particular application) that could not be done with previous approaches (Section 4.3 mentions this very briefly).
Detailed comments:
-----------------
l21: Actually -> Currently?
l22: I don't think "topical" is the appropriate word here; perhaps "contested"?
l23: [Mathematics] have -> has
l24: [their] efficiency -> its
l29: the profile are fitting -> the profiles are fitted?
l36: [amounts] to estimate -> to estimating
l45: in the aim to grant -> with the aim of granting?
l50: temporary [efficient] -> temporarily
l54: little [assumptions] -> few
l54: to deal with -> (to be) dealt with
l55: his -> its
l56: i.e -> i.e.
Equation between lines 81-82: t_R^m in the subscript in the final term should be t_R^{m-1} (similarly in Equation (*))
l91: I count 2m-2 constraints, not 2m+1, but may be mistaken
l93: constrain -> constraint (also repeated a couple of times further down)
l106: "we decree the less constrains possible": I am not sure of the meaning here; perhaps "we impose/require the fewest constraints possible"?
l118: in the fraction, I think t_{R,i}^{l-1} should be t_R^{l-1} (based on the constraints above)
l131: used to chemotherapy monitoring -> either "used for/in chemotherapy monitoring" or "used to [do something] in chemotherapy monitoring"
l134: n = 2 -> m = 2 (if I understand right)
l139: especially -> effectively / essentially (also further down)
l154: \phi_i^l (1), as mentioned earlier in the comments
l156: *** -> fin?
l160: I think (but may be wrong) that the composition should be the other way around, i.e. \gamma_i^1 = \gamma_0^1 \circ \phi_i^1, if \phi is the time warping here
l167: [interested] to -> in
l177: independents -> independent
l183-185: is it not the case that, given z and \theta, the measurements are (conditionally) independent?
l241: the table 1 -> Table 1
Figure 2: there is a thick dark green line with no observations - is that intentional?
Table 1: I found the caption a little hard to understand, but I am not sure what to suggest as an improvement
l245: the biggest [...], the best -> the bigger [...], the better
l257: Symetric -> Symmetric
l260: is 14, 50 intended to be 14.50?
l260: Figure 3a, -> no comma
l264: early on [the progression] -> early in / early on in
l264: to consider with quite reserve -> to be considered with some reserve
Figure 3: (last line of caption) large scale -> large range?
l280: "we have explicit it": I'm not sure what you mean here
l284: Last -> Lastly


Efficient Modeling of Latent Information in Supervised Learning using Gaussian Processes
Often in machine learning, data are collected as a combination of multiple conditions, e.g., the voice recordings of multiple persons, each labeled with an ID. How could we build a model that captures the latent information related to these conditions and generalize to a new one with few data? We present a new model called Latent Variable Multiple Output Gaussian Processes (LVMOGP) that allows to jointly model multiple conditions for regression and generalize to a new condition with a few data points at test time. LVMOGP infers the posteriors of Gaussian processes together with a latent space representing the information about different conditions. We derive an efficient variational inference method for LVMOGP for which the computational complexity is as low as sparse Gaussian processes. We show that LVMOGP significantly outperforms related Gaussian process methods on various tasks with both synthetic and real data.

# Update after author feedback and reviewer discussion
I thank the authors for feedback. I maintain my assessment and do not recommend publication at this stage.
The core contribution is representing conditions with latent variables, and deriving a VI algorithm to cope with intractibility. This is interesting, but the discussion around it could be much improved.
Some possible improvements are addressed in the author feedback, eg I'm not sure how Fig 1 could have been understood without the complementary explanation brought up in the feedback.
Beyond what has been addressed in the author feedback, some work is needed to make this paper appealing (which the idea under study, the method and the results seem to call for):
- clarifying the mathematical formulation, eg what forms of $k_H$ are we examining, provide a full probabilistic model summary of the model, point out design choices
- pointing out differences or similarities with existing work
- remove gratuitous reference to deep learning in intro (it detracts)
- make sure that all important questions a reader might have are addressed
# Overall assessment
The issue addressed (modelling univariate outputs which were generated under different, known conditions) and the modelling choice (representing conditions as latent variables) are interesting.
I see no reason why to even mention and work with the (restrictive) parameterization of section 2.2, and not immediately the parameterization allowing "missing data" section 4. The latter is more interesting and equally simple.
Overall, the paper is reasonably clear, but seems lacking in originality and novelty.
## Novelty
Wrt work on multi-output / multi-task GP, I can see that:
- this model has latent variables, not task IDs, and so can generalize over tasks (called conditions in the paper)
- the computational treatment using inducing points helps
The two seem very close though. The assumption of Kronecker structure is a staple. In particular, experiments should try compare to these methods (eg considering every condition a distinct task).
The application of Titsias 2009 seems straightforward, I cannot see the novelty there, nor why so much space is used to present it.
## Presentation and clarity
1. Paper requires English proofreading. I usually report typos and mistakes in my reviews, but here there are too many grammar errors.
2. Motivating example with braking distance is helpful.
2. I don't understand fig 1 c,d at all: why do we not see the data points of fig 1a? where do these new points come from? what is depicted in the top vs the bottom plot?
2. References: the citation of Bussas 2017 is wrong. The complete correct citation is more like: Matthias Bussas, Christoph Sawade, Tobias Scheffer, and Niels Landwehr. Varying-coefficient models for geospatial transfer learning.. Machine Learning, doi:10.1007/s10994-017-5639-3, 2017.


Learning Chordal Markov Networks via Branch and Bound
We present a new algorithmic approach for the task of finding a chordal Markov network structure that maximizes a given scoring function. The algorithm is based on branch and bound and integrates dynamic programming for both domain pruning and for obtaining strong bounds for search-space pruning. Empirically, we show that the approach dominates in terms of running times a recent integer programming approach (and thereby also a recent constraint optimization approach) for the problem. Furthermore, our algorithm scales at times further with respect to the number of variables than a state-of-the-art dynamic programming algorithm for the problem, with the potential of reaching 20 variables and at the same time circumventing the tight exponential lower bounds on memory consumption of the pure dynamic programming approach.

The authors present a branch and bound algorithm for learning Chordal Markov networks. The prior state of the art algorithm is a dynamic programming approach based on a recursive characterization of clique tress and storing in memory the scores of already-solved subproblems. The proposed algorithm uses a branch and bound algorithm to search for an optimal chordal Markov network. The algorithm first uses a dynamic programming algorithm to enumerate Bayesian network structures, which are later used as pruning bounds. A symmetry breaking technique is introduced to prune the search space.
The special property of Chordal Markov networks necessitates that no score pruning can be done. Given that this step is exponential in time and space. It is unlikely that learning CMSL can scale to even medium-size datasets (>20-30variables).
What is the definition of decomposable DAG? This is important for understanding other concepts defined later on, such as ordered decomposable DAG.
The symmetry breaking rule is useful. What is its relation to the symmetry breaking rules introduced in (van Beek&Hoffmann 15) for learning BNs? The authors referenced that paper and should have discussed the relevance.
The paper did not give a clear picture on how much time is spent on computing scores, and how much time is spent on optimization. Also, the total runtime of the proposed algorithm includes computing optimal BNs and segment trees. How much do they take relative to the total runtime?  The aggregated time does not provide a complete picture.


Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
Recent work has shown that state-of-the-art classifiers are quite brittle, in the sense that a small adversarial change of an originally with high confidence correctly classified input leads to a wrong classification again with high confidence. This raises concerns that such classifiers are vulnerable to attacks and calls into question their usage in safety-critical systems. We show in this paper for the first time formal guarantees on the robustness of a classifier by giving instance-specific lower bounds on the norm of the input manipulation required to change the classifier decision. Based on this analysis we propose the Cross-Lipschitz regularization functional. We show that using this form of regularization in kernel methods resp. neural networks improves the robustness of the classifier with no or small loss in prediction performance.

This paper fills an important gap in the literature of robustness of classifiers to adversarial examples by proposing the first (to the best of my knowledge) formal guarantee (at an example level) on the robustness of a given classifier to adversarial examples. Unsurprisingly, the bound involves the Lipschitz constant of the Jacobians which the authors exploit to propose a cross-Lipschitz regularization. Overall the paper is well written, and the material is well presented.  The proof of Theorem 2.1 is correct. I did not check the proofs of the propositions 2.1 and 4.1.
This is an interesting work. The theorem 2.1 is a significant contribution.
The experimental section is not very insightful as very simple models are used on small/toy-ish datasets. Some claims in the paper are not consistent:
    - "the current main techniques to generate adversarial samples [3, 6] do not integrate box constraints"
There exist approaches integrating box constraints: see [1,2]
    - "It is interesting to note that classical regularization functionals which penalize derivatives of the resulting classifier function are not typically used in deep learning."
Penalizing the derivatives is a well-known strategy to regularize neural networks[3,4]. In fact, it has recently been exploited to improve the robustness of neural networks to adversarial examples[3].
I have the following question for the authors:
Theorem 2.1 says that the classifier decision does not change inside a ball of small radius for a given p-norm. Yet, we can still find adversarial examples even when the norm of the gradient is small (yes, we can. I have tried!). Does this mean the topology induced by the p-norm does not do justice to the perceptual similarity we have of the image and its adversarial version? In other words; Does close in terms of p-norm reflect "visually close"?

Overall I like the paper. Though it leaves me with mixed feelings due to weak experimental section, I lean on the accept side.
[1]  Intriguing properties of neural networks
[2]  DeepFool: a simple and accurate method to fool deep neural networks
[3]  Parseval Networks: Improving Robustness to Adversarial Examples
[4]  Double back propagation: increasing generalization performance


Estimating Mutual Information for Discrete-Continuous Mixtures
Estimation of mutual information from observed samples is a basic primitive in machine learning, useful in several learning tasks including correlation mining, information bottleneck, Chow-Liu tree, and conditional independence testing in (causal) graphical models. While mutual information is a quantity well-defined for general probability spaces, estimators have been developed only in the special case of discrete or continuous pairs of random variables. Most of these estimators operate using the 3H-principle, i.e., by calculating the three (differential) entropies of X, Y and the pair (X, Y ). However, in general mixture spaces, such individual entropies are not well defined, even though mutual information is. In this paper, we develop a novel estimator for estimating mutual information in discrete-continuous mixtures. We prove the consistency of this estimator theoretically as well as demonstrate its excellent empirical performance. This problem is relevant in a wide-array of applications, where some variables are discrete, some continuous, and others are a mixture between continuous and discrete components.
1 Introduction mutual information estimators, on both the theoretical as well as practical fronts [46,31,44,45,22,19,7,15,14,17,16].
The previous estimators focus on either of two cases -the data is either purely discrete or purely continuous. In these special cases, the mutual information can be calculated based on the three (differential) entropies of X, Y and (X, Y ). We term estimators based on this principle as 3H-estimators (since they estimate three entropy terms), and a majority of previous estimators fall under this category [19,16,46].
In practical downstream applications, we often have to deal with a mixture of continuous and discrete random variables. Random variables can be mixed in several ways. First, one random variable can be discrete whereas the other is continuous. For example, we want to measure the strength of relationship between children's age and height, here age X is discrete and height Y is continuous. Secondly, a single scalar random variable itself can be a mixture of discrete and continuous components. For example, consider X taking a zero-inflated-Gaussian distribution, which takes value 0 with probability 0.1 and is a Poisson distribution with mean 10 with probability 0.9. This distribution has both a discrete component as well as a component with density, and is a well-known model for gene expression readouts [24,37]. Finally, X and / or Y can be high dimensional vector, each of whose components may be discrete, continuous or mixed.
In all of the aforementioned mixed cases, mutual information is well-defined through the RadonNikodym derivative (see Section 2) but cannot be expressed as a function of the entropies or differential entropies of the random variables. Crucially, entropy is not well defined when a single scalar random variable comprises of both discrete and continuous components, in which case, 3H estimators (the vast majority of prior art) cannot be directly employed. In this paper, we address this challenge by proposing an estimator that can handle all these cases of mixture distributions. The estimator directly estimates the Radon-Nikodym derivative using the k-nearest neighbor distances from the samples; we prove 2 consistency of the estimator and demonstrate its excellent practical performance through a variety of experiments on both synthetic and real dataset. Most relevantly, it strongly outperforms natural baselines of discretizing the mixed random variables (by quantization) or making it continuous by adding a small Gaussian noise.
The rest of the paper is organized as follows. In Section 2, we review the general definition of mutual information for Radon-Nikodym derivative and show that it is well-defined for all the cases of mixtures. In Section 3, we propose our estimator of mutual information for mixed random variables. In Section 4, we prove that our estimator is 2 consistent under certain technical assumptions and verify that the assumptions are satisfied for most practical cases. Section 5 contains the results of our detailed synthetic and real-world experiments testing the efficacy of the proposed estimator.

Summary: This paper proposes an estimator for the mutual information of a pair (X,Y) of random variables given N joint samples. While most previous work has focused separately on the purely discrete and purely continuous methods (with significantly different methods used in each setting), the novel focus here on the case of combinations or mixtures or discrete and random variables; in this case, the mutual information is well-defined but cannot always be expressed in terms of entropy, making most previous estimators unusable. The proposed estimator generalizes the classic KSG estimator by defining the behavior at samples points with k-NN distance 0. The L_2 consistency of the estimator is proven, under a sort of continuity condition on the density ratio that allows the existence of atoms, and suitable scaling of k with N. Finally, experimental results on synthetic and real data compare the proposed estimator to some baseline estimators based on perturbing the data to make it purely discrete or continuous.
Main Comments:
This paper addresses a significant hole that remains in the nonparametric dependence estimation literature despite significant recent progress in this area; many current estimators are difficult or impossible to use with many data sets due to a combination or mixture of discrete and continuous components. While the proposed estimator seems like a fairly simple correction of the KSG estimator, and the consistency result is thus likely straightforward given the results of [16], the empirical section is strong and I think the work is of sufficient practical importance to merit acceptance. The paper is fairly well-written and easy to read.
Minor Comments/questions:
1) As far as I understand 3H estimators can work as long as each individual dimension of the variables is fully discrete or full continuous, but fail if any variable is a mixture of the two (i.e., if an otherwise continuous distribution has any atoms). It might be good to clarify this, and also to compare to some 3H estimators (e.g., in experiment II).
2) Lines 39-42: Perhaps there is a typo or I am misreading something, but it isn't clear to my why the given definition of X isn't entirely discrete (it appears to be supported on {0,1,2,...}).
3) Lines 129-130: could be probably clearer "the average of Radon-Nikodym derivative" should be "the average of the logarithm Radon-Nikodym derivative of the joint distribution P_{XY} with respect to the product distribution P_XP_Y."
4) As discussed in Appendix A, a key observation of the paper is that, to be well-defined, mutual information requires only the trivial condition of absolute continuity of the joint distribution w.r.t. the marginal distribution (rather than w.r.t. another base measure, as for entropy). I think it would be worth explicitly mentioning this in the main paper, rather than just in the Appendix.
5) Although this paper doesn't study convergence rates, could the authors conjecture about how to quantify parameters that determine the convergence rate for mixed discrete/continuous variables? For continuous distributions, the smoothness of the density is usually relevant, while, for discrete distributions, the size of the support is usually relevant. However, mixture distributions are discontinuous and have infinite support.
6) Theorem 1 assumes that k -> infinity as N -> infinity. Can the authors conjecture whether the proposed estimator is consistent for fixed values of k? I ask because of recent work [4,16,43] showing that the KL and KSG estimators are consistent even for fixed k.
7) For some reason, the PDF is encoded such that I cannot search, highlight, or copy text. Perhaps it was converted from a postscript? Anyhow, this made the paper significantly more difficult to review.
#### AFTER READING AUTHOR REBUTTAL ####
"H(X,Y) is not well-defined as either discrete entropy or continuous entropy" - I think


ALICE: Towards Understanding Adversarial Learning for Joint Distribution Matching
We investigate the non-identifiability issues associated with bidirectional adversarial training for joint distribution matching. Within a framework of conditional entropy, we propose both adversarial and non-adversarial approaches to learn desirable matched joint distributions for unsupervised and supervised tasks. We unify a broad family of adversarial models as joint distribution matching problems. Our approach stabilizes learning of unsupervised bidirectional adversarial learning methods. Further, we introduce an extension for semi-supervised learning tasks. Theoretical results are validated in synthetic data and real-world applications.

Adversarially Learned Inference (a.k.a. Adversarial Feature Learning)
is an interesting extension to GANs, which can be used to train a
generative model by learning generator G(z) and inference E(x)
functions, where G(z) maps samples from a latent space to data and
E(x) is an inference model mapping observed data to the latent
space. This model is trained adversarially by jointly training E(x)
and G(z) with a discriminator D(x,z) which is trained to distinguish
between real (E(x), x) samples and fake (z, G(z)) samples. This is an
interesting approach and has been shown to generate latent
representations which are useful for semi-supervised learning.
The authors highlight an issue with the ALI model, by constructing a
small example for which there exist optimal solutions to the ALI loss
function which have poor reconstruction, i.e. G(E(x)) can be very
different to x. This identifiability issue is possible because the ALI
objective imposes no restrictions on the conditional distributions of
the generator and inference models. In particular, this allows for
solutions which violate a condition known as cycle consistency, which
is considered a desirable property in unsupervised learning. They
propose adding a cross-entropy term to the loss, which enforces a
constraint on one of the two conditionals.
The ALICE training objective is shown to yield better reconstructions
than ALI on toy data. They also evaluate on the inception score, a
metric proposed in Salimans et al. (2017), which also shows better
performance for the model trained with the ALICE objective.
The description of how the model can be applied to semi-supervised
learning seemed a little strange to me. The authors consider a case
where the latent space z corresponds to the labels and semi-supervised
learning involves training a model with the ALICE objective, when some
paired (z,x) samples are available. They claim that these labelled
samples can be leveraged to address the identifiability issues in
ALI. While this setting holds for the experiments they present in
section 5.2, a more common application of this kind of model to
semi-supervised learning is to use the inference network to generate
features which are then used for classification. The results in the
ALI paper follow this approach and evaluate ALI for semi-supervised
learning on some standard benchmarks (CIFAR10 and SVHN). This is
possibly the strongest evidence in the ALI paper that the model is
learning a useful mapping from data space to latent space, so it would
be useful to see the same evaluation for ALICE.
In the ALI and BIGAN papers, it is shown that cycle consistency is
satisfied at the optimum when the E and G mappings are
deterministic. Similarly, in this paper the authors show that the
optimal solution to a model trained with cross-entropy regularisation
has deterministic mappings. In lines 228-233, the authors suggest that
a model trained with deterministic mappings is likely to underfit and
taking the alternative approach of training a model with stochastic
mappings and a cross-entropy regularisation gives faster and more
stable training. While this is an interesting claim, it was not
immediately clear to me why this would be the case from the
description provided in the paper.
This paper is on an interesting topic, highlighting a significant
issue with the ALI training objective and proposing a solution using
cross-entropy regularisation. It would be more convincing if the
authors had evaluated their model on the same set of experiments as
the original ALI paper, especially the semi-supervised learning
benchmarks, and provided more justification for the use of
cross-entropy regularisation over deterministic mappings


Group Additive Structure Identification for Kernel Nonparametric Regression
The additive model is one of the most popularly used models for high dimensional nonparametric regression analysis. However, its main drawback is that it neglects possible interactions between predictor variables. In this paper, we reexamine the group additive model proposed in the literature, and rigorously define the intrinsic group additive structure for the relationship between the response variable Y and the predictor vector X, and further develop an effective structure-penalized kernel method for simultaneous identification of the intrinsic group additive structure and nonparametric function estimation. The method utilizes a novel complexity measure we derive for group additive structures. We show that the proposed method is consistent in identifying the intrinsic group additive structure. Simulation study and real data applications demonstrate the effectiveness of the proposed method as a general tool for high dimensional nonparametric regression.

Note: Since the supplement appears to include the main paper, I simply reviewed that, and all line numbers below correspond to the supplement.
Summary:
This studies group-additive nonparametric regression models, in which, for some partitioning of the predictor variables, the regression function is additive between groups of variables; this model interpolates between the fully nonparametric model, which is difficult to fit, and the additive model, which is sometimes too restrictive. Specifically, the paper studies the problem where the group structure is not known in advance and must be learned from the data. To do this, the paper proposes a novel penalty function, based on the covering numbers of RKHS balls, which is then added to the kernel ridge regression objective. This results in an objective that can be optimized over both the group structure (which, together with the kernel determines a function space via direct sum of RKHSs over each group of variables) and the regression estimate within each group. Two algorithms are presented for approximately solving this compound optimization problem, and then theoretical results are presented showing (a) the rate at which the empirical risk of the estimate approaches the true risk of the true optimum, and (b) consistency of the group structure estimate, in that the probability it matches the true group structure approaches 1 as n -> infinity. Finally, experimental results are presented on both synthetic and real data.
Main Comments:
The key innovation of the paper appears to be recognizing that the complexity of a particular group structure can be quantified in terms of covering numbers of the direct sum space. The paper is comprehensive, including a well-motivated and novel method, and reasonably solid theoretical and empirical results. I'm not too familiar with other approaches to fitting models between the additive and nonparametric models, but, assuming the discussion in Lines 31-50 is fairly, complete, this paper seems like a potentially significant advance. As noted in the Discussion section, the main issue with the method appears to be difficulty solving the optimization problem over group structure when the number of variables in large. The paper is also fairly clearly written, aside from a lot of typos.
Minor Comments/Questions:
Just curious: is there any simple characterization of "interaction" between two variables that doesn't rely on writing the whole model in Equation (1)?
Line 169: I don't quite understand the use of almost surely here. Is this meant as n -> infinity?
Equation (3) is missing a summation (over i).
Line 205: Perhaps "translation invariant kernel" is a more common term for this than "convolutional kernel"
Typos:
Line 145: "on RHS" should be "on the RHS"
Line 152: "not only can the bias of \hat f_{\lambda,G} reduces" should be "not only can the bias of \hat f_{\lambda,G} reduce"
Line 160: "G^* exists and unique" should be "G^* exists and is unique"
Line 247: "turning parameters" should be "tuning parameters"
Algorithm 2: Line 1: "State with" should be "start with"
Line 251: "by compare" should be "by comparing".


Expectation Propagation with Stochastic Kinetic Model in Complex Interaction Systems
Technological breakthroughs allow us to collect data with increasing spatiotemporal resolution from complex interaction systems. The combination of highresolution observations, expressive dynamic models, and efficient machine learning algorithms can lead to crucial insights into complex interaction dynamics and the functions of these systems. In this paper, we formulate the dynamics of a complex interacting network as a stochastic process driven by a sequence of events, and develop expectation propagation algorithms to make inferences from noisy observations. To avoid getting stuck at a local optimum, we formulate the problem of minimizing Bethe free energy as a constrained primal problem and take advantage of the concavity of dual problem in the feasible domain of dual variables guaranteed by duality theorem. Our expectation propagation algorithms demonstrate better performance in inferring the interaction dynamics in complex transportation networks than competing models such as particle filter, extended Kalman filter, and deep neural networks.

This paper considers the variational approach to the inference problem on certain type of temporal graphical models. It defines a convex formulation of the Bethe free energy, resulting in a method with optimal convergence guarantees. The authors derive Expectation-Propagation fixed point equations and gradient-based updates for solving the optimization problem. A toy example is used to illustrate the approach in the context of transportation dynamics and it is shown that the proposed method outperforms sampling, extended Kalman Filtering and a neural network method.
In the variational formulation, the optimization problem is generally non-convex, due to the entropy terms. Typical variational methods bound the concave part linearly and employ an EM-like algorithm. The key contribution of this paper is to work out a dual formulation on the natural parameters so that the dual problem becomes convex. The derivations seem correct and this looks like an important contribution, if I have not missed some important detail. Authors should elaborate on the generality and applicability of their approach to other models.
I think it would be useful to compare their method with the standard double-loop approach [11] both in terms of describing the differences and in numerical comparison (in terms of error and cpu-time).
The authors state that they are minimizing the Bethe free energy, in found a bit confusing that they call this a mean field forward and backward in the algorithm.
Regarding the experiments, the state representation goes from a factorized state with the state of an agent being one component to location states in which the identities of the agents are lost and the components become strongly coupled. This is OK, but to me it contradicts a bit the motivation in the introduction, in which the authors state "we are not satisfied with only a macroscopic description of the system (...) the goal is (..) to track down the actual underlying dynamics".
Some details about the other methods would be good to know. For example, how many particles N where used for PF? How sensitive are the results (accuracy and time) w.r.t N? Also it is not clear how the NNs are trained. Is the output provided the actual hidden state?
The paper looks too dense. I think substantial part in the Introduction is not necessary and more space could be gain so that formulas do not need to be in reduced font and figure 1 can be made larger.
Other minor comments:
It should be stated in the beginning that x is continuous variable unless otherwise stated
Eq (3) no need to define g_v, since it is not used in the rest of the paper
line 130: better $h(x_{t-1},c_v)$ than $h(x_{t},c_v)$
Eq (4) I think $1-\tau$ should be $(1-\tau)$
Equations between 139-140. \psi(|x_{t-1,1}) -> \psi(x_{t-1,1})
Equations between 139-140. "F_dual" -> "F_Dual"
line 104: "dynamics" -> "dynamic"
line 126: remove "we have that"
line 193: -> "by strong duality both solutions exist and are unique"
line 194: "KKT conditions gives" -> "KKT conditions give"
Use \eqref{} instead of \ref{} for equations
Algorithm 1: the name can be improved
Algorithm 1: Gradient ascent: missing eqrefs in Eqs. ...
Algorithm 1: Gradient ascent: missing parenthesis in all $\langle f(x_t \rangle$
Figure 1
line 301: "outperforms against" -> "outperform"
line 302: "scales" -> "scale"


Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks
Matrix completion models are among the most common formulations of recommender systems. Recent works have showed a boost of performance of these techniques when introducing the pairwise relationships between users/items in the form of graphs, and imposing smoothness priors on these graphs. However, such techniques do not fully exploit the local stationary structures on user/item graphs, and the number of parameters to learn is linear w.r.t. the number of users and items. We propose a novel approach to overcome these limitations by using geometric deep learning on graphs. Our matrix completion architecture combines a novel multi-graph convolutional neural network that can learn meaningful statistical graph-structured patterns from users and items, and a recurrent neural network that applies a learnable diffusion on the score matrix. Our neural network system is computationally attractive as it requires a constant number of parameters independent of the matrix size. We apply our method on several standard datasets, showing that it outperforms state-of-the-art matrix completion techniques.

This paper generalizes a recent NIPS 2016 paper [10] by allowing convolutional neural network (CNN) models to work on multiple graphs.  It extracts local stationary patterns from signals defined on the graphs simultaneously.  In particular, it is applied to recommender systems by considering the graphs defined between users and between items.  The multi-graph CNN model is followed by a recurrent neural network (RNN) with long short-term memory (LSTM) cells to complete the score matrix.
Strengths of the paper:
* The proposed deep learning architecture is novel for solving the matrix completion problem in recommender systems with the relationships between users and the relationships between items represented as two graphs.
* The proposed method has relatively low computational complexity.
* The proposed method has good performance in terms of accuracy.
Weaknesses of the paper:
* The multi-graph CNN part of the proposed method is explained well, but the RNN part leaves open many questions.  Apart from illustrating the process using an example in Figure 3, there is not much discussion to explain why it can be regarded as a matrix diffusion process.  Does the process correspond to optimizing with respect to some criterion?  What is the effect of changing the number of diffusion steps on the completed matrix and how is the number determined?  This needs more detailed elaboration.
In addition, to demonstrate that incorporating the pairwise relationships between users or items helps, you may add some degenerate versions of RMGCNN or sRMGCNN by removing one or both graphs while keeping the rest of the architecture unchanged (including the RNN part).
Minor comments (just some examples of the language errors):
#18: “Two major approach”
#32: “makes well-defined”
#66: “Long-Short Term Memory”
#76: “turns out an NP-hard combinatorial problem”
#83-84: “to constraint the space of solutions”
#104-105: “this representation … often assumes”
#159: “in details”
Table 1 & Table 2: strictly speaking you should show the number of parameters and complexity using the big-O notation.
The reference list contains many arXiv papers.  For those papers that have already been accepted, the proper sources should be listed instead.


Concentration of Multilinear Functions of the Ising Model with Applications to Network Data
We prove near-tight concentration of measure for polynomial functions of the Ising model under high temperature. For any degree d, we show that a degreed polynomial of a n-spin Ising model exhibits exponential tails that scale as
Our concentration radius is optimal up to logarithmic factors for constant d, improving known results by polynomial factors in the number of spins. We demonstrate the efficacy of polynomial functions as statistics for testing the strength of interactions in social networks in both synthetic and real world data.

Summary:

  The paper considers concentration of bilinear functions of samples from an ferromagnetic Ising model under high temperature (satisyfing Dobrushin's condition which implies fast mixing). The authors show that for the case when there is no external field, that any bilinear function with bounded coefficients concentrate around a radius of O(n) around the mean where n is the number of nodes in the Ising Model. The paper also shows that for Ising models with external fields, bilinear functions of mean shifted variables again obey such concentration properties.
   The key technical contribution is adapting the theory of exhangeable pairs from [Cha05] which has been applied for linear functions of the Ising model and apply it to bilinear functions. Naive applications give concentration radius of n^{1.5} which the authors improve upon. The results are also optimal due to the lower bound for the worst case.
Strengths:
 a) The key strength is the proof technique - the authors show that the usual strategy of creating an exchangeable pair from a sample X from the Ising model and X' which is one step of glauber dynamics is not sufficient. So they restrict glauber dynamics to a desirable part of the configuration space and show that the same exchangeable trick can be done on this censored glauber dynamics. They also show that this glauber dynamics mixes fast using path coupling results and also by a coupling argument by chaining two glauber Markov Chains tightly. This tightly chained censored glauber dynamics really satisfies all conditions of the exchangeable pairs argument needed. The authors show this carefully by several Lemmas in the appendix. I quite enjoyed reading the paper.
b) I have read the proofs in the appendix. Barring some typos that can be fixed, the proof is correct to the best of my knowledge and the technique is the major contribution of the paper.
c) The authors apply this to test if samples from a specific distribution on social networks is from an Ising model in the high temperature regime. Both synthetic and real world experiments are provided.
Weaknesses:
  Quite minor weaknesses:
    a) Page 1 , appendix : Section 2, point 2 Should it not be f(X)- E[f(X)] ??
    b) Why is Lemma 1 stated ? It does not help much in understanding Thm 1 quoted. A few lines connecting them could help.
    c) Definition 1, I think It should be intersection and not union ?? Because proof of lemma 5 has a union of bad events which should be the negation of intersection of good events we want in Defn 1.
    d) Typo in the last equation in Claim 3.
    e) Lemma 9 statement. Should it not be (1-C/n)^t ??
    f) In proof of Lemma 10, authors want to say that due to fast mixing, total variation distance is bounded by some expression. I think its a bit unclear what the parameter eta is ?? It appears for the first time. It will be great if the authors can explain that step better .
    g) Regarding the authors experiments with Last.fm - Whats the practical use case of testing if samples came from an Ising model or not ? Actually the theoretical parts of this paper are quite strong. But I just wanted to hear author's thoughts.
    h) Even for linear functions of uniform distribution on the boolean hypercube, at a different radius there are some anti concentration theorems like Littlewood Offord. Recently they have been generalized to bilinear forms (by Van Vu , terry tao with other co authors). I am wondering if something could be said about anti concentration theorems for Bilinear forms on Ising Models ??


Adaptive Accelerated Gradient Converging Method under Hölderian Error Bound Condition
Recent studies have shown that proximal gradient (PG) method and accelerated gradient method (APG) with restarting can enjoy a linear convergence under a weaker condition than strong convexity, namely a quadratic growth condition (QGC). However, the faster convergence of restarting APG method relies on the potentially unknown constant in QGC to appropriately restart APG, which restricts its applicability. We address this issue by developing a novel adaptive gradient converging methods, i.e., leveraging the magnitude of proximal gradient as a criterion for restart and termination. Our analysis extends to a much more general condition beyond the QGC, namely the Hölderian error bound (HEB) condition. The key technique for our development is a novel synthesis of adaptive regularization and a conditional restarting scheme, which extends previous work focusing on strongly convex problems to a much broader family of problems. Furthermore, we demonstrate that our results have important implication and applications in machine learning: (i) if the objective function is coercive and semialgebraic, PG's convergence speed is essentially o( 1 t ), where t is the total number of iterations; (ii) if the objective function consists of an 1 , ∞ , 1,∞ , or huber norm regularization and a convex smooth piecewise quadratic loss (e.g., square loss, squared hinge loss and huber loss), the proposed algorithm is parameter-free and enjoys a faster linear convergence than PG without any other assumptions (e.g., restricted eigen-value condition). It is notable that our linear convergence results for the aforementioned problems are global instead of local. To the best of our knowledge, these improved results are first shown in this work.

Summary: This paper studies the convergence rate of the proximal gradient and its accelerated version under the more general Holderian error bound condition, further extending a line of recent efforts that has weakened the usual strong convexity assumption to the quadratic growth condition. The main contribution is a restarting scheme that eliminates the need of knowing the Holderian constant. The authors complement their theoretical claim with some numerical experiments, which seem to verify the efficiency of the proposed adaptive algorithm.
As far as I can tell, this work is a fine synthesis of some recent developments around semialgebraic functions, error bound, and the proximal gradient algorithm. The results are not too surprising given the existing known results. Nevertheless, it is a very welcome addition to the field. And given its practical nature, the proposed algorithm might be useful for a variety of applications (some toy demonstrations in the experiment).
Some comments:
While the technical proofs involve many pieces from relevant references, the general idea is in fact quite simple (and dates back at least to FISTA): one first derives some condition (e.g. Theorem 5) that is guaranteed to hold if the correct parameter is used, then in the algorithm one periodically checks this condition; if it ever falls short then we know our parameter of choice is too small and we increase it (the familiar doubling trick) and rerun the algorithm. This high level description of the adaptive algorithm perhaps worth explicitly mentioning in the revision (to give the less familiar readers a clear guidance).
Line 85: no need to mention that "g(x) > -\infty for all x" since the range is assumed to be (-\infty, \infty].
Theorem 1: the option I and II are only defined in the appendix. better mention them in the main text (or at least say "see appendix"). The calculations and notations in the proofs of Theorem 1 and 2 are a little sloppy: when theta > 1/2, the rate does not depend on epsilon (the target accuracy) at all? Some more details between say line 68 and line 69 are needed.
Section 4: Can one develop a similar proof for the objective value gap? Is it because of the DUAL algorithm you use to solve Eq (8) that limits the guarantee to the minimal gradient?
Can we say anything about the dependence of the constants theta and c on the dimension and the number of samples? It could be possible that a linear rate is in fact worse than a sublinear rate if such dependence is not carefully balanced.
Experiments: On some datasets (cpusmall and bodyfat), why proximal gradient takes so many more iterations? This looks a little unusual. How many of the proximal mappings are spent on searching for the step size (for each method)? Also, to give a fuller picture, maybe also include the result for the decrease of the objective values. Another observation here: is it really meaningful to consider an absolute epsilon = 10^{-6} say? For optimization per se, sure. But what if say we only care about the classification accuracy? Would it be possible that an solution at epsilon = 10^{-3} already gives us comparable classification accuracy? (This should be easy to verify since the authors did experiment with a classification dataset.)


Regularized Modal Regression with Applications in Cognitive Impairment Prediction
Linear regression models have been successfully used to function estimation and model selection in high-dimensional data analysis. However, most existing methods are built on least squares with the mean square error (MSE) criterion, which are sensitive to outliers and their performance may be degraded for heavy-tailed noise. In this paper, we go beyond this criterion by investigating the regularized modal regression from a statistical learning viewpoint. A new regularized modal regression model is proposed for estimation and variable selection, which is robust to outliers, heavy-tailed noise, and skewed noise. On the theoretical side, we establish the approximation estimate for learning the conditional mode function, the sparsity analysis for variable selection, and the robustness characterization. On the application side, we applied our model to successfully improve the cognitive impairment prediction using the Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort data.

The authors present a regularized modal regression method. The statistical learning view of this proposed method is studied and the resulting model is applied to Alzheimer's disease studies. There are several presentation and evaluation issues for this work in its current form.
Firstly, the authors motivate the paper using Alzheimer's disease studies and argue that modal regression is the way to analyze correlations between several disease markers of the disease. This seems very artificial. The necessity of use conditional mode for regression has nothing specific for the Alzheimer's application. The motivation for RMR makes sense without any AD related context. The authors then argue that the main contribution for the work is designing modal regression for AD (line 52). However, for the rest of the methods the discussion is about statistical learning -- no AD at all. This presentation is dubious. The authors need to decide if the work is statistical learning theory for modal regression OR an application of modal regression for AD (for the latter more applied journals are relevant and not this venue).
Moving beyond this overall presentation issue, if we assume that modal regression for AD is the focus of the paper, then the results should show precisely this performance improvement compared to the rest mean-based regression. However, as shown in section 4 this does not seem to be the case. In several cases the performance of LSR is very close to RMR. It should be pointed out that when reporting such sets of error measures, one should perform statistical tests to check if the performance difference is significant i.e., p-value differences between all the entries of Tables. Beyond, the tables the message is not clear in the Figures -- the visualizations need to be improved (or quantified in some sense). This non-appealing improvements of AD data evaluations is all the more reason that the authors should choose 'standard' regression datasets and show the performance gains of RMR explicitly (once this is done, the results of AD can be trusted and any small improvement can be deemed OK).


Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice
It is well known that weight initialization in deep networks can have a dramatic impact on learning speed. For example, ensuring the mean squared singular value of a network's input-output Jacobian is O(1) is essential for avoiding exponentially vanishing or exploding gradients. Moreover, in deep linear networks, ensuring that all singular values of the Jacobian are concentrated near 1 can yield a dramatic additional speed-up in learning; this is a property known as dynamical isometry. However, it is unclear how to achieve dynamical isometry in nonlinear deep networks. We address this question by employing powerful tools from free probability theory to analytically compute the entire singular value distribution of a deep network's input-output Jacobian. We explore the dependence of the singular value distribution on the depth of the network, the weight initialization, and the choice of nonlinearity. Intriguingly, we find that ReLU networks are incapable of dynamical isometry. On the other hand, sigmoidal networks can achieve isometry, but only with orthogonal weight initialization. Moreover, we demonstrate empirically that deep nonlinear networks achieving dynamical isometry learn orders of magnitude faster than networks that do not. Indeed, we show that properly-initialized deep sigmoidal networks consistently outperform deep ReLU networks. Overall, our analysis reveals that controlling the entire distribution of Jacobian singular values is an important design consideration in deep learning.

The article is focused on the problem of understanding the learning dynamics of deep neural networks depending on both the activation functions used at the different layers and on the way the weights are initialized. It is mainly a theoretical paper with some experiments that confirm the theoretical study. The core of the contribution is made based on the random matrix theory. In the first Section, the paper describes the setup -- a deep neural network as a sequence of layers -- and also the tools that will be used to study their dynamics. The analysis mainly relies on the study of the singular values density of the jacobian matrix, this density being computed by a 4 step methods proposed in the article. Then this methodology is applied for 3 different architectures: linear networks, RELU networks and tanh networks, and different initialization of the weights -- Ginibre ensemble and Circular Real Ensemble. The theoretical results mainly show that tanh and linear networks may be trained efficiently -- particularly using orthogonal matrix initialization -- while RELU networks are poorly conditioned for the two initialization methods. The experiments on MNIST and CIFAR-10 confirm these results.
First, I must say that the paper is a little bit out of my main research domain. Particularly, I am not familiar with random matrix theory. But, considering that I am not aware of this type of article, I find the paper well written: the focus is clear, the methodology is well described, references include many relevant prior works helping to understand the contribution, and the authors have also included an experimental section. Concerning the theoretical part of the paper, the proposed results seem correct, but would certainly need a deeper checking by an expert of the domain. Concerning the experimental section, the experiments confirm the theoretical results. But I must say that I am a little bit skeptical concerning the topic of the paper since all the demonstrations are focused on the simple deep neural network architectures with linear+bias transformations between the layers. It is a little bit surprising to see results on CIFAR-10 which is typically a computer vision benchmark where convolutional models are used, which is not the case here. So I have some doubt concerning the relevance of the paper w.r.t existing architectures that are now much more complex than sequences of layers. On another side, this paper is certainly a step toward the analysis of more complex architectures, and the author could argue on how difficult it could/will be to use the same tools (random matrix theory) to target for example convolutional networks.


Semi-Supervised Learning for Optical Flow with Generative Adversarial Networks
Convolutional neural networks (CNNs) have recently been applied to the optical flow estimation problem. As training the CNNs requires sufficiently large amounts of labeled data, existing approaches resort to synthetic, unrealistic datasets. On the other hand, unsupervised methods are capable of leveraging real-world videos for training where the ground truth flow fields are not available. These methods, however, rely on the fundamental assumptions of brightness constancy and spatial smoothness priors that do not hold near motion boundaries. In this paper, we propose to exploit unlabeled videos for semi-supervised learning of optical flow with a Generative Adversarial Network. Our key insight is that the adversarial loss can capture the structural patterns of flow warp errors without making explicit assumptions. Extensive experiments on benchmark datasets demonstrate that the proposed semi-supervised algorithm performs favorably against purely supervised and baseline semi-supervised learning schemes.

Summary:
The paper presents a semi-supervised approach to learning optical flow using a generative adversarial network (GAN) on flow warp errors.  Rather than using a handcrafted loss (e.g., deviation of brightness constancy + deviation from smoothness) the paper explores the use of a GAN applied to flow warp errors.
Strengths:
+ novel semi-supervised approach to learning; some concerns on the novelty in the light of [21]
+ generally written well
Weaknesses:
- some key evaluations missing
Comments:
Supervised (e.g., [8]) and unsupervised (e.g., [39]) approaches to optical flow prediction have previously been investigated, the type of semi-supervised supervision proposed here appears novel.  The main contribution is in the introduction of an adversarial loss for training rather than the particulars of the flow prediction architecture.  As discussed in Sec. 2, [21] also proposes an adversarial scheme.  A further comparative discussion is suggested.  Is the main difference in the application domain?  This needs to be clarified.  Also the paper claims that "it is not making explicit assumptions on the brightness constancy".  While the proposed approach does not use an Lp or robust loss, as considered in previous work, the concept of taking flow warp error images as input does in fact rely on a form of brightness constancy assumption.  The deviation from prior work is not the assumption of brightness constancy but the actual loss.
In reference to [39], it is unclear what is meant by the approach "requires significantly more sophisticated data augmentation techniques" since the augmentation follows [8] and is similar to that used in the current paper.
The main weakness of the paper is in the evaluation.  One of the main stated contributions (line 64) is that the proposed approach outperforms purely supervised approaches.  Judging from Table 3, this may be a gross misstatement.  For example, on Sintel-Clean FlowNet2 achieves 3.96 aEPE vs 6.28 by the proposed approach.
Why weren't results reported on the KITTI test sets using the test server?  Are the reported KITTI results including occluded regions (in other words all points) or limited to non-occluded regions?
The limitations of the presented training procedure (Sec. 4.4) should have been presented much earlier.  It would have been interesting to train the network with data all from the same domain, e.g., driving scenes.  For instance, rather than train on FlyingChairs and KITTI raw, it would have been interesting to train on KITTI raw (no ground truth) and Virtual KITTI (with ground truth) and evaluate on KITTI TEST.
Reported fine-tuned results should be presented for all approaches.
Minor point:
- line 22 "Due to the lack of the large-scale ground truth flow" --> "Due to the lack of large-scale ground truth flow"
- line 32: "With the limited quantity and unrealistic of ground truth"
Overall rating:
The training approach itself appears to be novel (a more detailed comparative discussion with respect to [21] is suggested) and will be of interest to those working on flow and depth prediction.  Based on the issues cited with the evaluation I cannot recommend acceptance at this point.
Rebuttal:
The reviewer acknowledges that the authors have addressed some of my concerns/suggestions regarding their evaluation. The performance is still below that of the state-of-the-art. Finetuning may help but has not been evaluated.  Is the performance deficit, as suggested by the authors, due to the capacity of their chosen network or is it due to their proposed GAN approach? Would the actual training of a higher capacity network with the proposed adversarial loss be problematic? These are topics to be explored.


Differentially private Bayesian learning on distributed data
Many applications of machine learning, for example in health care, would benefit from methods that can guarantee privacy of data subjects. Differential privacy (DP) has become established as a standard for protecting learning results. The standard DP algorithms require a single trusted party to have access to the entire data, which is a clear weakness, or add prohibitive amounts of noise. We consider DP Bayesian learning in a distributed setting, where each party only holds a single sample or a few samples of the data. We propose a learning strategy based on a secure multi-party sum function for aggregating summaries from data holders and the Gaussian mechanism for DP. Our method builds on an asymptotically optimal and practically efficient DP Bayesian inference with rapidly diminishing extra cost.

Title: Differentially private Bayesian learning on distributed data
Comments:
- This paper develops a method for differential privacy (DP) Bayesian learning in a distributed setting, where data is split up over multiple clients. This differs from the traditional DP Bayesian learning setting, in which a single party has access to the full dataset. The main issue here is that performing DP methods separately on each client would yield too much noise; the goal is then to find a way to add an appropriate amount of noise, without compromising privacy, in this setting. To solve this, the authors introduce a method that combines existing DP Bayesian learning methods with a secure multi-party communication method called the DCA algorithm. Theoretically, this paper shows that the method satisfies differential privacy. It also shows specifically how to combine the method with existing DP methods that run on each client machine. Experiments are shown on Bayesian linear regression, on both synthetic and real data.
- Overall, I feel that this method provides a nice contribution to work on Bayesian DP, and extends where one can use these methods to data-distributed settings, which seems like a practical improvement.
- I feel that this paper is written quite nicely. It is therefore easy to understand previous work, the new method, and how the new method fits in context with previous work.
- One small note: I might’ve missed it, but please make sure the acronym TA (for trusted aggregator, I believe) is defined the paper. I believe “TA” might be used in a few places without definition.


Successor Features for Transfer in Reinforcement Learning
Transfer in reinforcement learning refers to the notion that generalization should occur not only within a task but also across tasks. We propose a transfer framework for the scenario where the reward function changes between tasks but the environment's dynamics remain the same. Our approach rests on two key ideas: successor features, a value function representation that decouples the dynamics of the environment from the rewards, and generalized policy improvement, a generalization of dynamic programming's policy improvement operation that considers a set of policies rather than a single one. Put together, the two ideas lead to an approach that integrates seamlessly within the reinforcement learning framework and allows the free exchange of information across tasks. The proposed method also provides performance guarantees for the transferred policy even before any learning has taken place. We derive two theorems that set our approach in firm theoretical ground and present experiments that show that it successfully promotes transfer in practice, significantly outperforming alternative methods in a sequence of navigation tasks and in the control of a simulated robotic arm.

This paper presents a RL optimization scheme and a theoretical analysis of its transfer performance. While the components of this work aren't novel, it combines them in an interesting, well-presented way that sheds new light.
The definition of transfer given in Lines 89–91 is nonstandard. It seems to be missing the assumption that t is not in T. The role of T' is a bit strange, making this a requirement for "additional transfer" rather than just transfer. It should be better clarified that this is a stronger requirement than transfer, and explained what it's good for — the paper shows this stronger property holds, but never uses it.
Among other prior work, Theorem 1 is closely related to Point-Based Value Iteration, and holds for the same reasons. In fact, w can be viewed as an unnormalized belief over which feature is important for the task (except that w can be negative).
The corollary in Lines 163–165 can be strengthened by relaxing the condition to hold for *some* s, s' that have positive occurrences.
The guarantee in (10) only comes from the closest w_j. Wouldn't it make sense to simplify and accelerate the algorithm by fixing this j at the beginning of the task? Or is there some benefit in practice beyond what the analysis shows? Empirical evidence either way would be helpful.
Very much missing is a discussion of the limitations of this approach. For example, how badly does it fail when the feature-space dimension becomes large?
Beyond the two 2D experiments presented, it's unclear how easy it is to find reward features such that similar tasks are close in feature-space. This seems to go to the heart of the challenge in RL, rather than to alleviate it.
The connection drawn to temporal abstraction and options is intriguing but lacking. While \psi is clearly related to control and w to goals, the important aspect of identifying option termination is missing here.


Communication-Efficient Distributed Learning of Discrete Probability Distributions
We initiate a systematic investigation of distribution learning (density estimation) when the data is distributed across multiple servers. The servers must communicate with a referee and the goal is to estimate the underlying distribution with as few bits of communication as possible. We focus on non-parametric density estimation of discrete distributions with respect to the 1 and 2 norms. We provide the first non-trivial upper and lower bounds on the communication complexity of this basic estimation task in various settings of interest. Specifically, our results include the following:
1. When the unknown discrete distribution is unstructured and each server has only one sample, we show that any blackboard protocol (i.e., any protocol in which servers interact arbitrarily using public messages) that learns the distribution must essentially communicate the entire sample. 2. For the case of structured distributions, such as k-histograms and monotone distributions, we design distributed learning algorithms that achieve significantly better communication guarantees than the naive ones, and obtain tight upper and lower bounds in several regimes. Our distributed learning algorithms run in near-linear time and are robust to model misspecification.
Our results provide insights on the interplay between structure and communication efficiency for a range of fundamental distribution estimation tasks.
1. When the unknown discrete distribution is unstructured and each server has only one sample, we show that any blackboard protocol (i.e., any protocol in which servers interact arbitrarily using public messages) that learns the distribution must essentially communicate the entire sample. 2. For the case of structured distributions, such as k-histograms and monotone distributions, we design distributed learning algorithms that achieve significantly better communication guarantees than the naive ones, and obtain tight upper and lower bounds in several regimes. Our distributed learning algorithms run in near-linear time and are robust to model misspecification.
Our results provide insights on the interplay between structure and communication efficiency for a range of fundamental distribution estimation tasks.
1 Introduction

Summary; The paper studies the classical problem of estimating the probability mass function (pmf) of a discrete random variable given iid samples, but distributed among different nodes. The key quantity of interest is how much communication must be expended by each node (in a broadcast, but perhaps interactive, setting) to a central observer which then outputs the estimate of the underlying pmf. The main results of the paper areclearly stated and are  easy to follow.  The results mostly point out that in the worst case (i.e., no assumptions on the underlying pmf) there is nothing better for each node to do than to communicate its sample to the central observer.
The paper addresses a central topic of a long line of recent works on distributed parameter estimation. The distinguishing feature here is to have as few parametric assumptions on the underlying distribution as possible.
1. In the case when no assumption at all is made on the underlying distribution (other than the alphabet size), the main result is to show that essentially nothing better can be achieved than when all nodes communicate their raw samples (noninteractively) to the central observer. The upper bound is hence trivial and doesnt take much to analyze. The key innovation is in the lower bound where the authors consider a specific near-uniform distribution (where neighboring letters have probabilities that are parametrically (\delta_i in the notation of the paper) close to each other). Then the authors show that any (interactive) protocol allows the central observer to learn these \delta_i parameters and which itself is tantamount to learning the sample itself. The key calculation is to bound the mutual information between the transcripts and the \delta_i parameters which is then fed into a standard hypothesis testing framework (Fano's inequality). Although this machinery is standard, I found the setup of the hypothesis test creative and interesting and novel. Overall, this part of the paper is of fundamental interest (no assumptions at all on the underlying hypothesis) and nicely carried out.
2. I am less sure about the two `structured' settings considered in this paper: k-histograms and monotone pmfs. I can imagine monotone pmfs coming up in some application domain, but far less sure about why k-histograms makes sense.  The main challenging aspect of this setting is that only k intervals are allowed, but their position is arbitrary. I understand the formulation makes for pretty math (and fits in nicely with a bunch of existing literature), but cannot  see any canonical/practical setting where allowing this flexibility makes sense. I would appreciate it if the authors can elaborate on this point in the rebuttal stage.
3. In the context of the last sentence of the previous comment, I would also like to see some discussion by authors on any practical implications of this work. For instance the authors mention that the problem is very well motivated and cite some works from the literature [44, 40, 30, 54, 47]. Of these, I would guess that [47] is the only recent practical work (the others are either more than a decade old or also correspond to an already stylized setting) -- on this topic, can the authors please fix the reference [40] so that it includes its title? I would like to see concrete settings where the structured settings are motivated properly and the authors actually try out their algorithms (along with competing baselines) in that setting. In the context of this paper, I would like to see a (brief) discussion of how the settings might be related to the distributed computing revolution underlying modern data centers. Most NIPS papers  have a section where they discuss their  algorithms in the context of some concrete practical setting and report empirical results.
3. The schemes proposed in the two structured settings are more sophisticated than the baseline one (where the nodes simply communicate their samples directly), but are directly motivated from their centralized counterparts. In particular, the k-histogram setting uses ideas from [3] and approximation in the \A_k norm and the monotone setting from the classical work of Birge. Overall, the proofs are correct and represent a reasonable innovation over a long line of work on this topic.
Summary: I recommend this submission as far better suited to COLT or even SODA.


Doubly Stochastic Variational Inference for Deep Gaussian Processes
Gaussian processes (GPs) are a good choice for function approximation as they are flexible, robust to overfitting, and provide well-calibrated predictive uncertainty. Deep Gaussian processes (DGPs) are multi-layer generalizations of GPs, but inference in these models has proved challenging. Existing approaches to inference in DGP models assume approximate posteriors that force independence between the layers, and do not work well in practice. We present a doubly stochastic variational inference algorithm that does not force independence between layers. With our method of inference we demonstrate that a DGP model can be used effectively on data ranging in size from hundreds to a billion points. We provide strong empirical evidence that our inference scheme for DGPs works well in practice in both classification and regression.

This paper presents a doubly stochastic variational inference for deep Gaussian processes. The first source of stochasticity comes from posterior sampling, devised to solve the intractability issue of propagating uncertainty through the layers. The second source of stochasticity comes from minibatch learning, allowing the method to scale to big datasets, even with 1 billion instances.
The paper is in general well written and the authors make an effort to explain the intuitions behind the proposed method. However, the paper is not placed well in the related literature. The "related work" section is missing reference to [11] (presumably it was meant to be included in line 65) and the paper is completely missing reference to the very related (Hensman and Lawrence, 2014; Nested Variational Compression in DGPs). The idea of using SVI and mini-batch learning for DGPs has also been shown in (Frigola et al. 2014; Variational GP State-Space Models) and (Damianou, 2015; PhD Thesis).
To further expand on the above, it would be easier for the reader if the authors explained modeling and inference choices with respect to other DGP approaches. For example, using the exact conditional p(f|u) to obtain the cancellation is found in a number of approaches, but in the paper this is somehow presented as a new element. On the other hand, absorbing the noise \epsilon in the kernel is (to my knowledge) unique to this approach, however this design choice is underplayed (same holds for the mean function), stating that "it is for notational convenience", although in practice it has implications that go beyond that (e.g. no need to have q(h), h being the intermediate noisy layer). This is counter-intuitive, because these are actually neat ideas and I'd expect them to be surfaced more. Therefore, I suggest to the authors to clearly relate these details to other DGP approaches, to facilitate comprehension and theoretical comparison.
Regarding the technical aspect, I find the method sound and significant. The method is not very novel, especially given (Hensman and Lawrence, 2014), but it does introduce some nice new ideas and shows extensive and impressive results. It is perhaps the first paper promising "out-of-the-box" training of DGPs and this is a significant step forward - although given this strong claim it'd be great to also have the code at submission time, especially since it's < 200 lines.
It would be worth discussing or exploring some further properties of the method. For example, what does it imply to absorb the layer noise in the kernel? How does the model construction and sampling for inference affect the uncertainty calibration?
I find the experiments extensive, well executed and convincing. However, I didn't understand what line 223 means ("we used the input dimension..") and why can't we see a deeper AEPDGP (does that choice make it fair for one model but unfair for the other?). Since the method is relatively fast (Table 3) and performance improves with the number of layers, this begs the question of "how deep can you go?". It'd be great to see a deeper architecture in the final version (in [8] the authors make a similar observation about the number of layers).
Other comments:
- including an algorithm showing the steps of the sampling-based inference would greatly improve clarity
- line 176: Shouldn't f_i^l be f_i^{l-1} before "(recall..."?
- line 211: It's not clear to me what "whiten inputs and outputs" means and why this is done
- line 228: What does "our DGP falls back to the single-layer GP" mean? Learning the identity mapping between layers?
- line 264: Reference missing.
- line 311: Does "even when the quality of the approximation..." refer to having more inducing points per layer, or is it a hint about the within-the-layer approximation?
- which kernel is used for classification?


Hash Embeddings for Efficient Word Representations
We present hash embeddings, an efficient method for representing words in a continuous vector form. A hash embedding may be seen as an interpolation between a standard word embedding and a word embedding created using a random hash function (the hashing trick). In hash embeddings each token is represented by k d-dimensional embeddings vectors and one k dimensional weight vector. The final d dimensional representation of the token is the product of the two. Rather than fitting the embedding vectors for each token these are selected by the hashing trick from a shared pool of B embedding vectors. Our experiments show that hash embeddings can easily deal with huge vocabularies consisting of millions of tokens. When using a hash embedding there is no need to create a dictionary before training nor to perform any kind of vocabulary pruning after training. We show that models trained using hash embeddings exhibit at least the same level of performance as models trained using regular embeddings across a wide range of tasks. Furthermore, the number of parameters needed by such an embedding is only a fraction of what is required by a regular embedding. Since standard embeddings and embeddings constructed using the hashing trick are actually just special cases of a hash embedding, hash embeddings can be considered an extension and improvement over the existing regular embedding types.

This work describes an extension of the standard word hashing trick for embedding representation by using a weighted combination of several vectors indexed by different hash functions to represent each word. This can be done using a predefined dictionary or during online training. The approach has the benefit of being easy to understand and implement and greatly reduces the number of embedding parameters.
The results are generally good and the approach is quite elegant. Additionally, the hash embeddings seem to act as an effective regularizer. However, table 2 would benefit from describing the final selected vocabulary sizes as well as the parameter reduction provided by the hash embedding. Table 3 is also missing a joint state of the art model for the DBPedia dataset [1].
Line 255 makes the claim that the ensemble would train in the same amount of time as the single large model. However, this would only be true if each model in the ensemble had an architecture with fewer weights (that were not embedding weights). From the description, it seems that the number of non-embedding weights in each network in the ensemble is the same as that in the large model so that training time would be significantly larger for the ensemble.
Table 3 highlights the top 3 best models, however, a clearer comparison might be to split the table into embedding only approaches vs RNN/CNN approaches. It would also be interesting to see these embeddings used in the more context-sensitive RNN/CNN models.
Minor comments:
L9: Typo: million(s)
L39: to(o) much
L148: This sentence is awkwardly rewritten.
L207: What is patience?
L235: Typo: table table
Table 4: Were these obtained through summing the importance weights? What were the order of the highest/lowest weights?
[1] Miyato, Takeru, Andrew M. Dai, and Ian Goodfellow. "Virtual adversarial training for semi-supervised text classification." ICLR 2017.