Distributed Weight Consolidation: A Brain Segmentation Case Study
Collecting the large datasets needed to train deep neural networks can be very difficult, particularly for the many applications for which sharing and pooling data is complicated by practical, ethical, or legal concerns. However, it may be the case that derivative datasets or predictive models developed within individual sites can be shared and combined with fewer restrictions. Training on distributed data and combining the resulting networks is often viewed as continual learning, but these methods require networks to be trained sequentially. In this paper, we introduce distributed weight consolidation (DWC), a continual learning method to consolidate the weights of separate neural networks, each trained on an independent dataset. We evaluated DWC with a brain segmentation case study, where we consolidated dilated convolutional neural networks trained on independent structural magnetic resonance imaging (sMRI) datasets from different sites. We found that DWC led to increased performance on test sets from the different sites, while maintaining generalization performance for a very large and completely independent multi-site dataset, compared to an ensemble baseline.

Research is becoming an increasingly collaborative endeavor but experiments that result in data collection are still costly. The costs, together with privacy, regulations and practicality concerns lead to the current situation when large datasets are amassed across multiple research groups yet any single group rarely has enough data to train a machine learning algorithm. The paper proposes an algorithm for training machine learning models in these settings in the continual learning style, when a model is first trained on data at some of the sites and then the learned weights are aggregated and subsequent models are fine-tuned from that point. This is achieved thanks to a variational approximation and independence assumption on filter weights  My two major concerns are  1.  The unrealistic testing setup that goes against the offered decentralized research centers models. The test in the paper are conducted on a very few sites with a very large amount of data, and not on many sites with moderate data sizes. To gain an intuition whether the approach works one would need many more realistic setups for validation.  1.  The choice of brain segmentation problem to demonstrate PWC. It is a well known feature of most of the neural segmentation methods that unlike other deep learning algorithms they only need a few labeled examples, since each pixel/voxel is labeled and provides a signal. Thus, among all of the deep learning algorithms the paper seems to imply it will improve the chosen method seems to be the one benefiting the least. Furthermore, it would be good to see and compare with performance of the original MeshNet segmentation algorithm just on the data that was used at the final stages. I suspect, it may perform as well.  Other concerns:  1.  line 52, in fact, MeshNet, when ran uses quite a lot of memory, may be even more than other mentioned methods, but that memory is in the activations, not in useful weights. So may be the paper needs to be more explicit of the fact, that MeshNet is the only one of the mentioned models that is feasible to use in the proposed settings as all other models require transfer of Gigabytes of parameters for each trained model, while MeshNet's parameters amount to under a Megabyte? 2.  The cited papers of segmentation techniques for brain imaging are just the most recent. PyramidLSTM from 2015 is not cited, for example. 3.  It is unclear from the text whether the algorithm ships the weights over the network or what happens to them at all. it would be good to have that explained. 4.  Line 77-78 - editorial comment is left in the manuscript. 5.  independent values of w in spatial filter are hard to assign the meaning to. The whole purpose of the local spatial filter is to capture correlations in the input, so the weights cannot be independent. 6.  The notation is heavy and is not optimized for getting the point across. The use of indexing is excessive and leads to difficulties with interpreting even simple mathematics. Equation 16 is an example that could be greatly simplified but then it would took quite simple. 7.  Line 141 implies that parameter uncertainty increases with each consolidation, If that is try then does not that defeat the point of the work? 8.  The model uses a truncated set of FreeSurfer units, while the complete 104 ROI set in the atlas could be more useful for practicioners. 9.  line 153 provides citations for splitting each volume, but the provided references did not split, but sample subvolumes. 10. Lines 158-159 needs to qantified. 11. lines 171-173 the vanilla MeshNet should be also among the baselines. 12. it may be worth citing prior work on successfully executing the model that the authors describe here (ENIGMA 1) and potentially any of the papers that worked out the decentralized data setup in detail, including a similar to the proposed "hot potato" algorithm schema as well as the iterative one (COINSTAC papers 2). Without those citations the current language in the paper misleads the reader to think that the specific decentralized biomedical setup is also proposed by the authors of this paper.     1.  Thompson, P. M. and others (2014). The ENIGMA consortium     2.  Plis, SM et al., 2016. COINSTAC: a privacy enabled model  1.  lines 43-45 imply that the datasets were combined but line 153 states otherwise 2.  Lines 199-200 The vision does come across clear. 3.  line 203 - unsupported claim. 4.  Why the subvolumes size have been decreased from 38 to 32? 5.  The receptive field size of convolutional network is 37, which is bigger then 32 subvolume size  Minor:  -   line 94 - such elastic, should be such as elastic -   line 95 - reinterpretted - an extra t


Causal Inference with Noisy and Missing Covariates via Matrix Factorization
Valid causal inference in observational studies often requires controlling for confounders. However, in practice measurements of confounders may be noisy, and can lead to biased estimates of causal effects. We show that we can reduce bias induced by measurement noise using a large number of noisy measurements of the underlying confounders. We propose the use of matrix factorization to infer the confounders from noisy covariates. This flexible and principled framework adapts to missing values, accommodates a wide variety of data types, and can enhance a wide variety of causal inference methods. We bound the error for the induced average treatment effect estimator and show it is consistent in a linear regression setting, using Exponential Family Matrix Completion preprocessing. We demonstrate the effectiveness of the proposed procedure in numerical experiments with both synthetic data and real clinical data.

Summary:  The paper describes a novel method to compute average causal treatment effects via adjustment when a large number of noisy proxy covariates is available. It treats the (many) proxy covariates as noisy observations (measurement noise incl. missing values) of a few unobserved/underlying ‘true’ confounders that should ideally be used for adjustment. It then proposes to use low-rank matrix factorization as a preprocessing step on the many observed noisy covariates to uncover / reconstruct a much better approximation to these true confounders. It is subsequently shown on both synthetic and real-world data sets that using these low rank constructs as input to existing methods to compute treatment effects for complete/missing data significantly improves their performance.  Frequently encountered problem, very interesting idea, very well executed. Although I am aware that many approaches using latent class/factor analysis etc. on this type of data exist, to the best of my knowledge the proposed method and combination with ATE estimates is novel, with promising results and should be relevant to a wide audience.  As far as I can see there are no significant flaws in the theoretical results. Especially the bounds on the treatment effect estimates are impressive, although I did not check the derivation of eq.(3) and (4) in detail. One point of concern is that the method effectively assumes that the low-rank confounder constructs are independent: if we assume these correspond to psychometric factors like intelligence and personality traits (as in the examples provided), then this is probably unlikely to be true. However, in many other cases it may be sufficiently reasonable, and I suspect that even when it does not hold then it will still help to obtain a better ATE than using the myriad of noisy covariates as a starting point for adjustment.  The paper as a whole is well written, and good to follow, although it tends to get quite technical at some points, and might benefit from 1 or 2 explanatory figures.   In summary: clear accept.  ======= Details: 74: typo ‘marix’ 69-81: bit compact for the large body of related work 97: ‘.. treatment and control respectively’ => this suggest 0=treatment, 1=control … 115+ : perhaps a small figure depicting system + model would be nice here … 149: ‘.. so long as the .. not vanish too fast’ => given a concrete example of where this is the case 169: MCAR is usually a rather strong/unrealistic assumption in psychometric tests, but ok … 172: ‘.. exp.family noise mechanism’. => does this relate to the measurement noise or the intrinsic noise? 175: typo ‘anlytic’ 180: give some intuition on the purpose/implication of this assumption on \grad^2G(u) 192: typo ‘matirx’ 204: typo: ’an error bound’ or ‘error bounds’ 237: typo ‘numebr’ idem: try to give a concrete example where this is the case 246: nice bound … though I could not check this derivation :) Fig.1 : caption should be a bit more informative 260-269,eq.(5): how would you solve this for a combination of continuous and discrete covariates simultaneously?  338/Fig2: good & interesting results … can you explain the dramatic difference between Match(ing) with and without the MF preprocessing step? (initially without seems to get worse for increasing nr. of proxies?) idem: make caption more useful


Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects
We present Sequential Attend, Infer, Repeat (SQAIR), an interpretable deep generative model for videos of moving objects. It can reliably discover and track objects throughout the sequence of frames, and can also generate future frames conditioning on the current frame, thereby simulating expected motion of objects. This is achieved by explicitly encoding object presence, locations and appearances in the latent variables of the model. SQAIR retains all strengths of its predecessor, Attend, Infer, Repeat (AIR, Eslami et al., 2016), including learning in an unsupervised manner, and addresses its shortcomings. We use a moving multi-MNIST dataset to show limitations of AIR in detecting overlapping or partially occluded objects, and show how SQAIR overcomes them by leveraging temporal consistency of objects. Finally, we also apply SQAIR to real-world pedestrian CCTV data, where it learns to reliably detect, track and generate walking pedestrians with no supervision.

The authors propose a temporal extension of the structured image generative model AIR.  The model is composed of two parts: the first one, DISC, detects new objects at each time step, and is very similar to AIR. The only difference is that its predictions are conditioned on the updated latent variables of previous frames. The second one PROP, is responsable for updating or forgetting latent variables given a new image. It does so by employing two RNNs : the first one (temporal) updates the previous latent states. The second one (relational) models relations between objects, and is recurrent over the objects in the frame. The authors show extensive experimentation on a stochastic moving MNIST dataset and qualitative results on a dataset collected from static CCTV cameras. The structure of their model allows them to learn to detect and track objects in an unsupervised fashion, perform video prediction, and use the latent variables for solving simple tasks (addition, counting of objects).  The authors propose a compelling and ambitious approach that tackles the extremely important problem of structured generative modeling, where the structure can be used for unsupervised learning of high level recognition tasks. Incorporating structure in the model to reflect the prior that a scene is composed of objects that should be mainly described by their location and their appearance, and showing that the learned latent variables are interpretable and thus useful in down stream tasks, is an extremely appealing direction. At this point, the approach does not model the background, and has not been shown to be applicable in contexts where the camera is non static (as background subtraction is necessary), but it constitues a nice step towards in this important direction. It is nice that the authors show a real-world application of their method.  Unfortunately, the experimental evaluation is rather lacking, and I have several questions in regards to this matter: - the authors should compare to other VAE-flavoured state of the art video generative modelling methods, such as Denton et al. ICML 2018. In particular, the qualitative results of Figure 12 are clearly not state of the art. - how important is the relational RNN ? The authors transparently acknowledge that the method is complex and an ablation study would be very useful. It would also be helpful to show in a more tangible manner how this RNN is "implementing the explaining away phenomenon" ? l.122 -  "All objects are present through the sequence; the digits can move out of the frame, but always come back." l.152, 153 Why impose this constraint on the dataset ? What severely limits the model from dealing where cases when the digit does not come back ? Also, can the digits appear during the sequence and if not, why not ? If this showed failures of the model, it would be interesting to analyse why. - I don't agree with the justifications for not filling in Table 1: a classifier could be learned for VRNN for counting, just like in the addition experiment. And for completion, why not perform these experiments for the MLP-VRNN ; it is better in terms of KL divergence than its CONV counterpart. -  "We will show that the resulting model, which we name Sequential AIR  (Sequential Attend, Infer, Repeat (SQAIR )), retains the strengths of the original AIR formulation while outperforming it in both synthetic and real-world scenarios." I did not find the results for AIR on the CCTV dataset. - can you give a qualitative interpretation of the fact that the reconstruction metric of MLP-AIR is better than MLP-SQAIR ? - Figure 10: why are there two entirely lines input fully black inputs ? - "which signifies that the latent representation can be allocated to regions of the frames that need it, as opposed to describing the whole scene" l.170 This is not clear to me. Cumulatively, this leaves the reader a little bit unsatisfied (to summarize: missing results, no ablation study, missing comparison to an important state of the art paper, main results on a toyish dataset that could be slightly less toy). Also, it would strengthen the paper a lot to show a more thorough and quantitative evaluation on the real-world dataset. This is what leads me to the current decision of "Marginally below the acceptance threshold."    Regarding clarity of the method, the paper is generally well written, but I would like to have the following clarifications:  - l. 118 you specify that the temporal RNN has no access to the image. Then in appendix D, equation (14), as well as Figure 3, imply the opposite. Can you clarify ? - Figure 6 is ambiguous: the lower lines are predictions or reconstructions ?  Finally, a note that for this type of problem, accompanying videos are much appreciated to ease the qualitative evaluation of the samples and reconstructions, especially to appreciate temporal consistency.   Typos / errors :  - then should be than l.53 and l.188 - l.59 in the AIR model, n is at most N (a fixed hyperparameter of the method) - l.65: according to Figure 1, f_theta^dec 's input is z_t^what,i and output is g_t^i. Calling the decoder a "glimpse decoder" seems to imply some form of attention mechanism on the generated image, which is not the case. Also, z^where should be indexed with i. - l.97 vector valued should be -valued vector - l.103 logical flow of text would require that it is also explained that z^what and z^where are sampled during the propagation step -  images l.151 should be digits - l.160 Being similar to a VAE.  -------- Assessment after the rebuttal -------- The authors have addressed most of my concerns. The experimental protocol is still slightly lacking, as they do not experimentally validate the impact of using a relational RNN on top of the temporal RNN; and I still think that the toy dataset should have had certain properties to test the strengths of the proposed model (like appearing and disappearing digits.) However, like the authors and the other reviewers, I am convinced of the importance of the problem addressed, and I acknowledge that the authors have proposed a compelling and ambitious approach to tackle this problem. Therefore, with the clarifications and improvements that the authors have promised to bring, I now think that this paper is good and should be accepted.


A Spectral View of Adversarially Robust Features
Given the apparent difficulty of learning models that are robust to adversarial perturbations, we propose tackling the simpler problem of developing adversarially robust features. Specifically, given a dataset and metric of interest, the goal is to return a function (or multiple functions) that 1) is robust to adversarial perturbations, and 2) has significant variation across the datapoints. We establish strong connections between adversarially robust features and a natural spectral property of the geometry of the dataset and metric of interest. This connection can be leveraged to provide both robust features, and a lower bound on the robustness of any function that has significant variance across the dataset. Finally, we provide empirical evidence that the adversarially robust features given by this spectral approach can be fruitfully leveraged to learn a robust (and accurate) model.

The paper studies the very timely problem of feature robustness - it provides a new perspective on this problem with spectral approach. In particular, it wants to define functions that are robust to adversarial perturbations and vary across data points. Towards that goal, it shows connections between the spectral properties of the dataset, and the robustness of features.   Detailed comments:  - l.26: what are examples created in the real world? - l.48 why is it necessarily a good idea to disentangle the problems of accuracy, and robustness? any design that would try to address both simultaneously is necessarily bound to fail? - the notation for the distance metric,  is the same as the dimension of the input space - this is not ideal. - l.79 - why does the variance have to be close to 1 necessarily? - l.127 - the statement that there is no effort in trying to understand the geometry properties in the robustness problem is probably an overstatement (see the work of Alhussein Fawzi for example, as mentioned later) - the section between lines 166 and 175, contains quite a few notation problems, or imprecisions; it might be worth revising that part carefully. Also, it it not clear at this stage why the dataset is augmented with x? And does it mean that a new graph is built for each new x?  - at several places, the authors talk about suitable conditions for arguments or theorems (like in line 200 for example). These conditions should be clarified in the text - it would be a nice addition to have a sketch of the Th.2 in the main text. And actually, the discussion of Th.1 should come before Th 2 for better readability. - is there any constructive way to pick the correct threshold in lines 226-228? - the spectral view might appear as an overstatement in general - the authors mostly look at the Fiedler vector, and not at the full spectrum, in their analysis, and fonction construction. - the bounds defined by the authors, appear to be very loose, mostly because of the underlying assumptions: are they useful at all at the end? The authors should discuss in more details the relevance of the proposed bounds (esp. the lower one).  - Figure 2: this one is not very convincing: one can argue against the author’s statement that there is a clear (linear) relationship + outliers...  - recent works have studied the bounds on robustness, and relations to accuracy, e.g., the 2018 arxiv paper of Fawzi et al.. These works have to be discussed to provide a complete overview of the closely related literature, as they address some aspects of the key questions raised at l.38. - the are a few English typos, that should be corrected. E.g., lines 10,18, 133, 167 (index 0 misplaced), 169 (0 should be x_0??), 213, 336...  Overall, the paper takes an interesting, and new spectral perspective on robustness. They derive also new bounds, that could potentially bring interesting insights on the robustness problem. The classification function stays unfortunately trivial (and classical in graph-based problem), and is not learned or trained for the target task, as this was hinted in the introduction. As a result, the paper brings interesting insights, but fails to be completely convincing about the relevance of this theoretical study and its results. It could be accepted however, as it will probably trigger interesting discussions on a timely topic.


Learning and Inference in Hilbert Space with Quantum Graphical Models
Quantum Graphical Models (QGMs) generalize classical graphical models by adopting the formalism for reasoning about uncertainty from quantum mechanics. Unlike classical graphical models, QGMs represent uncertainty with density matrices in complex Hilbert spaces. Hilbert space embeddings (HSEs) also generalize Bayesian inference in Hilbert spaces. We investigate the link between QGMs and HSEs and show that the sum rule and Bayes rule for QGMs are equivalent to the kernel sum rule in HSEs and a special case of Nadaraya-Watson kernel regression, respectively. We show that these operations can be kernelized, and use these insights to propose a Hilbert Space Embedding of Hidden Quantum Markov Models (HSE-HQMM) to model dynamics. We present experimental results showing that HSE-HQMMs are competitive with state-of-the-art models like LSTMs and PSRNNs on several datasets, while also providing a nonparametric method for maintaining a probability distribution over continuous-valued features.

The paper develops a connection between quantum graphical models and inference with Hilbert space embeddings (HSE). It shows kernel formulations for sum rule and Bayes rule with quantum graphical models, connecting kernel mean embeddings as well as cross-covariance operators. The paper proposes to use HSE on top of HQMMs, arguing that it improves empirical performance against PSRNN and LSTMs.  The paper is nicely written and does a very good job at explaining difficult quantum concepts using tools that should be familiar to a machine learning audience. I personally really enjoyed reading and learning about the connections developed in the paper. I also think the work is very original and could lead to interesting follow up work.  I have a few questions regarding the experiments:  - Why not compare with a simpler baseline, such as regular HMM?  - There are plenty of existing standard datasets for sequence prediction / modeling, yet none of the experiments in this paper seem to use any of them, which makes it harder to convince others that the improvement is not because of weak baselines and unusual train/test splits.  - What are the per iteration BPTT efficiency and modeling complexity for HSE-HQMM compared with other baselines, namely LSTM and PSRNN?   - I am rather confused by the PTB experiments. The log-perplexity for all these methods are no more than 2, which means that the perplexity is below 10. However, the current SOTA for PTB perplexity should be significantly higher (in the 30-50 range).  Minor points:  Heat map resolution and aspect ratio. Typo on line 106: embeddingsx   ==========  Thanks for the clarifications!


Online Adaptive Methods, Universality and Acceleration Volkan Cevher EPFL
We present a novel method for convex unconstrained optimization that, without any modifications, ensures: (i) accelerated convergence rate for smooth objectives, (ii) standard convergence rate in the general (non-smooth) setting, and (iii) standard convergence rate in the stochastic optimization setting. To the best of our knowledge, this is the first method that simultaneously applies to all of the above settings. At the heart of our method is an adaptive learning rate rule that employs importance weights, in the spirit of adaptive online learning algorithms [12,20], combined with an update that linearly couples two sequences, in the spirit of [2]. An empirical examination of our method demonstrates its applicability to the above mentioned scenarios and corroborates our theoretical findings.

This paper considers the problem of designing universal gradient methods, that is designing iterative methods which given gradient or subgradient oracles automatically achieve the best rate for a given problem even without knowing all problem parameters beforehand. In particular, this paper provides a method they call AcceleGrad (which in turn is inspired from AdaGrad) which provided the initial distance of a point to the minimizer is appropriately estimated and an un upper bound on the norm of gradient is given, achieves the best rates possible for smooth minimization, non-smooth minimization, and the stochastic versions of each of these problems. Crucially, the algorithm does not need to be given the smoothness bound beforehand. The algorithm provides experiments to analyze the efficacy of the method.  This paper is well-written and the proposed algorithm is very clean and some of the analysis is quite nice. This paper does a good job of unifying the analysis of multiple convex optimization problems and has key properties that as they point out, no universal method had previously. The main weaknesses of the paper are that it claims universality and optimality fairly strongly in the beginning, but requires upper bounds on the gradient and the domain size up front. While this is not necessarily a major detriment the writing should make this abundantly clear in the beginning. Also, the experiments of the paper could be strengthened. Why the particular experimental settings are chosen and what they really say about the practicality of AcceleGrad could be improved. Furthermore, it isn’t clear there is something particular novel in the analysis or how well known the open problem being addressed is.   Nevertheless, this paper provides a very interesting algorithm that advances the state of the art for convex optimization and unifies the analysis of a number of techniques in a clever way. Consequently, it could make for a nice addition to the program.  Specific comments: - 65: “are explore” --> “are explored” - 78: “we assume to be given” --> “we assume we are given” - 83: “Note that we allow to choose point outside K” – I would clarify this further. - 83: “We also assume that the objective function I G” – throughout the paper or just here, I would be clear. - Figure 1 – what is rho and epsilon in the charts, would make this clearer.  EDIT: Thank you for your thoughtful response. Thank you for clarifying and for agreeing to make certain changes. However, I believe novelty in analysis requires more than being the first to analyze a particular algorithm, it requires an argument of why previous analysis techniques might be stuck, a simple approach might fail, etc. I still think this paper could be a nice addition to the program, but a more concrete case would be need to raise my score much more.


GIANT: Globally Improved Approximate Newton Method for Distributed Optimization
For distributed computing environment, we consider the empirical risk minimization problem and propose a distributed and communication-efficient Newton-type optimization method. At every iteration, each worker locally finds an Approximate NewTon (ANT) direction, which is sent to the main driver. The main driver, then, averages all the ANT directions received from workers to form a Globally Improved ANT (GIANT) direction. GIANT is highly communication efficient and naturally exploits the trade-offs between local computations and global communications in that more local computations result in fewer overall rounds of communications. Theoretically, we show that GIANT enjoys an improved convergence rate as compared with first-order methods and existing distributed Newton-type methods. Further, and in sharp contrast with many existing distributed Newton-type methods, as well as popular first-order methods, a highly advantageous practical feature of GIANT is that it only involves one tuning parameter. We conduct large-scale experiments on a computer cluster and, empirically, demonstrate the superior performance of GIANT.
severely limited bandwidth and high latency. As a result, it is imperative to reduce the communications between the center and a node or between two nodes. In such settings, the preferred methods are those which can perform expensive local computations with the aim of reducing the overall communications across the network.
Optimization algorithms designed for distributed setting are abundantly found in the literature. Firstorder methods, i.e, those that rely solely on gradient information, are often embarrassingly parallel and easy to implement. Examples of such methods include distributed variants of stochastic gradient descent (SGD) [17,27,47], accelerated SGD [35], variance reduction SGD [10,28], stochastic coordinate descent methods [5,13,20,31] and dual coordinate ascent algorithms [30,43,46]. The common denominator in all of these methods is that they significantly reduce the amount of local computation. But this blessing comes with an inevitable curse that they, in turn, may require a far greater number of iterations and hence, incur more communications overall. Indeed, as a result of their highly iterative nature, many of these first-order methods require several rounds of communications and, potentially, synchronizations in every iteration, and they must do so for many iterations. In a computer cluster, due to limitations on the network's bandwidth and latency and software system overhead, communications across the nodes can oftentimes be the critical bottleneck for the distributed optimization. Such overheads are increasingly exacerbated by the growing number of compute nodes in the network, limiting the scalability of any distributed optimization method that requires many communication-intensive iterations. To remedy such drawbacks of high number of iterations for distributed optimization, communicationefficient second-order methods, i.e., those that, in addition to the gradient, incorporate curvature information, have also been recently considered [16,36,29,45,7,15,38]; see also Section 1.1.
The common feature in all of these methods is that they intend to increase the local computations with the aim of reducing the overall iterations, and hence, lowering the communications. In other words, these methods are designed to perform as much local computation as possible before making any communications across the network. Pursuing similar objectives, in this paper, we propose a Globally Improved Approximate NewTon (GIANT) method and establish its improved theoretical convergence properties as compared with other similar second-order methods. We also showcase the superior empirical performance of GIANT through several numerical experiments.
The rest of this paper is organized as follows. Section 1.1 briefly reviews prior works most closely related to this paper. Section 1.2 gives a summary of our main contributions. The formal description of the distributed empirical risk minimization problem is given in Section 2, followed by the derivation of various steps of GIANT in Section 3. Section 4 presents the theoretical guarantees. The most commonly used notation is listed in Table 1. Due to the page limit, the readers can refer to the long version [41]; Section 5 provides a summary of our experiments. The proofs can be found in the long version [41].

The paper introduces GIANT, a distributed variant of Newton algorithm. The considered problem is important and the paper gives a nice contribution to the field of distributed optimisation. The paper is very clear and nice to read, and propose nice theoretical contributions and experiments, with a detailed bibliography and positioning with respect to priori work.  Here is my main criticism :  * Authors acknowledge that their approach is close to previous works, namely DANE, for which GIANT seem to coincide to DANE in the least-squares loss case. However, the rate obtained in the paper is much better, certainly thanks to the introduction of the incoherence assumption, which is well known in the field of compressed sensing and randomized linear algebra. I think that the paper would benefit from a more thorough comparison of approaches, and to what extent this row-coherence assumption is reasonable in practical assumptions. It would help also to understand where the gap between the rate of GIANT and DANE comes from, if it is either related to the different set of assumptions, or if it comes from an alternative proof technique.  * The numerical experiments are a bit disappointing. It proposes a comparison to good baselines, but on  medium scale datasets (epsilon, covertype) and then on larger datasets, constructed as extensions of cover type. Many large or huge datasets for binary classification are available : it would be really interesting and more transparent to see the performances of these distributed optimisation algorithms for more realistic problems, for which feature matrices are really different. Indeed, considering different simulation setups for features hardly accounts for the variability of features in realistic problems. Indeed, the process of augmentation of the cover type dataset is probably helping with the incoherence, since it consists in row duplications with small Gaussian noise and features are augmented with random Fourier features. Furthermore, no experiments / guarantees for sparse features matrices are provided. In big data setting, with very large n / large d, features are typically sparse (word counts for instance)  In summary, I like the paper and think that it deserves to be published, but I won't be shocked if it is not. I sincerely believe that it can be strongly improved by an extension of the experiments to several real-world large scale datasets, that are now commonly available in kaggle or UCI platforms.


SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator
In this paper, we propose a new technique named Stochastic Path-Integrated Differential EstimatoR (SPIDER), which can be used to track many deterministic quantities of interests with significantly reduced computational cost. Combining SPIDER with the method of normalized gradient descent, we propose SPIDER-SFO that solve non-convex stochastic optimization problems using stochastic gradients only. We provide a few error-bound results on its convergence rates. Specially, we prove that the SPIDER-SFO algorithm achieves a gradient computation cost of O min(n 1/2 −2 , −3 ) to find an -approximate first-order stationary point. In addition, we prove that SPIDER-SFO nearly matches the algorithmic lower bound for finding stationary point under the gradient Lipschitz assumption in the finite-sum setting. Our SPIDER technique can be further applied to find an ( , O( 0.5 ))-approximate second-order stationary point at a gradient computation cost ofÕ min(n 1/2 −2 + −2.5 , −3 ) .

Update: I have read the reviewer's response and I believe their proposed corrections makes the main result correct. Therefore, I now strongly support accepting this paper. I would be quite interested in seeing high-probability analysis that shows  how (with subgaussian assumptions) to avoid the need to randomly pick a past iterate, which seems to be something no sane method would do.  ---  The paper proposes a stochastic optimization scheme for smooth nonconvex functions. The scheme combines fixed-norm stochastic gradient steps with a novel approach to unbiased gradient estimation, based on summing estimates of the increments in the gradient and periodically resetting the summation using an accurate gradient estimate. The paper provides an analysis of this scheme, claiming rates of convergence to stationarity that improve upon the best known rates for both the fully stochastic and the finite-sum settings. Additionally, the algorithm is extended to guarantee convergence to approximate second-order stationary points, and a matching lower bound is given in the finite-sum setting.  In my opinion the proposed algorithm is original and elegant, and --- if the claimed rates of convergence were proven --- would make an excellent contribution to the active line of work on efficient scalable algorithms for nonconvex optimization. Unfortunately, I believe that the proof of the main result in the paper has fairly significant gaps, and cannot be accepted in its current state.  The above-mentioned gaps stem from incorrect statements on conditional expectations. In Lemma 6, everything is conditional on the event \|v^k\| \ge 2\epsilon. Hence, the transition from Eq. (25) to Eq. (26) is based on the claim \E [ \|v^k - \nabla f(x^k) \|^2 | \|v^k\| \ge 2\epsilon ] \le \epsilon^2. However, Lemma 2 does not, and in all likelihood cannot, establish such bound on the above conditional expectation. This issue repeats in Lemma 7, where in Eq. (28) the authors (implicitly) assert that  \E [ \|v^k - \nabla f(x^k) \|^2 | \|v^k\| \le 2\epsilon ] \le \epsilon^2, which is just as problematic. Worse, when Lemma 6 is applied in the proof of Theorem 1, the expectation bound is actually stated conditional on the event {\|v^i\| \ge 2\epsilon for every i \in [K]}, so a valid proof of Lemma 6 would need to show \E [ \|v^k - \nabla f(x^k) \|^2 | \|v^i\| \ge 2\epsilon, i=1,...,K ] \le \epsilon^2, which is very unlikely to be true or provable. A similar problem exists with the application of Lemma 7.  I encourage the authors to try and fix their analysis: in my opinion doing so will be challenging but by no means a lost cause. However, some structural changes in the algorithm are likely necessary. Namely, the algorithm in its current form promises small gradient (in expectation) at its last iterate, while all other comparable methods only give a guarantee on a randomly sampled iterate --- it is likely that a correct  version of the proposed method will have to do that as well. One possible approach to sidestepping difficulties with conditioning would be to add a sub-Gaussianity assumption on the noise and appeal to martingale concentration results.   Below are two additional comments: - What about the important special case where f is convex? Does the method attain the optimal eps^{-2} convergence rate to eps-approximate global minimia? What happens when f is non-smooth? - The parameter n_0 doesn't represent a batch size, and so its name is somewhat confusing. I think it would be better to leave \mc{S}_2 as a free parameter (ranging from 1 to 2\sigma/\epsilon or n^{1/2} in the stochastic and finite-sum cases respectively). Then we have q = \mc{S}_1 / \mc{S}_2 which I think is much clearer and more intuitive.


Inferring Latent Velocities from Weather Radar Data using Gaussian Processes
Archived data from the US network of weather radars hold detailed information about bird migration over the last 25 years, including very high-resolution partial measurements of velocity. Historically, most of this spatial resolution is discarded and velocities are summarized at a very small number of locations due to modeling and algorithmic limitations. This paper presents a Gaussian process (GP) model to reconstruct high-resolution full velocity fields across the entire US. The GP faithfully models all aspects of the problem in a single joint framework, including spatially random velocities, partial velocity measurements, station-specific geometries, measurement noise, and an ambiguity known as aliasing. We develop fast inference algorithms based on the FFT; to do so, we employ a creative use of Laplace's method to sidestep the fact that the kernel of the joint process is non-stationary.

-- Paper Summary  The primary challenges posed by the problem considered in this paper arise from three main issues, namely 1) the data may be aliased, 2) only the radial velocity is measured (while the full velocity has to be inferred) and 3) the data is generally linked to a specific weather station. While some of these issues have been previously addressed, the paper delivers a solution which jointly tackles the three issues. This proposal builds upon the KISS-GP framework, while also developing an efficient way to carry out Laplace inference when incorporating a wrapped normal likelihood. While it was previously common practice to subsample the data, this approach can handle the full dataset (500k points) without compromising on speed. This is particularly important since the kernel is non-stationary, where standard inference techniques are not always applicable.    -- Originality + Significance  While applied papers do feature at NIPS, I believe that the theoretic contributions developed in this paper are only of minor significance to the general community, which tends to value significant algorithmic contributions above advances in applied areas. The generality of the proposed improvements to existing GP methods also lose their impact due to the paper’s focus on a single dataset.  While there is some emphasis on the Laplace method being used to carry out exact inference, the authors then claim that this is no longer possible when incorporating the wrapped normal likelihood. The novelty of the Laplace computation is also unclear to me. Similar work was carried out in relation to grid-structured inputs and conjugate gradient for Laplace in ‘Fast Kronecker Inference in Gaussian Processes with non-Gaussian likelihoods (Flaxman et al, 2015)’ and ‘Preconditioning Kernel Matrices (Cutajar et al, 2016)’, but these papers are not discussed or cited here.  The overall scope of this work is also limited to a particular problem and its associated data. While the conclusion indicates that there may in fact be other similar problems which are worth investigating (e.g. inferring wind velocity from precipitation or wind particles), it is not immediately clear whether the model developed in this paper can be similarly successful in these other instances.   -- Quality  The derivations put forward in the paper appear to be correct, drawing from results which were previously established in the literature on structured kernel interpolation for Gaussian processes. The superior performance of the method is not altogether surprising given the success of its intermediate components showcased in other papers. The impact of using pseudo-observations in order to fill up incomplete grids should however be elaborated by at least re-iterating the motivations given in the original KISS-GP paper.  The results obtained using the proposed method are very promising in comparison with other naïve GP approaches. The running time comparison is also very useful in validating the authors’ intuition that the increase in predictive performance does not incur a penalty in speed. There is however very little context for how successful the results are beyond the narrow GP setting. The “6 different methods” highlighted in L234 are only the proposed method and the naïve GP method with 5 different training set sizes. As alluded to before, the performance improvements enabled by this proposed method are expected given their success in other domains.  Although the temporal aspect of this problem is not considered in the paper (even though it is implied in the introductory comment that such analysis could answer questions “over more than two decades”), a brief investigation into INLA methods may also be warranted. Scalable deep GPs may also be useful for modelling non-stationarity behaviour.  Finally, while I understand that such data may not be readily available, I think it would be beneficial to have additional results for the model on similarly structured datasets having similar aliasing issues for example. I would expect similarly good results on other such spatio-temporal problems, but it would be best if this could also be supplemented with actual experiments.   -- Writing and Clarity  I cannot fault the writing of this paper – it is extremely well written with nary a typo to be found (I only spotted L62). The problem is adequately set up, and the individual components of the model design are well-motivated and described in an orderly manner. The paper is also sufficiently detailed to properly understand without having to rely on the supplementary material. The notation is easy to follow and interpret.   -- Overall Recommendation  I appreciate that having an end-to-end scalable model which takes partial measurements as input and successfully models actual wind velocities is useful for specific problems and datasets in this particular domain. This is evidenced by the model’s success compared to standard GP models.  Nevertheless, even though the problem is indeed interesting and the paper is in a state fit for publication, I do not think NIPS is the ideal venue unless a wider variety of problems or data settings are explored. Essentially, the minor theoretic contributions must be compensated for by a wider and more comprehensive set of experiments.  -------------  -- Post-rebuttal  I thank the authors for replying to my comments. After reading the rebuttal and other reviews, I have bumped up my score from 4 to 6 in order to better reflect its quality as an application track paper. Nonetheless, for other reasons mentioned in the full-review, I do not feel inclined to argue more strongly for its acceptance.


Conditional Adversarial Domain Adaptation
Adversarial learning has been embedded into deep networks to learn disentangled and transferable representations for domain adaptation. Existing adversarial domain adaptation methods may not effectively align different domains of multimodal distributions native in classification problems. In this paper, we present conditional adversarial domain adaptation, a principled framework that conditions the adversarial adaptation models on discriminative information conveyed in the classifier predictions. Conditional domain adversarial networks (CDANs) are designed with two novel conditioning strategies: multilinear conditioning that captures the crosscovariance between feature representations and classifier predictions to improve the discriminability, and entropy conditioning that controls the uncertainty of classifier predictions to guarantee the transferability. With theoretical guarantees and a few lines of codes, the approach has exceeded state-of-the-art results on five datasets.
Despite their general efficacy for various tasks ranging from classification [12,51,28] to segmentation [43,50,22], these adversarial domain adaptation methods may still be constrained by two bottlenecks. First, when data distributions embody complex multimodal structures, adversarial adaptation methods may fail to capture such multimodal structures for a discriminative alignment of distributions without mode mismatch. Such a risk comes from the equilibrium challenge of adversarial learning in that even if the discriminator is fully confused, we have no guarantee that two distributions are sufficiently similar [3]. Note that this risk cannot be tackled by aligning distributions of features and classes via separate domain discriminators as [23,8,50], since the multimodal structures can only be captured sufficiently by the cross-covariance dependency between the features and classes [47,44]. Second, it is risky to condition the domain discriminator on the discriminative information when it is uncertain.
In this paper, we tackle the two aforementioned challenges by formalizing a conditional adversarial domain adaptation framework. Recent advances in the Conditional Generative Adversarial Networks (CGANs) [34,35] disclose that the distributions of real and generated images can be made similar by conditioning the generator and discriminator on discriminative information. Motivated by the conditioning insight, this paper presents Conditional Domain Adversarial Networks (CDANs) to exploit discriminative information conveyed in the classifier predictions to assist adversarial adaptation. The key to the CDAN models is a novel conditional domain discriminator conditioned on the crosscovariance of domain-specific feature representations and classifier predictions. We further condition the domain discriminator on the uncertainty of classifier predictions, prioritizing the discriminator on easy-to-transfer examples. The overall system can be solved in linear-time through back-propagation. Based on the domain adaptation theory [4], we give a theoretical guarantee on the generalization error bound. Experiments show that our models exceed state-of-the-art results on five benchmark datasets.

In this paper, the authors proposed a conditional adversarial domain adaptation framework, by considering a conditional discriminator which takes the multilinear maps of feature and network prediction as input. In addition, randomized multilinear conditioning was considered to reduce the input dimension with comparable performance.  Overall, the presentation is clear and easy to follow. The proposed method is technically correct. However, I am leaning towards rejection since the contribution and novelty of this paper is limited. Having multimodal distributions in both source and target domains is a well-known challenge in domain adaptation. This paper is clearly not the first work to explicitly or implicitly address such challenge under a conditional adversarial framework. There are multiple previous related works, such as: Hoffman et al., FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation, arXiv16 Chen et al., No More Discrimination: Cross City Adaptation of Road Scene Segmenters, ICCV17 Tsai et al., Learning to Adapt Structured Output Space for Semantic Segmentation, CVPR18 In particular, the second work leverages network prediction for class-wise domain alignment, while the third work imposed discriminators on both features and network predictions. Both previous work share certain similarity to this work in terms of incorporating conditional information in the discriminator. However, none of the above works were cited or compared in this paper.  The proposal of multilinear conditioning and randomized multilinear conditioning is a novel part of this work but the novelty/contribution doesn't seem enough for NIPS. In addition, this part lacks more principled theoretical analysis/ablation study why the proposed methods are better than other types of conditional input to discriminator.  Experiment of this work lacks comparison to multiple state-of-the-arts and GAN baselines on unsupervised domain adaptation, including: Hoffman et al., CyCADA: Cycle-Consistent Adversarial Domain Adaptation, ICML18 Zhu et al., Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks, ICCV17 Liu et al., Unsupervised Image-to-Image Translation Networks, NIPS17 The authors could have conducted UDA experiments on several digit datasets following the above works.  ================================================================================================ Updates:  I have carefully looked into the rebuttal, the other reviews, and the draft paper (again). I understand the authors have put in good effort trying to convince me, which I appreciate very much. However, for now my overall view towards the paper remains similar to my initial decision. I want to be clear that I have no problem with the technical contents of the paper. To me the proposed method makes sense and seems technically correct. My major concern lies in that: If the paper is accepted to NIPS, it just makes another mediocre work that does not distinguish itself from the abundant closely related literature with clear insights. Here are my reasons:  Within the context of this paper, one can decompose the contributions into two perspectives: 1. Discovering the fact that conditional information is used in adversarial training can help the UDA problem 2. Presenting a novel way how conditional information participates in adversarial training, and theoretically/empirically show why this is better than other proposed ways of incorporating conditional information participates.  I understand the importance of using conditional information in adversarial training based UDA. In fact many previous work have similarly discovered the multi-modal distribution issue and accordingly presented different ways to incorporate conditional information. One big problem with this paper is that it is written in such as way that one who is not an expert in this field is likely to feel that this work has contribution in item 1. An example is the way how the introduction, and related work are written, where the authors claim they get the inspiration from conditional generative methods without mentioning mentioning other conditional UDA methods at all. The title of this paper also strengthened such impression.  I understand the authors argued by emphasizing that their definition of "conditional adversarial" is actually based on item 2 not item 1 (the fact that both class label and feature are jointly taken as input to the same discriminator). However, in the original submission the authors mentioned that "Motivated by the conditioning insight, this paper presents Conditional Domain Adversarial Networks (CDAN) to exploit discriminative information conveyed in the classifier predictions to inform adversarial adaptation." There is no way one can be certain the definition is based on item 2. Such definition seems to fit perfectly to other conditional UDA methods as well.  Finally, Assuming that the authors resolves the above issues by completely rewriting the introduction and related work, current form of this work is not enough to support item 2 even if the rebuttals are taken into consideration. The authors have shown HOW this method is different from other conditional UDA methods. However, the theoretical insights WHY this method is better over others seems below the NIPS standard. The only solid theoretical analysis falls on proving the good approximation of multilinear mapping by inner product, of which similar theories (unbias estimator) were also well-established by previous random feature methods. On the empirical side, the authors did not show any segmentation applications where considerable number of conditional UDA methods were proposed. The performance gain from the proposed multilinear conditional adversarial module (which is the most important claim of contribution in this paper) over other similar works seems marginal if the priority loss is not included. It is also unclear how the proposed method performs on the digits experiments without priority loss. In addition, details on the encoder architecture (ResNet-50? AlexNet? Others?) are missing too. It is not surprising these factors could all contribute to the performance gain, and currently the only information one has is that the authors achieved good system-level results with the help of priority loss.  Besides the above concerns, for reference [1] in the rebuttal, the fact that the generation network jointly takes both classifier prediction and the feature as input to generate images that are further used as input to the discriminator also seems to fall into the author's definition of "conditional adversarial". The claim in the paper: "Towards adversarial learning for domain adaptation, only unconditional ones have been exploited while conditional ones remain unexplored." is therefore wrong.  I think the authors showed some improvements on the tasks at system level. For this I decide to increase my overall score to 5. However, my feeling is that the authors' response can not convince me much on the above concerns.


Insights on representational similarity in neural networks with canonical correlation
Comparing different neural network representations and determining how representations evolve over time remain challenging open questions in our understanding of the function of neural networks. Comparing representations in neural networks is fundamentally difficult as the structure of representations varies greatly, even across groups of networks trained on identical tasks, and over the course of training. Here, we develop projection weighted CCA (Canonical Correlation Analysis) as a tool for understanding neural networks, building off of SVCCA, a recently proposed method [22]. We first improve the core method, showing how to differentiate between signal and noise, and then apply this technique to compare across a group of CNNs, demonstrating that networks which generalize converge to more similar representations than networks which memorize, that wider networks converge to more similar solutions than narrow networks, and that trained networks with identical topology but different learning rates converge to distinct clusters with diverse representations. We also investigate the representational dynamics of RNNs, across both training and sequential timesteps, finding that RNNs converge in a bottom-up pattern over the course of training and that the hidden state is highly variable over the course of a sequence, even when accounting for linear transforms. Together, these results provide new insights into the function of CNNs and RNNs, and demonstrate the utility of using CCA to understand representations.

Edit after rebuttal: I thank the authors for addressing my concerns and provide further details, and I updated my overall score to vote that this paper should be accepted.  Summary: =======  The authors present an analysis of representations learned by neural networks using Canonical Correlation Analysis, building and expanding on the SVCCA paper[1]. I think the paper offered some interesting insights, that are likely of interest to the wider community, thus I am generally in favor of accepting this submission. But I am unsure about the clarity of the paper. Should these be addressed in the rebuttal, I am happy to update my score.  The authors first introduce "projection weighted CCA", their work-horse for the analysis. In my opinion, this is the weakest point of the paper. The authors often refer to [1] as the paper they build on, whose authors used SVD on the representation vectors to get rid of noise directions, and then used CCA to align representations of different networks. The authors here omitted the pruning-noise-via-SVD step, and instead weight the projection-directions to reduce noise. It is unclear to me why this change in methodology was needed -- if noise-directions within each layer's activation are reduced via SVD, then maybe it would not be necessary to down-weight uninformative CCA directions. I suggest the author expand on how their method of analysis differs from [1], and why they needed these changes.  My 2nd issue with the paper is that it gives very little details on experiments. It is impossible to reproduce the experiments in this paper, since there are too many details omitted. The authors do not specify the arcitectures of the networks they used, which would be crucial to better put the results into context. Furthermore, the experiments that lead to figure 2 are only rudimentarily described. I strongly suggest the authors add more details to make experiments reproducible (in the appendix, if space is an issue). I would also encourage the authors to provide the source code of their experiments to improve reproducability (though of course this is not a must).   Minor comments: ============== - Section 3.1 speculates that "solutions found by memorizing networks were as diverse as those found across entirely different dataset labelings".  It would be interesting (and presumably fairly easy for the authors) to test this instead of just speculating about it.  - I'm not convinced about Figure 4b: On the one hand, larger networks are more similar (fig 4a), on the other hand, larger networks achieve better accuracy. Isn't figure 4b simply a result of these two facts? I.e., there is a latent variable "network size" that influences both performance and CCA distance. One would need to eliminate the dependance on this latent variable to see how good the connection between CCA distance & performance is. It would be more convincing if more networks of the same size (but different test accuracy) would be plotted, instead of varying the size.  - The authors often use terms which are formally introduced only much later. E.g. the term "CCA vectors" is defined at line 113, but is first used from line 87 onwards.  - Equation (**) should be offset from the main text so it's easier to see (the authors later refer back to this equation, and it's hard to find )  - In the 3rd paragraph of Section 3.1, the fact that "the same set of random labels was used for all networks" is repeated in the next sentence as "the randomization of CIFAR 10 labels was consistent for all networks". Why is this mentioned twice right after each other. Do these mean different things?     References: ==========  [1]  Raghu et al, "SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability", NIPS 2017


Neural Ordinary Differential Equations
We introduce a new family of deep neural network models. Instead of specifying a discrete sequence of hidden layers, we parameterize the derivative of the hidden state using a neural network. The output of the network is computed using a blackbox differential equation solver. These continuous-depth models have constant memory cost, adapt their evaluation strategy to each input, and can explicitly trade numerical precision for speed. We demonstrate these properties in continuous-depth residual networks and continuous-time latent variable models. We also construct continuous normalizing flows, a generative model that can train by maximum likelihood, without partitioning or ordering the data dimensions. For training, we show how to scalably backpropagate through any ODE solver, without access to its internal operations. This allows end-to-end training of ODEs within larger models.

# Response to author feedback  My thanks to the authors for their responses to my comments and questions in their feedback and commitment to make several clarifications in response to the suggestions made. After reading their feedback and the other reviewer's comments I maintain my original assessment and feel this is an excellent submission that I'd strongly argue for including in the conference.  ---  # Summary and relation to previous work  This submission makes several related contributions:  * it provides a clear and readable introduction to the adjoint sensitivity methods for reverse-mode automatic differentiation (AD) through ODE solvers,  * motivated by previous links drawn between Res-Nets and ODE models (Lu, Zhong, Li & Dong, 2017), the direct use of ODE solvers as differentiable (and so learnable) components in machine learning models is proposed, with such 'ODE-Nets' offering advantages of improved memory efficiency during gradient calculation and adpative effective network depth,  * continuous normalising flows (CNFs) are proposed as an analogue to previous work on flow models e.g. normalising flows (Rezende & Mohamed, 2015) and NICE (Dinh, Krueger and Bengio, 2015), with CNFs claimed to provided the advantage of allowing more expressive flows while maintaining computational tractability and efficient reversibility of the flow (compared to normalising flows),   * a generative latent variable ODE-based time series model (L-ODE) trained with a variational objective is proposed as an alternative to standard recurrent neural network approaches, with more natural handling of irregularly time-sampled sequences.  # Strengths and weaknesses  Although the adjoint sensitivity method is an existing method, exposing this method to machine learning and computational statistics communities where, as far as I am aware it is not widely known about, is a worthwile contribution of this submission its own right. Given the ever increasing importance of AD in both communities, adding to the range of scientific computing primitives for which frameworks such as autograd can efficiently compute derivatives through will hopefully spur more widespread use of gradient based learning and inference methods with ODE models and hopefully spur other frameworks with AD capability in the community such as Stan, TensorFlow and Pytorch to implement adjoint sensitivity methods.  The specific suggested applications of the 'ODE solver modelling primitive' in ODE-Nets, CNFs and L-ODEs are all interesting demonstrations of some of the computational and modelling advantages that come from using a continuous-time ODE mode; formulation, with in particular the memory savings possible by avoiding the need to compute all intermediate states by recomputing trajectories backwards through time being a possible major gain given that device memory is often currently a bottleneck. While 'reversing' the integration to recompute the reverse trajectory is an appealing idea, it would have helped to have more discussion of when this would be expected to breakdown - for example it seems likely that highly chaotic dynamical systems would tend to be problematic as even small errors in the initial backwards steps could soon lead to very large divergences in the reversed trajectories compared to the forward ones. It seems like a useful sanity check in an implementation would be to compare the final state of the reversed trajectory to the initial state of the forward trajectory to check how closely they agree.  The submission is generally very well written and presented with a clear expository style, with useful illustrative examples given in the experiments to support the claims made and well thought out figures which help to give visual intuitions about the methods and results. There is a lot of interesting commentary and ideas in the submission with there seeming to be a lot of potential in even side notes like the concurrent mixutre of dynamics idea. While this makes for an interesting and thought-provoking read, the content-heavy nature of the paper and slightly rambling exploration of many ideas are perhaps not ideally suited to such a short conference paper format, with the space constraints meaning sacrifices have been made in terms of the depth of discussion of each idea, somewhat terse description of the methods and results in some of the experiments and in some cases quite cramped figure layouts. It might be better to cull some of the content or move it to an appendix to make the main text more focussed and to allow more detailed discussion of the remaining areas.  A more significant weakness perhaps is a lack of empirical demonstrations on larger benchmark problems for either the ODE-Nets or CNFs to see how / if the proposed advantages over Res-Nets and NFs respectively carry over to (slightly) more realistic settings, for example using the CNF in a VAE image model on MNIST / CIFAR-10 as in the final experiments in original NF paper. Although I don't think such experiments are vital given the current numerical experiments do provide some validation of the claims already and more pragmatically given that the submission already is quite content heavy already so space is a constaint, some form of larger scale experiments would make a nice addition to an already strong contribution.  # Questions   * Is the proposal to backward integrate the original ODE to allow access to the (time-reversed) trajectory when solving the adjoint ODE rather than storing the forward trajectory novel or is this the standard approach in implementations of the method?  * Does the ResNet in the experiments in section 7.1 share parameters between the residual blocks? If not a potentially further interesting baseline for the would be to compare to a deeper ResNet with parameter sharing as this would seem to be equivalent to an ODE net with a fixed time-step Euler integrator.  * What is the definition used for the numerical error on the vertical axis in Figure 4a and how is the 'truth' evaluated?  * How much increase in computation time (if any) is there in computing the gradient of a scalar loss based on the output of `odeint` compared to evaluating the scalar loss itself using your Python `autograd` implementation in Appendix C? It seems possible that the multiple sequential calls to the `odeint` function between pairs of successive time points when calculating the gradient may introduce a lot of overhead compared to a single call to `odeint` when calculating the loss itself even if the overall number of inner integration steps is similar? Though even if this is the case this could presumably be overcome with a more efficient implementation it would be interesting to get a ballpark for how quick the gradient calculation is currently.  # Minor comments / suggestions / typos:  Theorem 1 and proof in appendix B: while the derivation of this result in the appendix is nice to have as an additional intuition for how the expression arises, it seems this might be more succinctly seen as direct implication of the Fokker-Planck equation for a zero noise process or equivalently from Liouville's equation (see e.g. Stochastic Methods 4th ed., Gardiner, pg. 54). Similarly expressing in terms of the time derivative of the density rather than log density would perhaps make the analogy to the standard change of variables formula more explicit.  Equation following line 170: missing right parenthesis in integrand of last integral.  Algorithm 1: Looks like a notation change might have lead to some inconsistencies - terms subscripted with non-defined $N$: $s_N$, $t_N$, look like they would be more consistent if instead subscripted with 1. Also on first line vector 0 and scalar 0 are swapped in $s_N$ expression  References: some incorrect capitalisation in titles and inconsistent used of initials rather than full first names in some references.  Appendices  L372: 'differentiate by parts' -> 'integrate by parts'  L385: Unclear what is meant by 'the function $j(x,\theta,t)$ is unknown - for the standard case of a loss based on a sum of loss terms each depending on state at a finite set of time points, can we not express $j$ as something like  $$ j(x,\theta,t) = \sum_{k=1}^K \delta(t_k - t) \ell_k(x, \theta) $$  which we can take partial derivatives of wrt $x$ and then directly subsitute into equation (19)?   L394: 'adjoing' -> 'adjoint'


Gaussian Process Conditional Density Estimation
Conditional Density Estimation (CDE) models deal with estimating conditional distributions. The conditions imposed on the distribution are the inputs of the model. CDE is a challenging task as there is a fundamental trade-off between model complexity, representational capacity and overfitting. In this work, we propose to extend the model's input with latent variables and use Gaussian processes (GPs) to map this augmented input onto samples from the conditional distribution. Our Bayesian approach allows for the modeling of small datasets, but we also provide the machinery for it to be applied to big data using stochastic variational inference. Our approach can be used to model densities even in sparse data regions, and allows for sharing learned structure between conditions. We illustrate the effectiveness and wide-reaching applicability of our model on a variety of real-world problems, such as spatio-temporal density estimation of taxi drop-offs, non-Gaussian noise modeling, and few-shot learning on omniglot images.

Edit after the rebuttal period: I have read the other reviews and the author feedback. My evaluation and rating about this submission will remain unchanged.  Summary: The paper considers approximate inference and learning for the Gaussian process latent variables model in which some latent variables are observed. An alternative view of the model considered here is Gaussian process regression in which some dimensions of the inputs are unobserved. Variational inference based on pseudo-points is used. The posterior over the pseudo-points are approximately integrated out by quadrature or approximated by a Gaussian parameterised by a recognition model. Natural gradients and linear input and output warpings are also considered. Extensive experiments on density estimation and regression were provided to support the proposed inference scheme.  Comments:  The contribution of this work is, in my opinion, a well-executed framework for inference and learning in Gaussian process latent variable models with some latent variables being known beforehand. I think the contributions listed at the end of section 1 are incremental, as i. linear input and output warpings have been considered before in multiple contexts, for example: multi-task learning, ii. natural gradients for the variational approximation over the non-linear function have been considered before, e.g. Hensman et al's GP for big data paper. In addition, recognition models have been used before to parameterise the variational distribution in the GP latent variable model, and similar models have been considered before -- see the list at the end of the review. Having said this, this submission is well-structured and is an interesting and, perhaps, non-trivial combination of all these things that seems to work well.  re analytically optimal q(w): I agree this approach could be useful for low-dimensional w and there will be less variational parameters to deal with [and this could perform well as in the regression experiment considered], but I'm pedantically unclear about the bound being tighter here. The gaps between the bound and the true marginal likelihood are: in the free-form q(w) case: log \int dw p(w) \exp [ KL(q(f) || p(f|w, x, y)) ] in the fixed-form case: KL( q(f)q(w) || p(f, w|x, y) ) Is this true that the gap in the first case above is always smaller than that in the second case?  Some questions about the experiments: + linear model baselines should be included in the density modelling tasks + taxi trip density estimation: the test set has 4 folds with 50 taxi trips each -- this seems rather small compared to the whole dataset of 1.5 mil trips. + few shot learning on omniglot: the generated images are not very crisp. Is there some quantitative or qualitative comparison to alternative models here, e.g. VAE? + regression task: this seems like the only task the proposed model + inference technique have the competitive edge on average. Is this because the number of latent/input dimensions is small and no input + output warpings are used? The additional variational approximation for the input warping could make things worse in other tasks. + MNIST density estimation: why not use the full binarised MNIST as often considered in generative modelling literature? It is hard to judge the number provided table 2 as they are very different to what often reported. The Gaussian likelihood could be straightforwardly replaced by another likelihood for binary outputs, right? I'm also slightly puzzled by table 2 and the full table in the appendix: why is the first row for CVAE: sigma^2 optimised made bold? For fixed variances, GP-CDE is like 100-500 nats better than CVAE but for optimised variance, GP-CDE is, disappointedly, way worse compared to CVAE? Could you please enlighten me on this: perhaps the capacity of the GP-CDE model is not great, or is that limited by the inference scheme? Would CVAE + early stopping work much better than GP-CDE overall then?  Clarity: The paper is very well-written and the experiments seem to be well thought out.  Perhaps the following works should be mentioned and discussed in details, as they are closely related to this submission:  Lawrence, Neil D., and Joaquin Quiñonero-Candela. "Local distance preservation in the GP-LVM through back constraints", ICML 2006 -- this was briefly mentioned, but the idea of using recognition models for GP latent variables was first introduced in this paper, albeit only for MAP.  Thang Bui and Richard Turner, "Stochastic variational inference for Gaussian process latent variable models using back constraints", NIPS workshop on Black Box Learning and Inference, 2015 --- this paper uses recognition models for GP latent variable models  Zhenwen Dai, Andreas Damianou, Javier González, Neil Lawrence, "Variational Auto-encoded Deep Gaussian Processes", ICLR 2016 -- using structured recognition models for deep GPs.  A. Damianou and N. Lawrence, "Semi-described and semi-supervised learning with Gaussian processes", UAI 2015 --- This paper is very related to the submission here, though the set up is slightly different: some inputs are observed and some inputs are unobserved/missing and treated as latent variables.  Depeweg S., Hernández-Lobato J. M., Doshi-Velez F. and Udluft S., "Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networks", ICLR 2017 --- the set up in this paper is very similar to what proposed here, except that Bayesian neural networks were considered instead of GPs


Improved Expressivity Through Dendritic Neural Networks
A typical biological neuron, such as a pyramidal neuron of the neocortex, receives thousands of afferent synaptic inputs on its dendrite tree and sends the efferent axonal output downstream. In typical artificial neural networks, dendrite trees are modeled as linear structures that funnel weighted synaptic inputs to the cell bodies. However, numerous experimental and theoretical studies have shown that dendritic arbors are far more than simple linear accumulators. That is, synaptic inputs can actively modulate their neighboring synaptic activities; therefore, the dendritic structures are highly nonlinear. In this study, we model such local nonlinearity of dendritic trees with our dendritic neural network (DENN) structure and apply this structure to typical machine learning tasks. Equipped with localized nonlinearities, DENNs can attain greater model expressivity than regular neural networks while maintaining efficient network inference. Such strength is evidenced by the increased fitting power when we train DENNs with supervised machine learning tasks. We also empirically show that the locality structure of DENNs can improve the generalization performance, as exemplified by DENNs outranking naive deep neural network architectures when tested on classification tasks from the UCI machine learning repository.

UPDATED REVIEW: the authors responded to most comments. Although they provided details in the response, It's not clear to me if they will include discussions of (1) piecewise activation functions and in neuroscience e.g. two-compartment models in the related work (2) generalization (+learning curves) in the manuscript; I think both these things would improve the work. I still find it strange that no generalization results were presented in the original submission.  I agree with the other reviewers that the title sounds strange (typically you're advised not to use subjective descriptions like "great" or "amazing" in scientific work). I have updated my rating to 6.  ========================  PAPER SUMMARY:The authors propose a network architecture to model synaptic plasticity in dendritic arbours, which ends up looking to me like alternating layers of maxout and regular units, with some randomly-dropped connections. The proposed architecture is tested on MNIST, CIFAR, and a range of UCI classification tasks, and found to outperform alternatives. Insight about why this is the case is provided in the form of an analysis of expressivity, measured as number of linear regions in loss space.  STRENGTHS:   - quality: The paper is generally well-written and the structure and experiments are largely well-suited to evaluate the proposed architecture. The introduction and related work do a good job contextualizing the work. The experiments about expressivity are interesting and relevant.   - clarity: What is proposed and what is done are both clear. Details of datasets, experiments, and explanations of related work are generally very good.   - originality: To my knowledge, the proposed architecture is novel. Being somewhat familiar with the biology, I find piecewise linear/max operations a reasonable approximation intuitively.   WEAKNESSES:   - quality: It seems to me that the chosen "algorithm" for choosing dendrite synapses is very much like dropout with a fixed mask. Introducing this sparsity is a form of regularization, and a more fair comparison would be to do a similar regularization for the feed-forward nets (e.g. dropout, instead of bn/ln; for small networks like this as far as I know bn/ln are more helpful for optimization than regularization). It also seems to me that the proposed structure is very much like alternating layers of maxout and regular units, with this random-fixed dropout; I think this would be worth comparing to.  I think there are some references missing, in the area of similar/relevant neuroscience models and in the area of learned piecewise activation functions.  It would be reassuring to mention the computation time required and whether this differs from standard ff nets. Also, most notably, there are no accuracy results presented, no val/test results, and no mention is made of generalization performance for the MNIST/CIFAR experiments.    - clarity:  Some of the sentences are oddly constructed, long, or contain minor spelling and grammar errors .The manuscript should be further proofread for these issues. For readers not familiar with the biological language, it would be helpful to have a diagram of a neuron/dendritic arbour; in the appendix if necessary. It was not 100% clear to me from the explanations whether or not the networks compared have the same numbers of parameters; this seems like an important point to confirm.   - significance: I find it hard to assess the significance without generalizaion/accuracy results for the MNIST/CIFAR experiments.   REPRODUCABILITY: For the most part the experiments and hyperparameters are well-explained (some specific comments below), and I would hope the authors would make their code available.  SPECIFIC COMMENTS:   - in the abstract, I think it should say something like "...attain greater expressivity, as measured by the change in linear regions in output space after [citation]. " instead of just "attain greater expressivity"   - it would be nice to see learning curves for all experiments, at least in an appendix.    - in Figure 1, it would be very helpful to show a FNN and D-Net with the same number of parameters in each (unless I misunderstood, the FNN has 20 and the DNN has 16).   - There are some "For the rest part" -> for the rest of (or rephrase)   - missing references: instead of having a section just about Maxout networks, I think the related work should have a section called something like "learned piecewise-linear activation functions" which includes maxout and other works in this category, e.g. Noisy Activation Functions (Gulcehre 2016). Also, it's not really my field but I believe there is some work on two-compartment models in neuroscience and modeling these as deep nets which would be quite relevant for this work.   - It always bothers me somewhat when people refer to the brain as 'sparse' and use this as a justification for sparse neural networks. Yes, overall/considering all neurons the brain as one network it would be sparse, but building a 1000 unit network to do classification is much more analogous to a functional subunit of the brain (e.g. a subsection of the visual pathway), and connections in these networks are frequently quite dense. The authors are not the first to make this argument and I am certainly not blaming them for its origin, but I take this opportunity to point it out as (I believe) flawed. :)   - the definition of "neuron transition" is not clear to me - the sentence before Definition 2 suggests that it is a change in _classification_ (output space), which leads to a switch in the linear region of a piecewise linear function, but the Definition and the subsequent sentence seem to imply it is only the latter part (a change from one linear region of the activation function to another; nothing to do with the output space). If the latter, it is not clear to me how/whether or not these "transitions" say anything useful about learning. If it is the former (more similar to Raghu et al), I find the definition given unclear.    - I like the expressiveness experiments, but It would be nice to see some actual numbers instead of just descriptions.   - unless I missed it somehow, the "SNN" is never defined. and it is very unclear to me whether it refers to a self-organizing neural network cited in [12]. or a "sparse" neural network, and in any case what exactly this architecture is.   - also possible I missed it despite looking, but I could not find what non-linearity is used on D-Nets for the non-dendrite units   OVERALL ASSESSMENT: My biggest issue with the paper is the lack of mention/results about generalization on MNIST/CIFAR, and the ambiguity about fair comparison. If these issues are resolved I would be very willing to change my rating.  CONFIDENCE IN MY SCORE: This is the first time I've given a confidence of 5. With due credit to the authors, I believe I've understood most things about the paper, and I am familiar with the relevant work. Of course I'm not infallible and it's possible I've missed or misunderstood something, especially relating to the things I noted finding unclear.


Learning Temporal Point Processes via Reinforcement Learning
Social goods, such as healthcare, smart city, and information networks, often produce ordered event data in continuous time. The generative processes of these event data can be very complex, requiring flexible models to capture their dynamics. Temporal point processes offer an elegant framework for modeling event data without discretizing the time. However, the existing maximum-likelihood-estimation (MLE) learning paradigm requires hand-crafting the intensity function beforehand and cannot directly monitor the goodness-of-fit of the estimated model in the process of training. To alleviate the risk of model-misspecification in MLE, we propose to generate samples from the generative model and monitor the quality of the samples in the process of training until the samples and the real data are indistinguishable. We take inspiration from reinforcement learning (RL) and treat the generation of each event as the action taken by a stochastic policy. We parameterize the policy as a flexible recurrent neural network and gradually improve the policy to mimic the observed event distribution. Since the reward function is unknown in this setting, we uncover an analytic and nonparametric form of the reward function using an inverse reinforcement learning formulation. This new RL framework allows us to derive an efficient policy gradient algorithm for learning flexible point process models, and we show that it performs well in both synthetic and real data.

The paper "Learning Temporal Point Processes via Reinforcement Learning" proposes a new way to lean temporal point processes, where the intensity function is defined via recurrent neural networks rather than classical parametric forms. This enables a better fitting with the true generative process. It builds on the WGAN approach but rather than dealing with a minimax optimization problem, the authors propose to use a RKHS  formalization to find an analytic maximum for the expected cumulative discrepancy between processes underlying observed sequences and generated ones.   The results look convincing and the followed inverse reinforcement learning approach  elegant, but I am missing some justification and clarifications w.r.t. simpler models. Authors claim that their way of learning allow them to define better specified models than parametric based ones. But they still use parametric sampling of time events (e.g., using an exponential distribution for the next event). It is unclear to me how this method would compete against a baseline which would simply maximize the log-likelihood of observed sequences using a similar rnn outputing parametrizations of  exponential distributions. Does this parametrization not cancel all the benefit resulting from considering the point process formalization ? For me the benefit from using Temporal point processes is that they allow one to define better specified temporal sequence distributions than using classical parametric models such as exponential laws parametrized according to the past. The problem is that they usually induce intractable likelihood functions and lead to models from which it is difficult to sample. But for me the proposed way to circumventing the problem reduces the model to classical ones... Authors clearly should add more discussion about this point.  .   Other remarks:        - What is the baseline used to reduce the variance in the policy gradient algorithm ? A moving average of the rewards ? Authors should detail this in the paper, since they only mention a baseline without defining it afterward.        - Would it be possible to consider a marked version were different kinds of events would be observed in the sequences ?


Asymptotic optimality of adaptive importance sampling
Adaptive importance sampling (AIS) uses past samples to update the sampling policy q t . Each stage t is formed with two steps : (i) to explore the space with n t points according to q t and (ii) to exploit the current amount of information to update the sampling policy. The very fundamental question raised in this paper concerns the behavior of empirical sums based on AIS. Without making any assumption on the allocation policy n t , the theory developed involves no restriction on the split of computational resources between the explore (i) and the exploit (ii) step. It is shown that AIS is asymptotically optimal : the asymptotic behavior of AIS is the same as some "oracle" strategy that knows the targeted sampling policy from the beginning. From a practical perspective, weighted AIS is introduced, a new method that allows to forget poor samples from early stages.

In this manuscript the authors consider adaptive importance sampling (AIS) schemes for estimating the integral of some target function \phi. To estimate such an integral a probability distribution (called sampling policy) generates an i.i.d. data sample and the integral is estimated by the empirical average of the function \phi weighted by the sampling policy. The key ingredient in adaptive sampling is to do multiple iterations of this procedure, updating the sampling procedure between each iteration to optimize the estimation.  The authors point to many convergence results regarding AIS but stress that these are in the case where the number of iterations is fixed while the number of samples taken per iteration goes to infinity. They argue that a more natural and general setting is where the total number of samples of all iterations goes to infinity, which includes the previous case as well as the case where the sample size per iteration stays fixed but the total number of iterations tends to infinity. In this setting the authors use Martingale techniques to prove a central limit theorem for both standard and normalized AIS. Both results are independent of the exact update strategy.   Next the authors consider a specific class of sampling policies, where the sampling policy is picked from a parametrized set. They prove that under certain mild assumptions, by applying risk minimization one obtains a collection of sampling policies that will converge to the optimal sampling policy. From this they then deduce a central limit theorem for parametric sampling policies, using risk minimization. Finally, a weighted update scheme for sampling policies is introduced. The weights enable the exclusion of previous bad updates when sampling the new data points. Using numerical experiments the authors show that such weighted AIS perform very well in practice as well as display their results for AIS. This paper is well-written with a clear overall structure, well defined objects and clearly stated results. In particular, I appreciate the fact that the authors make specific reference to the exact results from other papers they use for their proofs, which make the proof checking so much easier. I agree with the authors that the setting where the total number of samples tends to infinity is much more natural and hence I think their results are significant. I do however have a small problem with the proof of Theorem 5, since it assumes independence between variables U_i and U_j which does not seem to hold. In addition, I felt that the section on experiments was a bit short, which makes it hard for other researchers to duplicate the results. Apart from these two point I only have some small comments which I have listed below.  In summary, I think the result on adaptive importance sampling presented in this paper are relevant and new, and the numerical experiments clearly show that weighted AIS could improve estimations. Therefore I would recommend this paper for acceptance in NIPS. ADDED AFTER FEEDBACK: My main concern regarding the proof of Theorem 5 was properly addressed and the authors agreed to improve/expand the experiments section. My overall recommendation regarding this paper remains unchanged. Comments main text: Line 1, [policy of stage t]: Within the first six words you introduce a variable that is not defined. Please consider changing the text to be more high level. Line 8, [raised in the paper]: I would change this to: raised in this paper. Line 17, [choice of the sampling…]: I would write: choice of a sampling… Algorithm (AIS): This is just an abstract representation of an AIS scheme, since no explicit update scheme is provided. I would be helpful to the reader if this was stated explicitly in the text. Line 57, [is there a weighted…]: I would change this to: is then a weighted. Line 61, [based on parametric]: I would write: based on a parametric. Line 82, [inspired from the]: I would change this to: inspired by the. Algorithm AIS at sample scale: Again, this seems to be an abstract algorithm. Please specify this in the text. The for loop refers to i but the index in the rest of the algorithm is j. Which one is it? Also, you explain what x_j is but what about q_{j-1}? Line 106: Here you use I without ever specifying that it is \int \phi, which creates confusion with the reader. Please define this. Proposition 1 and 2: Given the short proofs and the fact that these are just simple technical statements I would recommend merging these into one lemma. Line 118: It would help the reader if you recall that \phi maps into \mathbb{R}^p. Line 130: I do not understand what you mean with the part: \int \phi \pi, not \int \phi. Could you explain this a bit more. Line 140: What does (\phi \pi, \pi) stand for? I thought we only needed to replace \phi with \phi \pi in (4) and (5). Section 3: Here there is a huge jump with respect to the previous part. In particular, \theta_t is never defined not mentioned in the previous text. This makes it very difficult for the reader to digest this part. Please provide a bit more introduction here. Expression below line 156: I thought we were using parametric sampling policies q_{\theta}. How are the q_{s-1} related to these? This is particularly important since they appear again in equation (10). Line 161: Here there is a typo, compute twice. Line 179-180: The expression given her for R_t(\theta) seems to be exactly the same as (10). Line 208: Please remind the reader that V(q,\phi) is defined as in (3). Line 230: How do (8) and (9) work in practice? I thought we did not know q^\ast. Could you please add more explanation so that other researcher can reproduce these experiments. Comments supplementary material: Line 13, [using (4)]: Please be explicit when referring to equations in the main text to avoid unnecessary confusion. Also it might help to provide a bit more details on the use of Cesaro, for instance state that a_n = \sum_{j = 1}^n (V(q_j,\phi) - V_\ast) and b_n = n. Derivation below line 18: At the end of the first line there should be absolute value bars around \frac{\phi(x)}{q_{j-1}(x)}. In addition the equation sign on the next line should be a \leq sign. Line 24, [apply Theorem 1]: This is very confusing since the main text contains no Theorem 1. Please be very specific that you are referring to results that you will discuss later in this text. Line 25-26, [(H1)…(H2)]: Similar to the above comment, please be very specific that both these refer to statement that appear later in the text. Or preferably, move section 2.2 before the proof of Theorem 5. Derivation below line 31: Here it seems that something is going wrong. The first equality sign is only true when U_i and U_j are independent, which I do not think they are since they involve the updated versions of the sampling policy which depends on previous samples.  Theorem 2: I do not follow the proof, especially the use of the n^2. From the fact that Var(S_n) \le cn it immediately follows that E[(\frac{S_n – E[S_n]}{n})^2] goes to zero as n \to \infty. This then immediately implies concentration and hence the result. Also, given that this result is fairly straightforward, I would recommend changing this to a lemma.