Group Sparse Additive Machine
A family of learning algorithms generated from additive models have attracted much attention recently for their flexibility and interpretability in high dimensional data analysis. Among them, learning models with grouped variables have shown competitive performance for prediction and variable selection. However, the previous works mainly focus on the least squares regression problem, not the classification task. Thus, it is desired to design the new additive classification model with variable selection capability for many real-world applications which focus on high-dimensional data classification. To address this challenging problem, in this paper, we investigate the classification with group sparse additive models in reproducing kernel Hilbert spaces. A novel classification method, called as group sparse additive machine (GroupSAM), is proposed to explore and utilize the structure information among the input variables. Generalization error bound is derived and proved by integrating the sample error analysis with empirical covering numbers and the hypothesis error estimate with the stepping stone technique. Our new bound shows that GroupSAM can achieve a satisfactory learning rate with polynomial decay. Experimental results on synthetic data and seven benchmark datasets consistently show the effectiveness of our new approach.

This paper presents a new additive classification model (GroupSAM) with grouped variables based on group based additive models in kernel Hilbert spaces, and an upper bound for the generalization error is provided in Theorem 1 to show that GroupSAM converges to the optimal error with a polynomial decay under certain conditions. Experiments are performed on synthetic data and seven UCI datasets that have been modified in order to get some group structure in the feature space. The results provided in the paper show that the proposed method is able to take into account grouped features while yielding good performance in terms of classification accuracy.
As far as I know, the paper presents a novel method for group sparse nonlinear classification with additive models. However, the degree of contribution of this paper with respect to
Yin, Junming, Xi Chen, and Eric P. Xing. "Group sparse additive models." Proceedings of the... International Conference on Machine Learning. International Conference on Machine Learning. Vol. 2012. NIH Public Access, 2012.
Is not clear to me as the framework presented in the above paper seems to be easy to extend to classification settings. In fact, Yin et al. apply their group sparse additive model for regression to address a classification task (classification of breast cancer data) by taking the sign of the predicted responses. On the other hand, I appreciate authors’ efforts in providing a theoretical analysis on the generalization error bound of GroupSAM and tis convergence.
Regarding the empirical results, I do not think that the experiments show the real significance of the proposed method as they real-world data has been “arbitrarily” tuned to enforce group structure. I think that the use of real data with group structure by itself is mandatory in order to properly measure the contribution of this work. For example, in bioinformatics in can be useful to build predictive models that take into account genes’ pathway information.  I also miss comparisons with other parametric approaches such as Group Lasso or the GroupSpSAM model adapted to the classification setting. Additionally, I cannot see a clear advantage with respect to Gaussian SVMs in terms of classification accuracy. I also would like to know whether the Gaussian kernel parameter was properly tuned.
Finally, I would like to know how multiclass classification was performed in the toy example as the GroupSAM method is formulated in Section 2 for binary classification problems.
Overall, the paper is well written and it has a well structure. However, I think that the introduction does not clearly present the differences between the proposed approach and other sparse additive models and group sparse models in parametric settings (e.g. Group Lasso).


Non-parametric Structured Output Networks
Deep neural networks (DNNs) and probabilistic graphical models (PGMs) are the two main tools for statistical modeling. While DNNs provide the ability to model rich and complex relationships between input and output variables, PGMs provide the ability to encode dependencies among the output variables themselves. End-to-end training methods for models with structured graphical dependencies on top of neural predictions have recently emerged as a principled way of combining these two paradigms. While these models have proven to be powerful in discriminative settings with discrete outputs, extensions to structured continuous spaces, as well as performing efficient inference in these spaces, are lacking. We propose non-parametric structured output networks (NSON), a modular approach that cleanly separates a non-parametric, structured posterior representation from a discriminative inference scheme but allows joint end-to-end training of both components. Our experiments evaluate the ability of NSONs to capture structured posterior densities (modeling) and to compute complex statistics of those densities (inference). We compare our model to output spaces of varying expressiveness and popular variational and sampling-based inference algorithms.

The paper proposes Non-parametric Neural Networks (N3) a method that combines advantages of deep models for learning strong relations between input and output variables with the capabilities of probabilistic graphical models at modeling relationships between the output variables.
Towards this goal, the proposed method is designed based on three components:
a) a deep neural network (DNN) which learns the parameters of local non-parametric distributions conditioned on the input variables, b) a non-parametric graphical model (NGM) which defines a graph structure on the local distributions considered by the DNN. and, c) a recurrent inference network (RIN), which is trained/used to perform inference on the structured space defined by the DNN+NGM.
The proposed method is sound, well motivated and each of its components are properly presented.
The method is evaluated covering a good set of baselines and an ablation study showing variants of the proposed method. The evaluation shows that state of the art results are achieved by the proposed method. However, due to its technical nature, there is a large number of variables and terms that complicate the reading at times. Still this is just a minor point.
Overall all the paper proposes a method with good potential in a variety of fields. However, one of my main concern is that its evaluation is conducted in the self-defined task of analyzing the statistics of natural pixels from the MS COCO dataset. In this regard, I would recommend adding more details regarding the evaluation, especially on how performance is measured for that task. Moreover, given that the evaluation is related with visual data, I would highly recommend evaluating their method on a more standard task from computer vision.
In addition, very recently a paper with the very same title as this manuscript ("non-parametric neural networks") came out, i.e. Phillip and Carbonell, ICLR'17, while the method and goal approached in this paper is different from that from ICLR, I would strongly suggest either, a) make an explicit and strong positioning of your paper w.r.t. Phillip and Carbonell, ICLR'17, or b) consider an alternative (or extended) title for your paper. In this regard, I consider that your paper is better suited for the N3 title, however, this is a conflict that should be addressed adequately.
Finally, given the potential of the proposed method I would encourage the authors to release the code related to the experiments presented in the manuscript. Are there any plans for releasing the code?
I believe, this should encourage the adoption of the proposed method by the research community.
I would appreciate if the authors address the last three paragraphs in their rebuttal.


Online control of the false discovery rate with decaying memory
In the online multiple testing problem, p-values corresponding to different null hypotheses are observed one by one, and the decision of whether or not to reject the current hypothesis must be made immediately, after which the next pvalue is observed. Alpha-investing algorithms to control the false discovery rate (FDR), formulated by Foster and Stine, have been generalized and applied to many settings, including quality-preserving databases in science and multiple A/B or multi-armed bandit tests for internet commerce. This paper improves the class of generalized alpha-investing algorithms (GAI) in four ways: (a) we show how to uniformly improve the power of the entire class of monotone GAI procedures by awarding more alpha-wealth for each rejection, giving a win-win resolution to a recent dilemma raised by Javanmard and Montanari, (b) we demonstrate how to incorporate prior weights to indicate domain knowledge of which hypotheses are likely to be non-null, (c) we allow for differing penalties for false discoveries to indicate that some hypotheses may be more important than others, (d) we define a new quantity called the decaying memory false discovery rate (mem-FDR) that may be more meaningful for truly temporal applications, and which alleviates problems that we describe and refer to as "piggybacking" and "alpha-death." Our GAI++ algorithms incorporate all four generalizations simultaneously, and reduce to more powerful variants of earlier algorithms when the weights and decay are all set to unity. Finally, we also describe a simple method to derive new online FDR rules based on an estimated false discovery proportion.

This paper unifies and extends concepts related to the online false discovery rate (FDR) control. This is a recent trendy setting where null hypotheses are coming in a sequential manner over time and the user should make an (irrevocable) decision about rejecting (or not) the null coming at each time. The aim is to make a sequential inference so that the expected proportion of errors among the rejected nulls is bounded by a prescribed value $\alpha$ (at any stopping time).
My opinion is that this paper is of very high quality: the writing is good and easy to follow, the motivation is clear and interesting, the new procedures and concepts are clever and the proofs are really nicely written.
My only concern is that the strongest innovation (decaying memory procedure) is maybe not
sufficiently well put forward in the main paper. It only comes at the last 1.5 pages and there is no illustration for Alpha-death recovery. This concept seems however more important than the double-weighting of Section 4 that can be put in appendix safely (more standard). I would also suggest to put Section D.3 of the supplement in the main paper (illustration of Piggybacking).
Here are some other comments and typos :
 line 58 "will equal 1" (only with high probability);
 line 131 "mst";
 line 143 "keeping with past work";
 Notation "W(t)" I suggest to use W_t (if you agree);
 Lemma 1 f_i is not F_i measurable but just measurable (right?);
 Section D.1, line 469-470 :" mu_j follows 0" should read "mu_j=0";
 line 528 : equation (13) should read equation (10);
 line 529-530 : -phi_t (R_t-1) should read +phi_t (R_t-1).


Unbounded cache model for online language modeling with open vocabulary
Recently, continuous cache models were proposed as extensions to recurrent neural network language models, to adapt their predictions to local changes in the data distribution. These models only capture the local context, of up to a few thousands tokens. In this paper, we propose an extension of continuous cache models, which can scale to larger contexts. In particular, we use a large scale non-parametric memory component that stores all the hidden activations seen in the past. We leverage recent advances in approximate nearest neighbor search and quantization algorithms to store millions of representations while searching them efficiently. We conduct extensive experiments showing that our approach significantly improves the perplexity of pre-trained language models on new distributions, and can scale efficiently to much larger contexts than previously proposed local cache models.

This paper discusses an extensions to the recently proposed continuous cache models by Grave et al. The authors propose a continuous cache model that is unbounded, hence can take into account events that happened an indefinitely long time ago. While interesting, the paper fails to provide good experimental evidence of its merits. Its main statement is that this model is better than Grave et al., but then does not compare with it. It only seems to compare with cache models from the nineties (Kuhn et al.), although that is not even clear as they spend only one line (line 206) discussing the models they compare with. "the static model interpolated with the unigram probability distribution observed up to time t" does sound like Kuhn et al. and is definitely not Grave et al.
The authors also mention the importance of large vocabularies, yet fail to specify the vocabulary size for any of their experiments. I also don't understand why all the datasets were lowercased if large vocabularies are the target? This (and the above) is a real pity, because sections 1-3 looked very promising. We advise the authors to spend some more care on the experiments section to make this paper publishable.
Minor comments:
* line 5: stores -> store
* line 13: twice "be"
* line 30: speach -> speech
* line 31: "THE model"
* line 59: "assumptions ABOUT"
* line 72: no comma after "and"
* line 79: algorithm -> algorithmS
* line 79: approach -> approachES
* line 97: non-parameteric -> non-parametric
* line 99: "THE nineties"
* line 110: "aN update rule"
* line 127: Khun -> Kuhn
* line 197: motivationS
* line 197: "adapt to A changing distribution"
* line 211: "time steps"
* line 211: adaptative -> adaptive
* table 2: the caption mentions that the model is trained on news 2007, but in fact this varies throughout the table?
* line 216: interest -> interested
* line 235: millions -> million
* line 267: experimentS
* line 269: where do these percentages come from? they seem wrong...
* line 281: "THE static model"
* line 283: Set


Elementary Symmetric Polynomials for Optimal Experimental Design
We revisit the classical problem of optimal experimental design (OED) under a new mathematical model grounded in a geometric motivation. Specifically, we introduce models based on elementary symmetric polynomials; these polynomials capture "partial volumes" and offer a graded interpolation between the widely used A-optimal design and D-optimal design models, obtaining each of them as special cases. We analyze properties of our models, and derive both greedy and convex-relaxation algorithms for computing the associated designs. Our analysis establishes approximation guarantees on these algorithms, while our empirical results substantiate our claims and demonstrate a curious phenomenon concerning our greedy method. Finally, as a byproduct, we obtain new results on the theory of elementary symmetric polynomials that may be of independent interest.

The authors present a novel objective for experiment design that interpolates A- and D-optimality. They present a convex relaxation for the problem which includes novel results on elementary symmetric polynomials, and an efficient greedy algorithm based on the convex relaxation with performance guarantees for the original task. They present results on synthetic data, and present results on a concrete compressive strength task.
		  I am not an expert in this area but I believe I understand the contribution and that it is significant.
		  The paper is extremely well-presented, with minor points of clarification needed that I have identified below.

		  The main suggestion I have is that the authors more explicitly explain the benefits of the new method in terms of the properties of solutions that interpolate between A and D -- In 6.2, the authors present the ability to use their method to get a spars(er) design matrix as an important feature. This should be stated at the beginning of the paper if it is a true aim (and not simply an interesting side-effect) and the authors should review other methods for obtaining a sparse design matrix. (I presume that there are e.g. L1 penalized methods that can achieve sparsity in this area but I do not know.) In general, a more explicit statement of the practical benefits of the contributions in the paper would be useful for the reader.

		  l21. is a bit confusing regarding use of "vector" -- \epsilon_i is a scalar, yes?
		  l22. Gauss-Markov doesn't require \epsilon_i be Gaussian but it *does* require homoskedasticity. Clarify.
		  l45. "deeply study" - studied
		  l77. I think (2.1) is confusing with v as vectors unless the product is element-wise. Is this intended?
		  l107. "Clearly..." - Make a more precise statement. Is an obvious special case hard? What problem of known hardness reduces to this problem? If you mean "no method for finding an exact solution is known" or "we do not believe an efficient exact solution is possible" then say that.


Hunt For The Unique, Stable, Sparse And Fast Feature Learning On Graphs
For the purpose of learning on graphs, we hunt for a graph feature representation that exhibit certain uniqueness, stability and sparsity properties while also being amenable to fast computation. This leads to the discovery of family of graph spectral distances (denoted as FGSD) and their based graph feature representations, which we prove to possess most of these desired properties. To both evaluate the quality of graph features produced by FGSD and demonstrate their utility, we apply them to the graph classification problem. Through extensive experiments, we show that a simple SVM based classification algorithm, driven with our powerful FGSD based graph features, significantly outperforms all the more sophisticated state-of-art algorithms on the unlabeled node datasets in terms of both accuracy and speed; it also yields very competitive results on the labeled datasets -despite the fact it does not utilize any node label information.

The authors propose a kernel for unlabeled graphs based on the histogram of the pairwise node distances where the notion of distance is defined via the Eigen decomposition of the laplacian matrix. They show that such a notion enjoys useful uniqueness and stability properties, and that when viewed as an embedding it is a generalization of known graph embedding and dimensional reduction approaches.
The work would benefit from a clearer exposition (briefly anticipating the notion of "distance" or of invariance, uniqueness, stability and sparsity albeit in an informal way).
Figure 3 needs to be rethought as it is not informative in the current form (perhaps showing the histogram of the number of non zero elements in the two cases?)
The authors should be careful not to oversell the results of Theorem 1, as in practice it is not known how does the number of non unique harmonic spectra increases with the increase in graph size (after all graphs in real world cases have sizes well exceeding 10 nodes). Also it is not clear whether having near identical spectra could be compatible with large structural changes as measured by other sensible graph metrics.
Theorem 5 in the current form is not informative as it encapsulates in \theta a variability that is hard for the reader to gauge.
The stability analysis seems to be based on the condition that the variation of weights be positive; perhaps the authors want to add a brief comment on what could be expected if the variation is unrestricted?
An empirical analysis of the effect of the histogram bin size should probably be included.
The work of Costa and De Grave. "Fast neighborhood subgraph pairwise distance kernel." ICML 2010, seems to be related as they use the histogram of (shortest path) distances between (subgraphs centered around) each pair of nodes, and should probably be referenced and/or compared with.
There are a few typos and badly formatted references.
However the work is of interest and of potential impact.


Dilated Recurrent Neural Networks
Learning with recurrent neural networks (RNNs) on long sequences is a notoriously difficult task. There are three major challenges: 1) complex dependencies, 2) vanishing and exploding gradients, and 3) efficient parallelization. In this paper, we introduce a simple yet effective RNN connection structure, the DILATEDRNN, which simultaneously tackles all of these challenges. The proposed architecture is characterized by multi-resolution dilated recurrent skip connections, and can be combined flexibly with diverse RNN cells. Moreover, the DILATEDRNN reduces the number of parameters needed and enhances training efficiency significantly, while matching state-of-the-art performance (even with standard RNN cells) in tasks involving very long-term dependencies. To provide a theory-based quantification of the architecture's advantages, we introduce a memory capacity measure, the mean recurrent length, which is more suitable for RNNs with long skip connections than existing measures. We rigorously prove the advantages of the DILATEDRNN over other recurrent neural architectures. The code for our method is publicly available 1 .

The paper proposes to build RNNs out of recurrent layers with skip connections of exponentially increasing lenghts, i.e. the recurrent equivalent of stacks of dilated convolutions. Compared to previous work on RNNs with skip connections, the connections to the immediately preceding timestep are fully replaced here, allowing for increased parallelism in the computations. Dilated RNNs are compared against several other recurrent architectures designed to improve learning of long-range temporal correlations, and found to perform comparably or better for some tasks.
The idea is simple and potentially effective, and related work is adequately covered in the introduction, but the evaluation is somewhat flawed. While the idea of using dilated connections in RNNs is clearly inspired by their use in causal CNNs (e.g. WaveNet), the evaluation never includes them. Only several other recurrent architectures are compared against. I think this is especially limiting as the paper claims computational advantages for dilated RNNs, but these computational advantages could be even more outspoken for causal CNNs, which allow for training and certain forms of inference to be fully parallelised across time.
The noisy MNIST task described from L210 onwards (in Section 4.2) seems to be nonstandard and I don't really understand the point. It's nice that dilated RNNs are robust to this type of corruption, but sequences padded with uniform noise seem fairly unlikely to occur in the real world. In Section 4.3, raw audio modelling is found to be less effective than MFCC-based modelling, so this also defeats the point of the dilated RNN a bit. Also, trying dilated RNNs combined with batch normalization would be an interesting additional experiment.
Overall, I don't find the evidence quite as strong as it is claimed to be in the conclusion of the paper.
Remarks:
- Please have the manuscript proofread for spelling and grammar.
- In the introduction, it is stated that RNNs can potentially model infinitely long dependencies, but this is not actually achievable in practice (at least with the types of models that are currently popular). To date, this advantage of RNNs over CNNs is entirely theoretical.
- It is unclear whether dilated RNNs also have repeated stacks of dilation stages, like e.g. WaveNet. If not, this would mean that large-scale features (from layers with large dilation) never feed into fine-scale features (layers with small dilation). That seems like it could be a severe limitation, so this should be clarified.
- Figures 4, 5, 6 are a bit small and hard to read, removing the stars from the curves might make things more clear.
UPDATE: in the author feedback it is stated that the purpose of the paper is to propose an enhancement for RNN models, but since this enhancement is clearly inspired by dilation in CNNs, I don't think this justifies the lack of a comparison to dilated CNNs at all. Ultimately, we want to have models that perform well for some given task -- we do not want to use RNNs just for the sake of it.
That said, the additional experiments that were conducted do address my concerns somewhat (although I wish they had been done on a different task, I still don't really get the point of "noisy MNIST".) So I have raised the score accordingly.


Predictive-State Decoders: Encoding the Future into Recurrent Networks
Recurrent neural networks (RNNs) are a vital modeling technique that rely on internal states learned indirectly by optimization of a supervised, unsupervised, or reinforcement training loss. RNNs are used to model dynamic processes that are characterized by underlying latent states whose form is often unknown, precluding its analytic representation inside an RNN. In the Predictive-State Representation (PSR) literature, latent state processes are modeled by an internal state representation that directly models the distribution of future observations, and most recent work in this area has relied on explicitly representing and targeting sufficient statistics of this probability distribution. We seek to combine the advantages of RNNs and PSRs by augmenting existing state-of-the-art recurrent neural networks with PREDICTIVE-STATE DECODERS (PSDs), which add supervision to the network's internal state representation to target predicting future observations. PSDs are simple to implement and easily incorporated into existing training pipelines via additional loss regularization. We demonstrate the effectiveness of PSDs with experimental results in three different domains: probabilistic filtering, Imitation Learning, and Reinforcement Learning. In each, our method improves statistical performance of state-of-the-art recurrent baselines and does so with fewer iterations and less data.

The authors present a simple method for regularizing recurrent neural networks (specifically applied to low-dimensional control tasks trained by filtering, imitation, and reinforcement learning) that is inspired by the literature on Predictive State Representations (PSRs). In PSRs, processes with latent states are modeled by directly identifying the latent state with a representation of the sufficient statistics of future observations. This stands in contrast to RNNs, which use an opaque hidden state that is optimized by backpropagation to make the whole system model the target of interest through evolution of the hidden state. They propose a simple regularization method for RNNs inspired by PSRs, called Predictive State Decoders, which maintains the parametric form of a standard RNN, but augments the training objective to make the hidden state more directly predict statistics of distant future observations. While direct training of an RNN by backpropagation implicitly encourages the internal state to be predictive of the future, the authors demonstrate that this explicit additional regularization term gives significant improvements on several control tasks.
The idea specifically is to augment the standard training loss for an RNN controller (either for filtering, imitation learning, or reinforcement learning) with an additional term that maps the hidden state at each timestep through an additional learned parametric function F, and penalizes the difference between that and some statistics of up to 10 future observations. This encourages the internal state of the RNN to more directly capture the future distribution of observations without over-fitting to next-step prediction, and architecturally improves gradient flow through the network similar to skip-connections, attention, or the intermediate classifiers added in FitNets or Inception.
I like the main idea of the paper. It is simple, well-motivated, and connects RNN-based models to a large body of theoretically-grounded research in modeling latent state processes, of which the authors do a good literature review. The experiments and analysis are thorough, and demonstrate significant improvements over the baseline on many of the tasks in the OpenAI gym. The baselines use strong, well-known techniques and optimization methods for imitation learning and RL.
I have some concerns that the paper does not offer enough detail about the exact architectures used. For the filtering experiment, it is stated that phi is the identity function, which I assume is true for the future experiments. Additionally, unless I am missing something, the exact functional form of F, the decoder, is never mentioned in the text. Is this a feedforward neural network? Given that the predicted states can sometimes be 10 times the size of the observation dimension, have the authors thought about how to apply this method to models with very large dimensional observations? More detail would be required on exactly the form of F in a camera-ready.
Overall, I think this is a well-executed paper that demonstrates a nice synergy between the PSR and RNN methods for latent state process modeling. The method seems simple to implement and convincingly improves over the baselines.


TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
High network communication cost for synchronizing gradients and parameters is the well-known bottleneck of distributed training. In this work, we propose TernGrad that uses ternary gradients to accelerate distributed deep learning in data parallelism. Our approach requires only three numerical levels {−1, 0, 1}, which can aggressively reduce the communication time. We mathematically prove the convergence of TernGrad under the assumption of a bound on gradients. Guided by the bound, we propose layer-wise ternarizing and gradient clipping to improve its convergence. Our experiments show that applying TernGrad on AlexNet doesn't incur any accuracy loss and can even improve accuracy. The accuracy loss of GoogLeNet induced by TernGrad is less than 2% on average. Finally, a performance model is proposed to study the scalability of TernGrad. Experiments show significant speed gains for various deep neural networks. Our source code is available 1 .

The authors tried to reduce communication cost in distributed deep learning. They proposed a new method called Ternary Gradients to reduce the overhead of gradient synchronization. The technique is correct and the authors made some experiments to evaluate their method. Overall, the method is novel and interesting and may have significant impact on large scale deep learning. Some comments below for the authors to improve their paper.
1. Some notations and words should be made clearer.  For example, s is defined as the max value of abs(g), but the domain of the max operation fails to show up in the equation.
2. The author mentioned that they tried to narrow the gap between the two bounds in the paper for better convergence, but not mentioned for better convergence rate or better convergence probability. And it’s a pity that there’s no theoretical analysis of the convergence rate.
3.  One big drawback is that the author made too strong assumption, i.e. the cost function has only one single minimum, which means the objective function is convex. It’s rare in practice. To my best knowledge, there’re many efficient algorithms for convex optimization in distributed systems. Therefore, the proposed method will not have big impacts.
4.  The author should use more metrics in the experiments instead of just accuracy.


Optimistic posterior sampling for reinforcement learning: worst-case regret bounds
We present an algorithm based on posterior sampling (aka Thompson sampling) that achieves near-optimal worst-case regret bounds when the underlying Markov Decision Process (MDP) is communicating with a finite, though unknown, diameter. Our main result is a high probability regret upper bound ofÕ(D √ SAT ) for any communicating MDP with S states, A actions and diameter D, when T ≥ S 5 A. Here, regret compares the total reward achieved by the algorithm to the total expected reward of an optimal infinite-horizon undiscounted average reward policy, in time horizon T . This result improves over the best previously known upper bound ofÕ(DS √ AT ) achieved by any algorithm in this setting, and matches the dependence on S in the established lower bound of Ω( √ DSAT ) for this problem. Our techniques involve proving some novel results about the anti-concentration of Dirichlet distribution, which may be of independent interest.

Posterior Sampling for Reinforcement Learning: Worst-Case Regret Bounds
=======================================================================
This paper presents a new algorithm for efficient exploration in Markov decision processes.
This algorithm is an optimistic variant of posterior sampling, similar in flavour to BOSS.
The authors prove new performance bounds for this approach in a minimax setting that are state of the art in this setting.
There are a lot of things to like about this paper:
- The paper is well written and clear overall.
- The analysis and rigour is of a high standard. I would say that most of the key insights do come from the earlier "Gaussian-Dirichlet dominance" of Osband et al, but there are some significant extensions and results that may be of wider interest to the community.
- The resulting bound, which is O(D \sqrt{SAT}) for large enough T is the first to match the lower bounds in this specific setting (communicating MDPs with worst-case guarantees) up to logarthmic factors.
Overall I think that this is a good paper and will be of significant interest to the community.
I would recommend acceptance, however there are several serious issues that need to be addressed before a finalized version:
- This paper doesn't do a great job addressing the previous Bayesian analyses of PSRL, and the significant similarities in the work. Yes, these new bounds are "worst case" rather than in expectation and for communicating MDPs instead of finite horizon... however these differences are not so drastic to just be dismissed as incomparable. For example, this paper never actually mentions that PSRL/RLSVI used this identical stochastic optimism condition to perform a similar reduction of S to \sqrt{S}.
- Further, this algorithm is definitively not "PSRL" (it's really much more similar to BOSS) and it needs a specific name. My belief is that the current analysis would not apply to PSRL, where only one sample is taken, but PSRL actually seems to perform pretty well in experiments (see Osband and Van Roy 2016)... do the authors think that these extra samples are really necessary?
- The new bounds aren't really an overall improvement in big O scaling apart from very large T. I think this blurs a lot of the superiority of the "worst-case" analysis over "expected" regret... none of these "bounds" are exactly practical, but their main value is in terms of the insight towards efficient exploration algorithms. It seems a significant downside to lose the natural simplicity/scalability of PSRL all in the name of these bounds, but at a huge computational cost and an algorithm that no longer really scales naturally to domains with generalization.
- It would be nice to add some experimental evaluation of this new algorithm, perhaps through the code published as part of (Osband and Van Roy, 2016)... especially if this can add some intuition/insight to the relative practical performance of these algorithms.
==============================================================================
= Rebuttal comments
==============================================================================
Thanks for the author feedback, my overall opinion is unchanged and I definitely recommend acceptance... but I *strongly* hope that this algorithm gets a clear name separate to PSRL because I think it will only lead to confusion in the field otherwise.
On the subject of bounds I totally agree these new bounds (and techniques) are significant. I do believe it would give some nice context to mention that at a high level the BayesRegret analysis for PSRL also reduced the S -> sqrt{S}.
On the subject of multiple samples the authors mention “An Optimistic Posterior Sampling Strategy for Bayesian Reinforcement Learning, Raphael Fonteneau, Nathan Korda, Remi Munos”, which I think is a good reference. One thing to note is that this workshop paper (incorrectly) compared a version of PSRL (and optimistic PSRL) that resamples every timestep... this removes the element of "deep exploration" due to sampling noise every timestep. If you actually do the correct style of PSRL (resampling once per episode) then multiple samples does not clearly lead to improved performance (see Appendix D.3 https://arxiv.org/pdf/1607.00215.pdf )
On the subject of bounds and large T, I do agree with the authors, but I think they should make sure to stress that for small T these bounds are not as good and whether these "trailing" terms are real or just an artefact of loose analysis.


Hierarchical Clustering Beyond the Worst-Case
Hiererachical clustering, that is computing a recursive partitioning of a dataset to obtain clusters at increasingly finer granularity is a fundamental problem in data analysis. Although hierarchical clustering has mostly been studied through procedures such as linkage algorithms, or top-down heuristics, rather than as optimization problems, Dasgupta [9] recently proposed an objective function for hierarchical clustering and initiated a line of work developing algorithms that explicitly optimize an objective (see also [7, 22, 8]). In this paper, we consider a fairly general random graph model for hierarchical clustering, called the hierarchical stochastic block model (HSBM), and show that in certain regimes the SVD approach of McSherry [18]  combined with specific linkage methods results in a clustering that give an Op1q approximation to Dasgupta's cost function. Finally, we report empirical evaluation on synthetic and real-world data showing that our proposed SVD-based method does indeed achieve a better cost than other widely-used heurstics and also results in a better classification accuracy when the underlying problem was that of multi-class classification.

Summary of paper:
The paper studies hierarchical clustering in the recent framework of [1], which provides a formal  cost function for the problem. [4] proposed a hierarchical stochastic block model (HSBM) for this problem, and showed that under this model, one can obtain a constant factor approximation of Dasgupta’s cost function [1].
The present work generalises the HSBM model slightly, where they do not restrict the number of clusters to be fixed, and hence, the cluster sizes may be sub-linear in the data size. The authors prove guarantees of a proposed algorithm under this model. They further extend their analysis to a semi-random setting where an adversary may remove some inter-cluster edges.
They also perform experiments on some benchmark datasets to show the performance of the method.
Quality:
The paper is of decent quality. While the paper is theoretically sound, the experiment section is not very convincing. Why would one consider 5 flat cluster datasets and only one hierarchical dataset in a hierarchical clustering paper? Even for the last case, only a subset of 6 classes are used. Further, 3 out of 5 methods in comparison are variations of Linkage++, the other two being two standard agglomerative techniques. What about divisive algorithms?
Clarity:
Most parts of the paper can be followed, but in some places, the authors compressed the material a lot and one needs to refer to cited papers for intuitive understanding (for instance, Definition 2.1). There are minor typos throughout the paper, example in lines 41, 114, 126, 179, 201, 342.
Originality:
The paper derives a considerable portion from [4]. Though some extensions are claimed, I wonder if there is any technical challenges in proving the additional results (I did not see the details in supplementary).
For example, is there any difference between Linkage++ and the method in [4]? Apparently I did not find any significant difference.
The analysis on HSBM may allow growing k, but is there truly a challenge in doing so. At least for first step of algorithm, achieving n_min > \sqrt{n}  does not seem a big hurdle.
Analysis on the semi-random setting is new, but I wonder if there are any challenges in this scenario. Or does it immediately follow from combining HSBM analysis with standard techniques of semi-random graph partitioning?
Significance:
In general, hierarchical clustering in the framework of [1] has considerable significance, and [4] is also a quite interesting paper. However, I consider the present paper a minor extension of [4], with the only possible interesting aspect being the result in the semi-random setting.
---------------------------
Update after rebuttal:
I thank the authors for clarifying the technical challenges with proving the new results. Hence, I have now increased my score (though I am still concerned about the algorithmic and experimental contributions).
Regarding the experiments, I do agree that there are not many well-defined benchmark out there. The papers that I know are also from early 2000s, but the authors may want to check the datasets used here:
1. Zhao, Karypis. Hierarchical Clustering Algorithms for Document Datasets. Data Mining and Knowledge Discovery, 2005
2. Alizadeh et al. Distinct types of diffuse large B-cell lymphoma identified by gene expression profiling. Nature, 2000
Apart from these, UCI repository has Glass and Reuters that I know have been used for hierarchical clustering. Perhaps, there are also more recent document / gene expression datasets in recent data mining papers.


Maximum Margin Interval Trees
Learning a regression function using censored or interval-valued output data is an important problem in fields such as genomics and medicine. The goal is to learn a real-valued prediction function, and the training output labels indicate an interval of possible values. Whereas most existing algorithms for this task are linear models, in this paper we investigate learning nonlinear tree models. We propose to learn a tree by minimizing a margin-based discriminative objective function, and we provide a dynamic programming algorithm for computing the optimal solution in log-linear time. We show empirically that this algorithm achieves state-of-the-art speed and prediction accuracy in a benchmark of several data sets.

The authors of this paper present a new decision tree algorithm for the interval regression problem.  Leaves are partitioned using a  margin based hinge loss  similar to the L1-regularized hinge loss in Rigaill et al, Proc ICML 2013. However, the regression tree algorithm presented in this work is not limited to modeling linear patterns as the L1-regularized linear models in Rigaill et al.  For training the non linear tree model, a sequence of convex optimization subproblems are optimally solved in log-linear time by  Dynamic Programming (DP). The new maximum margin interval tree (MMIT) algorithm is compared with state-of-the-art margin-based and non-margin-based methods in several real and simulated datasets.
-	In terms of originality, the proposed margin based hinge loss is similar to the L1-regularized hinge loss in Rigaill et al, Proc ICML 2013. However, the regression tree algorithm presented in this work is not limited to modeling linear patterns as the L1-regularized linear models in Rigaill et al. MMIT achieves low test errors on nonlinear datasets and yields accurate models on simulated linear datasets as well.
-	In terms of significance: interval regression is a fundamental problem in fields such as survival analysis and computational biology. There are very few algorithms designed for this task, and most of them are linear models. A  method learning nonlinear tree models like Maximum Margin Interval Trees (MMIT) could be helpful for practitioners in the field.
-	It terms of quality, the paper is fairly executed, the authors compare MMIT with state-of-the-art margin-based and non-margin-based methods in several real and simulated datasets. The 11-page supplementary material contains proofs, pseudocode, details of the open source implementation (link was hidden for anonymity) and of the experiments.
-	In terms of clarity, this is a well written paper.
In summary, my opinion is that this is a well written and nicely organized work; the proposed method would be useful in real-world applications, and the novelty of the work satisfies the NIPS standards. Thus I recommend this work for publication.


Character-Level Language Modeling with Recurrent Highway Hypernetworks
We present extensive experimental and theoretical support for the efficacy of recurrent highway networks (RHNs) and recurrent hypernetworks complimentary to the original works. Where the original RHN work primarily provides theoretical treatment of the subject, we demonstrate experimentally that RHNs benefit from far better gradient flow than LSTMs in addition to their improved task accuracy. The original hypernetworks work presents detailed experimental results but leaves several theoretical issues unresolved-we consider these in depth and frame several feasible solutions that we believe will yield further gains in the future. We demonstrate that these approaches are complementary: by combining RHNs and hypernetworks, we make a significant improvement over current state-of-the-art character-level language modeling performance on Penn Treebank while relying on much simpler regularization. Finally, we argue for RHNs as a drop-in replacement for LSTMs (analogous to LSTMs for vanilla RNNs) and for hypernetworks as a de-facto augmentation (analogous to attention) for recurrent architectures.

The paper describes how to achieve state-of-the-art results on a small character prediction task. Some detailed comments below:
- Title: First of all, I would not call work on PTB using characters "language modeling", it is character prediction. Yes, theoretically the same thing but actual language model applications typically use much bigger vocabularies and much more data, and this is where most of the problems in language modeling lie. Replacing "language modeling" by "character prediction" would be a more appropriate title
- Results (experimental): I agree that PTB is OK to use as a database to test character prediction models. However, it would have been good to also test/show your findings on larger databases like for example the 1 Billion Google corpus or similar. Public implementations for training these relatively quickly on limited hardware exist and it would have made this a much stronger paper, as eventually I believe you hope that your findings will be in fact used for larger, real problems. To show in your paper that you can handle large problems is always better than finding reasons to not do it as you do in line 113-118 and in other places throughout the paper.
- Results (experimental): You list accuracy/perplexity/bpc -- it would be good to define exactly how these are calculated to avoid confusion.
- 3.2: 2-3 days of training for PTB is quite long, as you note yourself. It would have been nice to compare the different models you compare yourself over time by plotting perplexity vs. training time for all the models to compare how efficiently they can be trained.
- 5.1: You say that you are almost certainly within a few percent of maximum performance on this task -- why? Maybe that is true but maybe it isn't. Diminishing gains on an absolute scale can have many reasons, including the one that possibly many people do not work on PTB for character prediction anymore. Also, the last sentence: Why is accuracy better than perplexity to show relative improvement? BTW, I agree that accuracy is a good measure in general (because everybody knows it is bounded between 0 and 100%), but perplexity is simply the standard measure for these kinds of predictions tasks.
- 5.4: You say that how and when to visualize gradient flow has not been receiving direct treatment. Yes, true, but I would argue the main reason is that it is simply not so interesting -- what matters in training these networks is not a theoretical treatment of some definition of gradient flow but simply time to convergence for a given size model on a given task. This goes also back to my comment from above, what would have been more interesting is to show training time vs accuracy for several different networks and different sizes.
- 6: Why can you compare gradient norms of completely different architectures? I would argue it is quite unclear whether gradients of different architectures have to have similar norms -- the error surface could look very different. Also, even if you can compare them, the learning algorithm matters a lot -- SGD vs Adam vs Rprop or whatnot have all a completely different effect given a certain gradient (in some cases only the sign matters etc.). So overall, I would argue that gradient flow is simply not important, it would be better to focus to convergence optimizing learning algorithms, hyperparameters etc.


Incorporating Side Information by Adaptive Convolution
Computer vision tasks often have side information available that is helpful to solve the task. For example, for crowd counting, the camera perspective (e.g., camera angle and height) gives a clue about the appearance and scale of people in the scene. While side information has been shown to be useful for counting systems using traditional hand-crafted features, it has not been fully utilized in counting systems based on deep learning. In order to incorporate the available side information, we propose an adaptive convolutional neural network (ACNN), where the convolution filter weights adapt to the current scene context via the side information. In particular, we model the filter weights as a low-dimensional manifold within the high-dimensional space of filter weights. The filter weights are generated using a learned "filter manifold" sub-network, whose input is the side information. With the help of side information and adaptive weights, the ACNN can disentangle the variations related to the side information, and extract discriminative features related to the current context (e.g. camera perspective, noise level, blur kernel parameters). We demonstrate the effectiveness of ACNN incorporating side information on 3 tasks: crowd counting, corrupted digit recognition, and image deblurring. Our experiments show that ACNN improves the performance compared to a plain CNN with a similar number of parameters. Since existing crowd counting datasets do not contain ground-truth side information, we collect a new dataset with the ground-truth camera angle and height as the side information.

Summary of the Paper:
This work proposes to use adaptive convolutions (also called 'cross convolutions') to incorporate side information (e.g., camera angle) into CNN architectures for vision tasks (e.g., crowd counting). The filter weights in each adaptive convolution layer are predicted using a separate neural network (one network for each set of filter weights) with is a multi-layer perceptron. This network is referred to as 'Filter Manifold Network' which takes the auxiliary side information as input and predicts the filter weights. Experiments on three vision tasks of crowd counting, digit recognition and image deconvolution indicate the potential of the proposed technique for incorporating auxiliary information. In addition, this paper contributes a new dataset for crowd counting with different camera heights and angles.
Paper Strengths:
- A simple yet working technique for incorporating auxiliary information into CNNs with adaptive convolutions.
- With this technique, a single network is sufficient for different dataset settings (for instance, denoising with different blur kernel parameters) and authors demonstrated generalization capability of the proposed technique to interpolate between auxiliary information.
- Experiments on 3 different vision tasks with good results compared to similar CNN architectures with non-adaptive convolutions.
- A new dataset on crowd counting.
Weaknesses:
- There is almost no discussion or analysis on the 'filter manifold network' (FMN) which forms the main part of the technique. Did authors experiment with any other architectures for FMN? How does the adaptive convolutions scale with the number of filter parameters? It seems that in all the experiments, the number of input and output channels is small (around 32). Can FMN scale reasonably well when the number of filter parameters is huge (say, 128 to 512 input and output channels which is common to many CNN architectures)?
- From the experimental results, it seems that replacing normal convolutions with adaptive convolutions in not always a good. In Table-3, ACNN-v3 (all adaptive convolutions) performed worse that ACNN-v2 (adaptive convolutions only in the last layer). So, it seems that the placement of adaptive convolutions is important, but there is no analysis or comments on this aspect of the technique.
- The improvements on image deconvolution is minimal with CNN-X working better than ACNN when all the dataset is considered. This shows that the adaptive convolutions are not universally applicable when the side information is available. Also, there are no comparisons with state-of-the-art network architectures for digit recognition and image deconvolution.
Suggestions:
- It would be good to move some visual results from supplementary to the main paper. In the main paper, there is almost no visual results on crowd density estimation which forms the main experiment of the paper. At present, there are 3 different figures for illustrating the proposed network architecture. Probably, authors can condense it to two and make use of that space for some visual results.
- It would be great if authors can address some of the above weaknesses in the revision to make this a good paper.
Review Summary:
- Despite some drawbacks in terms of experimental analysis and the general applicability of the proposed technique, the paper has several experiments and insights that would be interesting to the community.
------------------
After the Rebuttal:
------------------
My concern with this paper is insufficient analysis of 'filter manifold network' architecture and the placement of adaptive convolutions in a given CNN. Authors partially addressed these points in their rebuttal while promising to add the discussion into a revised version and deferring some other parts to future work.
With the expectation that authors would revise the paper and also since other reviewers are fairly positive about this work, I recommend this paper for acceptance.


A Dirichlet Mixture Model of Hawkes Processes for Event Sequence Clustering
How to cluster event sequences generated via different point processes is an interesting and important problem in statistical machine learning. To solve this problem, we propose and discuss an effective model-based clustering method based on a novel Dirichlet mixture model of a special but significant type of point processesHawkes process. The proposed model generates the event sequences with different clusters from the Hawkes processes with different parameters, and uses a Dirichlet distribution as the prior distribution of the clusters. We prove the identifiability of our mixture model and propose an effective variational Bayesian inference algorithm to learn our model. An adaptive inner iteration allocation strategy is designed to accelerate the convergence of our algorithm. Moreover, we investigate the sample complexity and the computational complexity of our learning algorithm in depth. Experiments on both synthetic and real-world data show that the clustering method based on our model can learn structural triggering patterns hidden in asynchronous event sequences robustly and achieve superior performance on clustering purity and consistency compared to existing methods.

In this work, the authors introduce a novel method to cluster sequences of events. This method relies on a Dirichlet mixture model of Hawkes processes to describe the data and assign events to clusters. This differs from previous work using Hawkes Processes for sequential data by focusing directly on clustering the sequences themselves rather than sub-events; in particular, compared to prior work such as by Luo et al., 2015, the clustering is done as part of the modeling itself rather than applied after learning the parameters of the model, which leads to a sizable improvement in experimental results.
Quality: This paper is of good quality.
Clarity: This paper is overall clear, but contains some grammar and style mistakes - some have been noted below in the detailed comments, but the authors ideally should have their manuscript proof-read for English grammatical and style issues. Figure 2 is too small to be easily readable.
Originality: This paper introduces a new method to model sequences of events.
Significance: This paper appears to be significant.
Detailed comments:
- In section 2, it would be valuable to provide a formal definition of Hawkes Process; in particular, it would improve readability to put the intensity equation (line 67) in its own equation environment, and then refer to it by number in Section 3 when relevant.
- When comparing line 67 to Eq. 1, it seems that the sum over the event types (c_i) is missing; could you please clarify this?
- Line 156: you state that the runtime of the algorithm is linearly proportional to the number of inner iterations - could you write out the complexity explicitly?
- Along the same lines, could you please state the complexity of MMHP in section 4.4?
- The second paragraph of section 4.2 needs to be made clearer.
Clarity comments:
- You do not need to define R+ line 101
- line 23: "the previous two"
- line 39: "prior of the clusters"
- line 51: "the problems of overfitting"
- line 61: "A Hawkes"
- line 73: "was first proposed"
- line 77: "was used"
- line 100: "basis"
- line 109: "a lot of work" should be rephrased
- line 125: "goes to infinity"
- line 232: "state of the art"
- line 255: spacing


Information-theoretic analysis of generalization capability of learning algorithms
We derive upper bounds on the generalization error of a learning algorithm in terms of the mutual information between its input and output. The bounds provide an information-theoretic understanding of generalization in learning problems, and give theoretical guidelines for striking the right balance between data fit and generalization by controlling the input-output mutual information. We propose a number of methods for this purpose, among which are algorithms that regularize the ERM algorithm with relative entropy or with random noise. Our work extends and leads to nontrivial improvements on the recent results of Russo and Zou.

This paper derives upper bounds on the generalization error of learning algorithms in terms of the mutual information between the data S and the hypothesis W. Using Lemma 1, their first result is to show that the generalization error can be upper-bounded by a quantity that is proportional to the square root of the mutual information between S and W, as long as the loss is \sigma-subgaussian. Lemma 1 also enables the authors to immediately recover a previous result by Russo and Zou. They furthermore provide a high-probability bound and an expected bound on the absolute difference between the population risk and the empirical risk.
The next section then applies these bounds to specific algorithms: two of them have an hypothesis set that can be easily upper bounded; the next two have a bounded mutual info between S and W that can be upper bounded; then the authors discuss practical methods to effectively bound I(S;W); and finally, the generalization error of a composite learning algorithm is analyzed. Finally, a similar analysis is then employed to “adaptive data analytics”.
PROS:
- The authors attempt to bridge information-theoretic ideas with learning theory. This is something that I believe would enormously favor both communities.
- The first bounds on the generalization error based on the mutual information are intuitively appealing. (It is however unclear to me how novel they are relative to Russo and Zou's prior work).
CONS:
- The paper is very dense as it is, and would have benefited from more concrete, lively examples that the NIPS community can relate to in order to illustrate the results. In particular, I feel it's a terrible shame that most likely only minute fraction of the NIPS community will find interest in the results of this paper, just because of the way it is written. This is the main reason why I felt obliged to decrease my score.
- All the results seem to require the sigma-subgaussianity of the loss. How restrictive is this? Can you give concrete examples where the analysis holds and where it fails?


Semisupervised Clustering, AND-Queries and Locally Encodable Source Coding
Source coding is the canonical problem of data compression in information theory. In a locally encodable source coding, each compressed bit depends on only few bits of the input. In this paper, we show that a recently popular model of semisupervised clustering is equivalent to locally encodable source coding. In this model, the task is to perform multiclass labeling of unlabeled elements. At the beginning, we can ask in parallel a set of simple queries to an oracle who provides (possibly erroneous) binary answers to the queries. The queries cannot involve more than two (or a fixed constant number ∆ of) elements. Now the labeling of all the elements (or clustering) must be performed based on the (noisy) query answers. The goal is to recover all the correct labelings while minimizing the number of such queries. The equivalence to locally encodable source codes leads us to find lower bounds on the number of queries required in variety of scenarios. We are also able to show fundamental limitations of pairwise 'same cluster' queries -and propose pairwise AND queries, that provably performs better in many situations.

This paper explores a clustering problem of which the goal is to reconstruct the true labels of elements from possibly noisy & non-adaptive query-answers, each associating only a few elements. It derives lower bounds on the number of queries required to reconstruct the ground-truth labels exactly or partially in a variety of scenarios. It also proposes a so-called "AND-queries" algorithm that applies to arbitrary k-cluster case, and evaluates the performance of the algorithm under a real data set.
While the paper attempts to provide theoretical analyses for various scenarios as well as develops an algorithm together with a real-data simulation, it comes with limitations in presentation, clarity of the claimed results, and comparison to related works. Perhaps significant changes and possibly new developments are to be made for publication. See below for detailed comments.
1. (Repeating the same questions is not allowed): This constraint looks a bit restricted. Also it may degrade the performance compared to the no-restriction case, as allowing the same questions is one natural way to combat the noise effect. One proper way to justify the restricted model might be providing an extensive set of real-data simulations with a comparison to an algorithm tailored for the no-restriction case, and possibly showing a better performance of the proposed algorithm. If that is not the case, the no-restriction case is more plausible to investigate instead. The authors seem to concern about independence-vs-dependence issue for multiple answers w.r.t the same query, which might make analysis challenging. But this issue can be resolved focusing on a setting in which the same query is given to distinct annotators and therefore the answers from the annotators are assumed to be independent.
2. (Theorem 1): The noiseless setting with exact recovery was also studied in a recent work [Ahn-Lee-Suh] which also considers the XOR-operation ('same-cluster' queries) under a non-adaptive measurement setting. But the result in [Ahn-Lee-Suh] does not coincide with the one claimed in Theorem 1. For instance, when p=1/2 and \Delta is a constant, the query complexity in Theorem 1 reads m=n/log2, while the one in [Ahn-Lee-Suh] reads m=nlogn/\Delta. This needs to be clarified in details. Moreover, the proof of Theorem 1 is hard to follow.
3. (Theorem 3): The expression of the lower bound on \delta is difficult to interpret. In particular, it is difficult to infer the bound on m in terms of n and \delta, from which one can directly see the performance improvement due to the relaxed constraint, reflected in \delta.
4. (Theorem 4): The same comment given w.r.t. Theorem 3 applies here. In addition, how different is the upper bound from the lower bound in Theorem 3? Can be the gap unbounded?  Again the proof is hard to follow.
5. (Figure 2):  No comparison is made w.r.t. the same cluster query method – there is a comparison only to the same-cluster-query 'lower' bound. Wondering if there is a crossing point also when compared to the same-cluster-query 'method'. If that is the case, any intuition behind that? Actually the paper claims that the proposed approach outperforms the same-cluster query scheme, which cannot be supported from Figure 2.
6. (Algorithm 1): The proposed algorithm requires the prior knowledge on p or the relative sizes of clusters (in view of Theorem 6). How to obtain such information in practice? A proper justification might validate the practicality of the algorithm. Also the pseudo-code is hard to follow without relying on the detailed proof of Theorem 5, which is also difficult to grasp.
7. (Figure 3 & 4 & 5): No comparison is made w.r.t. the state of the arts such as the same-cluster-query method, which is crucial to support the claim made in several places: the AND-queries algorithm outperforms the same-cluster-query scheme.
[Ahn-Lee-Suh] K. Ahn, K. Lee, and C. Suh, “Community recovery in hypergraphs,” Proceedings of Allerton Conference on Communication, Control, and Computing, 2016.
------------
(Additional comments)
I have rebuttal. I found some comments properly addressed. For further clarification:
1. Still not clear as to whether repeating the same question is not helpful. Did the Nature paper show that it is not useful? Perhaps this can be clarified in a revision.
2. Re. the comparison to the same-query method: it would be clearer to include the detailed discussion as addressed in the response.


PredRNN: Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs
The predictive learning of spatiotemporal sequences aims to generate future images by learning from the historical frames, where spatial appearances and temporal variations are two crucial structures. This paper models these structures by presenting a predictive recurrent neural network (PredRNN). This architecture is enlightened by the idea that spatiotemporal predictive learning should memorize both spatial appearances and temporal variations in a unified memory pool. Concretely, memory states are no longer constrained inside each LSTM unit. Instead, they are allowed to zigzag in two directions: across stacked RNN layers vertically and through all RNN states horizontally. The core of this network is a new Spatiotemporal LSTM (ST-LSTM) unit that extracts and memorizes spatial and temporal representations simultaneously. PredRNN achieves the state-of-the-art prediction performance on three video prediction datasets and is a more general framework, that can be easily extended to other predictive learning tasks by integrating with other architectures.

This paper deals with predictive learning (mainly video prediction), using a RNN type of structure. A new (to my knowledge) variation of LSTM is introduced, called ST-LSTM, with recurrent connections not only in the forward time direction.
The predictive network is composed of ST-LSTM blocks. Each of these block resemble a convolutional LSTM unit, with some differences to include an additional input. This extra input comes from the last layer of the previous time step, and enters at the first layer of the current time step, it is then propagated through the layers at the current time step.
The resulting ST-LSTM is similar to the combination of two independent LSTM units, interacting through concatenation of the memories (figure 2(left)).
I couldn't find an explicit formulation of the loss (the closest I could find is equation 1). I am assuming that it is a MSE loss, but this should be confirmed, since other forms are possible.
Results are presented on the Moving MNIST (synthetic) and KTH (natural) datasets, showing both PSNR/SSIM and generations. The authors compare their PredRNN with other baselines, and show the best results on these datasets. On Moving MNIST, there is a comparison of different LSTM schemes, and ST-LSTM show the best results, which is a good result.
However, the experimental section could be stronger by having more datasets (the two datasets presented have little ambiguity in their future, it could be interesting to see how the model preforms in less contained dataset, such as Sports1m, UCF101 or the Google "Push" dataset, for instance). Although this paper present a complex (and, as it seems, good) structure for the predictor, the loss function is almost never mentioned. As (among others) [17] and [19] mention, a simple loss such as the MSE cannot work well when the future distribution is multimodal. The paper would also be stronger if compared with other methods (non LSTM-based), such as the ones presented in section 1.2. In particular, VPNs and GANs seem like strong competitors.
Notes:
- Figure 1: Although the figure is clear, I do not quite understand what the orange arrows mean (compared to the black arrows).
- Figure 2(right): As I understand it, each W box should also have an Xt input (not just W1)


Plan, Attend, Generate: Planning for Sequence-to-Sequence Models
We investigate the integration of a planning mechanism into sequence-to-sequence models using attention. We develop a model which can plan ahead in the future when it computes its alignments between input and output sequences, constructing a matrix of proposed future alignments and a commitment vector that governs whether to follow or recompute the plan. This mechanism is inspired by the recently proposed strategic attentive reader and writer (STRAW) model for Reinforcement Learning. Our proposed model is end-to-end trainable using primarily differentiable operations. We show that it outperforms a strong baseline on character-level translation tasks from WMT'15, the algorithmic task of finding Eulerian circuits of graphs, and question generation from the text. Our analysis demonstrates that the model computes qualitatively intuitive alignments, converges faster than the baselines, and achieves superior performance with fewer parameters.

The paper introduces the planning mechanism into the encoder-decoder model with attention. The key idea is that instead of computing the attention weights on-the-fly for each decoding time step, an attention template is generated for the next k time steps based on the current decoding hidden state s_{t-1}, and a commitment vector is used to decide whether to follow this alignment plan or to recompute it.
Overall, the paper is well written, and technical discussion are clear. The effect of the planning mechanism is demonstrated by Figure 2, where planning mechanism tends to generate smoother alignments compared to the vanilla attention approach. In this paper, the attention template (matrix A) is computed as eq(3). It would be interesting to see if some other type of features could be used, and if they could control the language generation, and produce more diverse and controllable responses. This might be evaluated on a conversation task, however, I understand that it is out of the scope of this paper.
In general, this is an interesting idea to me. A couple of minor questions:
-- In Table 1, why the Baseline + layer norm is missing? PAG and rPAG seem only outperform the baseline when layer norm is used.
--You used k=10 for all the experiments. Is there any motivation behind that? 10 time steps only correspond to 1 to 2 English words for character MT.
--Do you have any results with word-level MT? Does the planning approach help?
--Can you say anything about the computation complexity? Does it slow down training a lot?


Matching on Balanced Nonlinear Representations for Treatment Effects Estimation
Estimating treatment effects from observational data is challenging due to the missing counterfactuals. Matching is an effective strategy to tackle this problem. The widely used matching estimators such as nearest neighbor matching (NNM) pair the treated units with the most similar control units in terms of covariates, and then estimate treatment effects accordingly. However, the existing matching estimators have poor performance when the distributions of control and treatment groups are unbalanced. Moreover, theoretical analysis suggests that the bias of causal effect estimation would increase with the dimension of covariates. In this paper, we aim to address these problems by learning low-dimensional balanced and nonlinear representations (BNR) for observational data. In particular, we convert counterfactual prediction as a classification problem, develop a kernel learning model with domain adaptation constraint, and design a novel matching estimator. The dimension of covariates will be significantly reduced after projecting data to a low-dimensional subspace. Experiments on several synthetic and real-world datasets demonstrate the effectiveness of our approach.

The paper proposes a new matching algorithm for estimation of treatment effects in observational studies. The algorithm is appropriate when the number of covariates is large. It finds a projection of the original covariate space into a lower dimensional one and applies a standard matching approach such as nearest neighbor matching in this new space. The proposed low dimensional nonlinear projection has two objectives: (1) maximize separation between outcomes and (2) maximize the overlap between cases and controls. The new algorithm is evaluated on one synthetic and two real data sets. The results indicate the proposed method is better than several alternatives.
  Positives:
+ the proposed objective function is intuitively appealing
+ the experiments were well designed and the results are positive
+ the paper is easy to read
  Negatives:
- the proposed ordinal scatter discrepancy criterion is a heuristic that is sounds reasonable. However, it would require a deeper experimental or theoretical study to truly understand its advantages and drawbacks.
- Any type of causal study on the observational data needs to come with caveats and understanding of potential pitfalls. This paper is not discussing it and presents the algorithm only in a positive light. It would be useful for the paper to at least have a "buyer beware" discussion.
- There are several moving parts required to make the proposed algorithm work, such as discretizing the outcomes, selecting the kernel, deciding on the dimensionality, selecting the hyperparameters. It would be important to see how sensitive the algorithm is to those choices. From the presented results, it is fair to ask if the shown results might be due to some overfitting
- the paper will need to be carefully proofread -- there are numerous grammar mistakes throughout.


Task-based End-to-end Model Learning in Stochastic Optimization
With the increasing popularity of machine learning techniques, it has become common to see prediction algorithms operating within some larger process. However, the criteria by which we train these algorithms often differ from the ultimate criteria on which we evaluate them. This paper proposes an end-to-end approach for learning probabilistic machine learning models in a manner that directly captures the ultimate task-based objective for which they will be used, within the context of stochastic programming. We present three experimental evaluations of the proposed approach: a classical inventory stock problem, a real-world electrical grid scheduling task, and a real-world energy storage arbitrage task. We show that the proposed approach can outperform both traditional modeling and purely black-box policy optimization approaches in these applications.

--Brief summary of the paper:
The paper proposes a learning method for solving two-stage stochastic programming problems which involve minimizing f(x,y,z) w.r.t. z. The main idea of the paper is to learn a predictive model p(y|x;theta) such that the task's objective function f is directly optimized. In contrast, traditional approaches learn p(y|x;theta) to minimize a prediction error without considering f. The main technical challenge in the paper is to solve a sub-optimization problem involving argmin w.r.t. z, and the proposed method can do so in an efficient manner by assuming that the optimization problem is convex in z. The method is experimentally evaluated on two problems and it is shown to outperform traditional methods.
--Major comments:
The idea of adopting end-to-end learning to solve two-stage stochastic programming is interesting. However, I have a major concern for the proposed method which is the lack of convergence guarantees. Since the optimization problem is assumed to be convex in z, the obtained solution z*(x;theta) is supposed to be the "true" optimal if data is drawn from the true distribution p(x,y). However, a solution obtained using the predictive model p(y|x;theta) is unlikely to be true optimal unless p(y|x;theta) is the true conditional distribution p(y|x). (This issue is commonly known as model bias in the context of model-based reinforcement learning which usually involves non-convex objectives.) Since the proposed method does not theoretically guarantee that p(y|x;theta) converges to p(y|x) even when the model hypothesis is correct, it seems likely that even for a convex optimization problem the method may only find a sub-optimal solution. For this reason, I think having convergence guarantees or error bounds either for the predictive model or for the obtained solution itself are very important to theoretically justify the method and would be a significant contribution to the paper.
--Questions:
1) It is not clear why Algorithm 1 requires mini-batches training since Line 7 of the algorithm only checks the constraint for a single sample.
2) In the first experiment, why does the performance of the end-to-end policy optimization method depend on the model hypothesis when it does not rely on a predictive model?
--Minor suggestions:
1) In line 154 the paper argue that the model-free approach requires a rich policy class and is data inefficient. However, the model-based approach also requires a rich model class as well. Moreover, the model-based approach can suffer from model bias while the model-free approach cannot.
2) The applicability of the proposed method is quite limited. As mentioned in the paper, solving a sub-optimization problem with argmin is not trivial and convexity assumption can help in this regard. However, practical decision making problems may involve non-convex or unknown objective functions. A variant of the proposed method that is applicable to these tasks would make the method more appealing.
3) The last term of Eq.(4) should have an expectation over the density of x.
--Comments after author's response:
I feel more positive about the paper after reading the author’s response. Now I think that the proposed method is an important contribution to the field and I will increase my score. However, I am still not convince that the proposed method will be useful outside domains with convex objectives without empirical evidences.


Approximation Algorithms for 0 -Low Rank Approximation
We study the 0 -Low Rank Approximation Problem, where the goal is, given an m × n matrix A, to output a rank-k matrix A for which A − A 0 is minimized. Here, for a matrix B, B 0 denotes the number of its non-zero entries. This NPhard variant of low rank approximation is natural for problems with no underlying metric, and its goal is to minimize the number of disagreeing data positions. We provide approximation algorithms which significantly improve the running time and approximation factor of previous work. For k > 1, we show how to find, in poly(mn) time for every k, a rank O(k log(n/k)) matrix A for which
To the best of our knowledge, this is the first algorithm with provable guarantees for the 0 -Low Rank Approximation Problem for k > 1, even for bicriteria algorithms. For the well-studied case when k = 1, we give a (2+ )-approximation in sublinear time, which is impossible for other variants of low rank approximation such as for the Frobenius norm. We strengthen this for the well-studied case of binary matrices to obtain a (1 + O(ψ))-approximation in sublinear time, where ψ = OPT / A 0 . For small ψ, our approximation factor is 1 + o(1).

This paper considers the problem of approximating matrices when no clear metric is present on the data and the l_0 norm is used (the number of non-zero elements in the difference between the original data and the approximation).
A low-rank solution is proposed.
The paper doesn't seem to give a practical case where this is useful.
In fact, "low rank" implies some linearity and hence implicitly a metric, which lay not make a lot of sense if we can't interprete the data with a metric to start with.  E.g., for k=1 one can easily make the first row and then one element in each other row of A-A' equal to zero (so 2n-1 elements), but then predicting linearly the values of the other elements seems arbitrarily.
Also, if the elements in A are continuous variables drawn from a distribution with no linear properties, the probability that for any low rank approximation A' more than these minimal number of elements of A-A' can be made equal to *exactly* zero (the requirement of the l_0 norm) is equal to 0.
In that sense, if the data is drawn randomly from a continuous distribution, with very high probability, OPT^{(k)} is something like (n-k)(m-k).  If k is "low", then theorems as Theorem 2 talking about multiples of OPT^{(k)} are rather trivial (because O(k^2) OPT^{(k)} > 2.OPT^{(k)} > mn) for "random" data.
So maybe the problem is to "discover" that by some external cause the data is very close already to low rank, in the sense that a lot of elements happen to match exactly the linear low rank relation to other elements.
In amongst others line 272, the authors too indicate that the work is most interesting if A is already close to low rank.
The mathematical elaboration and randomized algorithms are quite interesting from a theory point of view.
Moving some details to the supplementary material, and making a consistent story connecting the presented algorithms to clear learning problems, could make this a nice paper.
Some details:
* Line 48: supplementary -> supplementary material
* Line 244: having two times "w.h.p." in the same statement is not really necessary (but could be a compact way to present the result).


Positive-Unlabeled Learning with Non-Negative Risk Estimator
From only positive (P) and unlabeled (U) data, a binary classifier could be trained with PU learning, in which the state of the art is unbiased PU learning. However, if its model is very flexible, empirical risks on training data will go negative, and we will suffer from serious overfitting. In this paper, we propose a non-negative risk estimator for PU learning: when getting minimized, it is more robust against overfitting, and thus we are able to use very flexible models (such as deep neural networks) given limited P data. Moreover, we analyze the bias, consistency, and mean-squared-error reduction of the proposed risk estimator, and bound the estimation error of the resulting empirical risk minimizer. Experiments demonstrate that our risk estimator fixes the overfitting problem of its unbiased counterparts.

Summary
The paper builds on the literature of Positive and Unlabeled (PU) learning and formulates a new risk objective that is negatively bounded. The need is mainly motivated by the fact that the state of the art risk formulation for PU can be unbounded from below. This is a serious issue when dealing with models with high capacity (such as deep neural networks), because the model can overfit.
A new lower-bounded formulation is provided. Despite not being unbiased, the new risk is consistent and its bias decreases exponentially with the sample size. Generalization bounds are also proven.
Experiments confirm that state of the art cannot be used with high-capacity models, while the new risk is robust to overfitting and can be applied for classification with simple deep neural networks.

Comments
The paper is very well written, easy to follow and tells a coherent story.
The theoretical results are flawless from my point of view; I have checked the proofs in the supplementary material. The theory is also supported by many comments helping intuition of the arguments behind the proofs. This is very good. I am only confused by the meaning of those two sentences in Section 4:
- Theorem 3 is not a necessary condition even for meeting (3): what is the condition the authors refer to?
- In the proof, the reduction in MSE was expressed as a Lebesgue-Stieltjes integral [29]: why is this relevant?
The authors successfully show that optimisation of this new risk formulation may be effective in practice, on deep neural networks, with only a few (positively) labeled data points. From this, it could help to see what is the point of view of the authors in comparing PU with current trends in deep learning, such as few-shots learning.
The dataset `epsilon` is never mentioned before talking about the model trained on it; I find that confusing for the reader.
Experimental results well address the main point of the paper (i.e. loweboundness/overfitting), but could be expanded. Fig. 1 shows two graphs with a very similar information: PU overfits when training a deep net. A curious reader who does not know about [16] could wonder whether the previous approach actually works at all. Why not showing the same graph (for one among the 2 losses) on a scenario where [16] is effective? For example, given the thesis of the paper, [16] should work fine if we train a linear model on MNIST, right?
Additionally, it would be interesting to see what is the effect of estimating the class-prior on performance. Experiments in Fig. 2 are designed to show that NNPU can be even better than PN. But what happens if the class priors are estimated by a state of the art method? Since the paper is mostly focus on the theory side, I think this lack is not a major issue for acceptance. Although it will be for any realistic implementation of the proposed method.
Minor
53: Menon et al. ICML 2015 is another valid method for estimating class priors other than [22, 23, 24]
56: Curiosity on notation: why not denoting R_n(g) as R_n^-(g) ? That would be symmetric with R_n^+ (same for the empirical risks)
237: performance of [35] on CIFAR10 has been surpassed, see
https://www.eff.org/ai/metrics or
http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html#43494641522d3130
I think it might be said that [35] is "among the best performing architectures for classifying on CIFAR10"


Identification of Gaussian Process State Space Models
The Gaussian process state space model (GPSSM) is a non-linear dynamical system, where unknown transition and/or measurement mappings are described by GPs. Most research in GPSSMs has focussed on the state estimation problem, i.e., computing a posterior of the latent state given the model. However, the key challenge in GPSSMs has not been satisfactorily addressed yet: system identification, i.e., learning the model. To address this challenge, we impose a structured Gaussian variational posterior distribution over the latent states, which is parameterised by a recognition model in the form of a bi-directional recurrent neural network. Inference with this structure allows us to recover a posterior smoothed over sequences of data. We provide a practical algorithm for efficiently computing a lower bound on the marginal likelihood using the reparameterisation trick. This further allows for the use of arbitrary kernels within the GPSSM. We demonstrate that the learnt GPSSM can efficiently generate plausible future trajectories of the identified system after only observing a small number of episodes from the true system.

The authors derive a variational objective for inference and hyperparameter learning in a GPSSM. The authors apply a mean field variational approximation to the distribution over inducing points and a Gaussian approximation with Markov structure to the distribution over the sequence of latent states. The parameters of the latter depend on a bi-RNN. The variational bound is optimised using doubly stochastic gradient optimisation.
The authors apply their algorithm to three simulated data examples, showing that particular applications may require the ability to flexibly choose kernel functions and that the algorithm recovers meaningful structure in the latent states.
Overall, the paper is well written and provides an interesting combination of sparse variational GP approximations and methods for armotised inference in state space models. However, it is difficult to judge how much the proposed algorithm adds compared to existing sparse variational GPSSM models, since the authors did not make any comparisons or motivate the algorithm with an application for which existing approaches fail. From the paper alone, it is not clear whether related approaches could perform similarly on the given examples or what extensions of the model make it possible to go beyond what previous approaches can do. Comparisons to related models and a more quantitative analysis of the given algorithm would be essential for this. (The authors include RMSE comparisons to a Kalman filter and autoregressive GP in the rebuttal)
Even though the motivation of the paper focuses on system identification, it would be interesting to see if the given approach is successful in correctly learning the model parameters (i.e. linear mapping of observations) and hyperparameters. Currently all results refer to the latent states or transition function. This would be interesting in order to evaluate biases that may be introduced via the choice of approximations and could also help with highlighting the need for the approximate inference scheme presented here (e.g. if a naive Gaussian parameterisation without the bi-RNN introduces more biases in learning).
Lastly, the authors do not apply their algorithm to any real data examples, which again would be useful for motivating the proposed algorithm.


Accuracy First: Selecting a Differential Privacy Level for Accuracy-Constrained ERM
Traditional approaches to differential privacy assume a fixed privacy requirement ε for a computation, and attempt to maximize the accuracy of the computation subject to the privacy constraint. As differential privacy is increasingly deployed in practical settings, it may often be that there is instead a fixed accuracy requirement for a given computation and the data analyst would like to maximize the privacy of the computation subject to the accuracy constraint. This raises the question of how to find and run a maximally private empirical risk minimizer subject to a given accuracy requirement. We propose a general "noise reduction" framework that can apply to a variety of private empirical risk minimization (ERM) algorithms, using them to "search" the space of privacy levels to find the empirically strongest one that meets the accuracy constraint, and incurring only logarithmic overhead in the number of privacy levels searched. The privacy analysis of our algorithm leads naturally to a version of differential privacy where the privacy parameters are dependent on the data, which we term ex-post privacy, and which is related to the recently introduced notion of privacy odometers. We also give an ex-post privacy analysis of the classical AboveThreshold privacy tool, modifying it to allow for queries chosen depending on the database. Finally, we apply our approach to two common objective functions, regularized linear and logistic regression, and empirically compare our noise reduction methods to (i) inverting the theoretical utility guarantees of standard private ERM algorithms and (ii) a stronger, empirical baseline based on binary search.

Summary:
This paper considers a different perspective on differentially private algorithms where the goal is to give the strongest privacy guarantee possible, subject to the a constraint on the acceptable accuracy. This is in contrast to the more common setting where the we wish to achieve the highest possible accuracy for a given minimum level of privacy.
The paper introduces the idea of ex-post differential privacy which provides similar guarantees to differential privacy, but for which the bound on privacy loss is allowed to depend on the output of the algorithm. They then present a general method for private accuracy constrained empirical risk minimization which combines ideas similar to the noise reduction techniques of Koufogiannis et al. and the Above Threshold algorithm. At a high level, their algorithm computes ERM models for increasing privacy parameters (using the noise reduction techniques, rather than independent instantiations of a private learning algorithm) and then uses the above threshold algorithm to find the strongest privacy parameter satisfying the accuracy constraint. The noise reduction techniques allow them to pay only the privacy cost for the least private model they release (rather than the usual sum of privacy costs for the first k models, which would be required under the usual additive composition laws) and the modified above threshold algorithm has privacy cost scaling only logarithmically with the number of models tested.
They apply this algorithm to logistic and ridge regression and demonstrate that it is able to provide significantly better ex-post privacy guarantees than two natural baselines: inverting the privacy privacy guarantees to determine the minimum value of epsilon that guarantees sufficiently high accuracy, and a doubling trick that repeatedly doubles the privacy parameter until one is found that satisfies the accuracy constraint.
Comments:
Providing algorithms with strong utility guarantees that also give privacy when possible is an important task and may be more likely to be adopted by companies/organizations that would like to preserve privacy but are unwilling to compromise on utility. The paper is clearly written and each step is well motivated, including discussion of why simpler baselines either do not work or would give worse guarantees.


Expectation Propagation for t-Exponential Family Using q-Algebra
Exponential family distributions are highly useful in machine learning since their calculation can be performed efficiently through natural parameters. The exponential family has recently been extended to the t-exponential family, which contains Student-t distributions as family members and thus allows us to handle noisy data well. However, since the t-exponential family is defined by the deformed exponential, an efficient learning algorithm for the t-exponential family such as expectation propagation (EP) cannot be derived in the same way as the ordinary exponential family. In this paper, we borrow the mathematical tools of q-algebra from statistical physics and show that the pseudo additivity of distributions allows us to perform calculation of t-exponential family distributions through natural parameters. We then develop an expectation propagation (EP) algorithm for the t-exponential family, which provides a deterministic approximation to the posterior or predictive distribution with simple moment matching. We finally apply the proposed EP algorithm to the Bayes point machine and Student-t process classification, and demonstrate their performance numerically.

The authors proposed an expectation propagation algorithm that can work with distributions in the t-exponential family, a generalization of the exponential family that contains the t-distribution as a particular case. The proposed approach is based on using q-algebra operations to work with the pseudo additivity of t-exponential distributions. The gains of the proposed EP algorithm with respect to assumed density filtering are illustrated in experiments with a Bayes point machine. The authors also illustrate the proposed EP algorithms in experiments with a t-process for classification in the presence of outliers.
Quality
The proposed method seems technically sound, although I did not check the derivations in detail. The experiments illustrate that the proposed method works and it is useful.
Clarity
The paper is clearly written and easy to follow and understand. I only miss an explanation of how the authors compute the integrals in equation (33) in the experiments. I assume that they have to integrate likelihood factors with respect to Student t distributions. How is this done in practice?
Originality
The paper is original. Up to my knowledge, this is the first generalization of EP to work with t-exponential families.
Significance
The proposed method seems to be highly significant, extending the applicability of EP with analytic operations to a wider family of distributions, besides the exponential one.