Collaborative Recurrent Autoencoder: Recommend while Learning to Fill in the Blanks
Hybrid methods that utilize both content and rating information are commonly used in many recommender systems. However, most of them use either handcrafted features or the bag-of-words representation as a surrogate for the content information but they are neither effective nor natural enough. To address this problem, we develop a collaborative recurrent autoencoder (CRAE) which is a denoising recurrent autoencoder (DRAE) that models the generation of content sequences in the collaborative filtering (CF) setting. The model generalizes recent advances in recurrent deep learning from i.i.d. input to non-i.i.d. (CF-based) input and provides a new denoising scheme along with a novel learnable pooling scheme for the recurrent autoencoder. To do this, we first develop a hierarchical Bayesian model for the DRAE and then generalize it to the CF setting. The synergy between denoising and CF enables CRAE to make accurate recommendations while learning to fill in the blanks in sequences. Experiments on real-world datasets from different domains (CiteULike and Netflix) show that, by jointly modeling the order-aware generation of sequences for the content information and performing CF for the ratings, CRAE is able to significantly outperform the state of the art on both the recommendation task based on ratings and the sequence generation task based on content information.

This paper proposes a joint model of sequential content signals (such as movie plot text) and user ratings.  A stochastic recurrent neural network (RNN) is used to model the former, then producing a latent representation v that serves as the item component in the factorization of the ratings matrix.  Several auxiliary techniques are proposed to improve performance: the aforementioned noise in the RNN, a dropout-like technique called 'wildcard denoising', and a Beta pooling operation.  Experiments against five baseline models are performed on the CiteULike and Netflix datasets for recommendation and sequence generation tasks.  The CRAE shows superior to competitive recall at k and superior BLEU scores.
Making recommendations by exploiting the plethora of text data that accompanies products seems like an under-explored area, especially using recent advances in deep language models.  I commend the authors for contributing to this research direction.  The CRAE seems to work well (at least in recall at k), performing at a high level on two real-world datasets.  However, I think this paper would be a better fit for a more applied conference, such as KDD or RecSys, because there is little novelty to the model's core components.  I'll address each individually, in order of (my perceived) importance:  1) Robust Recurrent Networks (RRN): The proposed RRN uses distributional activations that are backpropagated through directly.  If this development was indeed as novel as the paper seems to suggest, then my rating would have been higher (probably "poster level" in the 'novelty' category).  However, these exact techniques are used in "Fast Dropout Training" by S. Wang and C. Manning: http://nlp.stanford.edu/pubs/sidaw13fast.pdf  This paper is not cited but absolutely should be.  The RRN is simply a recurrent version of the Wang-Manning model.  Furthermore, considering the RNN from Wang & Manning's asymptotic dropout perspective, I'm not so sure the hidden activations should be drawn from normal distributions since traditional feedforward dropout does not work well in RNNs.  Alternatives preserving activations over time seem to work better (see https://arxiv.org/abs/1606.01305 and http://arxiv.org/abs/1512.05287).  2) "CRAE as a Hierarchical Bayesian Model": Despite the RRN's lack of novelty, Bayesian learning of a model as complex as the CRAE on datasets as large as Netflix would have been a laudable contribution.  However, only MAP estimation is performed, thus relegating the priors to regularization terms.  It's unclear how Bayesian learning of this model could be done at any realistic scale (which the authors admit, line 178) and so I don't think it's accurate to call it a Bayesian model (for example, one wouldn't call a neural network with L2 regularization a Bayesian neural network).  3) Wildcard denoising: Sampling a 'wildcard' word does seem to be a novel regularization technique that others may find useful.  Although, the development itself is not novel enough to significantly increase the paper's score.    4) Beta pooling:  The authors suggest pooling latent representations via partitioning the Beta distribution and weighting each vector by the corresponding area under the pdf.  This would be interesting if the Beta parameters were learned (which they can be, as the supplementary materials discuss), but they seem to be set a priori in the experiments (from line 186).  When the hyperparameters are fixed, the method reduces to weighted average pooling (I think?).  This is not significantly new enough to warrant an increase from 'sub-standard'.    Next, I'll move on to address the experiments.  The authors test the CRAE for making recommendations using recall at K, saying precision at K "is not suitable" "due to the fact that [a zero entry may denote] the user is not interested in the item, or that the user is not aware of its existence."  While I agree with this statement, the Netflix dataset has ratings--why not report precision at k for that data?


Conditional Generative Moment-Matching Networks
Maximum mean discrepancy (MMD) has been successfully applied to learn deep generative models for characterizing a joint distribution of variables via kernel mean embedding. In this paper, we present conditional generative momentmatching networks (CGMMN), which learn a conditional distribution given some input variables based on a conditional maximum mean discrepancy (CMMD) criterion. The learning is performed by stochastic gradient descent with the gradient calculated by back-propagation. We evaluate CGMMN on a wide range of tasks, including predictive modeling, contextual generation, and Bayesian dark knowledge, which distills knowledge from a Bayesian model by learning a relatively small CGMMN student network. Our results demonstrate competitive performance in all the tasks.

This paper presents conditional generative moment matching networks, an extension of GMMNs to conditional generation / prediction applications.  The key to the proposed method is the kernel embedding of conditional distributions and conditional MMD metric for measuring discrepancy of conditional distributions.  The proposed CGMMN was tested on classification, conditional generation and distilling Bayesian models.
The naive approach of extending GMMNs to conditional setting is to estimate a GMMN for each conditional distribution, and all these conditional distributions share parameters through the use of the same neural network.  The problem of this approach is that each conditional distribution only has very few examples, and in the case of continuous domain for the conditioning variables, each conditional distribution may only have one single example, causing data sparsity problem.  The proposed approach treats all the conditional distributions as a family and tries to match the model with the conditional embedding operator directly rather than matching each individual conditional distributions.  The advantage of the proposed approach seems clear, but in some cases I can still see the naive approach do a reasonable job, for example in conditional generation where the conditioning variable takes one of 10 values as in MNIST.  It would be interesting to compare to such a naive approach as a baseline.  It is nice to see that the conditional MMD objective has a nice formulation, and can be easily computed almost in the same way as the unconditional MMD as pointed out in Remark 1.  All the K matrices in equation 2 are independent from the model parameters, and backpropagation can be done in the same way as before.  One concern about the proposed approach is that the CMMD objective potentially requires big minibatches, otherwise the stochasticity of the minibatches may dominate the error, on the other hand big minibatches makes the computation of the objective expensive, especially the matrix inverses could be prohibitively expensive to compute.  Overall I think the CMMD provides a nice way to model conditional distributions.  The experiment results on classification and distillation of Bayesian models are also interesting.  The presentation in section 2.3 and 3.1 wasn’t easy to follow.  I have to dive into the referenced papers to understand more about what is actually going on.  There are a couple minor problems:  (1) on line 77, it is mentioned that the MMD estimator is unbiased, which is not true. (2) on line 90, the conditional embedding operator was not consistent with the one given in reference [29].  To get the form used in this paper it seems we need a different definition of the cross covariance operator.


Minimizing Quadratic Functions in Constant Time
A sampling-based optimization method for quadratic functions is proposed.
Our method approximately solves the following n-dimensional quadratic minimization problem in constant time, which is independent of n:
n×n is a matrix and d, b ∈ R n are vectors. Our theoretical analysis specifies the number of samples k(δ, ) such that the approximated solution z satisfies |z − z * | = O( n 2 ) with probability 1 − δ. The empirical performance (accuracy and runtime) is positively confirmed by numerical experiments.
complexity. However, we still need to compute the matrix-vector product of size n, which requires O(n) time. Clarkson et al. [7] proposed sublinear-time algorithms for special cases of quadratic function minimization. However, it is "sublinear" with respect to the number of pairwise interactions of the variables, which is O(n 2 ), and their algorithms require O(n log c n) time for some c ≥ 1.
Our contributions: Let A ∈ R n×n be a matrix and d, b ∈ R n be vectors. Then, we consider the following quadratic problem:
Here, ·, · denotes the inner product and diag(d) denotes the matrix whose diagonal entries are specified by d. Note that a constant term can be included in (1); however, it is irrelevant when optimizing (1), and hence we ignore it.
Let z * ∈ R be the optimal value of (1) and let , δ ∈ (0, 1) be parameters. Then, the main goal of this paper is the computation of z with |z − z * | = O( n 2 ) with probability at least 1 − δ in constant time, that is, independent of n. Here, we assume the real RAM model [6], in which we can perform basic algebraic operations on real numbers in one step. Moreover, we assume that we have query accesses to A, b, and d, with which we can obtain an entry of them by specifying an index. We note that z * is typically Θ(n 2 ) because v, Av consists of Θ(n 2 ) terms, and v, diag(d)v and b, v consist of Θ(n) terms. Hence, we can regard the error of Θ( n 2 ) as an error of Θ( ) for each term, which is reasonably small in typical situations.
Let ·| S be an operator that extracts a submatrix (or subvector) specified by an index set S ⊂ N; then, our algorithm is defined as follows, where the parameter k := k( , δ) will be determined later.

The authors prove that it is possible to approximately compute the minimum of quadratic functions by solving a sub-sampled instance in a constant number of coordinates. Their main tool is a proving a weak-regularity lemma sort of result for quadratic functions. They utilize tools developed for defining limits of dense graphs.  The authors present evidence for the method by solving several synthetic instances, and also apply the algorithm to quickly estimate Pearson divergence in several (synthetic) instances.
The main novelty of the paper is to in fact _prove_ that as long as the quadratic minimization problem has bounded coefficients, solving a subsampled instance can provide a meaningful approximation. The statement / proof is technically challenging, even though a lot of the tools were already developed in the context of graph limits.   I am of the opinion that the paper needs to do more to convince that this can in fact be practically useful. Theoretically, the requirement for number of partitions (and hence samples) to be 2^(1/eps^2) is necessary (see Bounds for graph regularity and removal lemmas, Conlon and Fox). Such a high sample complexity would render the result not useful in practice. The authors claim that their experiments suggest that much smaller sample complexities suffice. However, to me, the (synthetic) experiments seem unconvincing. The instances are drawn from a large random distribution. For these instances, I suspect that concentration of measure phenomenon already implies that the optimas are tightly concentrated. The authors should at least report the range of optima and the range of answers generated by the algorithm, and hopefully design more experiments that can demonstrate practical impact.   Minor comments / typos: - Lem 3.2 — There is something incorrect in the claim here. The dependence on K should be K^2 because of linearity, as can be seen by applying the lemma to f / K and g/K. I suspect the first line has a mistake. An alternate proof wouldThe proof could integrate over L = {x | h(x) <= tau}, whereby there would be no tau_1, tau_2 in the integrand. - Cor 3.4 — The second last claim ‘Since P^t is a refinement of the …,’ is the key component of the proof, and needs a proof / reference. - Lem 3.5 — The lhs should also have a factor L


Stochastic Gradient Methods for Distributionally Robust Optimization with f -divergences
We develop efficient solution methods for a robust empirical risk minimization problem designed to give calibrated confidence intervals on performance and provide optimal tradeoffs between bias and variance. Our methods apply to distributionally robust optimization problems proposed by Ben-Tal et al., which put more weight on observations inducing high loss via a worst-case approach over a non-parametric uncertainty set on the underlying data distribution. Our algorithm solves the resulting minimax problems with nearly the same computational cost of stochastic gradient descent through the use of several carefully designed data structures. For a sample of size n, the per-iteration cost of our method scales as O(log n), which allows us to give optimality certificates that distributionally robust optimization provides at little extra cost compared to empirical risk minimization and stochastic gradient methods.

This paper focuses on solving the robust empirical risk minimization problem, which is in a "min-max" structure and has the ability to automatically trade between loss and variance. Previously this problem is usually solved by using SGD after taking the dual of the inner supremum, however it might not work well because of the unbounded variance of the subgradients.  This paper presents a new solver for the robust empirical risk minimization problem. It is inspired by a mirror descent based method to solve two-player convex games. The solver consists of two alternating mirror descent/ascent steps, one is for minimization and the other one is for maximization. The main difference between this work and previous works is that the authors generalize this method to Cressie-Read family f-divergence constrained cases.   The authors proved regret bounds for different cases of f-divergences in the Cressie-Read family, including important cases like KL divergence and Chi-square divergence. The convergence shown is sublinear, in O(\sqrt T), which match the regret bounds of common stochastic methods. For the special case of Chi-square divergence, the authors uses a trick proposed in [11] to run the projection step in O(log n) time.  The authors also conduct experiments on synthetic and real data, and try to show that the proposed solver is better than SGD and IPM (interior point method) in terms of CPU time and convergence, also the robust solution does provided stability as expected.
Overall I think the work in this paper is valid and valuable. The first 3 sections are well written. However I think the experiment section looks weak and needs more refinements (see below) before the paper can be published.  In the experiments, the proposed algorithm uses the best step size given in section 3, but for SGD, Armijo's backtracking line search is used. I think this could be a reason why SGD is slower, because line search is slow. The authors should compare the two algorithms both using line search, or both using a fixed *best* step size.  In the introduction section the authors show that stochastic gradient may fail due to the unbounded objective and subgradients. It would be nice to show a dataset that has this problem, and how the two-player bandit mirror descent algorithm can improve that. This will make this paper stronger and more people will pick it up.  Some more explanations and discussions should be added for Figure 2a and 2b, and the legends on some figures can be made better (like in figure 1, "SGD-minmax" can be replaced with "proposed" to avoid confusion with the SGD algorithm being compared). Some missing references (shown as ?) should also be fixed.  In section 4, the texts do not explicitly say which Bregman divergence is used. Following the derivation (in KKT condition) I think it's $\frac{1}{2} \sum_1^n p_i^{2}$. Maybe you can make it more clear in section 4. Also, does this O(log n) trick for solving \lambda work for other Bregman divergences?


Constraints Based Convex Belief Propagation
Inference in Markov random fields subject to consistency structure is a fundamental problem that arises in many real-life applications. In order to enforce consistency, classical approaches utilize consistency potentials or encode constraints over feasible instances. Unfortunately this comes at the price of a tremendous computational burden. In this paper we suggest to tackle consistency by incorporating constraints on beliefs. This permits derivation of a closed-form message-passing algorithm which we refer to as the Constraints Based Convex Belief Propagation (CBCBP). Experiments show that CBCBP outperforms the conventional consistency potential based approach, while being at least an order of magnitude faster.

The authors propose belief propagation for discrete graphical models with certain consistency constraints, related to PN-potentials. The contribution lies in deriving closed-form solutions to the belief-propagation operations in this more general model and showing empirically that it is advantageous over naively doing belief-propagation on general factors explicitly modeling the consistency constraints.
General comments:  (i) The authors only solve a new special kind of higher order consistency constraints, generalizing soft PN-potentials, but not a truly general class of constraints, as indicated in the title or in the abstract.  (ii) I do not agree with what the authors say in lines 72 - 78. In case of MAP-inference, which is normally desired, the goal is to obtain a single assignment which satisfies all given linear constraints. The proposed model (i.e. computing marginals) is then less desirable. The relaxed model the authors optimize is simply a byproduct of looking for marginals instead of MAP-assignments (the added entropy is responsible for this). In case of vanishing entropy one gets the same model. Hence there certainly remains the disadvantage of a parameter in the PN-potential, but now hidden in the entropy. Additionally, when a MAP-solution is wanted, the proposed algorithm is in fact disadvantageous, as inference is not done w.r.t. the energy of a single MAP-solution and rounding results in some arbitrariness of the obtained solution.  (iii) Experimental comparison: - No comparison against solving MAP-inference with PN-potentials and existing inference algorithms is given. - Experiments are very small scale, e.g. image segmentation is only done on superpixels. I do not consider such microbenchmarks very informative. One can usually just plug everything into an off-the-shelf LP-solver in such cases. - To be able to really judge the algorithm, convergence plots would be helpful, but none are given. In general, I deem experimental comparison unconvincing: while many experiments have been performed, no real comparison against any algorithm other than CBP is performed.  (iv) Related work: Many references to work on inference with higher order potentials are lacking. To name a few: - Komodakis: Beyond pairwise energies: Efficient optimization for higher-order MRFs  - Tarlow et al.: HOP-MAP: Efficient Message Passing with High Order Potentials. - Kappes: Higher-order Segmentation via Multicuts E.g., the work of Kappes shows how to very efficiently include PN-potentials in an LP-relaxation. Generally, efficient ways for PN-potentials have been explored before, which is not acknowledged in the author's text.  (v) The work is rather incremental: The main contribution is the derivation of a new updating formula for convex belief propagation and consistency constraints.  Detailed comments: - The entropy H is never defined. - Notation is scattered around: Some is defined in section 2 and some is defined in section 3, but only informally in the text, making the article harder to read. - Line 8: What is the standard approach? - Line 71: which is expect -> which is expected - Line 96: One can also derive duals when the primal is not strictly convex (duals exist even for non-convex programs).


A Pseudo-Bayesian Algorithm for Robust PCA
Commonly used in many applications, robust PCA represents an algorithmic attempt to reduce the sensitivity of classical PCA to outliers. The basic idea is to learn a decomposition of some data matrix of interest into low rank and sparse components, the latter representing unwanted outliers. Although the resulting problem is typically NP-hard, convex relaxations provide a computationally-expedient alternative with theoretical support. However, in practical regimes performance guarantees break down and a variety of non-convex alternatives, including Bayesian-inspired models, have been proposed to boost estimation quality. Unfortunately though, without additional a priori knowledge none of these methods can significantly expand the critical operational range such that exact principal subspace recovery is possible. Into this mix we propose a novel pseudo-Bayesian algorithm that explicitly compensates for design weaknesses in many existing non-convex approaches leading to state-of-the-art performance with a sound analytical foundation.

The paper presents a variational algorithm for robust Bayesian PCA.
Update after author feedback: I thank the authors for the clarification of the purpose of the paper. Including that in the text might make it easier for all your readers to appreciate the paper.  ---------------------------------------  Technical quality:       The method seems reasonable. The experiments are weak: all the experiments appear at least partly synthetic in terms of added noise.         Novelty:       The method is an interesting novel combination of existing ideas. The main novelty appears to be a new bound on a joint column-wise and row-wise prior.        Impact:       On paper, the method looks very good and one would expect it to attract a lot of users. I am however underwhelmed by the experiments: the authors have do not seem to have been able to find a real practical killer application for their method which raises some doubt of how useful it will be in practice.        Clarity:       The paper is difficult to read because there appears to be too much material crammed in. There are many comments that could be expanded or supported by relevant references (e.g. lines 34, 82-83, Eq. (7) and its relation to ARD, line 135, proof of Proposition 1, ...). I feel the submission might work better as a journal paper than a brief conference paper.        I am not sure why the authors have chosen to call their method pseudo-Bayesian, which seems like a very uninformative name. The method seems like a good examples of a variational algorithm in the spirit of Jordan et al., i.e. not of the KL divergence minimising type. At least I would find the term "variational" much more informative as a characterisation.              The experiments are described very briefly and many important parameters and their values are not described, especially how r is chosen. The metrics used in all the experiments should be presented and the parameters defining success justified. The sensitivity of the results to any arbitrary parameters should be analysed.              There is no discussion on weaknesses and limitations of the method.


Dual Learning for Machine Translation
While neural machine translation (NMT) is making good progress in the past two years, tens of millions of bilingual sentence pairs are needed for its training. However, human labeling is very costly. To tackle this training data bottleneck, we develop a dual-learning mechanism, which can enable an NMT system to automatically learn from unlabeled data through a dual-learning game. This mechanism is inspired by the following observation: any machine translation task has a dual task, e.g., English-to-French translation (primal) versus French-to-English translation (dual); the primal and dual tasks can form a closed loop, and generate informative feedback signals to train the translation models, even if without the involvement of a human labeler. In the dual-learning mechanism, we use one agent to represent the model for the primal task and the other agent to represent the model for the dual task, then ask them to teach each other through a reinforcement learning process. Based on the feedback signals generated during this process (e.g., the languagemodel likelihood of the output of a model, and the reconstruction error of the original sentence after the primal and dual translations), we can iteratively update the two models until convergence (e.g., using the policy gradient methods). We call the corresponding approach to neural machine translation dual-NMT. Experiments show that dual-NMT works very well on English↔French translation; especially, by learning from monolingual data (with 10% bilingual data for warm start), it achieves a comparable accuracy to NMT trained from the full bilingual data for the French-to-English translation task.

This paper presents a model called "communication-based machine translation (CMT), where translation models for the two translation directions are improved jointly by forth- and back-translating monolingual data and collecting feedback after each sampled translation. The goal is to leverage monolingual data in an effective way for improving machine translation based on bilingual corpora. The model is evaluated on two language pairs and compared to a standard neural MT system and a system that is additionally trained on pseudo-bilingual data.
The same goal has been pursued by e.g. including a language model on the target side for re-scoring translations [1] or include it into the MT system via deep or shallow fusion [2]. The paper does not sufficiently review the work that has been done in this direction and only focuses on the recent work by Sennrich et al. Since the goal of exploiting monolingual data for MT has been in the focus of many works, more empirical comparisons are needed to demonstrate the superiority of their system. It would have been easy to e.g. use the same data set as in [2] to make a direct comparison. Also, there has been work on the unsupervised training of noisy-channel models [3] which needs to be mentioned. In addition, there are no comparisons to other communication-based learning scenarios in reinforcement learning. Instead of repeatingly describing the basic idea of having two players communicating via two translation models, the paper would benefit from a more careful and thorough review and comparisons of previous work. The authors conclude that this model gives new directions for learning the translation model from scratch from monolingual data, but I personally doubt that this will work, since both Ranzato et. al. and Shen et al. [4] who learn translation models from reinforcement learning observed that a warm-start model is required, because the feedback is too weak. In line 116 they assume that translation starts with "weak" models, but in the end they use "well-trained" (l 152) translation models. The only experiment which suppports that the improvement is really large with monolingual data and a "weak" model is Fr->En.  Also the extension to multiple languages in a translation chain is not as straight-forward as described, because the source of translation errors cannot be detected that easily anymore, so the updates might get very noisy. Although the metaphor of the two-players-game is nice, it does not align well with the actual algorithm. The assumption is that each player only understands one language, so scoring via a language model is possible, but the communication reward requires knowledge of the actual translation model, which is unavailable for the player in the described game. In order to accept the experiments as supporting their model, more details about the models (number of training instances - monolingual and bilingual) need to be reported and better comparisons have to be made (on the original data from the referred work and state-of-the-art as upper bound). To put it in a nutshell, they present a nice (but not so novel) idea, but the paper lacks sufficient (experimental) comparisons to closely related work for me to accept it.  [1] Holger Schwenk. 2012. Continuous space translation models for phrase-based statistical machine translation. In Martin Kay and Christian Boitet, editors, Proceedings of the 24th International Conference on Computational Linguistics (COLIN), pages 1071–1080. Indian Institute of Technology Bombay. [2] Gulcehre, C., Firat, O., Xu, K., Cho, K., Barrault, L., Lin, H. C., ... & Bengio, Y. (2015). On using monolingual corpora in neural machine translation. arXiv preprint arXiv:1503.03535. [3] Mylonakis, Markos, Khalil Sima'an, and Rebecca Hwa. "Unsupervised estimation for noisy-channel models." Proceedings of the 24th international conference on Machine learning. ACM, 2007. [4] Shen, S., Cheng, Y., He, Z., He, W., Wu, H., Sun, M., & Liu, Y. (2015). Minimum risk training for neural machine translation. arXiv preprint arXiv:1512.02433.


Approximate maximum entropy principles via Goemans-Williamson with applications to provable variational methods
The well known maximum-entropy principle due to Jaynes, which states that given mean parameters, the maximum entropy distribution matching them is in an exponential family has been very popular in machine learning due to its "Occam's razor" interpretation. Unfortunately, calculating the potentials in the maximumentropy distribution is intractable [BGS14]. We provide computationally efficient versions of this principle when the mean parameters are pairwise moments: we design distributions that approximately match given pairwise moments, while having entropy which is comparable to the maximum entropy distribution matching those moments. We additionally provide surprising applications of the approximate maximum entropy principle to designing provable variational methods for partition function calculations for Ising models without any assumptions on the potentials of the model. More precisely, we show that we can get approximation guarantees for the log-partition function comparable to those in the low-temperature limit, which is the setting of optimization of quadratic forms over the hypercube. ([AN06])

Considering Ising models (with no local fields), a very interesting twist on the standard variational inference approach is developed to show a convex relaxation and rounding that yields a multiplicative factor approximation to log Z. The factor is O(log chi(G)) where chi(G) is the chromatic number of the model graph. For fully attractive (ferromagnetic) models, this improves to a constant factor of 50.
The paper is clear and well written with helpful background and remarks [I have not checked the proofs in detail but the ideas are presented clearly and I trust that the details are correct]. The methods used and results obtained are very interesting. To my knowledge they are novel and may prove useful in other work. Note that a multiplicative bound on Z (hence an additive bound on log Z) would be much stronger and more useful, but as noted in the paper, this is likely impossible. It is not clear how useful these results will be in the near term in practice, but still they are theoretically important and provide good leads for future work.   The authors focus on `symmetric' Ising models with no local fields/singleton potentials, i.e. x_i \in {-1,+1} and E(x_i)=0 for all i. It is worth noting that if an Ising model does have local fields, it can always be transformed into a larger model without any local fields by adding one extra variable and encoding the original singleton potentials as edge potentials to the added variable. The larger model has exactly twice the partition function of the original model, e.g. see Weller ICML 2016, Uprooting and Rerooting Graphical Models.  Further observations on the following would be welcome: A longer comment on the differences between the analysis here and in Ris16 (so expand on lines 84-88 and 105-108). Are there any results/conjectures on what might be the best possible approximation bounds for these problems, so we can see how close these results are?  Minor points:  Section 2 is very helpful as a clear statement of the main results and prior related work. It would be good to tie it to the later text more neatly:  Definition 3.1 of the constant G should come before it is first used in Theorem 2.1. Clarify where exactly Theorems 2.1-2.4 are proved.  line 32: a -> an 33: from from -> than 35-38: very interesting, in 37: delete one 'there' 41: The -> A (there are other important reasons to compute Z) 45-46: Perhaps mention that we know that the Bethe partition function estimate lower bounds the true value (i.e. Z_B \leq Z) for ferromagnetic models, see Ruozzi NIPS 2012, The Bethe partition function of log-supermodular graphical models, and Weller and Jebara NIPS 2014, Clamping variables and approximate inference. 50: If the configuration with max score is not unique, then log Z -> the max * the multiplicity of the max 57: Perhaps mention that better results can be obtained for various restrictions such as bounded treewidth (Wainwright and Jordan 2004,  Treewidth-based conditions for exactness of the Sherali-Adams and Lasserre relaxations) or if certain minors are not present (Weller UAI 2016, Characterizing Tightness of LP Relaxations by Forbidding Signed Minors). 104: Perhaps move the footnote 3 to the end of 103 to save space. 118: theoretically much more poorly studied - not sure if that's true 121: delete 'what' 238: add space between bound) and of. Delete Same. 253: Insert 'a' before rounding 254, 256, 258 should the 2 be (2) as in \eqref References In several places, caps should be fixed, e.g. Grothendieck -> {G}rothendieck. I believe [BB] should be [BB08] from NIPS 2008.


Minimax Optimal Alternating Minimization for Kernel Nonparametric Tensor Learning
We investigate the statistical performance and computational efficiency of the alternating minimization procedure for nonparametric tensor learning. Tensor modeling has been widely used for capturing the higher order relations between multimodal data sources. In addition to a linear model, a nonlinear tensor model has been received much attention recently because of its high flexibility. We consider an alternating minimization procedure for a general nonlinear model where the true function consists of components in a reproducing kernel Hilbert space (RKHS). In this paper, we show that the alternating minimization method achieves linear convergence as an optimization algorithm and that the generalization error of the resultant estimator yields the minimax optimality. We apply our algorithm to some multitask learning problems and show that the method actually shows favorable performances.

This paper presents a new non-parametric tensor regression method based on kernels. More specifically, the authors proposed a regularization based optimization approach with alternating minimization for non-parametric tensor regression model [15]. Moreover, the theoretical guarantee for the proposed method is presented the paper. Through experiments on various datasets, the proposed method compares favorably with existing state-of-the-art.
This paper presents a new non-parametric tensor regression method based on kernels. More specifically, the authors proposed a regularization based optimization approach with alternating minimization for non-parametric tensor regression model [15]. Moreover, the theoretical guarantee for the proposed method is presented the paper. Through experiments on various datasets, the proposed method compares favorably with existing state-of-the-art.  The paper is clearly written and easy to read. I understand the key contribution of this paper is the theoretical analysis of the non-parametric tensor regression. However, the experimental section of this paper can be improved (See comments below). Overall, the proposed method is simple yet effective and theoretically guaranteed. I am happy to see the paper in NIPS.  Detailed comments: 1. In experiments, the performance of linear and nonlinear methods becomes similar when the number of samples increases. Are there any gap between linear and nonlinear methods with bigger sample size (e.g., 5000)? Also, including more comparison to other state-of-the-art linear tensor regression methods would be useful.  2. Related to the first question. If the performance of the linear and nonlinear methods are similar when the number of samples are large. Then, the experiments with web related data is not so practically interesting (though it is worth validating the method), since the web-data tends to be hundred millions of data points. That is, we might need to simply use a linear method for web-related datasets. Thus, if authors can include some experiments with high-dimensional and small number of samples datasets such as bioinformatics related datasets, the proposed method is more practically interesting. 3. How scale the AMP method with respect to the number of dimension? The method still works for d = 10^4 or more? 4. How the performance of the scaled latent convex regularization method? For completeness, it would be great to include it in either main text or supplementary materials. 5. How fast the AMP algorithm? There is no time comparison in Fig 1. (Author claimed the AMP method is faster in the main text without time comparison). 6. How to initialize the proposed method?


An equivalence between high dimensional Bayes optimal inference and M-estimation
When recovering an unknown signal from noisy measurements, the computational difficulty of performing optimal Bayesian MMSE (minimum mean squared error) inference often necessitates the use of maximum a posteriori (MAP) inference, a special case of regularized M-estimation, as a surrogate. However, MAP is suboptimal in high dimensions, when the number of unknown signal components is similar to the number of measurements. In this work we demonstrate, when the signal distribution and the likelihood function associated with the noise are both log-concave, that optimal MMSE performance is asymptotically achievable via another M-estimation procedure. This procedure involves minimizing convex loss and regularizer functions that are nonlinearly smoothed versions of the widely applied MAP optimization problem. Our findings provide a new heuristic derivation and interpretation for recent optimal M-estimators found in the setting of linear measurements and additive noise, and further extend these results to nonlinear measurements with non-additive noise. We numerically demonstrate superior performance of our optimal M-estimators relative to MAP. Overall, at the heart of our work is the revelation of a remarkable equivalence between two seemingly very different computational problems: namely that of high dimensional Bayesian integration underlying MMSE inference, and high dimensional convex optimization underlying M-estimation. In essence we show that the former difficult integral may be computed by solving the latter, simpler optimization problem.

The goal of this paper is to find a solution to the minimum mean squared error problem (MMSE) by establishing an equivalence between high-dimensional Bayes integral and high-dimensional convex optimization. This paper addresses the open issues of comparing optimal Bayesian inferences with optimal M-estimators and investigates the class of M-estimators for non-additive noise models. The authors consider the modern high-dimensional set-up where N,P both diverge keeping the ratio N/P fixed, an area that has received a lot of attention in the recent past. The connection between Bayes optimal inference and M-estimators is derived analytically through approximate message passing algorithm.
I am not an expert in this area but I liked the author’s approach of connecting M-estimation and Bayes optimal inference through the fixed points of bAMP and mAMP, and I believe this should lead to some advancement in this area through the extension to non-additive noise. I particularly like the correspondence between MAP inference and M-estimators through the smoothing terms. This approach has also the added advantage of adapting to the high-dimensional (and potentially sparse parameters through approximating log-concave parameter distribution) situation as well as the low dimensional situation through the measurement density and certain parameters going to zero. A minor question in this context: the authors stated that the M-estimation will be optimal in the case of a flat parameter distribution such as a Gaussian with very high variance – I am curious to know how it behaves if the parameter distribution is heavy-tailed (such as Cauchy) instead, as some authors in Bayesian literature suggest using Cauchy as an alternative to normal with high variance as non-informative priors (Gelman, A (2006) & Bhadra et al. (2016)). Although I admit this might be beyond the scope of the current discussion.   References:  Gelman, A. (2006) Prior distributions for variance parameters in hierarchical models, Bayesian Analysis.  Bhadra et al. (2016) Default Bayesian Analysis with Global-Local Shrinkage Priors, arXiv preprint arXiv:1510.03516


Using Social Dynamics to Make Individual Predictions: Variational Inference with a Stochastic Kinetic Model
Social dynamics is concerned primarily with interactions among individuals and the resulting group behaviors, modeling the temporal evolution of social systems via the interactions of individuals within these systems. In particular, the availability of large-scale data from social networks and sensor networks offers an unprecedented opportunity to predict state-changing events at the individual level. Examples of such events include disease transmission, opinion transition in elections, and rumor propagation. Unlike previous research focusing on the collective effects of social systems, this study makes efficient inferences at the individual level. In order to cope with dynamic interactions among a large number of individuals, we introduce the stochastic kinetic model to capture adaptive transition probabilities and propose an efficient variational inference algorithm the complexity of which grows linearly -rather than exponentially-with the number of individuals. To validate this method, we have performed epidemic-dynamics experiments on wireless sensor network data collected from more than ten thousand people over three years. The proposed algorithm was used to track disease transmission and predict the probability of infection for each individual. Our results demonstrate that this method is more efficient than sampling while nonetheless achieving high accuracy.

The paper introduces a stochastic kinetic model that is combined with a variational inference algorithm in order to model social dynamics on an individual centre. The authors initially contrast Hidden Markov models and Stochastic Kinetic Models, before formally deriving their model. The essential benefit of the model is that it scales linearly with the number of observed individuals.  The proposed algorithm is evaluated based on empirical data on epidemy dynamics. Compared to sampling approaches, the novel model and its variants promise superior performance both in terms of efficiency and accuracy.
Overall, given the results, the paper presents a promising approach for evaluating social dynamics in large data sets as an alternative to sampling approaches.  The paper is densely written, making it hard to follow at times. There are quite a number of grammatical errors/typos, which make reading the formal sections of the paper challenging, especially if not strongly acquainted with the area. A good example would have helped to guide the reader.  Some typos that I noticed (I eventually stopped noting them down): - Line 55: `... in the large social systems.' (remove the) - Line 70: CHMM has not been introduced as an acronym - Line 72: `... defnied ...' - Line 98: `... probability of (an) event.' - Check paragraph starting in Line 103 for readability.  - Line 107: Is `Variational Inference to ....' correct (Given the use in the remainder of the text, should it be perhaps be `for' or `with'?) - Line 196-198: Check sentence for correctness. - Line 200: `symptom' Missing `s'? - Line 229: `(explain the task later)' Is that a comment? - Table 1: `Poeple' --> People   The evaluation exclusively concentrates on the comparison to sampling techniques (Gibbs, particle filtering). It would be worthwhile to compare it to other approaches introduced as part of the related work in order to get a more differentiated comparison. Moreover, how does the model perform if you pick larger or smaller subsets than the chosen 20% and 10% of the evaluated datasets? (In other words, why did you choose those numbers?)  Overall, the paper makes interesting promises, but those should be better contextualised with the status quo of the field, and be presented in a clearer fashion.


Disease Trajectory Maps
Medical researchers are coming to appreciate that many diseases are in fact complex, heterogeneous syndromes composed of subpopulations that express different variants of a related complication. Longitudinal data extracted from individual electronic health records (EHR) offer an exciting new way to study subtle differences in the way these diseases progress over time. In this paper, we focus on answering two questions that can be asked using these databases of longitudinal EHR data. First, we want to understand whether there are individuals with similar disease trajectories and whether there are a small number of degrees of freedom that account for differences in trajectories across the population. Second, we want to understand how important clinical outcomes are associated with disease trajectories. To answer these questions, we propose the Disease Trajectory Map (DTM), a novel probabilistic model that learns low-dimensional representations of sparse and irregularly sampled longitudinal data. We propose a stochastic variational inference algorithm for learning the DTM that allows the model to scale to large modern medical datasets. To demonstrate the DTM, we analyze data collected on patients with the complex autoimmune disease, scleroderma. We find that DTM learns meaningful representations of disease trajectories and that the representations are significantly associated with important clinical outcomes.

In short, the main contribution of the paper is a framework for learning a low-dimensional representation of time series data.   In this work, the authors propose a probabilistic model that learns a low-dimensional representation of univariate time-series data. The authors learn the model using a stochastic variational inference algorithm tailored to their application, which allows the model to scale to large datasets. Applied to a real world dataset of patients with a complex autoimmune disease, the model appears to learn clinically meaningful representations of disease trajectories.
Overall, this is an important and interesting problem. Increasingly, researchers in ML and healthcare are studying methods for learning from longitudinal (time series) data (as opposed to more traditional approaches that consider only snapshot data). The work is well motivated, and clearly presented. Moreover, the authors include experiments on real data pertaining to patients with a complex chronic disease. Evaluation is two-fold. First look at clusters obtained by clustering low-dimensional representation, and then test relationship between low-dimensional representation and outcome. However, I do have a few concerns regarding this evaluation:   *The authors state that the median number of observations is 2 for TSS data and 3 for PFVC data. This is somewhat concerning given that the proposed model is primarily meant to model trajectories. If half of the patients have only two or fewer measurements, is this really a trajectory? Have the authors experimented with other datasets, or perhaps simulated data?  *Another related concern, is whether it makes sense clinically to compare patients who have 2 measurements with patients who have 22 measurements? Are these even clinically comparable? How does the number of measurements vary across clusters?  *In this particular example, it seems like we could characterize PFVC and TSS trajectories based on extent of changes in measurements and when they occur. Do we really need DTMs to help us identify declining trajectories vs. recovering trajectories? Again, I can see that while these trajectories are relatively simple, there may be disease processes with more complex trajectories to which the proposed method could be applied. Simulations may help show this.   *Quantitative evaluation only sheds so much light on the ability of the low dimensional representation to identify important clinical subgroups. In order to quantitatively evaluate clusters, why not look at outcome purity within each cluster? This idea is somewhat explored in the last figure of the paper, but could be interesting to show the frequency of outcomes within each of the clusters in Figures 1 & 2.  *In the introduction DTM is described as a method for modeling the trajectory of a single clinical marker. How do the clusters captured by PFVC and TSS compare? Can the proposed method capture trends in multivariate trajectories?


Efficient Nonparametric Smoothness Estimation
Sobolev quantities (norms, inner products, and distances) of probability density functions are important in the theory of nonparametric statistics, but have rarely been used in practice, due to a lack of practical estimators. They also include, as special cases, L 2 quantities which are used in many applications. We propose and analyze a family of estimators for Sobolev quantities of unknown probability density functions. We bound the finite-sample bias and variance of our estimators, finding that they are generally minimax rate-optimal. Our estimators are significantly more computationally tractable than previous estimators, and exhibit a statistical/computational trade-off allowing them to adapt to computational constraints. We also draw theoretical connections to recent work on fast two-sample testing and empirically validate our estimators on synthetic data.

The paper proposed a non-parametric estimator for Sobolev quantities (norms, inner products, and distances) of probability density. The estimator operates in the Fourier (frequency) domain, and it unbiasedly estimates the related quantities that are truncated to a certain frequency.   For the theoretical aspect, the convergence rate of the mean square error is provided, which matches several previous results. Moreover, the asymptotic distributions are derived for the inner product estimator and the distance estimator when $p=q$.   For the practical aspect, the computational complexity is emphasized which in linear w.r.t. the sample size $n$. This makes the proposed estimator more computationally tractable than previous estimator. Also, simulations are provided that validates the theory.
Sobolev quantities (norms, inner products, and distances) of probability density are important to a lot of applications in machine learning and signal processing. This paper proposed a non-parametric estimator for such quantities with the computational complexity linear in the sample size $n$. The convergence rate of the mean square error, and the asymptotic distributions are provided for such estimator. Moreover, a few simulations are provided that validates the theory.  Generally speaking, I think this is a pretty nice theoretical paper. The idea is simple and well presented: the estimator looks at the probability distribution in its Fourier domain, and provides an unbiased estimate of the desired quantities that are truncated to a certain frequency. Here are a few suggestions & comments.   1. My biggest concern is the usefulness of the proposed estimator. Choosing to estimate the frequency-truncated version of the Sobolev quantities seems a natural idea to me. However, I am a little less convinced why we should use the proposed estimator in practice instead of using some other methods. For example, we can look the quantity in some other domain rather than the Fourier domain.   The only reason that convinces me is its linear computational complexity. I would be better convinced if some optimality guarantees are presented. For example, some converse results that show the estimator is indeed rate optimal. Or some real world data experiments that show the proposed estimator is robust in practice. Some intuitions suggesting the Fourier domain is reasonable to look at will also help.   Also, I admit I am not an expert in the non-linear functional estimation of continuous distributions. It would be helpful if the authors can clarify a little more why estimating the Sobolev quantities, especially higher order Sobolev quantities, are very useful. A few examples will work here.   2. The paper uses Gaussian distribution and uniform distribution for simulation. It seems a little weird to me to use the Gaussian distribution, which violates the bounded support assumption. Also, it would be better to use some less smooth distributions. Of course, the paper would be much stronger if there were some real-world data experiments presented.  3. There are a few typos in section 7, which, of course, are just some minor issues. For example, line 195 and line 196, I believe it is equation (13) that we should look at instead of (14). Also, line 196 and 199, I think in the expression of $Z_n$, the exponent should be positive instead of negative.


Combinatorial Multi-Armed Bandit with General Reward Functions
In this paper, we study the stochastic combinatorial multi-armed bandit (CMAB) framework that allows a general nonlinear reward function, whose expected value may not depend only on the means of the input random variables but possibly on the entire distributions of these variables. Our framework enables a much larger class of reward functions such as the max() function and nonlinear utility functions. Existing techniques relying on accurate estimations of the means of random variables, such as the upper confidence bound (UCB) technique, do not work directly on these functions. We propose a new algorithm called stochastically dominant confidence bound (SDCB), which estimates the distributions of underlying random variables and their stochastically dominant confidence bounds. We prove that SDCB can achieve O(log T ) distribution-dependent regret andÕ( √ T ) distribution-independent regret, where T is the time horizon. We apply our results to the K-MAX problem and expected utility maximization problems. In particular, for K-MAX, we provide the first polynomial-time approximation scheme (PTAS) for its offline problem, and give the firstÕ( √ T ) bound on the (1− )-approximation regret of its online problem, for any > 0.

Paper 903 generalizes the well-studied problem of stochastic combinatorial multi-arm bandit (CMAB) to the setting where the reward function is a Lipschitz - though not necessarily linear - function of the random realization of the arms. At each round, the algorithm selects a subset S - called a 'super arm - of m base arms, and receives a reward R() which is a Lipschitz and monotone function of the rewards of all individual arms. Like linear CMAB, the subset S is relies in some restricted class of subsets (say, [m]-choose-K), and the rewards of each arm assumed to be independent, and identically distributed over rounds.  The authors introduce an algorithm which obtains O(log T)-instance dependent and O(sqrt(T))-worst-case simple regret with respect to alpha * the reward of the best of simple arm S^*, which makes exactly one call to an offline alpha-approximation oracle per round. The paper establishes upper bounds even when each individual arm has an arbitrary, compactly supported distribution, which qualitatively match those for linear CMAB. Moreover, 903 introduces a PTAS for the offline oracle when the reward function is the max of all the arms selected - which they denote the K-MAX problem.
 All in all, the paper is well written, the proofs are clear, and the results are compelling. Though the algorithm is a natural extension of the linear CMAB algorithm in [Kveton et. al. 15], its key innovation is maintaining pointwise estimate of the arm's CDF, and pointwise lower confidence interval of the arm's CDF, which corresponds to the distribution of a random variable which stochastically dominates the given arm. This tweak makes the algorithm and its analysis easy to grasp, while still preserving connections to the lienar CMAB problem.   In what follows, we adopt the notation of the present work: T is the horizon, m the number of arms, K the largest cardinality of a super-arm, M the max-reward of a set, alpha the approximation factor, C the lipschitz constant of the reward function. When the distributions are finitely supported, they present an upper bound whose asymptotic alpha-regret in the limit of the horizon T-> \infinity does not depend on s, the size of the largest support of one arms distribution. Moreover, by appropriately discretizing the algorithm for finitely supported distributions, they extend their result to an upper bound for generalized CMAB with arbitrary (possibly non-discrete) distributions, which does not require foreknowledge of the time horizon.  Under the relatively mild assumption C / M = O(1), this upper bound matches the worst-case regret lower bounds for linear CMAB of O(sqrt(mKt)) up to a factor of sqrt(logT). This  reviewer encourages the authors to more fully elaborate how their guarantees compare with known lower bounds in their dependence in the parameter K, m, and T, so future readers can appreciate the strength of their result: i.e, for  general Lipschitz functions introduce no further worst-case statistical difficulty in the CMAB game than do linear functions.   Paper 903 gives detailed attention to the K-MAX loss function, where the goal is to select the subset of arms which maximizes Exp[max_{i in S} ] over all subsets S of size K. The work presents a PTAS for a 1 - epsilon approximation oracle for every epsilon, yielding an online algorithm for obtaining vanishing 1-epsilon regret which runs in polynomial time. The construction and analysis of the PTAS are very clever and original even though (as is true of many PTAS's). While the run-time of the algorithm ( O(m^O((1/eps^2)))) is still prohibitive for any large m and moderate epsilon (for example, in light of the Gollovin and Streeter work, it only makes sense to take epsilon < 1/e, which yields a run time of a high degree polynomial in m), the authors do include experiments which verify that the PTAS is successful in yielding vanishing regret. It would have been interesting to see a comparison in run-time between the PTAS and brute-force search for small values of K, m, and small discretizations.   Overall this is fantastic contribution: clearly written, original, and ostensibly correct. The only real weakness of the work is that the analysis is rather pessimistic, as the O(log-T) instance upper bounds only depend on the worst-case gaps between subsets, given by Delta_{min}. This gap dependence can be very, very loose. For example, consider the K-MAX reward function, decisision [m]-choose-K, and suppose all the arms are Bernoulli. In this special case, choosing the best subset of arms amounts to choosing those with the top-means. To make this concrete, suppose that there are K arms with mean mu + epsilon, and the remaining m-K arms have meaning mu. Then S^* is the top K arms,the gap between any arm i in S^* and j not in S^* is epsilon, but the gap in reward between S^* and any S \ne S^* is as small as Delta_{min} = (mu + epsilon)^K - (mu + epsilon)^{K-1}epsilon = epsilon(mu + epsilon)^{K-1}. Thus, the present works upper bound obtains a guarantee of the form O( m K log T/epsilon * 1/(mu + epsilon)^{K-1}). When mu + epsilon are bounded away from 1, 1/(mu + epsilon)^{K-1} = exp(-Omega(K)).  However, there are algorithms for top-K identification see [Kalyanakrishnan, Shivaram, et al. "PAC subset selection in stochastic multi-armed bandits." Proceedings of the 29th International Conference on Machine Learning (ICML-12). 2012] finds that the best subset of arms can be identified with probability 1-delta using at most O(log(K/delta) m/epsilon^2) samples. If we consider each pull of a super arm as one "sample", then [Best-of-K Bandits, Max Simchowitz, Kevin Jamieson, Benjamin Recht, COLT, 2016]. or [Top Arm Identification in Multi-Armed Bandits with Batch Arm Pulls, Kwang-Sung Jun, Kevin Jamieson, Robert Nowak, Xiaojin Zhu, AISTATS, 2016. PDF] can identify the best arm using O(log(K/delta) m/ (K*epsilon^2)). These can be turned into a low-regret algorithm with regret of roughly O( m Delta_{max}/K epsilon^2 * log KT), where Delta_{max} = (mu+epsilon)^K - mu^K, which is no larger than K * epsilon; and hence, the run time of adapting top-K identification is no worse than O(mlog(KT)/epsilon). Hence, in the regime where mu + epsilon is bounded away from 1, we see that the CMAB algorithm in this worst has guarantees which are exponential-worse than using top-K.   Obviously, there are sacrifices to be made for generality (indeed, the Bernoulli case is trivial and the innovation of this paper is that it works for non-Bernoulli distribution!). Moreover, I understand that even the linear CMAB upper bounds dont't have such a granular dependence on problem parameters. However, it would be nice to include further comments on the dependence on problem parameters, and potential looseness therein, in future revision. As an example of a highly-granular dependence on problem parameter, I recommend that you cite [Best-of-K Bandits, Max Simchowitz, Kevin Jamieson, Benjamin Recht, COLT, 2016]. The second half of that work provides upper bounds for the Bernoulli K-MAX problem under various partial feedback models (the first half is concerned with hardness results when the arms are not independent). While Paper 903's upper bound of O(mKlog T/Delta_min) can be exponentially (in K) loose for the semi-bandit feedback regime (as described above),  the upper bounds and lower bounds in Simchowitz '16 for Bernoulli K-MAX under bandit feedback (only observing the reward of a super-arm, rather than the rewards of its constituents) do in fact incur a dependence on so called "information" occlusion terms, which are related to the gaps in rewards between super-arms.   One other possibly area for improvement I seemed to find was that the algorithm for discrete distributions incurs an up-front (i.e., independent of T) regret of O(alpha * m * s), which seems quite hefty for short horizons. By adjusting the lower confidence bound in the line 12 of algorithm to look like O(sqrt(log (t * s)/T_i), it seems like one may be able to reduce this up-front regret to O(alpha * m) (i.e., no dependence on s), perhaps at the expense of a slight dependence on s in the asymptotic regret. I understand that the point was to have no dependence on s in the asymptote, but perhaps tweaking the confidence interval might allow for some nice interpolation. This may also yield improved analysis of the discretization for non-finitely supported distributions.    ------- After Rebuttal period ------- Have reviewed authors comments and maintain my initial review.


Kernel Observers: Systems-Theoretic Modeling and Inference of Spatiotemporally Evolving Processes
We consider the problem of estimating the latent state of a spatiotemporally evolving continuous function using very few sensor measurements. We show that layering a dynamical systems prior over temporal evolution of weights of a kernel model is a valid approach to spatiotemporal modeling, and that it does not require the design of complex nonstationary kernels. Furthermore, we show that such a differentially constrained predictive model can be utilized to determine sensing locations that guarantee that the hidden state of the phenomena can be recovered with very few measurements. We provide sufficient conditions on the number and spatial location of samples required to guarantee state recovery, and provide a lower bound on the minimum number of samples required to robustly infer the hidden states. Our approach outperforms existing methods in numerical experiments.

This paper describes the problem of needing to model a spatiotemporal process using only a small number of physical locations.  The problem is rephrased using systems theory to provide a mechanism for identifying the optimal observation locations. Algorithms and associated proofs are provided to define the strategy and bound the minimum required complexity as a function of the complexity of the evolution of the system.  This is compared to nonstationary kernel-based approximation methods in numerical experiments which show significant improvements in training time.
Overall, not bad.  I think many of the paragraphs are far too long, and that often hurts the readability, but I think that the content is solid.  The introduction of Banach spaces adds nothing, which actually is part of a trend in this paper to add theory where it is not needed.  If the presentation could be tightened up some, with more of a practical slant, this paper would be very good.  As it is, the reason I gave a Technical score 2 rather than 3 was primarily for the issues with the proofs I list below, and not for the actual discussion in the main paper.  The experiments look fine, although Figure 3 is a bit confusing.  I have one significant fear about this paper, and that is why I am placing it here rather than toward the end of my comments: all the comments I have ever read about the Jordan canonical form suggest that it is numerical unstable to compute.  In a setting where that factorization of an admittedly inaccurate matrix ("Note that in practice, the matrix hat A needs to be inferred ...") is interacting with a kernel matrix which is notoriously subject to ill-conditioning, how can one be certain that the algorithms in this article will not be subject to instability?  Some acknowledgement of this should be included, if only to warn potential practitioners, or some proof as to why this will not be an issue.  The lack of discussion of numerical issues was the reason for a rating of 3 instead of 4 on the Potential Impact score.  I am not an expert in systems theory, and as such I may be way off base here, but I feel like much of the discussion regarding the proofs is unnecessarily abstract.  For example, Definition 1 could simply be "K has no zero valued columns", I think. In turn, Remark 1 could simply be "All positive definite kernels which are always positive produce shaded matrices", which would exclude oscillatory or compactly supported kernels, as I think it must.  Also, is a 1974 book really the most up to date reference available for cyclic index of a matrix?  If it is, maybe there are more popular tools for conducting these proofs?  Likewise, the immediate value of Proposition 1 is lost because by that point, the reader may have forgotten the assumption N < < M and be confused as to why one would need to prove that the product of two full rank matrices is full rank (as I did, initially).  Also, an important component of the proposition, that L > = M, is stated almost as a throwaway at the end, rather than as a condition of the setting; it would probably make more sense to start with that as an assumption, along with the assumption that Upsilon contains all (or at least M) unique values.  Also, I think there's a typo in the proof of proposition 1, where a bold K hat is written on line 83, but never defined or heard from again.  And if A hat is full rank with no repeated eigenvalues, I think the Lambda matrix just has M lambda values in it, because the Jordan decomposition would simply be the spectral decomposition?  That would be easier to comprehend than the confusing introduction of the Lambda_O term.  I also would prefer cleaner notation for the matrix-matrix products in this proof, where the use of [K ... K] implies something other than what (I think) is meant. O_Upsilon is defined as a matrix with NL rows and M columns, but the product written directly beneath it implies that the matrix should have N rows and M columns.  That of course can not be true since an NxM matrix with N < < M will not have linearly independent columns.  What I think is meant is a block diagonal matrix of size NL x ML with L copies of K along the diagonal.  Then the decomposition would be                     [K          [Lambda^tau_1 O_Upsilon =     ...                ...                              K] Lambda^tau_L] Or maybe I do not understand what O_Upsilon is, in which case the notation defining it is confusing.  And I think the same issue extends to line 88 where the logic is applied to the Jordan decomposition.  Also, the Lambda^t notation is not obvious: that matrix is not being raised to the t power?  And, actually, the statement about the sizes of P and Lambda^t is not correct or even consistent with the sizes you wrote earlier - here the resulting matrix would be size NxML, which would certainly not have independent columns.  The notation for the first part of the proof of proposition 2 is also confusing, but it at least looks consistent.  The proof of proposition 3 also seems fine if perhaps a bit confusing - maybe a cleaner presentation in more than 1 paragraph would help?  I think it should be possible to show that, with probability 1, the columns of the K matrix, for all positive definite kernels with only positive values, will be linearly independent for points drawn randomly from the domain.  This was shown to some degree in the differential equation setting by [Hon et al, On unsymmetric collocation by radial basis functions], and for the approximation setting by [Hangelbroek et al, Nonlinear approximation using Gaussian kernels]. I mention this only to alert the authors - no such proof need appear in this article.  Minor points: * Figure 5(b) says that it is a box plot, but does not look like one. * The phrase "much lesser" is used. * K, and variants with hats and twiddles and circles, is used too much.  It makes the content hard to discern. * spelling: nonistropic * The last paragraph of page 2 starts with a poorly written sentence


A Probabilistic Model of Social Decision Making based on Reward Maximization
A fundamental problem in cognitive neuroscience is how humans make decisions, act, and behave in relation to other humans. Here we adopt the hypothesis that when we are in an interactive social setting, our brains perform Bayesian inference of the intentions and cooperativeness of others using probabilistic representations. We employ the framework of partially observable Markov decision processes (POMDPs) to model human decision making in a social context, focusing specifically on the volunteer's dilemma in a version of the classic Public Goods Game. We show that the POMDP model explains both the behavior of subjects as well as neural activity recorded using fMRI during the game. The decisions of subjects can be modeled across all trials using two interpretable parameters. Furthermore, the expected reward predicted by the model for each subject was correlated with the activation of brain areas related to reward expectation in social interactions. Our results suggest a probabilistic basis for human social decision making within the framework of expected reward maximization.

In this paper authors propose the POMDP model to simulate human decision making in the Public Goods Game (PGG). They explain the machinery of the model, its fit to behavioral PGG data and comparison to other similar models. They also suggest that the model has neural correlates with reward expectations correlated with activity in two brain areas.
This is an interesting modeling and model-based analysis study, providing insights into the machinery of human social decision making and possibly its neural correlates. The paper is generally well written and combines advances that could be interesting to both experimental and modeling audience. However, some of its aspects (particularly interpretation of estimated parameters and fMRI analyses) should be improved for it to be acceptable to NIPS.  More specific comments: - What is the meaning of groups of 5 individuals if computer generates actions of 19 others? More details about how these actions are generated would be helpful. - Biological plausibility of the used value iteration algorithm should be discussed. - Equation 1 is hard to understand: what is the meaning of the second E on each line? - Why is it assumed that all group members have the same theta_c as the subject? - Regarding table 1: the models should be better explained. What are the free parameters/their number for each model? That should normally be included in model comparison. Which differences are statistically significant? - fMRI small volume correction (SVC) based on 10mm (radius or diameter?) spheres should be justified based on literature. How is the region center chosen for this purpose?  - There is a typo in the first T value - it should be 3.45 I suppose - T values are too small to be called "strong correlation". Generally, whole brain analyses are preferable to SVC. - At the beginning of 5.3 do you mean "the best FITTING alpha_0 and beta_0"? - Do alpha and beta themselves have no neural correlates? Either way this should be reported. Alpha and beta have nice mathematical grounding but their suitability for psychological interpretation is less clear. It seems ratios or sums could be more informative.  --- Response to author rebuttal: Thanks for addressing some of my concerns. However, I would like to clarify a few aspects which should be addressed in the final version. * While the use of SVC is understandable, it may lead to missing important effects in brain areas outside of the defined ones, whose relevance could also be justified and whose peak T-values may even be higher than of the reported ones. For example, many SVC-based studies of memory focus on hippocampus and exclude parahippocampal gyrus or prefrontal cortex, which were shown to have pronounced activity patterns in whole-brain studies. For this reason please provide whole-brain statistics as well (if necessary, in supplementary material), including peaks outside of the defined area with larger T values than reported. * If ratios and sums of alpha and beta are more informative, why not correlate them with neural data or behavior instead of simply mentioning this aspect in the discussion? If there are no effects, this should be reported as well.


Bayesian Optimization for Probabilistic Programs
We present the first general purpose framework for marginal maximum a posteriori estimation of probabilistic program variables. By using a series of code transformations, the evidence of any probabilistic program, and therefore of any graphical model, can be optimized with respect to an arbitrary subset of its sampled variables. To carry out this optimization, we develop the first Bayesian optimization package to directly exploit the source code of its target, leading to innovations in problem-independent hyperpriors, unbounded optimization, and implicit constraint satisfaction; delivering significant performance improvements over prominent existing packages. We present applications of our method to a number of tasks including engineering design and parameter optimization.

The paper considers Bayesian optimization for probabilistic programs (PPs). The specific interest is in marginal maximum a posterior (MMAP) estimate for a subset of the parameters in the model defined by a PP. Large parts of the paper are devoted to describing the programmin workflow and the specific PP Anglican.
The paper presents a implementation of Bayesian optimization to an interesting problem. I liked the content of the paper in general but not the presentation and had some doubts on the methods (as detailed below). However, the dislike of presentation might be my personal preference.   I'm not expert on Probabilistic Programming systems in broad sense (I'm confident in using STAN and few others). Hence, I find it very difficult to follow the presentation which relies on syntax for Anglican. I'm sorry but I don't fully understand the syntax in figures 2 and 3 and the long paragraphs of syntax style text - or at least I would need to first study the respective manual. For example in lines 119-120, I don't understand what "(sample:id dist)" means. However, the "intellectual" content in figures and text is syntax independent - and I think it should be in order to justify publication. For this reason I think authors should revise the paper and explain what they do in plain words and standard mathematics notations. For example, there is no need to use awkward "[:power-dis]" to denote distribution when authors could just write \theta \sim p(\theta).  I think the paper does not read very well in general. Authors use very condensed sentences that are hard to understand. For example, I did not understand the following: - line 144: "...sampling tested \theta values..." What are "tested \theta values"? - The list in lines 143-151 is too vague at this point. Could you, at least, refer to later sections when talking about things not defined yet (e.g. line 148). - line 200: "...combination...", what kind of combination; sum, product, ...? - line 201: "...significantly higher weigth". What is significant? How do you give weight to a kernel; do you have a prior over them and you sample GP's with different kernels? - line 204: What does mean "...linearity of GPs is exploited to convert the produced samples into a single GP."? - line 226: "If points are observed... target for the GP..." what are the points referred here and how are they observed? And what is target for the GP? - In line 166 and later authors talk about "problem independent hyperprior" but they never define that. The black box nature of their algorithm is an essential novelty here and the hyperprior plays major role there. Hence, it should be carefully justified.  Then a more fundamental question. In the Introduction and Discussion (and in few other places) authors talk about searching for marginal maximum a posterior estimate but as far as I've understood correctly they search for maximum marginal likelihood. In line 143 authors talk about "A code transformation allowing the marginal likelihood (ML) to be evaluated at a fixed \theta". This is also what seems to be possible with algorithm in Figure 3. I cannot find details on where the prior for \theta comes into play in this algorithm. This might be a problem of presentation that I criticized above and, if so, should be corrected. One specific point is line 208 where \hat{u}_m^* is defined to be the maximum expected posterior density of \theta. However, earlier in line 138 it was defined to be _log_ marginal _likelihood_.  I am a bit skeptical that the idea presented in section 4.4. works in general setting. How do you justify the choice of r_{\infty} for example? Let me tell an example from a real problem I'm working with currently. I'm interested in finding the (real) maximum a posterior estimate for \theta \in \Re^2. The prior is really vague and likelihood is really narrow leading to a situation where the meaningful posterior mass would cover only about 1% of authors hypercube. More spefically, the hypercube (in log \theta space) would be [0 1] x [0 1] and the MAP is around [0.9 0.9] and log p([0.9 0.9]+\epsilon*[1 1]|Y) + 10 < log p([0.9 0.9]|Y)  for all epsilon greater than 0.1. Hence, the posterior mass is practically negligible if the difference between log posteriors is greater than 10 and a naive BO algorithm failes here. Another question relates to the dimension of \theta. How large can it be?  Few minor specific comments: - Figure 3 caption: what does "probability p(y)=y according to value y = (C, \theta)" mean? Moreover, "C" is introduced much later than its first appearance in text or in reference to Figure 3.  - line 137: "\hat{u}_m^*" the same thing is denoted by \hat{w} in line 195 - line 138: "log marginal likelihood log p(\hat{\theta}_m^*|Y)"? isn't this log marginal posterior? - line 163: definition/explanation of q-cond comes much later so I was at first confused here. - the font size in figures 5 and 6 is too small  To summarize. I think the paper puts too much emphasis on "coding" before assuring that the theory behind the method really works. The paper could make a good example if formulated so that it concentrates on the method.


Agnostic Estimation for Misspecified Phase Retrieval Models
The goal of noisy high-dimensional phase retrieval is to estimate an s-sparse parameter β * ∈ R d from n realizations of the model Y = (X β * ) 2 + ε. Based on this model, we propose a significant semi-parametric generalization called misspecified phase retrieval (MPR), in which Y = f (X β * , ε) with unknown f and Cov(Y, (X β * ) 2 ) > 0. For example, MPR encompasses Y = h(|X β * |) + ε with increasing h as a special case. Despite the generality of the MPR model, it eludes the reach of most existing semi-parametric estimators. In this paper, we propose an estimation procedure, which consists of solving a cascade of two convex programs and provably recovers the direction of β * . Our theory is backed up by thorough numerical results.

The paper presents a generalization of ‘phase retrieval’ models that encompasses a wider range of data generated from sparse parameter vectors by using a single index model (SIM) with an unknown link function. These models encompass the traditional phase retrieval as well as the Wirtinger Flow models of data acquisition, with a covariance condition on the observations and the square of the inner product instead of just on the covariance of the observations and the inner product of the predictor vector X and the parameter B. The authors show that several different conditions will lead to the model they describe so that the class of MPR (misspecified phase retrieval) models is ‘sufficiently broad’. They then provide a two-step algorithm to solve these misspecified models – the first step involves solving a regularized semidefinite program that was relaxed from a nonconvex problem which tries to find an estimate for the direction of B (the desired parameter vector) using an empirical surrogate for the mean. They later prove that this step has a supremum which is bounded by the inverse of the size of the problem. The second step is an l1 – regularized refinement step which they show does not increase the computational complexity of the overall problem given few enough repetitions. The authors also have minimax bounds on the class of link functions that they define for additive SIMs. They back this up with a few numerical experiments.
The acronym AGENT seems like a bit of a stretch to me. I would like to see more numerical experiments as well as comparing to more than just Wirtinger Flow. I know that there are more classical methods for solving problems in classes like these (though perhaps without the same guarantees!), and would maybe like to see it commented on a little more or compared to in your numerical results. The numerical results section feels somewhat threadbare – we do see how the reinforcement strategy is effective, but the lack of comparisons in computational complexity/effort with other established methods makes it hard to see the benefits in computation; the agnosticism to the model is cool but I think it would also be cool to see exactly how AGENT works on the different models in the same plot for comparison. I think the idea is fairly unique – having a general algorithm that provably works with high probability for different models given enough samples is meritorious.  If the authors never deal with the complex domain, then classifying the paper as relevant for misspecified phase retrieval seems somewhat like a misnomer to me.  Another thing that might be helpful is defining the domain the Y and X reside in, since you have phase retrieval applications, specifying that in certain cases Y is in the real domain vs the complex domain while X has to be in the complex domain (since in the abstract you define B to be wholly real) is fairly important, especially for making comparisons to Wirtinger Flow.   I think one thing that would be helpful for the clarity of the reader is to insert reminders for different definitions to avoid frequent backtracking, for example line 167 instead of “satisfy (2.1)”, maybe “satisfy the covariance condition (2.1)”.  I don’t quite understand why in line 190 that by making this different observation we just can apply l1 regularized least squares (without an empirical estimate for the mean).  Another question I have is regarding equation (3.2), you say that Y~ depends on two directions, but B^ perp is a hyperplane – I think I understand what you mean but this could be slightly confusing?  The authors mix notation for the outer product, in line 211 and equation 3.4, the authors use one notation, but in line 221 the authors simply spell it out. I’m not sure how helpful this is for clarity.  Notationally sometimes Y bar is used and sometimes mu is used to denote the average. Is one for the empirical average and one for the ‘true’ expectation?


Density Estimation via Discrepancy Based Adaptive Sequential Partition
Given iid observations from an unknown absolute continuous distribution defined on some domain Ω, we propose a nonparametric method to learn a piecewise constant function to approximate the underlying probability density function. Our density estimate is a piecewise constant function defined on a binary partition of Ω. The key ingredient of the algorithm is to use discrepancy, a concept originates from Quasi Monte Carlo analysis, to control the partition process. The resulting algorithm is simple, efficient, and has a provable convergence rate. We empirically demonstrate its efficiency as a density estimation method. We also show how it can be utilized to find good initializations for k-means.

The authors propose an adaptive partitioning method to fit a piecewise constant density estimate. They bound the integration error and get standard Monte Carlo bounds (O(1/\sqrt{n})) for their method. The method (or a variant---I am not quite sure) is tested on a set of synthetic data and a k-means initialization task.
Novelty: there are a fair number of adaptive partitioning density estimation methods in existence that the authors omit (Darbellay and Vajda, 1999; Petralia, Vogelstein, and Dunson, 2013; etc). Some of these are designed for high dimensions, some function in lower dimensions. In general, this space is fairly well studied but there is room for serious theoretical analysis. A much stronger result than the standard Monte Carlo integration rates would be something like rates in Hellinger or Lp distance, which are generally not dimension-free. These rates and associated traits (adaptivity to intrinsic dimension, shape constraints, etc) are generally much more predictive of how a method performs in the wild than simple MC integration rates.  Impact: this method relies on star discrepancy to make partition selections and bound the error. However, this metric is generally impractical as it requires a supremum over all d-dimensional sub-rectangles, which is computationally burdensome. The authors suggest a work around, but this destroys their rate guarantees. This method would be quite slow in its original form and not theoretically supported in its modified form.  Clarity: I had some trouble following this paper. Aside from some random typos, some of the ideas were not well explained. For instance, what is the intuition behind using gaps? How does the scaling step relate? Additionally, I found a few of the experimental numbers puzzling. KDE computations were quite slow, while computations with this method were quite fast. Given the general simplicity of a KDE and the complexity of star discrepancy, I would expect those numbers to be reversed. There must be some aspect of the implementation that is not well explained.


Satisfying Real-world Goals with Dataset Constraints
The goal of minimizing misclassification error on a training set is often just one of several real-world goals that might be defined on different datasets. For example, one may require a classifier to also make positive predictions at some specified rate for some subpopulation (fairness), or to achieve a specified empirical recall. Other real-world goals include reducing churn with respect to a previously deployed model, or stabilizing online training. In this paper we propose handling multiple goals on multiple datasets by training with dataset constraints, using the ramp penalty to accurately quantify costs, and present an efficient algorithm to approximately optimize the resulting non-convex constrained optimization problem. Experiments on both benchmark and real-world industry datasets demonstrate the effectiveness of our approach.

The paper proposes a model and algorithm to solve linear classification problems on datasets with some common side-constraints that are often ignored.
I really like the basic premise of the paper. I am no expert on deployed systems, but it seems like these authors know about real-world systems (and have access to some proprietary real datasets) and have thought carefully about actual issues that come up when making a classifier.  The paper proposes (1) a new mindset, (2) a new model, and (3) an algorithm to solve the model. At times, I think any one of these components is a bit thin, and it’s not always clear why certain decisions (especially for the algorithm) were made — in particular, the numerical examples are provided to validate the new mindset but do not justify the algorithmic decisions. But overall, I think this is sufficiently novel that I am supportive.  A few specific points:  - I found the descriptions of the issues on page 1 and 2 (e.g., the churn example) quite interesting  - Page 3 and further in the paper, referring to “Problem 2” is a bit confusing when it is only labeled by an equation number (most authors define a problem environment, like a theorem). I also didn’t see r_p and r_n defined (line 103) — are these s_p and s_n, or a (un)normalized version?  - The randomized prediction (line 105) didn’t seem like it was fully explored. Does this affect training or only the final prediction? After vaguely introducing this concept, I felt like it was ignored until the numerical experiment plots, and I do not understand exactly what you are doing.  - Fig 1 was designed to make the convex bounds intuitive, but this means that for z >= 1/2, at the next step, there is no actual dependence on this component of z? With the hinge loss, at least there is still dependence. Intuitively this makes your scheme sound like a bad idea.    — Furthermore, discussing Davenport (line 178-180), you say that ramp-loss is similar or slightly better in accuracy than hinge-loss, but much faster. This doesn’t make sense to me, since to solve the ramp-loss you must solve a sequence of hinge-loss problems, so how can it be faster? Could you elaborate?  - Section 2.2 uses SVM solvers inside a cutting-plane routine. This may be a good idea, but you could describe why you didn’t pursue alternatives. Everything is piecewise linear, except the regularization which is quadratic, so you could solve everything with an off-the-shelf QP solver, which seems much simpler. Perhaps you prefer existing SVM solvers since you know good ones, and many of them naturally take in datasets in specified formats (such as libsvm format)?  A bit of discussion on your design decisions would be useful.


Deep Learning for Predicting Human Strategic Behavior
Predicting the behavior of human participants in strategic settings is an important problem in many domains. Most existing work either assumes that participants are perfectly rational, or attempts to directly model each participant's cognitive processes based on insights from cognitive psychology and experimental economics. In this work, we present an alternative, a deep learning approach that automatically performs cognitive modeling without relying on such expert knowledge. We introduce a novel architecture that allows a single network to generalize across different input and output dimensions by using matrix units rather than scalar units, and show that its performance significantly outperforms that of the previous state of the art, which relies on expert-constructed features.

The paper deals with modeling human behavior in game theoretic laboratory settings. Contrary to other related works, this work aims to learn human behavioral models without the assistance of domain experts and (more importantly), without the strong game theoretic assumption of rationality. The paper is of moderate to high relevance as it proposes an improvement to the state of the art where current solutions involve carefully handcrafted representations and solutions. As opposed to this work, where the representation is learned through repeated interactions. The setting and problem is very relevant, as it addresses a problem that will face every AI agent in the real world.
The technical aspects of the work are sound and explained with a decent amount of detail (much appreciated).   The experimental findings lack solid conclusions and need further analysis. Specifically, the experimental setup does not detail what kind of normal form games will be used, the properties of general sum games vary a lot and the implications are profoundly different across classes of games. The results presented do not show much as they are now. With such vague description of the setup 1) results are not reproducible and 2) cannot assess if the proposal generalizes well across classes of games (competitive, cooperative, mixed interest, social dilemmas, coordination/anti-coordination, etc). Furthermore, the analysis of the results is somewhat disappointing. It is very intriguing that the best performing model includes only one action response layer. I wonder if these findings suggest that subjects actually use 1 level models? It would have been informative to comment on how these findings relate to those in behavioral game theory and extend on related results on this field.  In conclusion, the work is interesting and I have no question that it advances the state of the art. But the experimental setup, analysis and conclusions could be improved.   Typos: - 107. "To apply a such a" - 123. "to represent the all the" - 314. "did not need them to achieve set a new "  Detailed questions: - In a footnote you say you ignore salient effects, it would be good if you give some examples of the type of effects. - Section 3. You never explained r is for row player and c for column player. - 226. You say \lambda is a scalar sharpness parameter, why do you need this?