Dual Space Gradient Descent for Online Learning
One crucial goal in kernel online learning is to bound the model size. Common approaches employ budget maintenance procedures to restrict the model sizes using removal, projection, or merging strategies. Although projection and merging, in the literature, are known to be the most effective strategies, they demand extensive computation whilst removal strategy fails to retain information of the removed vectors. An alternative way to address the model size problem is to apply random features to approximate the kernel function. This allows the model to be maintained directly in the random feature space, hence effectively resolve the curse of kernelization. However, this approach still suffers from a serious shortcoming as it needs to use a high dimensional random feature space to achieve a sufficiently accurate kernel approximation. Consequently, it leads to a significant increase in the computational cost. To address all of these aforementioned challenges, we present in this paper the Dual Space Gradient Descent (DualSGD), a novel framework that utilizes random features as an auxiliary space to maintain information from data points removed during budget maintenance. Consequently, our approach permits the budget to be maintained in a simple, direct and elegant way while simultaneously mitigating the impact of the dimensionality issue on learning performance. We further provide convergence analysis and extensively conduct experiments on five real-world datasets to demonstrate the predictive performance and scalability of our proposed method in comparison with the state-of-the-art baselines.

The paper introduces a new algorithm for kernel-based online learning, where (a) the number of support vectors is controlled by discarding the least significant ones, and (b) information of the discarded support vectors is partially retained by maintaining a random projection of them in a lower-dimensional space.  Regret bounds are derived in terms of the "budget" of retained support vectors and the dimensionality of the random projections.  Experiments show that compared to previous methods, the new algorithm obtains a very good trade-off between prediction accuracy and memory use.
The rebuttal answered all my technical questions, in particular the one about probability and expectations in Theorems 1(ii) and 3.  My scores (which were given assuming the technicalities can be fixed) remain the same.  *** Original review below ***  The algorithm is as far as I know an original and intuitively appealing combination of techniques previously introduced in this quite active subarea.  I quite like it.  The theoretical results do not feel that exciting.  The regret bounds seem more or less what one might expect, and there is no comparison to bounds for other algorithms.  I understand that there are no directly comparable bounds since you have both budget size and dimensionality of random projections as significant parameters, but still some discussion would be nice.  The dependence on budget B in your bounds could perhaps be pointed out more explicitly.  The experiments are compelling.  The relevant competing algorithms are included, and the data sets are sufficiently varied.  Detailed comments:  Algorithm 1: On line 4, you include all indices in I_t, even when alpha_t is zero.  Is this really intended?  On the other hand, b is increased only for non-zero alpha_t.  You also fail to mention where b is decreased, but it would seem if you really do this as written then the arg min in Algorithm 2 might just catch a lot of zeros and not actually free up any budget.  Theorem 1: I did not find a definition for sigma_mu in [15].  There was sigma_P where P is a distribution.  But what is mu here, anyway?  Theorem 2: "the following statement guarantees" seems wrong  line 169: "where" should be "were"?  Experiments: Is it reasonable to use the same values for hyperparameters such as B for all the algorithms?  Would it make any difference in the results if each algorithm was individually tuned?  line 232: Pegasos is not mentioned anywhere else as far as I can see  line 257, "due to additional exponential operators":  Another factor that causes more computation in Logit than in Hinge is that the gradient is always non-zero.  reference [15]: "Infomration"  Appendix, proof of Theorem 2: To get to (4) you need the previous bound to hold for all j, but if I understand correctly the current bound is just for a fixed j.  Proof of Theorem 2, last chain of inequalities: To get here you seem to need E[alpha^2 beta^2] \leq E[alpha^2] E[beta^2] , which seems intutively reasonable but would require a proof.  Similar issue appear in proof of Theorem 6 (top of page 5).  page 5, lines 7 and 8: The term with 8A^2 seems to have a factor 1/lambda missing.  Proof of Theorem 7: There is no "Lemma 2" here.  Appendix, Section 3: Your use of O notation is a bit sloppy.  In particular, from f_1 = O(g_1) and f_2 = O(g_2) one cannot conclude f_1-f_2 = O(g_1-g_2).


Kernel Bayesian Inference with Posterior Regularization
We propose a vector-valued regression problem whose solution is equivalent to the reproducing kernel Hilbert space (RKHS) embedding of the Bayesian posterior distribution. This equivalence provides a new understanding of kernel Bayesian inference. Moreover, the optimization problem induces a new regularization for the posterior embedding estimator, which is faster and has comparable performance to the squared regularization in kernel Bayes' rule. This regularization coincides with a former thresholding approach used in kernel POMDPs whose consistency remains to be established. Our theoretical work solves this open problem and provides consistency analysis in regression settings. Based on our optimizational formulation, we propose a flexible Bayesian posterior regularization framework which for the first time enables us to put regularization at the distribution level. We apply this method to nonparametric state-space filtering tasks with extremely nonlinear dynamics and show performance gains over all other baselines.

This paper proposes theoretical results characterizing posterior mean embeddings as the solution of a vector-valued regression problem. As in RegBayes which formulates a posterior distribution (via Bayes' rule) as the solution to an optimization problem, this paper proposes kRegBayes which does the same to kernel Bayes' rule of Fukumizu et al., 2013. For computing a conditional mean embedding operator, a new regularization technique is also proposed and is shown to be faster and consistent.
# Justifications of the ratings  Considering the complexity of this work which builds upon a number of existing technical papers, the authors did an excellent job in the writing. The writing is very clear and precise. The paper is self-contained. A potential academic impact from this work is from the optimizational formulation of kernel Bayesian inference which increases understanding and allows one to also add a regularization to the inferred posterior embedding. This can open up a new research direction for the kernel Bayes' rule as well as other techniques related to conditional mean embedding.  # Questions  1. Section 3.1 mainly deals with how to estimate the loss in (3) without using an i.i.d. joint sample to approximate the expectation (as we do not have it). What is mysterious to me is the absorption of \mu into f. Originally in (3), \mu is what we seek. Now in (4), the loss in (3) is rewritten in such a way that  the original \mu (a conditional mean embedding operator) is now in f, and a new \mu_{(X,Y)} appears (a joint embedding). I guess in the end f is treated as if it does not have \mu. Could you please elaborate a bit on what is going on here?   2. Line 152: f is assumed to be in a product RKHS. Is it really in the space? f looks like an unbounded function. What further conditions (e.g., on H_X and H_Y, or the domains) do we need for this to be true?  3. In theorem 3, does |\mathcal{X} \times \mathcal{Y}| < \infty mean the domains \mathcal{X}, \mathcal{Y} are discrete and not just that \mathcal{X} and \mathcal{Y} are bounded? If \mathcal{X}, \mathcal{Y} are indeed discrete, isn't this result fairly restrictive? Related: What are \mathcal{X}, \mathcal{Y} in the experiments?  4. Section 4 discusses a possibility of adding a regularization to the posterior embedding. I am not sure why the regularization (second term) in (10) is a sensible one. The description states that this is to encourage the posterior to be concentrated at some specified points. Why and when do we want this? I understand that this is just one way to do regularization under this framework. If there are more, what are others?   # Minor comments  * The proofs in the appendix are quite technical. It would be much better if   the paper contains a short proof sketch for each new result.  * Reference [14] has a wrong order of coauthors? * In section 2.2 and others, if I understand correctly \mu_{(YX)} is the same   as C^{\pi}_{YX}. It might make it a bit more readable to just use only C^{\pi}_{YX}. * Line 136: arg sup should be arg inf. * Line 234: By (30), you mean (10)?  ------------ after the rebuttal ---------- I read the authors' response.


Architectural Complexity Measures of Recurrent Neural Networks
In this paper, we systematically analyze the connecting architectures of recurrent neural networks (RNNs). Our main contribution is twofold: first, we present a rigorous graph-theoretic framework describing the connecting architectures of RNNs in general. Second, we propose three architecture complexity measures of RNNs: (a) the recurrent depth, which captures the RNN's over-time nonlinear complexity, (b) the feedforward depth, which captures the local input-output nonlinearity (similar to the "depth" in feedforward neural networks (FNNs)), and (c) the recurrent skip coefficient which captures how rapidly the information propagates over time. We rigorously prove each measure's existence and computability. Our experimental results show that RNNs might benefit from larger recurrent depth and feedforward depth. We further demonstrate that increasing recurrent skip coefficient offers performance boosts on long term dependency problems.

In this work the authors define three different complexity measures for RNNs: recurrent depth, feedforward depth and the recurrent skip coefficient. They give definitions for these three measures within a precise graph-theoretic framework. They then present experiments on various sequential tasks for RNN models with different complexity according to these measures.
A significant portion of the paper is spent defining in a very rigorous way three complexity measures which are fairly obvious, and in my opinion do not need the heavy graph-theoretic machinery the authors make use of.    For example, the authors give intuitive and clear definitions of recurrent depth and skip coefficient (respectively, as the average maximum number of non-linear transformations per time step, and the length of the shortest path from time i to time (i+n)). These are well illustrated with Figure 1 and the formal definitions the authors give in terms of the graph-theoretic framework do not add anything fundamental. Formalizing everything in a rigorous framework could be useful if the authors were proving new and useful theorems, but the theorems they provide are limited to proving the computability of the the measures they define, which seems more of a technical detail.  It is not clear what conclusions to draw from the experiments. The investigation of the different architectures in Figure 2a and 2b seems like an interesting direction, but the experiments are not very systematic and there are some methodological issues.  Questions/comments: -Why do the authors evaluate the architectures in Figure 2a on PennTree and Text8, the architectures in Figure 2b on Text8 and MNIST? Wouldn't it make more sense to evaluate the same set of architectures on all the datasets and see if certain architectures consistently work better on certain types of problems? -are the models whose results are in Table 1 (right) the same size as those reported in Table 1 (left) for Text8? It is not clear if the difference in performance is due to the differences in architecture or model size.  - It is not clear if the improvements with recurrent depth reported in Table 1 (right) are statistically significant, error bars over several initializations would help. - In Section 4.3, the results for MNIST should be included in the main paper, not the appendix. Also, they should be presented in a table like Table 1 (right).  - In Section 4.4, the results on the adding and copying problem should be presented in a table, not the text.  - In Table 2, top left: it is surprising that the results for permuted MNIST are *better* than the unpermuted version, since presumably this increases the timescale of the dependencies. Do you have an explanation? - Sentences like line 294-296: "We varied...found that this kind of improvement persists" should refer to specific results.  It isn't clear what insights the experiments give. The authors say in the conclusion "we find empirical evidence that increasing recurrent depth might yield performance improvements", yet increasing recurrent depth does not help for sequential MNIST and it is not tried for the copy or addition task.  They also say "increasing feedforward depth might not help on long term dependency tasks", which is unsurprising since it should not have a big influence on vanishing or exploding gradients. Their third point is that "increasing recurrent skip coefficient can largely improve performance on long-term dependency task", which was already known (for example, the Clockwork RNN).


Learning a Metric Embedding for Face Recognition using the Multibatch Method
which proves that the multibatch estimate is unbiased.
Next, we turn to analyze the variance of the multibatch estimate. let I ⇢ [k] 4 be all the indices i, j, s, t s.t. i 6 = j, s 6 = t, and we partition I to I 1 [ I 2 [ I 3 , where I 1 is the set where i = s and j = t, I 2 is when all indices are different, and I 3 is when i = s and j 6 = t or i 6 = s and j = t. Then:
For every r, denote by A (r) the matrix with A (r)
Observe that for every r,
Therefore,
Let us momentarily fix r and omit the superscript from A (r) . We consider the value of E ⇡ A ⇡(i),⇡(j) A ⇡(s),⇡(t) according to the value of q.
• For q = 1: we obtain E ⇡ A 2 ⇡(i),⇡(j) which is the variance of the random variable r r`i,j (z) r r L(z).
• For q = 2: When we fix i, j, s, t which are all different, and take expectation over ⇡, then all products of off-diagonal elements of A appear the same number of times in E ⇡ A ⇡(i),⇡(j) A ⇡(s),⇡(t) . Therefore, this quantity is proportional to P p6 =r v p v r , where v is the vector of all non-diagonal entries of A. Since P p v p = 0, we obtain (using Lemma 1) that P p6 =r v p v r  0, which means that the entire sum for this case is non-positive.
• For q = 3: Let us consider the case when i = s and j 6 = t, and the derivation for the case when i 6 = s and j = t is analogous. The expression we obtain is
. This is like first sampling a row and then sampling, without replacement, two indices from the row (while not allowing to take the diagonal element). So, we can rewrite the expression as:
where we denoteĀ i = E j6 =i A i,j and in the inequality we used again Lemma 1.
Finally, the bound on the variance follows by observing that the number of summands in I 1 is k 2 k and the number of summands in I 3 is O(k 3 ). This concludes our proof.

The paper addresses an important engineering challenge of developing a metric learning protocol which is computationally cheap while providing near state of the art performance. The task that the paper focuses on is face recognition.  The paper proposes a method to sample a dense set of loses to back prop. These loses are pairwise between k samples from different classes. The method is named multibatch, and the paper shows that the variance decreases as the square of the number of samples leading to faster convergence. The architecture proposed jointly aligns the faces and extracts a discriminative signature.  On the theoretical side, the paper presents an interesting view that metric learning can be seen as a harder problem. The multibatch method is a practical way to get SGD to converge quicker on the loss.
Major comments:   1) This reviewer's main concern is that although impressive performance on the LFW protocol is showcased in a very competitive setting, a atleast one or two more databases might be required to demonstrate that the protocol can scale up. Given that the practical contribution is primarily in terms of training through multibatch, more empirical evidence might be required. This reviewer imagines the results would generalize to other datasets, but given the strong focus of the paper on face recognition specifically, perhaps one or two more dataset results might be needed. This is the only major reason for the score of 2 on technical quality.  2) As most deep learning studies, this reviewer feels that perhaps the paper lacks a bit of intuition as to why a certain architecture worked. The final architecture employs a Network in network model, however given the fact that the paper aims to minimize model and computational complexity, it is a bit confusing (perhaps I m missing something) as to why the final model still seems more complex than a standard CNN. It would be more informative to know what other simpler models in fact did not work. I know that the paper mentions a VGG model that performed marginally better. This need not be in too much detail, however any discussion of that sort would provide more completeness and structure to the study.   Minor comments:  There are errors in the paper that have cropped up, eg. Fig 4 caption: LFW A thorough proofread is suggested.  Fig 2 is hard to read, could be made bigger  The experimental section could perhaps be given more structure, with small headings for each aspect of the experiment. It would improve readability as opposed to a big block of text.   Is the architecture shown in Table 2 in order of being applied to the input ? The exact ordering would be more informative than a simple listing for parameter count purposes.


Generating Videos with Scene Dynamics
We capitalize on large amounts of unlabeled video in order to learn a model of scene dynamics for both video recognition tasks (e.g. action classification) and video generation tasks (e.g. future prediction). We propose a generative adversarial network for video with a spatio-temporal convolutional architecture that untangles the scene's foreground from the background. Experiments suggest this model can generate tiny videos up to a second at full frame rate better than simple baselines, and we show its utility at predicting plausible futures of static images. Moreover, experiments and visualizations show the model internally learns useful features for recognizing actions with minimal supervision, suggesting scene dynamics are a promising signal for representation learning. We believe generative video models can impact many applications in video understanding and simulation.

The paper describes a generative adversarial convolutional neural network for video (a block of 32 frames x 64x64 pixels). The architecture is divided into two streams one for static background and one for the moving foreground that are combined together using a mask that is also generated. The loss is a standard generative adversarial loss. The paper shows video generation experiments on four scene categories with qualitative results and quantitative user-study evaluating the realism of the generated videos. In addition, the paper uses trained representation on 5000 hours of Flickr videos for action classification demonstrating that the resulting representation serves as a better initialization for training  action classification models.
Strengths:  - The addressed problem of generative models of videos is interesting, timely and difficult. - Novelty: The proposed two stream architecture (static background / moving foreground + mask) for video generation is novel (albeit somewhat incremental over the previous static image generation GANs). Nevertheless, I like the proposed extension. - Experiments: The set of experiments is fairly comprehensive (generation, classification, user study). The results are encouraging, but but the visual quality of the generated results is quite poor and the action recognition results are much below the current state-of-the-art on the considered UCF dataset.  Weaknesses:  - The paper is somewhat incremental. The developed model is a fairly straighforward extension of the GAN for static images. - The generated videos have significant artifacts. Only some of the beach videos are kind of convincing.  The action recognition performance is much below the current state-of-the-art on the UCF dataset, which uses more complex (deeper, also processing optic flow) architectures.  Questions: - What is the size of the beach/golf course/train station/hospital datasets? - How do the video generation results from the network trained on 5000 hours of video look?  Summary: While somewhat incremental, the paper seems to have enough novelty for a poster. The visual results encouraging but with many artifacts. The action classification results demonstrate benefits of the learnt representation compared with random weights but are significantly below state-of-the-art results on the considered dataset.


Variational Autoencoder for Deep Learning of Images, Labels and Captions
A novel variational autoencoder is developed to model images, as well as associated labels or captions. The Deep Generative Deconvolutional Network (DGDN) is used as a decoder of the latent image features, and a deep Convolutional Neural Network (CNN) is used as an image encoder; the CNN is used to approximate a distribution for the latent DGDN features/code. The latent code is also linked to generative models for labels (Bayesian support vector machine) or captions (recurrent neural network). When predicting a label/caption for a new image at test, averaging is performed across the distribution of latent codes; this is computationally efficient as a consequence of the learned CNN-based encoder. Since the framework is capable of modeling the image in the presence/absence of associated labels/captions, a new semi-supervised setting is manifested for CNN learning with images; the framework even allows unsupervised CNN learning, based on images alone.

This paper describes an approach to perform semi-supervised learning on images. The authors specifically consider classification and captioning tasks, although the method is applicable to other tasks that take single images as inputs.  The proposed approach consists of a type of variational autoencoder (VAE) where an encoder maps images to a latent feature space and a decoder then reconstructs images based on these latent features. The latent features are then used as inputs for a classification or text generation model appropriate for the task. This architecture enables semi-supervised training, either by pre-training the autoencoder in an unsupervised way and then training the task-specific model (two-step training) or by training end-to-end the autoencoder and the task-specific model as a single model on labeled examples and only the autoencoder on unlabeled examples (joint training).  For the autoencoder, the proposed approach uses a convolutional network (CNN) as the encoder, a deep generative deconvolutional network (DGDN) as the decoder, for the task-specific models it uses a Bayesian SVM for classification or a gated recurrent unit (GRU) for captioning.  The paper differs from similar approaches described in the literature for its use of stochastic pooling and unpooling in the encoder and the decoder, respectively. Also, it doesn't make use of non-linear activation functions after each layer, as it is usual, rather it uses tanh activation functions to compute stochastic pooling probabilities and in the final Gaussian sampling layer in the encoder. The use of a Bayesian SVM for classification is also unusual, although, together with stochastic unpooling, it was already found in the original DGDN paper.
The paper is clearly written and the approach is sound, although it appears as a straightforward combination of existing approaches, specifically VAE, DGDN and image captioning by RNN.  The approach includes some architectural details, specifically the stochastic pooling/unpooling and the lack of non-linear activations between convolutions in the encoder and the decoder, which differ from standard approaches described in the literature. The authors don't motivate these choices. It would be advisable if these choices were motivated by theoretical arguments or individual experimental comparisons. On the other hand, the choice of the Bayesian SVM as a classifier, while unusual, is motivated by experimental comparison against the more common softmax classifier.  The experimental section evaluates the model on supervised classification, semi-supervised classification and captioning in comparison with existing methids.  For supervised classification, the paper compares the proposed approach to the DGDN approaches, but not to plain supervised approaches or other semi-supervised approaches that could have provided reasonable baselines. The authors report comparable test errors and faster test speeds to the DGDN methods on standard benchmark datasets. No significance estimates are provided. Since the test errors are close, significance tests could have helped to better assess these results.  For semi-supervised classification, the paper compares the proposed approach to transductive SVM, standard non-convolutional VAE-based semi-supervised classifiers and Ladder network. The MNIST dataset is used. They outperform TSVM and VAE and slightly underperform Ladder network (though still within the significance range, which is provided this time). This result is interesting, although a direct comparison with convolutional VAE should have been performed to assess the usefulness of the architectural variant introduced by this approach. They also compare their method against standard image classification CNNs on Imagenet.  As a side remark,it shall be noted that neither MNIST nor Imagenet are true semi-supervised classification datasets, as in more natural scenarios, such as domain adaptation, the distributions of the labeled and unlabeled examples are often different. Experiments on native semi-supervised datasets would allow a better evaluation of the method robustness to distribution difference.  For semi-supervised captioning, the paper reports experiments on standard dataset with the addition of source images from Imagenet as unlabeled examples. They compare against purely supervised approaches in the literature, obtaining good, though not SOTA results. No significance estimates are given and the table is difficult to read due to lack of bolding.  In conclusion the paper presents an interesting, though not particularly novel, approach that may be valuable and does not present any obvious flaws, but more theoretical motivation or experimental validation would be advisable to assess its worth.


Combinatorial semi-bandit with known covariance
The combinatorial stochastic semi-bandit problem is an extension of the classical multi-armed bandit problem in which an algorithm pulls more than one arm at each stage and the rewards of all pulled arms are revealed. One difference with the single arm variant is that the dependency structure of the arms is crucial. Previous works on this setting either used a worst-case approach or imposed independence of the arms. We introduce a way to quantify the dependency structure of the problem and design an algorithm that adapts to it. The algorithm is based on linear regression and the analysis develops techniques from the linear bandit literature. By comparing its performance to a new lower bound, we prove that it is optimal, up to a poly-logarithmic factor in the number of pulled arms.

This paper studies a online linear optimization problem with combinatorial action sets and 'semi-bandit' feedback. At any given time, the reward noise of individual arms is correlated. The paper provides a regret bound for a linear UCB algorithm that scales with the degree of correlation and interpolates between two extreme cases studied in the prior literature: the case with no correlation, and the case where observations on different arms may be perfectly correlated.
This paper makes an interesting contribution. It's natural for there be correlation between arm outcomes, e.g. the realized time to traverse neighboring edges on a graph, and it's valuable to pin down the dependence of regret bounds on this correlation. This result helps link and unify separate results in the literature.   There are few limitations to the results. First, the proposed algorithm requires solving an optimization problem that can be NP hard. It's not clear the algorithm is implementable. Second, it seems this UCB approach may be statistically suboptimal in some settings. When possible, the most efficient way to gather information would be to select base arms with relatively uncorrelated outcomes. The algorithm won't do this. Finally, the regret bounds depend on the inverse mean-gaps that can be incredibly small. (e.g. the difference between the shortest and second shortest path in a graph) Gap-independent results would be nice here.    -- Here are some additional comments---  • It would be nice to provide a better motivation for the problem, and better intuition for the results. For example, it's very natural that one attains m times more information with no correlation than under perfect correlation.  • Do you need to depend on the largest correlation coefficient? Presumably, the reward of two base arms could be perfectly correlated without performance degrading substantially. The issue arises when there is a great deal of correlation on average.   • The proof looks familiar from other UCB analyses, but I find it very hard to read. I'm not sure this is necessary.   • The lower bound statement is not precise. It's not clear precisely what it means that "there exists a problem with gaps Delta" (all gaps = Delta? Is a gap a formal term? ). Also arms cannot  be sub-Gaussian, you mean that their reward noise is sub-Gaussian.


Exploiting Tradeoffs for Exact Recovery in Heterogeneous Stochastic Block Models
The Stochastic Block Model (SBM) is a widely used random graph model for networks with communities. Despite the recent burst of interest in community detection under the SBM from statistical and computational points of view, there are still gaps in understanding the fundamental limits of recovery. In this paper, we consider the SBM in its full generality, where there is no restriction on the number and sizes of communities or how they grow with the number of nodes, as well as on the connectivity probabilities inside or across communities. For such stochastic block models, we provide guarantees for exact recovery via a semidefinite program as well as upper and lower bounds on SBM parameters for exact recoverability. Our results exploit the tradeoffs among the various parameters of heterogenous SBM and provide recovery guarantees for many new interesting SBM configurations.

The paper focuses on a heterogeneous extension of the well-known stochastic block models, by considering the case where the communities might vary significantly in both size and connectivity. The authors develop sufficient recovery conditions for both convex relaxation approaches and a modified maximum likelihood estimate,  as well as some general lower bounds.  A few concrete examples have been provided to illustrate their results. In particular, the paper is able to accommodate the scenario where some communities have vanishingly small size.   The results are interesting, which provide a practically relevant extension of the prior work. In particular, I like the examples listed in Section 3,  which help the readers develop a better understanding of the tradeoff in heterogeneous SBM.    The flip side: the paper falls short of providing explanation for the key conditions presented in their main theorems (e.g. Theorem 1).  While both upper and lower bounds on the performance of MLE have been presented, I couldn't find discussion concerning the gap between these bounds,  making it difficult for me to evaluate the tightness of these results.  Also, I think the paper would benefit from some numerical experiments.
 1.  It would be good to provide some operational meaning of the notion "relative density".   2.  Some interpretation about the sufficient recovery condition in Theorem 1 is missing. What does the recovery condition mean?  For example, the first condition might be interpreted as the mean difference (or signal strength) being larger than the variation.  In general,  I hope that the authors can try their best to provide intuition for each of the key conditions, which would be very helpful for the readers.   3. Around Equation (2.5), the authors turn their attention from the original maximum likelihood estimate to a modified MLE independent of the parameters.  While this modified version is perhaps a more practically appealing algorithm,  I'm wondering whether there is any statistical loss that occurs due to this modification. It would be good to provide some comparisons (either positive or negative).   4.  The lower bound in Theorem 4 also needs some interpretation.  At first glance it looks quite different from the sufficient recovery condition given in Theorem 3.  The authors might want to discuss the tightness of both Theorem 3 and Theorem 4 -- do they match at least for some special cases?    5.  While the paper is mostly concerned with the scaling results,  can the authors comment on the possibility of obtaining tight pre-constants?  6.  Page 5,  the expression "sufficient conditions for when the exact recovery is impossible" doesn't read well; perhaps just "necessary conditions for exact recovery".    7.  One of the main results is the power of convex relaxation even when some communities have very small sizes.  Can the authors provide some numerical experiments using convex programming to illustrate the practical relevance of the theoretical prediction?


Probabilistic Inference with Generating Functions for Poisson Latent Variable Models
Graphical models with latent count variables arise in a number of fields. Standard exact inference techniques such as variable elimination and belief propagation do not apply to these models because the latent variables have countably infinite support. As a result, approximations such as truncation or MCMC are employed. We present the first exact inference algorithms for a class of models with latent count variables by developing a novel representation of countably infinite factors as probability generating functions, and then performing variable elimination with generating functions. Our approach is exact, runs in pseudo-polynomial time, and is much faster than existing approximate techniques. It leads to better parameter estimates for problems in population ecology by avoiding error introduced by approximate likelihood computations.
The N-mixture model [22] is a simple model with a Poisson latent variable for which no exact inference algorithm is known: (a) the model, (b) the prior and posterior for = 20, ⇢ = 0.25, y 1 = 2, y 2 = 5, y 3 = 3, (c) a closed form representation of the generating function of the unnormalized posterior, which is a compact and exact description of the posterior.
probability ⇢, so each observation y k is Binomial(n, ⇢). From these observations (usually across many sites with shared ), the scientist wishes to infer n and fit and ⇢.
This model is very simple: all variables are marginally Poisson, and the unnormalized posterior has a simple form (e.g., see Figure 1b). However, until recently, there was no known algorithm to exactly compute the likelihood p(y 1:K ). The naive way is to sum the unnormalized posterior p(n, y 1 , . . . , y K ) over all possible values of n. However, n has a countably infinite support, so this is not possible. In practice, users of this and related models truncate the infinite sum at a finite value [22]. A recent paper developed an exact algorithm for the N-mixture model, but one with running time that is exponential in K [8]. For a much broader class of models with Poisson latent variables [5,7,11,15,28], there are no known exact inference algorithms. Current methods either truncate the support [5,7,11], which is slow (e.g., see [4]) and interacts poorly with parameter estimation [6,8], or use MCMC [15,28], which is slow and for which convergence is hard to assess. The key difficulty with these models is that we lack finite and computationally tractable representations of factors over variables with a countably infinite support, such as the posterior distribution in the N-mixture model, or intermediate factors in exact inference algorithms.
The main contribution of this paper is to develop compact and exact representations of countably infinite factors using probability generating functions (PGFs) and to show how to perform variable elimination in the domain of generating functions. We provide the first exact pseudo-polynomial time inference algorithms (i.e., polynomial in the magnitude of the observed variables) for a class of Poisson latent variable models, including the N-mixture model and a more general class of Poisson HMMs. For example, the generating function of the unnormalized N-mixture posterior is shown in Figure 1c, from which we can efficiently recover the likelihood p(y 1 = 2, y 2 = 5, y 3 = 3) = F (1) = 0.0025. For Poisson HMMs, we first develop a PGF-based forward algorithm to compute the likelihood, which enables efficient parameter estimatation. We then develop a "tail elimination" approach to compute posterior marginals. Experiments show that our exact algorithms are much faster than existing approximate approaches, and lead to better parameter estimation.
Related work. Several previous works have used factor transformations for inference. Bickson and Guestrin [2] show how to perform inference in the space of characteristic functions (see also [17]) for a certain class of factor graphs. Xue et al. [26] perform variable elimination in discrete models using Walsh-Hadamard transforms. Jha et al. [14] use generating functions (over finite domains) to compute the partition function of Markov logic networks. McKenzie [18] describes the use of PGFs in discrete time series models, which are related to our models except they are fully observed, and thus require no inference.

This paper generalizes the classical variable elimination algorithm in latent variable models from latent variables with finite discrete values or Gaussian distribution to latent variables with countably infinite discrete values, in particular the Poisson latent variable models. The authors observe that the key property that enables fast and exact inference is that operations like conditioning, marginalization and multiplication are "conjugate", producing exact and compact representation. For models with countably infinite factors, they appeal to probability generating function to achieve analogous "conjugacy" so that these operations boast exact and compact representation. This enables variable elimination in such models.
The idea of probability generating function achieving similar forms of "conjugacy" as Gaussian random variables and (finite) discrete random variables is very interesting. This enables a fast and exact inference algorithm on Poisson latent variable models.         The formulation involving probability generating function does not seem to be constrained to Poisson random variables but all the simulations and real application pertain to Poisson HMM. It is unclear if there's anything special with Poisson that enables better parameter estimates. It is also unclear if there are other Poisson latent variable models other than Poisson HMMs with wide applications. Would be great to have this question answered in the article.        The comparison of the proposal, in the simulation experiments, is only with truncation in performance and with MCMC in speed. Truncation is known to perform bad when the rate is above the truncation threshold. It would be considered less than complete if it is not compared with MCMC in accuracy. MCMC is known to be particularly slow in convergence. Comparison of computational cost might as well be with truncation and variational inference. By the way, MCMC is also exact but only with sampling -- it is not approximate.         The real experiment is not too satisfying as well. It does not compare with either MCMC or truncation or present any new findings with this new proposal. Thus the merit of this proposal is unclear in real experiments as well. Apart from parameter estimation accuracy, it would also be very interesting to see predictive accuracy in real experiments because all models are wrong and it's interesting to see if some are useful.


Budgeted stream-based active learning via adaptive submodular maximization
Active learning enables us to reduce the annotation cost by adaptively selecting unlabeled instances to be labeled. For pool-based active learning, several effective methods with theoretical guarantees have been developed through maximizing some utility function satisfying adaptive submodularity. In contrast, there have been few methods for stream-based active learning based on adaptive submodularity. In this paper, we propose a new class of utility functions, policy-adaptive submodular functions, which includes many existing adaptive submodular functions appearing in real world problems. We provide a general framework based on policy-adaptive submodularity that makes it possible to convert existing poolbased methods to stream-based methods and give theoretical guarantees on their performance. In addition we empirically demonstrate their effectiveness by comparing with existing heuristics on common benchmark datasets.

This paper studies the budgeted active learning problem (as under a cardinality constraint) in the stream-based setting. The paper assumes that data arrives in a stream in a RANDOM order, and provide analysis for both the steam and secretary settings, when the objective function satisfies a notion called policy-adaptive submodularity. This property is an extension of submodularity to the adaptive setting, but stricter than adaptive submodularity (in the sense that it considers the gain of a policy as opposed to the gain of a test; however in the adaptive setting a partial realization changes not only the distribution of a test outcome, but also the structure of a policy, and this policy-adaptive submodularity imposes a stronger constraint). The authors show that several adaptive submodular objective functions in active learning also satisfies policy-adaptive submodualriy, and then proposes algorithms stream-based learning. Finally the results are backed-up by the evaluation of the stream and secretary algorithms against pool-based algorithm on UCI WDBC and MNIST dataset.
Pros:           The presentation is mostly clear. The paper shows that one can apply the proposed streaming algorithm, without changing the commonly used objective functions used in pool-based active learning setting. Proofs are sound, and experimental results show that the proposed algorithms work reasonably well in comparison with the pool-based setting.            Cons:                      The stream-based adaptive sensor placement application does not appear convincing to me.                      Is the condition $range(\pi) \subseteq V \setminus B$ in Def 3.1 necessary?                      Policy-adaptive submodularity is used for providng a lower bound on the expected gain of a policy on a random sequence of data points (Lemma B.7). It would be useful and more insightful if the authors can provide intuition in the main paper of why adaptive submodularity is not sufficient for the proof. Is policy-adaptive submodilarity (provably) necessary? or are there examples of adaptive submodular functions (which are not policy-adaptive submodular) that can not achieve the provided theoretical guarantee?                      Data arriving at a random order seems to be a strict constraint. It's a necessary assumption for the (adaptive secretary) algorithm to work provably well, because the algorithm discards the first 1/e data points in each segment, which can contain the most beneficial points in each segment. The authors point out the need of a new algorithm when the data arrives in a fixed order; it would be helpful (for completeness) to provide some very brief discussion on this (i.e., what applications fall into this category; why it's challenging; if there is any insight, or perhaps to point possible directions for solving this problem).                      Editorial issues:           Example A.6, There are two joint probabilities of P[Ya = +1, Yb = +1]. One of them should be P[Ya = +1, Yb = -1].           Ln 103, k labeled instances of the high utility --> k labeled instances of high utility           Lemma B.4, Let f: .. --> Let g: ...


Learning Structured Sparsity in Deep Neural Networks
High demand for computation resources severely hinders deployment of large-scale Deep Neural Networks (DNN) in resource constrained devices. In this work, we propose a Structured Sparsity Learning (SSL) method to regularize the structures (i.e., filters, channels, filter shapes, and layer depth) of DNNs. SSL can: (1) learn a compact structure from a bigger DNN to reduce computation cost; (2) obtain a hardware-friendly structured sparsity of DNN to efficiently accelerate the DNN's evaluation. Experimental results show that SSL achieves on average 5.1× and 3.1× speedups of convolutional layer computation of AlexNet against CPU and GPU, respectively, with off-the-shelf libraries. These speedups are about twice speedups of non-structured sparsity; (3) regularize the DNN structure to improve classification accuracy. The results show that for CIFAR-10, regularization on layer depth reduces a 20-layer Deep Residual Network (ResNet) to 18 layers while improves the accuracy from 91.25% to 92.60%, which is still higher than that of original ResNet with 32 layers. For AlexNet, SSL reduces the error by ∼ 1%.

This paper proposes using group sparsity on various CNN parameters to speed up their execution. The CNN is pre-trained using a classical baseline, then starting from the corresponding parameters, group sparsity constraints are applied during retraining. The inactive groups are then removed, and the final network is fine tuned without the sparsity constraints. This simple method gives good speed-ups without much loss in accuracy (and even some time improvements).
Using group sparsity to turn off redundant parts of a CNN and improve its speed seems like a good idea. Indeed,  significant speed-ups are obtained in a large variety of experiments, with little loss in accuracy and even sometimes a small improvement. The authors use group sparsity on several axes, including the number of filters and channels used, the shape of the filters (I didn't really understand how the authors deactivate efficiently certain filters sites, this should be clarified). and the number of layers (using shortcuts to prevent the network from being disconnected). The idea explored in the paper is thus rather straightforward, but it is a good  and probably useful one. However, unless I missed something, there are many details missing: How is the group sparsity optimisation performed within the CNN training? As importantly, how are the regularization weights chosen? Are they cross-validated? For example, Table 2 reports results with "different strengths of structured sparsity regularisation". These missing details are important, because in the end there must be a balance between speed and accuracy. If the network is to be used in practice it is not enough to duplicate baseline results on test data. The authors should clarify these points.


The Multiple Quantile Graphical Model
We introduce the Multiple Quantile Graphical Model (MQGM), which extends the neighborhood selection approach of Meinshausen and Bühlmann for learning sparse graphical models. The latter is defined by the basic subproblem of modeling the conditional mean of one variable as a sparse function of all others. Our approach models a set of conditional quantiles of one variable as a sparse function of all others, and hence offers a much richer, more expressive class of conditional distribution estimates. We establish that, under suitable regularity conditions, the MQGM identifies the exact conditional independencies with probability tending to one as the problem size grows, even outside of the usual homoskedastic Gaussian data model. We develop an efficient algorithm for fitting the MQGM using the alternating direction method of multipliers. We also describe a strategy for sampling from the joint distribution that underlies the MQGM estimate. Lastly, we present detailed experiments that demonstrate the flexibility and effectiveness of the MQGM in modeling hetereoskedastic non-Gaussian data.

This paper studies the Multiple Quantile Graphical Model (MQGM), which is a multiple quantile regression for high-dimensional, non-parametric distributions. It offers a much richer conditional distribution estimators, based on multiple quantile levels. The authors established theoretical and empirical supports that the MQGM is capable of representing the exact conditional independencies, even for heteroskedastic, non-Gaussian variables.
In this paper titled "The Multiple Quantile Graphical Model", the authors introduced the MQGM, which offers a much broader class of conditional distribution estimates by introducing a set of quantile levels . One key contribution of this paper is that the proposed MQGM asymptotically identifies the exact conditional independencies under some conditions, as the size of the graph grows. There is some empirical results of the proposed algorithm and other alternatives.   The strength of the paper: + an interesting and important problem + the idea is simple and sound + theoretical proof  My concern is mainly for significance of the key idea. The proposed method is very natural, and at a high level, introducing a set of quantile levels and providing more expressive class of estimates , which is not new. The main novelty here might be using multiple quantiles for non-parametric, continuous data. However, it is hard to catch what are the main challenges (compared to related methods) and how to handle it. One suggestion is that intuition about the above point could be made far detailed.  The overall presentation of the paper is good, but the presentation of the reference part is quite hard to follow. The review of relevant work is elegant, but that would be better if less relevant citations can be deleted. Also, it would even be better to follow if the brief description of the algorithm, called ADMM is given in the main paper.  [Typos] line 190: correpsonding -> corresponding


Learning to Poke by Poking: Experiential Learning of Intuitive Physics
We investigate an experiential learning paradigm for acquiring an internal model of intuitive physics. Our model is evaluated on a real-world robotic manipulation task that requires displacing objects to target locations by poking. The robot gathered over 400 hours of experience by executing more than 100K pokes on different objects. We propose a novel approach based on deep neural networks for modeling the dynamics of robot's interactions directly from images, by jointly estimating forward and inverse models of dynamics. The inverse model objective provides supervision to construct informative visual features, which the forward model can then predict and in turn regularize the feature space for the inverse model. The interplay between these two objectives creates useful, accurate models that can then be used for multi-step decision making. This formulation has the additional benefit that it is possible to learn forward models in an abstract feature space and thus alleviate the need of predicting pixels. Our experiments show that this joint modeling approach outperforms alternative methods.

This paper presents an approach on how to learn to arange four objects by poking them with a robotic arm from visual input alone. The authors's model-based method jointly learns the forward and inverse dynamics. Instead of reconstructing images, the forward model predicts the feature vector of a layer in the deep neural network, that is supposedly more abstract due to the regularizing nature of the inverse model. They evaluate the policies defined by the trained inverse models on a simulation and on a lot of real-world data by measuring relative location error obtained through template matching. The paper concludes with a short section about active data collection instead of random sampling.
Pros: + A good idea to move away from generative models when they're not necesary + Real-world experiment with lots of data provides credible proof of concept  Cons: - 'Strong' baseline is undefined. It's hard to judge the performance especially given the variances.  - Unclear evaluation metric, because location and position terms seem to be mixed (l. 211, 212, 215 and Fig. 5 caption) and the influence of rotation is not clear. How exactly are the distances between object locations defined (d_ft and d_it) in terms of position and angle? - Using the median might indicate outliers that are not being discussed.  Questions / Suggestions: - Separated evaluation in terms of position and angle might be useful. The shown trajectories seem to get the position right, but the angle is in some cases quite wrong. It would also be interesting to see if the angle is present in the representation x_t by comparing the activations while rotating the objects for the same position. - The claim in line 114 that, given x and u, a forward model is unique, only holds for single-agent Markov environments. What happens if the robot's arm is occluding the object?  Minor issues: - The 'intuitive' part of intuitive physics is not clearly explained in the introduction. - Lots of language errors. Random or additional 'the' in many sentences (132, 208, 226, Fig4 caption). - The three streams in Figure 3 are not easily identifiable. - The oracle evaluation is explained in line 264, after everything has already been discussed. It might make sense to do this beforehand. - Reference [10] is Sascha Lange, as in [9].


Towards Unifying Hamiltonian Monte Carlo and Slice Sampling
We unify slice sampling and Hamiltonian Monte Carlo (HMC) sampling, demonstrating their connection via the Hamiltonian-Jacobi equation from Hamiltonian mechanics. This insight enables extension of HMC and slice sampling to a broader family of samplers, called Monomial Gamma Samplers (MGS). We provide a theoretical analysis of the mixing performance of such samplers, proving that in the limit of a single parameter, the MGS draws decorrelated samples from the desired target distribution. We further show that as this parameter tends toward this limit, performance gains are achieved at a cost of increasing numerical difficulty and some practical convergence issues. Our theoretical results are validated with synthetic data and real-world applications.
sampling, with a non-uniform conditional slicing distribution. Based on this relationship, we develop theory to analyze the newly proposed broad family of auxiliary-variable-based samplers. We prove that under this special family of distributions for the momentum in HMC, as the distribution becomes more heavy-tailed, the one-step autocorrelation of samples from the target distribution converges asymptotically to zero, leading to potentially decorrelated samples. While of limited practical impact, this theoretical result provides insights into the properties of the proposed family of samplers. We also elaborate on the practical tradeoff between the increased computational complexity associated with improved theoretical sampling efficiency. In the experiments, we validate our theory on both synthetic data and with real-world problems, including Bayesian Logistic Regression (BLR) and Independent Component Analysis (ICA), for which we compare the mixing performance of our approach with that of standard HMC and slice sampling.

This paper derives a framework in which a type of HMC and slice sampling form special cases (in one dimension), motivating a family of HMC samplers with generalized Gaussian momentum distributions, which are shown similarly to be associated to particular generalizations of slice samplers.
The connection between these two separate auxiliary variable Monte Carlo methods is interesting; some previous work has been done in this general area (e.g. Slice Sampling on Hamiltonian Trajectories, ICML 2016), although the specific connection presented in this paper is novel as far as I’m aware.  Overall the quality of the presentation is fairly good. One point I found confusing was that the general set-up for MG-HMC and MG-SS (including explicit statements of the algorithms) is given in Section 3 entirely for the one-dimensional case. In Section 5 and beyond, the methods are used in multidimensional cases, without much explanation given as to how the algorithms/derivations as stated in the previous sections adapt for more than one dimension (e.g. when Hamiltonian trajectories are not necessarily closed) - perhaps it could be made explicit in the main paper exactly how the samplers generalize to more than one dimension? Some description of how this generalizes to more than one dimension is given in the appendix, although there is relatively little detail and/or justification of the points made here as far as I can see - i.e. what happens if the potential isn’t decomposable across dimensions, and what does “hyper-diamond” mean in the footnote of p10? It’s also not clear to me what is meant by “almost certainly” in line 457 - there are cases where the trajectories are not space-filling (e.g. Gaussian kinetic and potential energies).  The theoretical results in the main paper are clearly presented, and demonstrate a nice property of the one-dimensional analytic MG slice sampler for a large class of potential functions. However, there appears to be some discrepancy between the results as stated in the main paper, and the assumptions used in the proofs in the appendix. For example, it seems to be the case that the chain is assumed to be in its stationary distribution in order to prove Theorem 1, but this is not mentioned in the main paper - it would be good to get clarification from the authors on this point, and whether there are any other conditions needed to obtain the results as stated in the main text.  The analytic toy example of the exponential distribution is a nice addition that demonstrates some of the theoretical results explicitly in a particular case. Again, it is not clear to me from the paper whether these results would generalize to more than one dimension. It is mentioned in the paper that when the slice region is disconnected, there may be some discrepancy between the performance of analytic MG-HMC and analytic MG-SS, although it is not made clear whether the ACF results earlier in the section would be expected to generalize. It’s also not clear to me whether these results generalize to multidimensional versions of the samplers.  My two main concerns about the paper are: i) whether the set-up of Sections 2 and 3 is particular to the one-dimensional case, and whether the multi-dimensional implementation of MG-HMC described in Section 5 bears much relation to the analytic MG-HMC samplers, and generalized slice samplers, described earlier in the paper; ii) what is the full list of conditions needed to ensure the validity of some of the results in the appendix, and whether the statements of results in Section 4 of the main paper reflects these conditions.  The appendix appears to contain quite a few typos that should be corrected e.g. line 455 “If uniformly sample”, line 464 “in each of the case”, line 465 “in cases more than one dimension” etc.  With regards to experiments, the simulation studies provide a nice illustration of the one-dimensional theoretical results discussed earlier, and indicate where numerical issues cause practical performance to deviate from the theoretical expectation. The Bayesian logistic regression experiment also demonstrates that improved mixing can be achieved when taking a value of the parameter "a" not equal to 0.5, which nicely demonstrates that the general MG-HMC framework introduced yields samplers that can out-perform HMC with traditional Gaussian momentum.  Minor points: Line 166: MG-SS mentioned before being introduced - possible typo? Line 259: In the second equation, I think there is a missing sign(p)_d on the right-hand side.


Strategic Attentive Writer for Learning Macro-Actions
We present a novel deep recurrent neural network architecture that learns to build implicit plans in an end-to-end manner purely by interacting with an environment in reinforcement learning setting. The network builds an internal plan, which is continuously updated upon observation of the next input from the environment. It can also partition this internal representation into contiguous sub-sequences by learning for how long the plan can be committed to -i.e. followed without replaning. Combining these properties, the proposed model, dubbed STRategic Attentive Writer (STRAW) can learn high-level, temporally abstracted macro-actions of varying lengths that are solely learnt from data without any prior information. These macro-actions enable both structured exploration and economic computation. We experimentally demonstrate that STRAW delivers strong improvements on several ATARI games by employing temporally extended planning strategies (e.g. Ms. Pacman and Frostbite). It is at the same time a general algorithm that can be applied on any sequence data. To that end, we also show that when trained on text prediction task, STRAW naturally predicts frequent n-grams (instead of macro-actions), demonstrating the generality of the approach.

This paper proposes a way for an agent to learn so-called "macro-actions" (where one macro-action is a sequence of low-level actions), to help long-term planning in reinforcement learning tasks. The core idea consists in predicting the sequence of actions to perform in the next N steps. The agent follows this sequence until a new prediction is made, which occurs at "re-planning" steps also decided by the model. More precisely, a re-planning step triggered at time t consists in predicting for each future time step t+1, ..., t+N both a distribution probability over actions and a probability of triggering another re-planning (using an attention mechanism to actually update predictions only for a small number of time steps). The main experiments are on reinforcement learning tasks (2D maze and Atari Learning Environment (ALE)), showing the benefits of learning such macro-actions. Importantly, better results are obtained by adding noise to the state representation, making agent predictions more robust.
The paper is clearly written (up to a few notation issues detailed below), with proper references to related work. It addresses a very relevant problem in reinforcement learning: automatically learning high-level abstractions of actions to allow better long-term planning. To the best of my knowledge the approach taken here is original, and the ideas presented are sound, making use of several "hot" methods in recent machine learning applications (Asynchronous Advantage Actor-Critic, attention mechanisms, LSTMs, Convolutional Neural Networks...). Experimental results are promising, indeed demonstrating that the proposed algorithm helps the agent perform better in most tasks.  Overall I believe this is very interesting research that might eventually lead to significant advances in the field. It does not however, in my opinion, reach that point in the current state of this submission. Although there are many relevant experimental results shown here (and no room left for more due to the 8-page limit), I still feel after reading the paper that it is not entirely clear why and how it works. Here are some examples of questions that are not answered in the current state of the paper: - How useful is the attention mechanism? It looks like the agent is systematically re-planning only the next few steps anyway (see e.g. Fig. 5 and supplementary video) - In the maze experiments, is the improvement over LSTM brought by the long-term planning part, the noisy state representation, or both? - Is the "action repeat of 4" in ALE still useful for such a model, that is supposed to be able to learn by itself if and when it is good to commit to multiple consecutive actions? (I know it is standard practice in ALE but it would be interesting to see if your algorithm can do better) - Extra computational cost aside, would the agent perform better or worse if it re-planned at each time step at evaluation time? (but not during training as in 5.5) - What would Fig. 6 look like over 500 epochs instead of 100? (there is no justification for looking only at the first 100 epochs) What would it look like across more games? - What are typical macro-actions learned in ALE?  Other small remarks / questions: - Are actions also sampled during evaluation, or is the highest scoring action used? - Similar question for the noise added in STRAWe (is it also added during evaluation?) - "a matrix D of the same size as At, which contains the patch p scaled and positioned according to ψ": with the rest set to 0? - Please clarify earlier (in paragraph of l. 145) how the gradient flows through g_t (instead of waiting until the last sentence of section 4): it is confusing in Alg. 1 because it looks like a useless multiplication by 1 and the link to c_1^t-1 is unclear at that point. - The formatting of eq. 2 makes it hard to read - Eq. 3 is confusing: should it be "t=1" below the sum? Could you please define 1_gt? Should it be zeta instead of phi? What is c_t[t]? (by the way it should be c^t, not c_t) - "The most natural solution for our architecture would be to create value-plan containing the estimates": the wording "would be" makes it sound like you are doing something different, but as far as I can tell it is what is being done here. Just use "... is to create a..."? - l. 254: "Notice how corners and areas close to junctions are highlighted" => not all corners are, maybe add "some"? - l. 263: "every 2 million training steps" => better say "every two epochs" since Fig. 3b is plotted in epochs - What are the blue / black horizontal lines in Fig. 5? - Why does replanning occur in Fig. 5 even though it looks like the next two re-planning bits (last row in the figure) are off in all frames?  Typos and notation issues: - l. 7: "re-planing" - l. 43: "it is committed its action plan" - l. 57: "in genral" - l. 60: "enviroment" - C^t and d^t are used in some places instead of A^t and c^t: in Fig. 1, Fig. 2, l. 143 and Fig. 5 - l. 111: "it used to update" - l. 117: "A grid of K × A of" - l. 118: "co-ordinates" - l. 131: "h is two-layer" - l. 138 the last c_t should be c^t, and the last psi^c should be psi_t^c - l. 219 "STRAW(e)" should be STRAWe - l. 236: "it's" - l. 258: "uses a larger 15 x 15 random mazes" - Fig. 4: "From top to bottom" should be "From left to right" - Fig. 5: "drasticly" and "resotres" - l. 300: "and is beneficial to re-plan often"  Update after author feedback: Thanks for the response. I hope some of the questions I asked above can be answered in future work. Regarding the attention mechanism, thanks for pointing out the differences in planning horizon in the video. I still believe though it would be interesting to better understand how it contributes to the overall agent behavior: do you really need 3 attention parameters (position / stride / width) or would a single planning length be enough? (maybe even fixed for a given game?) Also do the read & write attention parameters need to be the same? Thank you also for mentioning the results when re-planning at each step. It would be interesting to add a remark about it in a final version (but please clarify why internal states differ, as I am not sure why this is the case).


Coresets for Scalable Bayesian Logistic Regression
The use of Bayesian methods in large-scale data settings is attractive because of the rich hierarchical models, uncertainty quantification, and prior specification they provide. Standard Bayesian inference algorithms are computationally expensive, however, making their direct application to large datasets difficult or infeasible. Recent work on scaling Bayesian inference has focused on modifying the underlying algorithms to, for example, use only a random data subsample at each iteration. We leverage the insight that data is often redundant to instead obtain a weighted subset of the data (called a coreset) that is much smaller than the original dataset. We can then use this small coreset in any number of existing posterior inference algorithms without modification. In this paper, we develop an efficient coreset construction algorithm for Bayesian logistic regression models. We provide theoretical guarantees on the size and approximation quality of the coreset -both for fixed, known datasets, and in expectation for a wide class of data generative models. Crucially, the proposed approach also permits efficient construction of the coreset in both streaming and parallel settings, with minimal additional effort. We demonstrate the efficacy of our approach on a number of synthetic and real-world datasets, and find that, in practice, the size of the coreset is independent of the original dataset size. Furthermore, constructing the coreset takes a negligible amount of time compared to that required to run MCMC on it.

This paper aims to develop a method to construct a coreset for Bayesian logistic regression. A coreset is a weighted subset of data, which is hopefully much smaller than the original dataset, and which yields provably similar results when existing inference algorithms are applied to it.  Intuitively, this procedure leverages a k-clustering of the data (which must first be constructed) to compute a weight for each data point and to compute the total size of the coreset. The coreset is then resampled from the initial dataset using the computed weights.  This paper restricts its focus to the Bayesian logistic regression model only.  Furthermore, the theory restricts the parameter space to be within a Euclidean ball of a pre-specified radius R (as opposed to unbounded R^D space of parameters, which is the typical setting for logistic regression). The authors also provide negative theoretical results showing that their theoretical framework would not work on an unbounded parameter space, to justify the bounded parameter assumption.  Empirically, the authors provide empirical results on three datasets, which aim to show that inference on coresets performs better than on random subsampling of the data, and provides a good approximation to inference on the full data when the coreset size grows large enough.
* I have one major gripe with the practicality of using coresets versus recent scalable inference strategies, such as minibatch inference methods.  Note that in minibatch inference methods, at each iteration, a small subset of the data is sampled from the full dataset and used to make an update; these methods take advantage of the redundancy in data to perform inexpensive updates. In this paper, coresets reduce the total dataset size by, in some sense, approximating the dataset with a smaller group of (weighted) examples. However, when coresets are used in existing inference algorithms (such as these minibatch algorithms), it seems to me that a very similar procedure will occur: a small subset of this approximate, weighted dataset will be drawn, and used to make an update. I am not convinced this would actually speed up inference (i.e. decrease the number of iterations needed or the time of each iteration), and I don't see a great justification given in this paper. In a sense, I feel that the main thing happening here is that the data is approximated in a smaller/compressed fashion; I can see how this might help with data storage concerns, but I don't see a great justification for why it would appreciably speed inference over existing minibatch methods (especially considering a coreset must be constructed before inference can proceed, which adds additional inference time to this method). One way to demonstrate this would be with timing comparison plots that explicitly show that coresets yield faster inferences given large datasets when compared to minibatch methods---however, no direct experiments of this sort are given.  To summarize this complaint, I feel there is some disconnect between what this paper showed and what it set out to do: we know that smaller datasets typically allow for faster inference, and we know that coresets produces smaller (weighted) datasets. However, it’s not convincingly shown that when coresets are used with (or in comparison with) recent efficient methods, they actually allow for faster inference.   * After the coreset is constructed, the authors intend for existing inference algorithms to be applied to the coreset. However, it is not immediately clear to me how a weighted set of data can be directly used with existing inference algorithms. One could duplicate or resample data points based on their weight, and use this new dataset directly in existing inference algorithms --- is this what happens? Or is there a general-purpose way to use the coreset weights in existing inference algorithms?  * Regarding the definition of an \epsilon-coreset: on page 3, an \epsilon-coreset is defined to be a coreset in which the difference between the log marginal likelihoods of the coreset and the full data are bounded.  Intuitively, I feel the most straightforward goal is to design a coreset in which the approximate coreset posterior is similar to the posterior of the full dataset (i.e. the coreset posterior and true posterior are bounded). However, I have no intuition about how bounding the difference between the log marginal likelihoods corresponds to this goal. Perhaps these two bounds do directly correspond in some way; if so, I feel the paper should do more to explicity state how well an \epsilon-coreset posterior approximates the true posterior. Or at least provide more justification about why bounding the log marginal likelihoods is the appropriate goal.  * Regarding empirical results: on page 7, the main body of empirical results show that a big enough subset eventually yields a good approximation to the test log-likelihood. However, as I describe above in more detail, the results do not seem to focus on what I feel is the direct goal of this work (which is to allow for faster inference in Bayesian logistic regression by first constructing a coreset). Results of this sort would aim to show that a coreset can be constructed, and inference on this coreset completed, in a faster time than it would take to perform inference on the full original dataset. I feel that results of this sort are particularly important since there exists many popular minibatch algorithms developed in recent times, in which each sample can typically be generated very quickly. It would be nice to see if running these minibatch inference algorithms on a coreset versus the entire dataset has a meaningful speedup.  * Experimental comparisons: there were not many experimental comparisons. The main experimental comparison seems to be random subsampling --- it slightly confused me that (I don't believe) this comparison was explicitly described in the paper. However, I assume this is just inference run on a random (uniformly drawn) subset of data of size M.  * What is a good way to determine the parameter space radius R (which is required as an input to the coreset algorithm)? The authors mention that restricting the parameter space to a Euclidean ball is justified since the data is usually preprocessed (to be mean zero and variance 1), which causes each parameter component to be small (e.g. absolute value less than 4) --- however, this statement is not rigorously justified, and I am not sure if it is always true. Is there any theoretically justified way to choose a correct R?  * It seems like the coreset procedure described in this paper is mainly dependent on how well the data can be approximated by a set of clusters.  Theoretically speaking, is there any intuition about a good way to choose the number of clusters, and is the procedure dependent upon this choice?  I do, however, notice that the choice of k over a range is demonstrated empirically.


A Multi-Batch L-BFGS Method for Machine Learning
The question of how to parallelize the stochastic gradient descent (SGD) method has received much attention in the literature. In this paper, we focus instead on batch methods that use a sizeable fraction of the training set at each iteration to facilitate parallelism, and that employ second-order information. In order to improve the learning process, we follow a multi-batch approach in which the batch changes at each iteration. This can cause difficulties because L-BFGS employs gradient differences to update the Hessian approximations, and when these gradients are computed using different data points the process can be unstable. This paper shows how to perform stable quasi-Newton updating in the multi-batch setting, illustrates the behavior of the algorithm in a distributed computing platform, and studies its convergence properties for both the convex and nonconvex cases.

In supervised learning, one is interested in minimizing the empirical risk where efficient optimization algorithms become the key. First-order methods such as stochastic gradient descent and its variants are reasonably well understood admitting efficient implementation and parallelization techniques. However there has been a recent interest in making second-order methods such as Newton's method or L-BFGS method efficient for such large-scale problems. This paper is along this direction, presenting a new variant of the stochastic L-BFGS method  that is efficient and robust in mainly two settings: The first arises in the presence of node failures in a distributed computing environment, the second occurs when one uses an adaptive batch size that varies over iterations for accelerating learning. The main idea is to form the Hessian estimate based on the overlap between consecutive batches (the intuition why this works is that we have less limitation in choosing the second-order information matrix compared to an estimate of the true gradient). This way, the Hessian corrections and updates with L-BFGS become more stable. The results are also illustrated and supported by numerical experiments.    I would recommend this paper to be accepted. My minor concerns/questions are below:  1) If the strong convexity lower-bound mu_1 is very small, then it seems that the resulting limit in Theorem 3.2 will be worse than the limit obtained by a first-order version of your algorithm when one would choose simply H_k = I. Could you please add a few lines to clarify this point?   2) I would suggest to add a few more references to the intro regarding recent work on stochastic L-BFGS for the sake of completeness of the literature survey. Examples include [Moritz, Nishihara, Jordan, 2016], [Mokhtari, Ribeiro, 2014]   3) To my knowledge, [Schmidt, Roux, Bach] paper appeared in Math. Programming recently. I would suggest to give a reference to the Math. Programming journal instead of the reference to the arxiv version.   4) Typo in line 59: use  --> used
The paper is readable and well-written on an interesting topic that has received much recent attention.


Coevolutionary Latent Feature Processes for Continuous-Time User-Item Interactions
Matching users to the right items at the right time is a fundamental task in recommendation systems. As users interact with different items over time, users' and items' feature may evolve and co-evolve over time. Traditional models based on static latent features or discretizing time into epochs can become ineffective for capturing the fine-grained temporal dynamics in the user-item interactions. We propose a coevolutionary latent feature process model that accurately captures the coevolving nature of users' and items' feature. To learn parameters, we design an efficient convex optimization algorithm with a novel low rank space sharing constraints. Extensive experiments on diverse real-world datasets demonstrate significant improvements in user behavior prediction compared to state-of-the-arts.

The paper introduces a model to predict the time of interactions between users and items, and the item which is the most promising for a user. This model builds upon a feature representation of users and items which shifts along time and is a weigthed sum of (i) past actions, (ii) past external features, and (iii) representations of users/items which did interact in the past. The law of time between interactions is represented by a Hawkes process based on the dot product between users and items representations.  The model is slightly changed to allow for a loss function (negative log likelihood) convex in the parameters. Experiments show that the proposed model predicts the interaction time with a much better accuracy than previous similar model (LowRankHawkes) while it similarly predicts the item to interact with.
Modeling the evolution of tastes/properties of users/items along time is a challenging problem for Machine Learning. This paper proposes a solution to model that evolution in a continuous manner, without relying on cyclicity or constant shift assumptions. Experimental results support the proposed model.  The paper was a pleasure to read. It puts apart technical points to focus on what is needed to understand the approach and use it. As an example, Section 4.2 recalls the conditional gradient algorithm but does not give closed forms of the gradients.        Still, I was surprised/disappointed by three points:  1) Is matrix eta necessary ? This matrix contradicts the fact that a user/item evolve along time. Did you measure by any way the necessity for eta ? By example you could look at the Frobenius norm of eta compared to the Frobenius norm of $\Sum_ix_iX_i$, or indicates prediction results of the same model without eta.  2) Theorem 1 is not a Theorem. Theoretical results on nuclear norm minimization to recover rank minimization require assumptions on the matrices and on the observed entries. Such assumptions are not listed in current paper. The authors only give a sketch proof on how to support the proposed reparameterization.  3) Conclusions of Theorem 1 are excessive. Under some hypothesis, we probably can prove that the algorithm learns a matrix X of (low-)rank $k$, but that rank will be greater than the rank of A, B, C and D. Thereafter, X_1, X_5, X_6 and X_7 can live in different column spaces.     To increase the practical usability of the proposed model, I'm looking forward extensions of that work (i) considering incremental update of the model to manage the never-ending flow of data, (ii) managing the exploration/exploitation tradeoff while using that model to recommend interactions to users.        ****** typos ******  Figure 1(a): at the bottom, isn't it $I_j(t_k^***u***)$ as it feeds user $u$ representation (if so, correct also the caption).  Figure 1(a): Arrows are a bit messy. I would expect the arrow from $U_u(t_1^u)$ to go to $I_i(t_l^u)$ as the effect of the interaction is taken into account in the future. Similarly, I would not expect any arrow from $q_j^{u,i}$ to $I_i(t_1^u)$; there is already an arrow from $q_j^{u,i}$ to $I_i(t_l^u)$. If these remarks make sens, there is also two arrows at time $t_l^u$ which should disappear.  Figure 1(b): I would expect an arrow from node $j$ to represent the fact that the representation of that item is poorly aligned with the representation of user $u$.  L.161: "[...] item $i$ at time ***$t$*** than [...]"  L.162: "[...] item $i$ than item ***$j$***. [...]"


The Parallel Knowledge Gradient Method for Batch Bayesian Optimization
In many applications of black-box optimization, one can evaluate multiple points simultaneously, e.g. when evaluating the performances of several different neural networks in a parallel computing environment. In this paper, we develop a novel batch Bayesian optimization algorithm -the parallel knowledge gradient method. By construction, this method provides the one-step Bayes optimal batch of points to sample. We provide an efficient strategy for computing this Bayes-optimal batch of points, and we demonstrate that the parallel knowledge gradient method finds global optima significantly faster than previous batch Bayesian optimization algorithms on both synthetic test functions and when tuning hyperparameters of practical machine learning algorithms, especially when function evaluations are noisy.

This work provides a method to perform parallel Batch Bayesian Optimization by computing a Bayes-optimal batch of configurations to evaluate next. The proposed method measures the utility of evaluating a batch of configurations using the knowledge gradient (q-KG). To efficiently maximize the acquisition function they present a method based on infinitesimal perturbation analysis to estimate the gradient of q-KG. In 3 experiments they show, that q-KG performs competitive with existing parallel Bayesian optimization methods on problems without observation noise and it outperforms these methods on problems with observation noise.
I do like the idea of performing parallel hyperparameter optimization to allow the efficient use of multi-core environments, but some aspects remain unclear to me and I am not convinced by the experimental results. Please find feedback for each of the points above and some questions at the end:  # Technical quality The paper used common artificial benchmark function to show superiority of the proposed method. Additional to these cheap-to-evaluate problems the paper also presents results for optimizing a CNN and logistic regression for Cifar10/MNIST. In the experiment section I would like to see an experiment showing wall-clock time or an analysis on how much one can gain from using this parallel method compared to simple random search or sequential optimization. Also I do not see, why q should be set to 4 for all experiments.  # Novelty/Originality The paper proposes to use the knowledge gradient (a generalization of expected improvement) as an acquisition function and infinitesimal perturbation analysis (IPA) to maximize q-KG. Both methods themselves are not new and the paper refers to "Parallel Bayesian global optimization of expensive functions", which already shows how to use IPA to select a batch of configurations using expected improvement.  # Impact/Usefulness Parallel hyperparameter tunings method are highly relevant for (real-world) applications from many disciplines, such as machine learning, computer vision, biology and robotics.  # Clarity/Presentation The technical details and the background section are described well and the experiment section contains everything necessary to comprehend the performed comparisons.  Questions to the authors:   1.) As Wang et al. (2015) seems to be very relevant to this work and code seems to be available online, have you compared to this method? (Or is this "parallel EI in MOE", then you should call it q-EI to distinguish it from Spearmint's parallel method). To avoid confusion it would be beneficial to clearly state the differences in the paper.   2.) Nowadays multi-core systems allow for much more than 4 parallel runs. For example     * "Scalable Bayesian Optimization Using Deep Neural Networks", ICML'15 used up to 800 parallel runs of DNGO     * "Parallel Algorithm Configuration", LION'12 used 25 SMAC runs in parallel      * "Practical Bayesian Optimization of Machine Learning Algorithms", NIPS'12 used up to 10 parallel runs of Spearmint      Do you have any insights to what extent your method scales and how much one can gain from more parallel runs?   3.) Right now your method is supposed to run synchronously. Would it be possible to extend it to run asynchronously, such as for example Spearmint does?   4.) In the end, wall-clock time matters. Have you evaluated how your method compares to random search or simple sequential Bayesian optimization with respect to wall-clock time? How large is the overhead for computing a new batch?    5.) What does "iterations" as the x-label in your plots mean. Is it the number of function evaluations or number of batches? And does the x-axis include the initial design? If yes, why do not all methods start from the same value?  Minor comments:  * Is there a z^(1:n) missing in eq (4.1)?  * "this architecture has been carefully tuned by the human expert, and achieve a test error" -> achieves  * Parenthetical references are not nouns, e.g. "[16] suggests parallelizing" should be "Snoek et al. [16] suggest parallelizing".  * I don't think "Wang et al. (2015)" is the correct reference for IPA, as they do not introduce the method, but use and apply it.


Mistake Bounds for Binary Matrix Completion
We study the problem of completing a binary matrix in an online learning setting. On each trial we predict a matrix entry and then receive the true entry. We propose a Matrix Exponentiated Gradient algorithm [1] to solve this problem. We provide a mistake bound for the algorithm, which scales with the margin complexity [2, 3] of the underlying matrix. The bound suggests an interpretation where each row of the matrix is a prediction task over a finite set of objects, the columns. Using this we show that the algorithm makes a number of mistakes which is comparable up to a logarithmic factor to the number of mistakes made by the Kernel Perceptron with an optimal kernel in hindsight. We discuss applications of the algorithm to predicting as well as the best biclustering and to the problem of predicting the labeling of a graph without knowing the graph in advance.

The paper concerns completing binary matrix in the online manner: in each trial, the learner predicts the label in {-1,1}, the nature reveals the outcome and the learner suffers 0/1 loss. The interpretation is that each row is a task, and each column is an object, and the complexity of the problem is measures in terms of the margin complexity of the matrix. It is assumed that the problem is consistent in the sense that the entry seen multiple times alway receives the same label. The proposed algorithm to solve this problem is a version of the Matrix Exponentiated Gradient algorithm: a mistake bound in terms of margin complexity is provided, and it is shown that the algorithm makes a number of mistakes which is comparable up to a logarithmic factor to the number of mistakes made by the Kernel Perceptron with an optimal kernel in hindsight. Some applications are discussed: predicting as well as the best biclustering, and predicting the labeling of a graph without knowing the graph in advance.
The paper is clearly written and easy to follow. The technical quality is sounds and the proofs seem correct (except one issue in the proof of Lemma A.4, mentioned below).   I am not entirely sure, whether there are any underlying assumptions on the data apart from the fact that the labels related to a given matrix entry are consistent (i.e., the same)? The authors claim that their setting is less general than in the previous papers on this topic, as they assume that there exists an underlying consistent predictor. Given the labels are consistent for each entry, is it sufficient to say that a consistent predictor exists? If so, I do not find this setting to be very restrictive, as in the real-life applications of matrix completion we often observe a given entry only once. I would be glad to get some feedback from the authors on this question.  While the basic mistake bound for matrix exponentiated gradient algorithm makes use of the analysis the previous work on MEG, the embedding of the comparators (sign-consistent matrices) to the space of positive matrices, and the outcomes to the space of symmetric matrices looks very interesting and original. I also liked the interpretation of the comparator class in terms of linear predictors with any weights over any features for each object, which permits competing with the Kernel Perceptron with an optimal kernel in hindsight.  One drawback is computational complexity of the algorithm, which can hinder the use of the algorithm in practical applications. I think, however, that the computational complexity could be somewhat improved by making use of the fact that the observed outcome matrices have rank one. Then, the eigenvalue decomposition of the exponent in the definition of W^(t) (Algorithm 1) can be maintained and updated using a faster algorithm for rank-one update of the eigenvalues and eigenvectors.  I think the paper brings significant contribution to matrix completion and online learning, and thus I am in favor of accepting it for publication.  Minor remarks:  - Lemma A.4: How is Tr(W^(t-1) X^(t)) bounded in the lemma when \hat{y} = -1? It seems to lower-bounded by 1/(n+m), while according to the Algorithm it is below 1/(n+m) when \hat{y} = -1.  - I think that part of the proof of Theorem 3.1, lines 129-135, is a quite standard bound on quantum relative entropy for matrices with unit trace (see, e.g., [1]), so it is probably not necessary to present it in full detail.  - l. 88-89: "the quantity 1/ mc(U) represents the margin of the worst linear classifier in the best embedded space" -- why is it the worst linear classifier -- from the definition, it looks as if the inverse margin is minimized (= the margin is maximized) over both the classifier's weights and the embedding; shouldn't it be "the best linear classifier"?   ============================  I have read the rebuttal, which clarified all the issues raised in my review. Thank you.


Fast Distributed Submodular Cover: Public-Private Data Summarization
In this paper, we introduce the public-private framework of data summarization motivated by privacy concerns in personalized recommender systems and online social services. Such systems have usually access to massive data generated by a large pool of users. A major fraction of the data is public and is visible to (and can be used for) all users. However, each user can also contribute some private data that should not be shared with other users to ensure her privacy. The goal is to provide a succinct summary of massive dataset, ideally as small as possible, from which customized summaries can be built for each user, i.e. it can contain elements from the public data (for diversity) and users' private data (for personalization). To formalize the above challenge, we assume that the scoring function according to which a user evaluates the utility of her summary satisfies submodularity, a widely used notion in data summarization applications. Thus, we model the data summarization targeted to each user as an instance of a submodular cover problem. However, when the data is massive it is infeasible to use the centralized greedy algorithm to find a customized summary even for a single user. Moreover, for a large pool of users, it is too time consuming to find such summaries separately. Instead, we develop a fast distributed algorithm for submodular cover, FASTCOVER, that provides a succinct summary in one shot and for all users. We show that the solution provided by FASTCOVER is competitive with that of the centralized algorithm with the number of rounds that is exponentially smaller than state of the art results. Moreover, we have implemented FASTCOVER with Spark to demonstrate its practical performance on a number of concrete applications, including personalized location recommendation, personalized movie recommendation, and dominating set on tens of millions of data points and varying number of users.

The paper considers the submodular cover problem in the MapReduce setting. In this problem, given a submodular function f(S), the goal is to find a set S of size as small as possible while satisfying the constraint that the function value f(S) is at least a given value Q. In the previous work, an algorithm named DISCOVER is proposed for this problem but it needed sqrt(m, |OPT|) rounds, where m is the number of machines and OPT is the optimal solution. In this work, a new algorithm is proposed that needs only a polylogarithmic number of rounds.  The basic idea is to simulate the descending threshold algorithm in the centralized setting. The elements are distributed on m machines. Given the same threshold and the current solution, all machines return elements whose marginal gains exceed the threshold. If the number of such element is too large, each machine only returns a small random subset of the ones it finds.  The paper also gives 3 motivating examples of submodular cover with multiple simultaneous constraints, which can be reduced to the above case via known reduction.
The new algorithm significantly improves over the previous work [13] in both theoretical bounds and experimental performance.  The motivation seems strange to me as it is optimizing the size of the union of the summaries of the users. It would not be surprising that this size is much larger than the size of the summary each user might see and the two sizes might not even be correlated.  The privacy motivation confuses me as it seems the reduction from multiple functions to the single function setting is known (the paper cites [15,16] for this reduction). It would be much better to position the paper as an improved algorithm for distributed submodular cover as it is the main contribution of the paper. The model considered in this paper is centralized computation in the sense that the system has access to all relevant information (no private information is withheld from the system) and no formalization of privacy is considered so the discussion regarding privacy is rather distracting from the main message (e.g. the solution presented to an user is affected by the present of other users due to the joint objective, how much privacy is lost?).   After the rebuttal, I am still very confused about the privacy consideration. The authors claim in the rebuttal that the presence or absence of user B does not affect summary for user A but I think this is not true. Consider a simple scenario where the ground set is simply {1,2}, both 1 and 2 are public. The utility functions are both modular. For user A, it is f_A({1}) = 2, f_A({2}} = 1. For user B, it is f_B({1}) = 0 and f_B({2}) = 1. The target utility is Q=1. If the system only has user A, it can return {1}. However, if the system supports both A and B, it will return {2}. Thus, user A can detect the presence of user B based on the his summary.