Finite Sample Prediction and Recovery Bounds for Ordinal Embedding
The goal of ordinal embedding is to represent items as points in a low-dimensional Euclidean space given a set of constraints like "item i is closer to item j than item k". Ordinal constraints like this often come from human judgments. The classic approach to solving this problem is known as non-metric multidimensional scaling. To account for errors and variation in judgments, we consider the noisy situation in which the given constraints are independently corrupted by reversing the correct constraint with some probability. The ordinal embedding problem has been studied for decades, but most past work pays little attention to the question of whether accurate embedding is possible, apart from empirical studies. This paper shows that under a generative data model it is possible to learn the correct embedding from noisy distance comparisons. In establishing this fundamental result, the paper makes several new contributions. First, we derive prediction error bounds for embedding from noisy distance comparisons by exploiting the fact that the rank of a distance matrix of points in R d is at most d + 2. These bounds characterize how well a learned embedding predicts new comparative judgments. Second, we show that the underlying embedding can be recovered by solving a simple convex optimization. This result is highly non-trivial since we show that the linear map corresponding to distance comparisons is non-invertible, but there exists a nonlinear map that is invertible. Third, two new algorithms for ordinal embedding are proposed and evaluated in experiments.

This paper addresses the problem of finding potentially low dimensional embedding from distance comparisons known as ordinal embedding. The authors provide novel generalization bounds based on finite quantitiesfor ordinal embedding, as far as I understand the fiel, this is novel. They also study a particular model based on maximum likelihood estimation where they justify that a correct embedding can be recevered. An experimental evaluation is provided on synthetic data that includes 2 new algorithms.
Pros: -Interesting and a priori novel theoretical analysis for Ordinal Embedding Cons: -Paper sometimes hard to follow where some parts deserve more attention -The noise model the authors propose to study is not clear -Experimental evaluation limited to synthetic data with comparison to only one existing method  Overall: Good and interesting theoretical results that are novel for this topic. But the paper lacks of discussion/precision on some parts and the practical aspects are relatively limited.  Comments:  -The bounds proposed in this paper appear to be novel and are interesting. I wonder wether there exists a link with U-statistics that can be used to justify that a true risk can be efficiently estimated  by considering a random subset from all the possible triplets. Check the following reference for example: *S. Clemencon, I. Colin and A. Bellet. Scaling-up Empirical Risk Minimization: Optimization of Incomplete U-statistics. Journal of Machine Learning Research 17(76):1-36, 2016.  The results in Section 4 are interesting, but the authors do not discuss their implication from a practical standpoint. We do not know if they are purely theoretical to justify the consistency of the method with no ambition to deduce some practical algorithms or if there are some perspectives to develop new well-founded approach to recover the embedding.  -The authors mention the presence of a noise model but I was unable to find a description or discussion about this noise model in the paper. If the authors have in mind a classic Gaussian noise, this must be clearly mentioned and discussed accordingly in the paper. In the supplementary caption of Figure 3 page 14, I was not able to understand what means \alphaM for describing the noise since \alpha and M have not been introduced before.  ==After rebuttal == Ok for the answer. You may anyway specify more clearly your point here. Since $f$ takes as argument the distance between elements in the triplet (which is reasonable for defining the true labeling), one could think that the noise may depend on this distance which is not what you meant here.   -The experimental setup is limited to synthetic data but i would have been interesting to study the behavior of the approach on real data.  The notion of prediction error is not explicitly mention in the setup, I guess that this corresponds to the evaluation over the test set of the difference between the risk R  in Section 2 of the learned model and the true R defined. If so, I think that an additional experiment showing the accuracy obtained on the test set by counting the number of times the distances between in each triplet are correctly ordered could be interesting to assess if the embedding is relevant in addition to a good estimation of the true matrix G^*.  I am not completely convinced by the approach "Nuclear Norm PGD Debiased" since it exhibits a performance similar or worse than "Nuclear PGD". Anyway the two proposed algorithms are presented quickly is Section 5. However, they deserve more discussion. In particular, the remark on performance gap between convex and non convex solutions at the end of Section 5.  The algorithms are not sufficiently presented and their properties are not sufficiently discussed. I think that the authors could move a part of the proofs of justifications of Section 4 into the supplementary material to have more place to discuss other aspects of the paper.  Finally, the experimental section seems disconnected from the results presented in Section 4.  Trying to assess in practice the quantities presented in this section from the model obtained in the experiments could strengthen the global coherence of the paper.  -The paper lacks of a general conclusion that could present the perspective and replace the work with respect to existing work in the state of the art.  At the end, it is difficult for me to really assess the quality of the contribution in comparison to existing work in the literature and I think the paper misses a deeper discussion on existing work.  -Other remarks: *l51: has a a non-empty -> has a non-empty *l77: in the definition of all the triplets, T, you may need to add the constraint: i < j  *l89: If you assume that the set of triplets T is infinite, the notation \frac{1}{T} is not accurate in the definition of the expectation *l98: later we will link define $\ell$ -> choose between link and define


Optimal Tagging with Markov Chain Optimization
Many information systems use tags and keywords to describe and annotate content. These allow for efficient organization and categorization of items, as well as facilitate relevant search queries. As such, the selected set of tags for an item can have a considerable effect on the volume of traffic that eventually reaches an item. In tagging systems where tags are exclusively chosen by an item's owner, who in turn is interested in maximizing traffic, a principled approach for assigning tags can prove valuable. In this paper we introduce the problem of optimal tagging, where the task is to choose a subset of tags for a new item such that the probability of browsing users reaching that item is maximized. We formulate the problem by modeling traffic using a Markov chain, and asking how transitions in this chain should be modified to maximize traffic into a certain state of interest. The resulting optimization problem involves maximizing a certain function over subsets, under a cardinality constraint. We show that the optimization problem is NP-hard, but has a (1− 1 e )-approximation via a simple greedy algorithm due to monotonicity and submodularity. Furthermore, the structure of the problem allows for an efficient computation of the greedy step. To demonstrate the effectiveness of our method, we perform experiments on three tagging datasets, and show that the greedy algorithm outperforms other baselines.

The current paper is devoted to the problem of maximizing the probability of reaching a given vertex under a Markov random walk on a graph by adding a set of new incoming links from a given set of feasible links under a budget constraint. Authors prove that the problem is NP-hard and propose a greedy algorithm, which is reasonable due to the submodularity of the reaching probability with respect to the chosen set of links. Though the paper is clear and results look correct, the paper does not seem novel enough in the theoretical part and does not seem to have a proved impact for practical tasks. At this stage, I recommend to reject it.
Optimization of the link structure for PR is not a new topic. Apart from papers mentioned in Related work, there are also those not reviewed, including “PageRank Optimization by Edge Selection” by Csaji et al., “Maximizing PageRank with New Backlinks” by Olsen, “PageRank Optimization in Polynomial Time by Stochastic Shortest Path Reformulation” by Csaji et al.  **The novelty** of the study is questionable.  The probability of reaching the target state $\sigma$ can be viewed as the state’s stationary probability for the graph, where the added edges are directed to the state $\sigma$ and the matrix of transition probabilities is raised to an appropriate power. This observation does not immediately reduce the problem of the paper to a known task, however, it may partially explain the similarity between the theoretical part and the works of Olsen, where the stationary probability is maximized.     In particular, Section 4 resembles the work “Maximizing PageRank with New Backlinks” (not cited in the paper), where M. Olsen considered a reduction of a Markov chain optimization problem to the independent set problem, which is equivalent to the vertex cover problem.  Theorems 5.1, 5.3 are reasonable, but very simple and resemble Lemmas 1,2 from [15].  **The practical example** with tags and objects looks a bit questionable, because of the two reasons:  1) To use the proposed solution in practice, one needs first to choose the set of feasible tags for the object. It should be conducted with some objective, and we can assume that it is performed by some ranking algorithm on the graph. Note that the development of such an algorithm is the task of the current paper. Therefore, we can assume in practice that the feasible set coincides with the whole set of tags. Furthermore, this means that objects do not differ in the sense that the properties of the object are not taken into account in the problem formulation.  2) I did not get why maximizing the probability of reaching an item is a well-suited objective for the application. Why is it better than the PageRank value?   **Experiments:**  I would be appreciated to see a stronger baseline like the following one. Compute, for each tag $\tau$, the probability $P(\tau)$ of reaching $\tau$ for the Markov chain for the initial graph. Choose $k$ tags $\tau$ with the maximal value of $q_{\tau \sigma}^+ P(\tau)$. This heuristic seems to be an equivalent of the one in [15] (“high PageRank values compared to the outdegree”, see Fig. 1). Note that the Low degree baseline outperforms the High Degree one for all datasets according to Figure 1. While the degree of a tag itself is an ambiguous factor, a more complicates heuristic would perform much better.  **Other comments:**  The presentation could be significantly optimized in terms of space occupied by different parts. In particular, Abstract and Introduction could be shortened significantly. For example, some sentences are redundant: compere “When an item is tagged, positive probabilities are assigned to transitioning from the tag to the item and vice versa” at line 51 and “Intuitively, tagging an item causes probability to flow from the tag to the item, on account of other items with this tag.” at line 54. It is said 3 times that “the optimization problem is NP-hard” and so on.  Equation in step 5 of Algorithm 1 is something not standard.  The equation for $B$ in line 255 looks wrong.  **Questions:**  Could you compare the proof in Section 4 with the technique of Olsen? Is the idea of the proof in Section 4 still novel after the work of Olsen?  50: “Our framework can incorporate many web search click models [9, 3].” Which click models from [9] are incorporated in particular? I believe, that paper do not study any Markov chain models.  64: “we argue that maximizing the probability of reaching an item, as opposed to maximizing the proportion of time an infinite random walk spends in the item’s state, is a better suited objective for the applications we consider.” Why? I don’t think so and argue the opposite.   **Misspellings:**  46: “a a set” 79: "One the main roles" -> "One of the main roles" 84: “[5].While” Space is missed 111: Eq. 1: $\max_{S\in [n]}$ should be $\max_{S\subset [n]}$     ----- **UPDATE on Author Feedback**  - A: Thank you for suggesting the additional baseline - however it actually does appear in our paper (“One Step” in Fig. 1).  It implies from the text (l. 284) that One Step baseline is much simpler and definitely different from that I proposed. In fact, you obtain One Step baseline, if you substitute the *initial distribution* $\pi$ instead of the *probabilities $P(\tau)$ of reaching $\tau$* used in the baseline I proposed. I believe, 1) The scoring in my baseline is a better approximation of the true gain, 2) It is still much more efficient than the Greedy algorithm, 3) It is analogous to a baseline from [15].  - A: We agree that it would be practically wise to first select a feasible set of tags. However, we do not see this as a caveat, since finding a set of K potential tags with k << K << n seems like a very reasonable user-initiated preliminary step.  It is still not clear, which objectives could prevent a user from choosing K=n while not preventing from choosing K=k.   - A: As our model supports arbitrary transition probabilities from tags to items, these can be set according to click models. For instance, in the cascade model, for item i with rank r in tag t we set Pr(t -> i) = (1-p_1)*(1-p_2)*...*(1-p_{r-1})*p_r. We will elaborate on this.  Now this is clear. Still, I read [9] and believe this paper is NOT about cascade click models (unlike [3]). Please, be more precise.


Automatic Neuron Detection in Calcium Imaging Data Using Convolutional Networks
Calcium imaging is an important technique for monitoring the activity of thousands of neurons simultaneously. As calcium imaging datasets grow in size, automated detection of individual neurons is becoming important. Here we apply a supervised learning approach to this problem and show that convolutional networks can achieve near-human accuracy and superhuman speed. Accuracy is superior to the popular PCA/ICA method based on precision and recall relative to ground truth annotation by a human expert. These results suggest that convolutional networks are an efficient and flexible tool for the analysis of large-scale calcium imaging data.
in shape, size, spacing, and resting-level fluorescence. Additionally, different neuroscience studies may require differing ROI selection criteria. Some may require only cell bodies [5,11], while others involve dendrites [6]. Some may require only active cells, while others necessitate both active and inactive cells [10]. Some neuroscientists may wish to reject slightly out-of-focus neurons. For all of these reasons, a neuroscientist may spend hours or days tuning the parameters of nominally automated methods, or may never succeed in finding a set of parameters that produces satisfactory results.
As a way of dealing with these difficulties, we focus here on a supervised learning approach to automated ROI detection. An automated ROI detector could be used to replace manual ROI detection by a human expert in the traditional workflow, or could be used to make the basis learning algorithms more reliable by providing good initial conditions for basis images. However, the usability of an automated algorithm strongly depends on it attaining high accuracy. A supervised learning method can adapt to different ROI selection criteria and generalize them to new datasets. Supervised learning has become the dominant approach for attaining high accuracy in many computer vision problems [12].
We assemble ground truth datasets consisting of calcium imaging videos along with ROIs drawn by human experts and employ a precision-recall formalism for quantifying accuracy. We train a sliding window convolutional network (ConvNet) to take a calcium video as input and output a 2D image that matches the human-drawn ROIs as well as possible. The ConvNet achieves near-human accuracy and exceeds that of PCA/ICA [6].
The prior work most similar to ours used supervised learning based on boosting with hand-designed features [13]. Other previous attempts to automate ROI detection did not employ supervised machine learning. For example, hand-designed filtering operations [14] and normalized cuts [15] were applied to image pixel correlations.
The major cost of supervised learning is the human effort required to create the training set. As a rough guide, our results suggest that on the order of 10 hours of effort or 1000 annotated cells are sufficient to yield a ConvNet with usable accuracy. This initial time investment, however, is more than repaid by the speed of a ConvNet at classifying new data. Furthermore, the marginal effort required to create a training set is essentially zero for those neuroscientists who already have annotated data. Neuroscientists can also agree to use the same trained ConvNets for uniformity of ROI selection across labs.
From the deep learning perspective, an interesting aspect of our work is that a ConvNet that processes a spatiotemporal (2+1)D image is trained using only spatial (2D) annotations. Full spatiotemporal annotations (spatial locations and times of activation) would have been more laborious to collect. The use of purely spatial annotations is possible because the neurons in our videos are stationary (apart from motion artifacts). This makes our task simpler than other applications of ConvNets to video processing [16].

This paper presents the first fully supervised approach to cell detection of two-photon imaging dat, based on a fully convolutional framework (ZNN). The results are compelling and seems not far from the performance of human annotators. Results are compared to a common PCA/ICA approach but not to the state of the art in unsupervised cell detection (e.g., CMNF). The output of this scheme is limited to the spatial domain (unlike other approach which in one complex step determine also spike activity). Performance for awake behaving mice primary visual cortex and medial entorhinal cortex.
NOTE Following the rebuttal and reading of the other reviews I lowered the "Technical Quality" which is now 2 (originally 3).  The authors of Ref1 describe cases where a human annotator could fail to reliably detect and demix cell boundaries. Therefore to claim superiority over such unsupervised methods the current paper needs to compare performance against the performance of such approaches (see Ref1, e.g. using). It is generally required to compare against CMNF in order to claim state of the art accuracy.   It is unclear whether the paper used re-evaluation in order to correct the seemingly erroneous ground truth used for annotation. (see specific comment 8 below).    It would be generally impressive if the authors can replace or extend the large temporal smoothing and downsampling with network’s kernel that achieve the same result in a supervised manner, e.g., using 3D kernels. If the authors already experienced with 3D training and the results ere not convincing this must be included in the paper. I see this point quite critical in the evaluation of the paper as the temporal smoothing is widely not justified (as I explain in my specific comments below).    Specific Comments:   1. 27-28. The authors suggest that CNMF and PCA/ICA are inadequate for some datasets. This point is very weak and incomplete. What are these datasets? And in particular how the current paper is superior over such methods (e.g., CNMF) on these unspecified datasets? 2. 28-29. The authors say that unsupervised methods have not fully replaced the traditional approach. However approaches like CNMF were published in 2016 and other laboratories are likely to share their code in open source pipelines this and next year. Therefore the claim that the traditional approach is still in practice is unjust.  3. 34-39. The text suggests that different experimental setups and scientific goals would require different cell detection outcomes and hence unsupervised approach will depend on parameter tuning. While this may be a true case, it is unclear how much of this burden is actually eliminated by preparation of human ground truth — it seems that all parameter tuning are parametrization of the model, and in the end will require less human effort than ground truth preparation. Moreover, once a laboratory fixed the parameters for several setups the tuning on new datasets (marginal effort) is likely to be greatly reduced. Therefore, I recommend the authors to suppress this paragraph.  4. 39-40. The authors point that current automatic methods require proofreading. However accuracy is what matters — hence also the suggested supervised method as well end up requiring proofreading. Please correct my impression or delete this point.   5. 51. The authors compare to a common method (PCA/ICA) that is not state of the art. First, it should be noted by the authors. Second, it will strengthen the paper if the authors compare to Ref1. 6. 71-77 can be suppressed.  7. 99-101. I find this explanation too heuristic. First at this point the author is unaware of the temporal sampling rate and cannot appreciate the smoothing effect. Second, the usefulness of making the dataset more manageable in size should be marginal where sophisticated methods could perform great (fast and accurate) without data reduction. Also, the procedure used (rectangular filter with max-pooling operation) is nonstandard and require better reasoning. Finally, 132 is read as a drawback of this “Severe reduction.” And then comes 133 which simply goes against the paper’s own method. This should be fixed by introducing better reasoning.  8. 110-115 is confusing. First the V1 dataset is presented. Then a sentence about human annotation time. Then the MEX dataset is presented. Following sentence is about both datasets. And then another seemingly general sentence about human annotation time (I suppose for the MEC dataset since numbers contradict the ones for V1; or could be different estimates of the same human time?).   9. 127 “an an” typo.  10. 132. The choice to heavily down sample in time was the authors’ and hence the explanation in 132 should be revised and made more convincing (the authors can say that other applications were inferior in performance, or come up with a alternative reasoning — perhaps related to the non-isotropic nature of the problem and the different noise distributions in space vs time. 11. 134. Space typo.  12. 136. The large difference of depth between the two datasets should be succinctly explained.  13. 177-179. I strongly disagree with the claim that the reevaluation and the margins reported translate to “nearing human performance.” Instead I suggest the authors repeat this scheme with another human expert and then let the two expert reconcile their disagreements into a mutual call (as done in other fields). It is unfair to let the human be convinced by the algorithm’s output and this is certainly an invalid evaluation metric. Also it is unclear that the reported margin shows near human performance even it this F score was obtained in a valid way. Finally the F1 score of the export w.r.t his own judgment being 0.89 does not represent a standard value for the F score that should be attained.   14. 194-197. Given the above reservations it will contribute if the authors can repeat the “human interrogation” procedure with the PCA/ICA result and see if naive humans are convinced to change their opinion.    References:  Ref1 ([9] in the submitted paper) Simultaneous Denoising, Deconvolution, and Demixing of Calcium Imaging Data (Neuron 2016)


Bayesian optimization for automated model selection
Despite the success of kernel-based nonparametric methods, kernel selection still requires considerable expertise, and is often described as a "black art." We present a sophisticated method for automatically searching for an appropriate kernel from an infinite space of potential choices. Previous efforts in this direction have focused on traversing a kernel grammar, only examining the data via computation of marginal likelihood. Our proposed search method is based on Bayesian optimization in model space, where we reason about model evidence as a function to be maximized. We explicitly reason about the data distribution and how it induces similarity between potential model choices in terms of the explanations they can offer for observed data. In this light, we construct a novel kernel between models to explain a given dataset. Our method is capable of finding a model that explains a given dataset well without any human assistance, often with fewer computations of model evidence than previous approaches, a claim we demonstrate empirically.

This paper introduces a novel approach to kernel choice, especially in the case of Gaussian processes, through Bayesian optimization. A very interesting kernel between kernels is proposed. While the paper focuses on GPs, the method could be used more generally, and the techniques developed (approximating the model evidence, a "kernel kernel") are potentially useful beyond this setting.
I think the clarity of your presentation can be strengthened.   1) Line 80 - hyperprameters -> hyperparameters 2) Line 66 - adopted -> adapter 3) Line 83 - "principle approach" -> principled approach 4) Footnote 1 - maybe I'm one of the authors who's to blame, but I'm perfectly comfortable thinking of model selection as hyperparameter learning. Sure, I'd like to integrate out the hyperparameters (and thus average over a class of models), but if I can't do that, then I'm happy to learn them and select a single model. Indeed, I'd question your claim that the Bayesian perspective is one of Bayesian model selection---you're just moving one more level up the hierarchy (hyper-hyper parameters!), but wouldn't it be even better if we could integrate over different models M? This might actually be possible within your framework---you're using Bayesian optimization to maximize a function, but the recent probabilistic numerics work on Bayesian quadrature suggests you could instead use BO to do integration! 5) I was a little confused about how you maximize the acquisition function---where does the set of candidate models C come from, and what exactly do you have to do for each candidate model. I think run an optimization to approximate the model evidence, is that right? If so, it's confusing me because the model evidence approximation isn't discussed until later -- page 5 while the acquisition function is discussed on page 3. 6) The priors you suggest are reasonable, though I wonder if it'd be better to use a distribution like Student t (see some work by Vehtari and colleagues) in place of a Normal. Do you have any citations to back up your choices? Why sd = 0.7? 7) I'd like a little more discussion in Section 4.3. Would you suggest using Laplace in place of the use of BIC in Duvenaud et al? You should provide a page number in Murphy, as I'm curious to know more about BIC vs Laplace. 8) Section 4.4 is very interesting. Eq. 6 looks like what happens when you create a cross-covariance of two SE kernels (e.g. Melkumyan and Ramos, IJCAI 2011) though of course the setup is different. I'm also curious whether if there's an alternative approach to creating a kernel kernel by considering the maximum mean discrepancy between the distributions. 9) After rereading Section 5.1, I'm still a little confused about what you're evaluating. You should more clearly say what your method is, and why/what the mean prediction / k-nearest neighbors regression are doing. I think what you've left implicit is exactly what the ground truth is to which you're comparing. Once you say that, and say what your method is doing, everything should become clear.


Safe Policy Improvement by Minimizing Robust Baseline Regret
An important problem in sequential decision-making under uncertainty is to use limited data to compute a safe policy, which is guaranteed to outperform a given baseline strategy. In this paper, we develop and analyze a new model-based approach that computes a safe policy, given an inaccurate model of the system's dynamics and guarantees on the accuracy of this model. The new robust method uses this model to directly minimize the (negative) regret w.r.t. the baseline policy. Contrary to existing approaches, minimizing the regret allows one to improve the baseline policy in states with accurate dynamics and to seamlessly fall back to the baseline policy, otherwise. We show that our formulation is NP-hard and propose a simple approximate algorithm. Our empirical results on several domains further show that even the simple approximate algorithm can outperform standard approaches.

The paper considers the problem of robust MDP. In particular, it considers that we have an uncertain model of transition probabilities of the MDP (with the rectangular uncertainty) as well as a baseline policy. The goal is to find a new policy such that it is guaranteed to be no worse than the baseline. This is called the safe policy improvement.  A robust approach would find a policy that maximizes the worst case performance. To provide a safe policy, it has to compare it with the performance of the baseline. Since the performance of the baseline isn’t known, one can compare it with the best possible performance of the baseline. However this might be an overly pessimistic approach because the choice of the worst model for the robust policy is not necessarily the same as the choice of the best model selected to evaluate the baseline policy.  A more reasonable approach is to compare the regret of the policy compared to the baseline. The regret of a policy is defined as the difference of the performance of the baseline policy with the performance of the policy under the same probability model. By finding the policy that maximizes the negative worst case regret within the uncertainty set, we ensure that the resulting policy is safe, and hopefully less conservative than the previous approach.  The paper provides a performance bound for this solution (Theorem 5). It also shows that in general such a policy is randomized (Theorem 3). The difficulty, however, is that the corresponding optimization problem is NP-Hard (Theorem 6).  To circumvent this challenge, the paper suggests a heuristic approximation to this problem. The approximation is based on the fact that if there is no uncertainty for the MDP when actions are selected according to the baseline policy, the problem becomes a usual robust MDP problem (Proposition 7), which can be solved in polynomial time. The approximation is that the algorithm enforces the uncertainty for the actions selected by the baseline policy to be zero, as if there is no error in their model. This type of approximation is reasonable when the data is collected mostly from following the baseline policy with some occasional exploratory randomness to select other actions. In that case, the estimated model for the baseline is indeed much more accurate.  Note that this is still an approximation. The paper does not provide a guarantee on the quality of this approximation, but it empirically shows that the performance is better than a usual robust MDP approach that doesn’t change the uncertainty set (Algorithm 2). It is also much better than a certainty-equivalence approach that does not take the uncertainty into account (Algorithm 1).
Brief summary of evaluation:  Technical quality: The results seem to be technically sound. The technical tools are more or less standard. Novelty/originality: The optimization formulation seems to be novel. The derivation of the error bounds is more or less standard. Potential impact or usefulness: This might be a stepping stone for further results, but I doubt it would have a huge real-world impact as it is, because a) it works only for finite MDPs and b) the solution doesn’t have a guarantee. Clarity and presentation: It is relatively well-written.  ===  This is a reasonably good paper. It addresses an important problem that is of interest to the real-world applications of reinforcement learning. It formulates the problem as an optimization problem that is much more reasonable than a conventional robust MDP formulation. It is unfortunate that the solution to the new optimization problem is NP-Hard (Theorem 6). However showing that this is the case is important. Also providing the theoretical guarantee on how well we can expect the optimal solution to behave is important too (Theorem 5).  The suggested algorithm (Algorithm 1) is indeed very simple. It would be much better to have a theoretical guarantee on the performance of the approximate algorithm, but in lieu of that, the empirical results are reassuring.  Some detailed comments:  - Algorithm 1 (L4): Shouldn’t the term rho(pi_B,xi) be removed? It is constant anyway.  - L198: The reason #2 is not clear.  - L69: Please specify that the paper is about finite state and action spaces.  - L159 (Theorem 5): The norms as well as e_{\pi*} and e_{\pi_B} are not defined.  - L90: What does a randomized uncertain set mean?  - Please compare the choice of regret as the objective to be minimized with the following work:  Regan and Boutilier, “Regret-based Reward Elicitation for Markov Decision Process,” USI, 2009. The paper is not about the uncertainty in P, but they choose regret too.  === After Rebuttal: Thank you for the response.


CliqueCNN: Deep Unsupervised Exemplar Learning
Exemplar learning is a powerful paradigm for discovering visual similarities in an unsupervised manner. In this context, however, the recent breakthrough in deep learning could not yet unfold its full potential. With only a single positive sample, a great imbalance between one positive and many negatives, and unreliable relationships between most samples, training of Convolutional Neural networks is impaired. Given weak estimates of local distance we propose a single optimization problem to extract batches of samples with mutually consistent relations. Conflicting relations are distributed over different batches and similar samples are grouped into compact cliques. Learning exemplar similarities is framed as a sequence of clique categorization tasks. The CNN then consolidates transitivity relations within and between cliques and learns a single representation for all samples without the need for labels. The proposed unsupervised approach has shown competitive performance on detailed posture analysis and object classification.

This paper proposes an unsupervised learning method for deep networks based on classifying samples into a pre-defined set of cliques. An optimization procedure is proposed to partition the samples into batches that respect the known similarity and dissimilarity relationships between the clique samples, and leave out pairs with unknown relationship. The method is iterated to refine the learned similarity network, starting with HOG-LDA similarity in the first iteration. Results are demonstrated by nearest-neighbor classification using the learned similarity in a number of datasets.
The paper reads well and is technically sound, especially the well-defined optimization problem in Sec. 2.3. The idea of consolidating transitivity relationships into batches is an interesting angle in similarity learning.  Excluding Section 2.3, however, the method as a whole feels a bit ad-hoc. There doesn't seem to be any reason why the assignment to batches could not be performed on samples directly, rather than cliques (other than that it might speed things up). It would make for a more well-justified method to introduced it in terms of the raw individual samples. Using cliques instead of samples could then be presented as an engineering trick to reduce the problem size.  The starting point of the paper seems to be that initial similarities are unreliable, and so there should be no label (training signal) assigned to most of them. It would be more satisfying to relate the optimization objective to this goal explicitly, framing it as an approximation of the full problem if possible.  The paper's presentation is good overall. The description in Section 2.2 is too terse, however, making reproducibility suffer; it is also missing supporting references (e.g. "complete-linkage clustering").  The experiments are adequate, especially the fact that the authors used the learned similarities for nearest-neighbor classification/prediction in challenging tasks. The comparisons with ground-truth nearest-neighbors and supervised methods are especially illuminating, placing the unsupervised learning results into perspective.  However, there could be a better experimental breakdown of the relative contributions of the aspects of the pipeline, since it is hard to untangle the different components. Simply training a classifier on cliques (suggested in line 141, "Forcing them into different classes ...") would be a good measure of the influence of the cliques clustering on the final result. Another important baseline would be training a simple two-stream network end-to-end on the same data. The authors claim that these two approaches would incorporate incorrect relationships, so it is important to show how the proposed method helps in that regard.  It is also debatable whether visualizing merged RGB images really shows that the similarities are meaningful; it is possible there is high pixel-level similarity, but not necessarily semantic similarity. For example, it would be possible for clustering based on RGB values alone to obtain similar visualizations. Another option would be to compute objective measures, like cluster purity, to complement and validate these results.


Large-Scale Price Optimization via Network Flow
This paper deals with price optimization, which is to find the best pricing strategy that maximizes revenue or profit, on the basis of demand forecasting models. Though recent advances in regression technologies have made it possible to reveal price-demand relationship of a large number of products, most existing price optimization methods, such as mixed integer programming formulation, cannot handle tens or hundreds of products because of their high computational costs. To cope with this problem, this paper proposes a novel approach based on network flow algorithms. We reveal a connection between supermodularity of the revenue and cross elasticity of demand. On the basis of this connection, we propose an efficient algorithm that employs network flow algorithms. The proposed algorithm can handle hundreds or thousands of products, and returns an exact optimal solution under an assumption regarding cross elasticity of demand. Even if the assumption does not hold, the proposed algorithm can efficiently find approximate solutions as good as other state-of-the-art methods, as empirical results show.

This paper considers the problem of maximizing the seller's revenue by finding the optimal pricing strategy, given certain demand models. In particular, a binary quadratic program, first proposed by Ito and Fujimaki, is applied to formulate this price optimization problem. The authors show that the gross profit function (mapping the prices of all items to the revenue) is supermodular if the demand model satisfies the substitute-goods property. In this case the objective function of the binary quadratic programming problem is submodular, which can be reduced to a minimum s-t cut (or max-flow) problem, which is known to be solvable (e.g. Dinic’s algorithm as it is used by the authors in their experiments).   In the case that the substitute-goods property fails to hold, the authors provide an approximation algorithm by constructing an approximation function which bounds the objective function from below. Specifically, the objective function of the binary quadratic program is decomposed into a supermodular and submodular terms, and the supermodular part is replaced by another function which is a lower bound. Then, the authors provide a proximal gradient method to compute this lower bound, with the number of iterations bounded by O(1/sqrt(epsilon)). The authors claim that in practice the convergence can be observed in about 10 iterations.   Finally, the authors provide many experiments results about their proximal gradient method based algorithm. They show that their algorithm is considerably faster than the previous algorithms, and produces better pricing strategy (Table 2). In addition, they show that (in Figure 1) the approximated value of the objective function is close to the exact optimal value (by deriving upper bound of it).
For the first part with substitute goods property, unfortunately, I do not see much novelty in it. The binary quadratic programming formulation was already proposed in Ito and Fujimak’s work, while the reduction from the unconstrained binary quadratic programming problem to the minimum cut is “well known” (as also remarked by the authors).   For the second part, the idea of providing a lower bound on the objective function and using proximal gradient method is interesting, but its theoretical part is not well-developed. In particular, there is no theoretical analysis regarding “how well” does the lower bound approximate the exact optimal value. Although empirical results given in Figure 1 address this issue, but I was expecting at least an asymptotic bound to be given. Nevertheless, the empirical results given in the last section seems to be convincing.   ---- Post discussion comments ----  I have read all the reviews and the corresponding authors’ responses. My sense is that the paper is perhaps somewhat stronger than my initial assessment, even if it is only a modest contribution on the theory side.   For the first part of the paper about the reduction to min-cut, after authors’ feedback 1) and 2), I can now see some novelties. I initially had the impression that they are simply applying existing techniques (the last reviewer has the same impression and raises the same issue). I think the authors should focus more on Proposition 2 which reveals the relationship between supermodularity of gross profit function and the substitute-goods property, because I believe this contribution is even more significant. (I am actually convinced that Substitute-Goods Property is important and practical.)   The second part of the paper should be the main part, and I think that the theory part could be better developed, and some other reviewers have a similar opinion. The authors claimed that the approximation gap is bounded by (1^T A^+ 1) / 2 in their responses, but I think this is a weak bound, at least in a theoretical sense. The entries of matrix A^+ are related to functions f_{ij} defined in Equation (1), which can be very large, and I think it is not surprising that this piece was omitted.   I think this is a reasonable paper. I would say it is definitely borderline, but it would be OK to accept.


Multivariate tests of association based on univariate tests
For testing two vector random variables for independence, we propose testing whether the distance of one vector from an arbitrary center point is independent from the distance of the other vector from another arbitrary center point by a univariate test. We prove that under minimal assumptions, it is enough to have a consistent univariate independence test on the distances, to guarantee that the power to detect dependence between the random vectors increases to one with sample size. If the univariate test is distribution-free, the multivariate test will also be distribution-free. If we consider multiple center points and aggregate the center-specific univariate tests, the power may be further improved, and the resulting multivariate test may have a distribution-free critical value for specific aggregation methods (if the univariate test is distribution free). We show that certain multivariate tests recently proposed in the literature can be viewed as instances of this general approach. Moreover, we show in experiments that novel tests constructed using our approach can have better power and computational time than competing approaches.

The authors propose a modular framework for designing omnibus tests of independence between multivariate distributions. Given (X_i,Y_i) pairs, the authors propose designating a center point z_x and z_y for each, reducing to a bivariate problem by computing (||X_i-z_x||, ||Y_i-z_y||) for each i, and testing whether the resulting distances are independent using some off-the-shelf test of independence for the reduced problem. A similar construction is given for reducing to a univariate K-sample test if X is categorical with K categories. More generally, we can repeat the same process for several different choices of central points and adjust appropriately for multiplicity. This approach is appealing both for its generality and its computational simplicity, since using off-the-shelf bivariate independence tests lets us look up critical values in a table rather than having to compute them by, e.g., permutations.  The authors show under mild conditions that their tests are consistent for almost every choice of central points z_x and z_y. They evaluate their methods in simulation, showing favorable power as compared to competing approaches.
The paper is well-written and well-structured. The organization is clear in the introduction and executed well throughout. The statements of the theorems are clear and there are few typos.  The technical level is solid. The theorems are interesting, though I would have liked to see more refined results. I am by no means an expert in this area but your theorems did not, in my mind, establish that anyone would want to use the test. Power --> 1 against a fixed alternative, with an infinite sample, is a very low bar that would be cleared by many truly terrible tests (to name a terrible idea off the top of my head: Pearson's chi^2 test after discretizing the data into a multidimensional grid, where the discretization gets finer and finer as the sample size grows).  The paper gave a nice, general framework encompassing a plethora of tests we could try but stopped short of giving specific recommendations, citing the need for further empirical and theoretical work to determine what tests will work well for various data types and against various alternatives. But one has certain alternatives in mind (as I suspect one would in most scientific applications) then why use an omnibus test?  My biggest reservation has to do with practical utility, which is, in my view, questionable. In what concrete applications will your approach actually represent an improvement on already-available methods? And what do we really learn when an omnibus test of independence rejects the null? Formally rejecting independence, unless we can make a concrete statement about how the distributions differ, doesn't seem very useful -- for example, was there ever any doubt whether men's and women's microbiomes were identically distributed, and what do we learn by declaring that they are not?  All but one of the simulation scenarios were for two-sample tests in dimension 2. The other example was for dimension 5, but there you were comparing a very heavy-tailed distribution to a very light-tailed one, so it is unsurprising that a procedure based on the distribution of inter-point distances, or distance from the centroid, would do well. These are very low-dimensional examples, leaving the reader to wonder whether your tests will have any appreciable power in intermediate or high dimensions.  On this point, I did not feel that the AGP experiment added anything to the paper; it sounds like the null was very obviously not true, and every test was able to pick that up very easily. It would be more interesting to compare the power of different approaches by subsampling the data.  Bottom line: there are several valid ways to make the case for your method over already-existing methods: rigorous theory, heuristic arguments, simulations, or on a real data set that represents an important use case. But in my judgement you haven't succeeded on any one of these counts. Your theoretical results can't make the case for your method over other consistent methods; you don't really give heuristic arguments about why or when your method should be more powerful; your simulations are unconvincing because they don't really go beyond dimension 2, and in your real data experiment all methods apparently have good power.  Minor comments:  Table 1: The numbers in this table should be reported with most two significant digits. For example, 250 rejections on 500 tries should give estimate 0.5 and standard error .022. 100 rejections on 1000 tries in the first column gives estimate 0.1 and standard error 0.013. More generally, please report standard errors (if only by reporting a maximum standard error of 0.022 for all values given in the table).  Prop 2.1: this is stated much more generally than what you seem to actually need; taking the time to parse the definitions involved briefly derailed my otherwise-smooth progress through the paper. Can you state it in a simpler way?  - Please state somewhere that you mean Euclidean distance (I assume you do) - L.77: R^p formatted differently than elsewhere - Labels on Fig. 1 are very small - You define the term "distribution-free" as "the null distribution of the test statistic does not depend on the marginal distributions of X and Y." In your proposal to combine p-values via e.g. Bonferroni, the above definition would not apply: the minimum p-value's null distribution does, in fact, depend on the dependence of the p-values (of course, the test still works). Please resolve this issue, e.g. by modifying your definition to say "the critical value of the test statistic ..." - L.150: "two sample" -> "two-sample" - L.219: the multivariate t-distribution is usually defined not as a product of univariate t-distributions, but rather as a multivariate normal divided by an independent chi-squared distribution. I would call this a "product t-distribution" or something. - L.281: "hypohtesis"


Learning What and Where to Draw
Generative Adversarial Networks (GANs) have recently demonstrated the capability to synthesize compelling real-world images, such as room interiors, album covers, manga, faces, birds, and flowers. While existing models can synthesize images based on global constraints such as a class label or caption, they do not provide control over pose or object location. We propose a new model, the Generative Adversarial What-Where Network (GAWWN), that synthesizes images given instructions describing what content to draw in which location. We show high-quality 128 × 128 image synthesis on the Caltech-UCSD Birds dataset, conditioned on both informal text descriptions and also object location. Our system exposes control over both the bounding box around the bird and its constituent parts. By modeling the conditional distributions over part locations, our system also enables conditioning on arbitrary subsets of parts (e.g. only the beak and tail), yielding an efficient interface for picking part locations.

The core idea is to take advantage of labeled data which associates images with captions as well as keypoint and bounding box information in order to train conditional GANs which generate an image given the other information. The other important idea is to train another GAN to generate the missing conditioning variables, given any subset of the others (at least this was done for the keypoints, but I presume that the same principle could be applied elsewhere). A sophisticated convolutional architecture is proposed in both the generator and discriminator parts in order to take advantage of the special meaning of the conditioning variables (bounding box and keypoints).
This is a very interesting paper with really impressive results!  One thing I would have liked to see more is a "deconstruction" of the different elements in the proposed approach, to see which ingredients matter more. Of course, part of the difficulty in such an exercise is the current lack of a quantitative evaluation procedure for GANs (or other likelihood-free generative models).  Something that concerns me is the complexity of the architecture and training procedure used in these experiments. As a condition for accepting this paper, I would like the authors to confirm that they will post their code for running these experiments. Otherwise, I feel that it would be very difficult to reproduce the experiments (or a long appendix would need to be added to clarify all the experimental details). I will raise my "clarity" rating if the authors confirm that they will publish their code.  For the same reason, I would expect that the modified dataset used for the MHP experiment be released publically.  There appear to be strange "repeated patterns" or textures showing up in some of the generated images, like in fig 8 and fig 4. These seem to be artifacts and do not correspond to the training distribution. Do you have any clue about what is going on?   Minor comments and requested clarifications:  eqn between lines 99 and 100: this is not the objective typically used to actually train GANs (please clarify, mention the actual objective).  sec. 3.2: why not optimize directly \phi and \Phi wrt the GAN objective? Is there any fine-tuning of these functions with the GAN objective? If so, does it make a difference?  Fig 2 and 3. Clarify what non-linearities are used in hidden layers and in output layers. I am assuming rectifiers for the former and tanh for the latter, but please clarify.  line 152: how was the text embedding t obtained? Like in sec 4.1?  line 165: I suspect that the T should be a K.  line 178: "a denoising autoencoder over keypoints would not suffice" --> actually this is probably not true, since one could run a Markov chain associated with the denoising autoencoder in order to sample from it (see "Generalized denoising auto-encoders as generative models" NIPS 2013).  line 189: how are (x,y) set in k for unobserved keypoints?  line 210: is this pre-training the same as for CUB?  line 214: why keep the # of averaged captions always the same at training time and variable at test time? I would vary that # at training time too.  line 232+: typo "an" should be "and"


Ancestral Causal Inference
Constraint-based causal discovery from limited data is a notoriously difficult challenge due to the many borderline independence test decisions. Several approaches to improve the reliability of the predictions by exploiting redundancy in the independence information have been proposed recently. Though promising, existing approaches can still be greatly improved in terms of accuracy and scalability. We present a novel method that reduces the combinatorial explosion of the search space by using a more coarse-grained representation of causal information, drastically reducing computation time. Additionally, we propose a method to score causal predictions based on their confidence. Crucially, our implementation also allows one to easily combine observational and interventional data and to incorporate various types of available background knowledge. We prove soundness and asymptotic consistency of our method and demonstrate that it can outperform the state-ofthe-art on synthetic data, achieving a speedup of several orders of magnitude. We illustrate its practical feasibility by applying it to a challenging protein data set.

This paper considers learning a causal structure through constraint testing, representing that structure as an 'ancestral' object rather than the more common directed acyclic graphs.  I think this is essentially equivalent to representing each DAG by its transitive closure (perhaps the authors could comment on this).
Overall this is a nice method to an important problem.  The setup seems quite flexible, so I can imagine incorporating other kinds of information relatively easily.  The use of ancestral models rather than Bayesian networks is sensible and novel to my knowledge - though see below for a comment about recovering a BN.    The paper is well written, seems technically solid and novel, and provides a method that could be very useful.  Some questions and suggestions for improvement are given below.  ----  COMMENTS  Page 2 line 46 - could you be a bit more precise when saying 'keep the computational burden low'.  Relatedly, it would be helpful to discuss how quickly the number of ancestral objects grows asymptotically, and whether Chickering's result about learning BNs being NP-hard applies to ancestral objects as well (is this is easy to check).  Lemma 2 - it would be useful to have here a signpost to the discussion after Theorem 1.  The list of rules immediately had me wondering: "how did they come up with this list and is it complete?"  In the simulations, I think it's a bit unreasonable to throw in the red curve (which simulates what happens if some interventional data are available), since none of the other methods claim to be able to deal with this.  This should be removed from the plot, as it is misleading.  What's going on with CFCI and FCI in the plots - they seem to have essentially no points on the plot.  Is this because it's too computationally hard to run FCI with different test sizes?  If so, perhaps using RFCI instead would help (I'm not sure if there is a CRFCI though!).  The description of the bootstrapping is a bit hard to follow - is this somewhat like stability selection?  Perhaps you could use the supplementary material to explain this better.  For the interventional setting, the method of Peters, Buhlmann and Meinshausen (2016) [invariant prediction] would seem like the most sensible comparison.  It can deal with a very flexible class of interventions, so would make a good counterpoint to the discussion at the end of Section 3.  Given the ancestral object, is it hard then just to recover a directed graph by doing some additional tests?  Perhaps this is harder than it seems because parts of the graph may not have an ordering at all, but I'd be interested to know the authors' thoughts.  ----  TYPOS ETC  Though not intentional, a naive reading of the second paragraph of the introduction gives the impression that 'hybrid methods' are a creation of this paper!  There are, of course, many other hybrid methods available for learning Bayesian networks and therefore causal structures.  p5 line 191 - "allow to deduce" -> "allow one to deduce"  Figure 3 means nothing to me - what is it supposed to represent?  Could you give the ancestral structure discovered in the appendix?  It would be good to compare to the 'ground truth' and the Sachs method.


A Constant-Factor Bi-Criteria Approximation Guarantee for k-means++
This paper studies the k-means++ algorithm for clustering as well as the class of D sampling algorithms to which k-means++ belongs. It is shown that for any constant factor β > 1, selecting βk cluster centers by D sampling yields a constant-factor approximation to the optimal clustering with k centers, in expectation and without conditions on the dataset. This result extends the previously known O(log k) guarantee for the case β = 1 to the constant-factor bi-criteria regime. It also improves upon an existing constant-factor bi-criteria result that holds only with constant probability.

The authors consider the famous k-means problem; Given a set of points P in a metric space with distance function d, the goal is to output k centers such that the sum of the squared distances of each point to its closest center is minimized.  The authors focus on bi-criteria approximation algorithms; The algorithms are allowed to output beta k centers (instead of k) and the approximation ratio is computed with respect to the optimal solution (that uses k centers). In the general case, the optimal solution using k centers can be arbitrarily worse than the solution using beta k centers for beta > 1.  The goal of the paper is to prove that a algorithms of a certain kind -- called D-sampling algorithms -- produce constant factor bi-criteria approximation.  D-sampling algorithms iteratively pick the set of centers at random according to some probability distribution based on the cost of the points with respect to the current set of centers.  This work is directly in the line of D. Arthur and S. Vassilvitskii [4] whose proved that the D-sampling algorithms produce an O(log k)-approximation (not bi-criteria). The main result of this work is to analyze the D-sampling algorithms in the bi-criteria regime.  More concretely, they show an approximation guarantee of 1+min{O((k-2)/(beta-1)k), log k} for D-sampling algorithms allowing beta k centers.
I find the result interesting and it is a nice addition to the literature but I do not find it very surprising in itself. Intuitively, D-sampling algorithms do not need to know k in advance, thus, if the algorithm is efficient for finding k centers (which we know it is from [4]), allowing it to pick beta k centers "has to" make the cost decrease significantly (as if the algorithm was solving an instance of the problem for beta k centers). Moreover, the result was known in the constant probability case (rather than expectation).  One of the main issue of this paper is the lack of comparison with previous work. There are at least 4 recent papers that tackle the k-means problem (and two of them tackle the bi-criteria version) and 3 of them are not cited (all on arxiv): 1-- A bi-criteria approximation algorithm for k Means. [20] 2-- On Variants of k-means Clustering. 3-- Local search yields approximation schemes for k-means and k-median in Euclidean and minor-free metrics. 4-- Local Search Yields a PTAS for $k$-Means in Doubling Metrics.  How does it compare to the recent bi-criteria approximation algorithms of 1 and 2? Does D-sampling algorithms provide a better bi-criteria approximation? It is stated that the algorithms in 1 have dependency in n^O(log(1/epsilon)/epsilon^2). However, local search with neighborhood of size 1 has running time O(n k) (and constant factor approximation guarantee) which is comparable to D-sampling algorithms. Thus, according to the sentence "The main difference between Corollary 1 and the bounds in [20] is the extra factor of r_D^l." it seens that D-sampling is worse.  Another issue is about the motivation of this work. Are bi-criteria D-sampling algorithms used in practice? This is true for not bi-criteria versions, are there references for the bi-criteria case?  One interesting question is whether allowing an additional number of clusters, say k + c, would be sufficient to significantly improve the approximation ratio of log(k) (this is true for the case of k-median for example).  Other relevant remark:  "Alternatively, a simpler algorithm (like k-means++) that has a constant-factor bi-criteria guarantee would ensure that the trade-off curve generated by this algorithm deviates by no more than constant factors along both axes from the optimal curve."  It is shown in 2 that local search (which is rather simple) is such an algorithm and this holds even when centers are restricted to be input points (at the cost of a constant factor).


Dimension-Free Iteration Complexity of Finite Sum Optimization Problems
Many canonical machine learning problems boil down to a convex optimization problem with a finite sum structure. However, whereas much progress has been made in developing faster algorithms for this setting, the inherent limitations of these problems are not satisfactorily addressed by existing lower bounds. Indeed, current bounds focus on first-order optimization algorithms, and only apply in the often unrealistic regime where the number of iterations is less than O(d/n) (where d is the dimension and n is the number of samples). In this work, we extend the framework of Arjevani et al. [3, 5]  to provide new lower bounds, which are dimension-free, and go beyond the assumptions of current bounds, thereby covering standard finite sum optimization methods, e.g., SAG, SAGA, SVRG, SDCA without duality, as well as stochastic coordinate-descent methods, such as SDCA and accelerated proximal SDCA.

The authors provide lower bounds for smooth finite sum optimization which apply regardless of the dimension of the problem, and which match upper bounds given by certain iterative optimization methods like SDCA, SVRG, SAG, etc.          The authors begin with a discussion of the limitations of many established lower bounds for finite sum optimization - namely that they apply only when the number of iterations is less than d/n where d is the dimension of the problem and n is the number of components in the sum, and that this condition can be unrealistic in many problems. To avoid a dependence on the dimension of the problem, their lower bound proof requires making assumptions about the form of iterate updates made by the optimization algorithm. They define the notion of an oblivious, CLI algorithm, where updates are linear functions of previous iterates and oracle results, with parameters that are fixed (or scheduled) before the algorithm begins. In this framework, they show that for a certain class of functions parametrized by \eta, the kth iterate of the algorithm is an order-k polynomial in \eta. From here, they use existing function approximation theory to prove the existence of a function in their class for which the algorithm converges slowly to the optimum, completing the lower bound proof.
        This was a very well-presented paper. It clearly presents a shortcoming of current lower bounds for finite sum optimization (that they apply only in large dimensions) and addresses it with a lower bound which is independent of the dimension of the problem. There are only a few very small issues with the presentation (I will mention those at the end of this section), and overall it is very clear.         This paper is an extension of [5] from single function optimization to finite sum, and uses the same analysis techniques in its proofs. Thus, it does not seem to be a massive leap forward in terms of novelty. Nevertheless, the authors' result here on finite sum optimization demonstrates the limitations of oblivious optimization algorithms, and clears up any questions about the complexity of finite sum optimization in low dimensions for many currently hot algorithms.          Two small mistakes:         - on line 193 in equation (10), C \in R^d rather than C \in R^{dxd}         - on line 202 in equation (12), it should be q^T w rather than q


Towards Conceptual Compression
We introduce convolutional DRAW, a homogeneous deep generative model achieving state-of-the-art performance in latent variable image modeling. The algorithm naturally stratifies information into higher and lower level details, creating abstract features and as such addressing one of the fundamentally desired properties of representation learning. Furthermore, the hierarchical ordering of its latents creates the opportunity to selectively store global information about an image, yielding a high quality 'conceptual compression' framework.

This paper introduces a new approach to unsupervised representation learning for images based on recurrent neural nets. The main innovation is to alter the cost function to encourage modeling of high-level features rather than fine details. This leads to an effective framework for lossy compression of images.
This is obviously a high quality paper with solid technical contributions and impressive results. The main question for a paper like this one is whether it is award-worthy or not. This is a weighty question because awards function not just to highlight the best results, but also as exemplars of the highest standards in scientific research. From this point of view, I heartily support acceptance of this paper, but vote against a best paper award for reasons detailed below.   Best features of this paper: 1. Compression  To me, the focus on compression and the high level insights that emerge from this focus are the key feature of the paper. Research in unsupervised learning is often focused on perplexity/log likelihood scores to the exclusion of all else. While in principle these things are intimately related (through arithmetic coding, as the authors point out), the focus on a more operational task, lossy compression, reveals a different way of thinking about unsupervised learning. The same LL scores may, nevertheless, lead to drastically different results from the point of view of "conceptual compression". Even if this notion is only specified informally in this paper, I think it is a useful contribution to the field.   It seems to me that compression has made a comeback (or never left?) with many papers talking about factorial codes as a motivation for unsupervised learning. (As in the recent InfoGAN paper, arxiv:1606.03657, which I mention again below.) The standard way of thinking is that in an independent or disentangled basis, (lossless) compression would become trivial. While that is an attractive notion, this paper points to a different path that doesn't appear to necessarily require any explicit disentangling. The point that very little information is contained at the conceptual level, with most contained at the detail level, is intuitive but difficult to express in the factorial coding point of view. What I find most exciting here is that there appears qualitatively to be a difference between building a latent representation based on the factorial idea (infoGAN) and the "conceptual" approach here. This makes for an exciting time for unsupervised learning.   2. Results I appreciate the effort to find and report on new standardized benchmarks that are more challenging than MNIST. I found the comparisons with JPEG compression compelling. I expect future developments will require equal parts of innovation for quantifying these types of comparisons, but I think the results here were enough to communicate a new and significant contribution. As a clarification here: the innovation is to use lossy compression in conjunction with a specific type of loss function to encourage and evaluate the learning of abstract details in unsupervised representation learning. There is certainly a great deal of existing work on lossy compression for images, and I think it would be appropriate to at least cite a review on that topic.    The following critiques are, as mentioned above, geared towards justifying a high, but not top, score for this paper.   3. Reproducibility Compared to some disciplines, you could say that the work presented here is highly reproducible. On the other hand, computer science can be held to a higher standard and I believe it should be. I did not see any mention of code availability (even if the link would have to be omitted for anonymity). For comparison, a contemporaneous paper, the infoGAN paper, provides documented code with running examples that reproduce paper figures. Without any expertise implementing deep nets, I was able to get the code from that paper running in a few minutes. The implementation consisted of a few hundred lines of code, the main ingredients of which I could easily recognize. If, on the other hand, I had to reproduce this code based on the description in the paper, it would take, at best, months of work and, at worst, I would find that there were relevant details about initialization or hyper-parameters that were essential but incompletely described. For the long-term vitality of research in this field, I believe code availability is an essential consideration for top research, even if it is not (yet) a requirement for publishing.   4. Concepts I appreciated the overall framing of the paper and got a lot out of it. However, I am still dubious about the suggestion that the top latent factors specify a concept like "cat" or "dog", while the subsequent factors fill in detail. It looks to me more like the top factors specify "light, orange, oriented smudge", and the subsequent layers specify details that make it recognizable as a cat or a dog later. I would be surprised if there were consistently a "cat" neuron that appears in this framework, for instance.  The flip side of the "concept" framing is that this model "learns to fill in detail later". It seems to me that the inclusion of convolutional structure is doing the heavy lifting here, as it does in most recent work, and is not an intrinsic advantage of the framework. I guess the test of this would be to see if this assertion continues to hold on non-image data. Regardless, the results are very interesting and further speculation and research on how and what is represented in this framework are warranted.   p.s. The authors might want to check out this paper, http://arxiv.org/pdf/1503.03585.pdf. In particular they discuss various multi-scale approaches and the "dead leaf" images are a good test of this. It might be nice to nod in this direction and clarify how your work differs from these efforts to model images at multiple scales.    edit: Based on the other reviews and the responses, it seems that more technical detail were omitted than I understood. I'm lowering my confidence and clarity score, but still argue for acceptance.


Clustering with Bregman Divergences: an Asymptotic Analysis
Clustering, in particular k-means clustering, is a central topic in data analysis. Clustering with Bregman divergences is a recently proposed generalization of k-means clustering which has already been widely used in applications. In this paper we analyze theoretical properties of Bregman clustering when the number of the clusters k is large. We establish quantization rates and describe the limiting distribution of the centers as k → ∞, extending well-known results for k-means clustering.

This paper present an asymptotic result for optimal k-means clustering, when there is infinite data and as the number of clusters goes to infinity. Essentially, if we process (15), the limiting distribution is proportional to the eigenvalues of the Hessian of the generator of the divergence.
There are several points that need clarification, from the most important to the less important one:  * One key missing point is the reason to exist of the k^{2r/d} factor which appears in (14). The only motivation is intuition, but I do not get it at all. It must be introduced rigorously.   * In (47) of the supplementary information, the authors claim that the difference between eigenvalues (max - min) of the Hessian is slammer then epsilon. If I am not mistaken, epsilon in (45), epsilon can be chosen arbitrarily small. If this is the case, then (47) is wrong, even for Mahalanobis, since is enforces the Hessian to be essentially spherical. Or if it is right, then the result cannot hold for any Bregman divergence, as claimed by the authors. This is important, because if there is a mistake, this invalidates (50) (supplement), and therefore Lemma 2, which is central to the paper.  * In page 7, the authors claim “In addition, we only verify r = 1 cases here, since  the Bregman clustering algorithm, which utilizes Lloyd’s method, cannot address Bregman quantization problems with r != 1”. This, I suspect, is wrong, or the authors need to make it formal. Bregman divergences are exhaustive for the population minimiser as an average (right population minimiser) or f-mean (left population minimiser). Generalisations of Bregman divergences exist that allow to satisfy additional properties that Bregman divergences would not satisfy, such as population minimisers that are invariant to rotations of the coordinate axes [1] (and references therein). It turns out that in this case, even when there is no equivalent Bregman divergence, Lloyd’s quantisation can still be used (see for example the optimisation of the left population minimiser in Total Bregmen divergences) !  Two suggestions, in decreasing order of importance:  * I strongly encourage the authors to pursue drilling down Lemma 3 and one if its implication that I can see, in the finite sample / finite k regime: there seems to be a seeding mechanism in the distribution defined in (20), and I wonder whether this seeding distribution would be better than the state of the art in this field, which is basically the infamous k-means++ algorithm [2] and its extensions to various Bregman divergences and Beyond [1,3].  * Additional remark: the “Remark” before Section 4 follows the well known fact that f-means associated to the KL and IS divergences are zero-attracting. I suggest the authors also put this in context with the paper’s results: there must be a link between the properties of the f-mean associated to the divergence and the limit distribution of centroids (Section 3.2).  Typos:  Pg 11 supplement: existance -> existence  Ref [2] is incomplete  Additional reference:  [1] Richard Nock, Frank Nielsen, Shun-ichi Amari: On Conformal Divergences and Their Population Minimizers. IEEE Trans. Information Theory 62(1): 527-538 (2016)  [2] David Arthur, Sergei Vassilvitskii: k-means++: the advantages of careful seeding. SODA 2007: 1027-1035  [3] Richard Nock, Raphaël Canyasse, Roksana Boreli, Frank Nielsen: k-variates++: more pluses in the k-means++. ICML 2016: 145-154


Exploiting the Structure: Stochastic Gradient Methods Using Raw Clusters *
The amount of data available in the world is growing faster than our ability to deal with it. However, if we take advantage of the internal structure, data may become much smaller for machine learning purposes. In this paper we focus on one of the fundamental machine learning tasks, empirical risk minimization (ERM), and provide faster algorithms with the help from the clustering structure of the data. We introduce a simple notion of raw clustering that can be efficiently computed from the data, and propose two algorithms based on clustering information. Our accelerated algorithm ClusterACDM is built on a novel Haar transformation applied to the dual space of the ERM problem, and our variance-reduction based algorithm ClusterSVRG introduces a new gradient estimator using clustering. Our algorithms outperform their classical counterparts ACDM and SVRG respectively.

The paper introduces two new algorithms: cluster version of accelerated coordinate descent methods (Cluster ACDM) and cluster version of stochastic variance reduced gradient method (Cluster SVRG). The basic idea is to study how it is possible  to improve these methods using additional information on the data; in particular: clustering information. The authors define several notions of raw clusters (different for the two methods; but related) – and explain how these can be efficiently computed. Using the clustering information, the SVRG and ACDM methods are then accelerated. The resulting complexity results are better than the results for the previous variants – as they do not make use of such extra closeting information. The theoretical findings are sufficiently conclusively confirmed by experiments; with insightful commentary.   For ACDM, the new approach can be seen as a new preconditioner enhanced by a novel Haar transform (specific rotations on each cluster highlighting average direction in each cluster and suppressing individual directions in a stochastic sense); and for SVRG, the the new Cluster SVRG method can be seen as an algorithm interpolating between SVRG and SAGA; with further reduction of variance of the stochastic gradient enabled by reduction in “outdatedness” via utilizing clustering information.  In general, this is a beautifully written paper, with elegant new results in an important area of ML.
The initial motivation seems to be the work of Hoffman et al on the use of clustering to speedup stochastic methods for ERM. Their method was not proved to converge to the optimal due to the use of biased stochastic gradients. Also, that work seemed to work only for small clusters due to the approach chosen. This papers goes a long way to develop the basic idea into a satisfying theoretical framework which also gives rise to efficient implementations.  This paper is truly a pleasure to read – a very fine example of academic exposition. The results are innovative, and yet intuitively explained. The theory seems correct; and the experiments show that the methods work better than the competing approaches considered. I believe that several people might pick up this work in an attempt to use it / extend the results.     Question 1: Can Assumption B.3 be removed or replaced by something that can be verified from assumptions on the problem only (and not on the algorithm itself)? Can you prove that in some cases \xi is finite? What value does it have? How large can it be?  Question 2: What are the challenges associated with extending the Cluster ACDM results to non-quadratic losses?  Question 3: Is Haar transform necessary? What other transforms can have the same effect on the dual problem?  Small issues: 1. Line 60: improve -> expect 2. Line 125: can be -> to be 3. Line 178: H_{cl} is not from R^n 4. Lines 263 and 266: same as -> same time as 5. Line 296: ride -> ridge


Theoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative Learning
In PU learning, a binary classifier is trained from positive (P) and unlabeled (U) data without negative (N) data. Although N data is missing, it sometimes outperforms PN learning (i.e., ordinary supervised learning). Hitherto, neither theoretical nor experimental analysis has been given to explain this phenomenon. In this paper, we theoretically compare PU (and NU) learning against PN learning based on the upper bounds on estimation errors. We find simple conditions when PU and NU learning are likely to outperform PN learning, and we prove that, in terms of the upper bounds, either PU or NU learning (depending on the class-prior probability and the sizes of P and N data) given infinite U data will improve on PN learning. Our theoretical findings well agree with the experimental results on artificial and benchmark data even when the experimental setup does not match the theoretical assumptions exactly.

The paper presents a theoretical analysis of when learning from positive and unlabelled data may be superior to learning from fully labelled data. The analysis relies on upper bounds of the estimation errors for various risks. Experiments largely agree with the theoretical analysis.
   The basic problem studied in the paper concerns learning from data which is only partially labelled, but nonetheless doing better than with fully labelled data. In the particular scenario where one has a pool of unlabelled data in lieu of one of the classes, the paper seeks to quantify the impact this has on the estimation error of the learned classifier. Determining the degradation or lack thereof when learning from unlabelled data is interesting, and thus the paper seems well motivated. The machinery used to illustrate its messages are fairly standard -- the key quantities, namely the estimation errors for each scenario, are derived from a simple Rademacher analysis -- however, the final results appear novel, with implications worked through in various scenarios.    The papers hinges on the simple facts that (a) different risks (for PN/PU/NU learning) may be seen as employing different weightings on individual risks for the positive and negative class, and (b) these ratings are reflected in appropriate terms for the estimation error. This, combined with the fact that in practice one is on the regime where unlabelled data is plentiful, is used to argue for the potential benefit is of learning from positive and unlabelled data. Quantifying the precise benefit that having a large amount of data from one "meta-class" buys you, and how it offsets the fact that this "meta-class" doesn't correspond to the actual underlying label, is nice.   The estimation error analysis is fairly standard, and simply exploits the different weightings on the per-class risks. As a lacuna, the comparison of the various scenarios ends up being based on the upper bounds provided by the Rademacher analysis; this isn't ideal, as it implicitly assumes tightness of the bounds. This aside, the comparison of the of various settings in Thm 6 seems fairly unsurprising: one finds that depending on the relation of the number of observed samples to the base rate, either the positive and unlabelled or the fully labelled scenarios will be preferable. I admittedly did not get a lot out of the specific comparisons provided under various constraints on the number of samples; perhaps it would be better to introduce some practical scenario, and then try to motivate some particular relationship between the number of samples. Additionally, it seems natural to explicitly consider the case where n_- = n_u, where the bound for PN will dominate, to make clear that the benefit of PU data is purely from the ability to collect large numbers of samples from the unlabelled set.   The asymptotic analysis in 3.3 aims to illustrate that with large amounts of unlabelled data, positive and on label learning can dominate the fully laden counterparts. The observation that one of PU or NU will dominate PN is intuitive in hindsight, but perhaps not obvious a-priori. I felt here that one could perhaps start off by considering the case where n_u = infty, before introducing the limit-based analysis. This would perhaps more directly hint at the power of the positive and unlabelled scenario.   One limitation of the present analysis is that it restricts attention to losses satisfying the symmetry condition (2). It wasn't too clear why this was strictly necessary. What about losses satisfying ell(-1, v) - ell(1, v) = v, e.g. those considered in [5]? As I understand, the latter showed a risk equivalence for such losses as well. Is the point that in this case one doesn't have the exact same loss on the PU data, which perhaps complicates analysis somewhat?     Other comments:   - a citation for the empirical observation that learning from positive and unlabelled data performs better than positive and negative data seems prudent.    - for the two sample positive and unlabelled setting, consider also citing  Ward et al. Presence-only data and the EM algorithm. Biometrics, 2009.    - the statement that the two sample scenario is more general than the one sample scenario is a bit unclear. The distributions that one observes in each scenario seem quite different. How exactly does one cast the noisy label scenario in terms of drawing from the positive conditional and the marginal?    - there are a few improper uses of \mapsto instead of \to.    - it wasn't entirely clear why one has the dependence on \pi in \hat{R}_pn. In practice surely one just uses 1/(n+ + n-) as a common scaling factor for the positive and negative risks, with no need to actually know the true value of the base rate. It seems that the paper employs this formulation in order to get an easy comparison to the bounds for the positive and unlabelled case? I note that otherwise Thm 3 for example would be nothing more than a standard Rademacher bound for the empirical risk, e.g. (Boucheron et al.), Thm 4.1.    - it is said that [1] establishes that the 01 loss can be directly estimated in the positive and unlabelled setting; but that paper was only for the OS setting? And what is the specific theorem reference?    - it could be made clearer in the theorem statements that one crucially relies on the loss satisfying the condition (2).    - in the proof of corollary 5, it is unclear why the discussion proceeds onto the case where the function class is of the form (6). Isn't the proof complete at this stage? Is the idea is simply to show a direct proof for a special case?    - Thm 6 could perhaps be demoted to Lemma 6?    - the proof of lemma 8 could surely borrow from existing Rademacher analysis e.g. (Boucheron et al.), rather than invoking McDiamird and Talagrand from scratch.


A Comprehensive Linear Speedup Analysis for Asynchronous Stochastic Parallel Optimization from Zeroth-Order to First-Order
Asynchronous parallel optimization received substantial successes and extensive attention recently. One of core theoretical questions is how much speedup (or benefit) the asynchronous parallelization can bring to us. This paper provides a comprehensive and generic analysis to study the speedup property for a broad range of asynchronous parallel stochastic algorithms from the zeroth order to the first order methods. Our result recovers or improves existing analysis on special cases, provides more insights for understanding the asynchronous parallel behaviors, and suggests a novel asynchronous parallel zeroth order method for the first time. Our experiments provide novel applications of the proposed asynchronous parallel zeroth order method on hyper parameter tuning and model blending problems.

The paper provides a general analysis of a wide class of asynchronous stochastic algorithms, including stochastic gradient descent, and stochastic coordinate descent, all in the lock-free case. Serious numerical examples are provided to back up their claims of near linear speedup.
This is the best paper I have read for NIPS this year. I think it is a great contribution. Not only is a powerful framework and theorem provided, but the authors take the time to show how to apply it to other cases, and they discuss interesting features (for example, they make sure the results make sense in light of previous results, and discuss the effects of the variance in the gradient estimate). The authors instill confidence to the reader that the authors did a good job critically examining their own results.  The numerical examples are not the typical small-scale test that claim this is the best algorithm ever, but rather they are well-down tests with good implementations on real-world datasets, and they do not claim to be the “best” algorithm ever, but rather test the stated claim of the paper — namely, that the asynchronous nature leads to speedup benefits up to a certain point.  My main suggestion is that the authors re-proof the paper, as there are still many grammar mistakes (for example, many missing articles before nouns).  I also must mention that I did not have time to read the appendix, so I am taking the proofs at the authors’ word.   Other minor issues I noticed:  - “sever” is listed several times, and should be “server” as in the title of [17].  - You might want to put the equation on line 107 as its own numbered equation, and mention that this is actually a change in the algorithm when mu=0.   - Line 113, I don’t see where \hat{x}_k has been defined, so I am a bit confused about where this fits into the discussion.  - Equation (4), I don’t see any coordinate indices, just the iteration indices, so it seems like this is a consistent read, not an inconsistent read. Maybe this is a notational issue? I’m confused.  - Line 150, the choice of gamma for the Theorem to hold requires a lot of known parameters. Could you analyze the algorithm assuming that gamma is this quantity, up to a fixed constant?  Or at least run numerical examples showing the robustness of the algorithm to incorrect choices in gamma?  - Line 241 is not a complete sentence.


More Supervision, Less Computation: Statistical-Computational Tradeoffs in Weakly Supervised Learning
We consider the weakly supervised binary classification problem where the labels are randomly flipped with probability 1 − α. Although there exist numerous algorithms for this problem, it remains theoretically unexplored how the statistical accuracies and computational efficiency of these algorithms depend on the degree of supervision, which is quantified by α. In this paper, we characterize the effect of α by establishing the information-theoretic and computational boundaries, namely, the minimax-optimal statistical accuracy that can be achieved by all algorithms, and polynomial-time algorithms under an oracle computational model. For small α, our result shows a gap between these two boundaries, which represents the computational price of achieving the information-theoretic boundary due to the lack of supervision. Interestingly, we also show that this gap narrows as α increases. In other words, having more supervision, i.e., more correct labels, not only improves the optimal statistical accuracy as expected, but also enhances the computational efficiency for achieving such accuracy.

This paper proposes to investigate the theoretical limits, both from a statistical point of view (minimax theory) and from a computational point of view the limits of estimators in the context on weakly supervised learning. In their model, a fix proportion of the labels are draw according to a uniform sampling (the complement are the true labels). The authors provides bound showing that there are regimes where the corruption of the label slows down both the minimax rates and the computationally feasible rate. Their technical tools really on testing theory as well as on a "computational (oracle) model".
This paper is interesting and deals with new kind of results introducing computational aspects in standard minimax theory. The phenomenon illustrated is new to me, and present some limitation of the computationally tractable algorithm w.r.t. "theoretical" ones that could be considered in the classical minimax theory. However, due to the relative novelty of the framework, it would be important that basic definitions and properties be better presented.   specific comments:   l18:remove / clarify this part. In the following there is only one model investigated. Please introduce only the one used (no missing values later on...).  l24/27/29: "See e.g. [...] for a survey" repeated 3 times in 3 lines...  l43: by a standard reduction argument [39]. Please provide details page/results for books.  l49: log d/n is ambiguous. Please use adapted parenthesis everywhere.  l74: the drawback raised "hinge on planted clique" should be described with more details and explain why the authors do not suffer the same issue here.  l86: notations - > notation. Also it could be given earlier since some quantities where used before being introduced (eg max sign, \Omega etc.)  l138 (Definition 2.2). This definition could be more commented and written more carefully. For instance, what the index T stands for is not that clear, especially in (2.5) there is no T, though the line below refers to R(\xi,n,T,\eta) as the set of oracles satisfying (2.5)...  I believe that this part is for the community the one with the main novelty, so spending more time on describing it, and giving examples could be helpful. For instance the point raised on line 154 could be explained with more details.  l167: why is \eta chosen of the order d^\eta?  l214: (3.7) could be written with the expression given in the Appendix, l477. That would make it easier to understand where it the test is coming from.  l225: "the" is missing before "test function".  l234: "not computationally tractable": this term is not defined/precise here. To me it is tractable... in a long long time...So this difficulty should be linked with the framework introduced by the authors.  l276: in in  - > in  l283: the quantity z_q_j and \tilde z_q_j are not clearly introduced. What are they?  l296: "the" is missing before "test function", and "an diagonal" should be "a diagonal"  l302: q(Y,X,Y',X) should be q(Y,X,Y',X').  l314: I guess if one estimate \Delta_\mu an estimator of this quantity should be written \widehat{\Delta_\mu} rather than \Delta_\hat{\mu}  l398: the year is wrong in the citation, it should be 2009  Appendix:  l420: Pinsker's inequality could be given a (book) reference for the interested reader.  l454: ".." one dot is enough.  l466: "we introduce random vector W" - >  we introduce the random vector W  l482: the lemma cited give the inequality for the operator norm. Please provide the similar version for the l2 norm considered, or the way to derive it from the other.  l489: I suspect a typo in the control of the probability. Please double check.  l492: "Recall that \phi_1" - >  Recall that \phi_2  l585: please use align correctly.  l629: the sentence is weird...  l640: before you used another sign for the max of two quantities. Please be consistent.