Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
The blind application of machine learning runs the risk of amplifying biases present in data. Such a danger is facing us with word embedding, a popular framework to represent text data as vectors which has been used in many machine learning and natural language processing tasks. We show that even word embeddings trained on Google News articles exhibit female/male gender stereotypes to a disturbing extent. This raises concerns because their widespread use, as we describe, often tends to amplify these biases. Geometrically, gender bias is first shown to be captured by a direction in the word embedding. Second, gender neutral words are shown to be linearly separable from gender definition words in the word embedding. Using these properties, we provide a methodology for modifying an embedding to remove gender stereotypes, such as the association between the words receptionist and female, while maintaining desired associations such as between the words queen and female. Using crowd-worker evaluation as well as standard benchmarks, we empirically demonstrate that our algorithms significantly reduce gender bias in embeddings while preserving the its useful properties such as the ability to cluster related concepts and to solve analogy tasks. The resulting embeddings can be used in applications without amplifying gender bias.

Word embedding approach is wildly used in many machine learning tasks. Because it is learned from huge collected language dataset like Google News articles, the serious stereotypes may be inherent in the dataset and they typically result in the bias model. The authors initiate the study of stereotypes in word embeddings and try to develop two metrics to quantify gender stereotypes through crowdsourcing. In addition, the authors focus on gender stereotypes and propose an algorithm for learning a transformation to remove it from the embedding while preserving useful properties in a low-dimensional subspace. A experiment is shown to demonstrate that the proposed approach addresses the gender stereotypes issue and does not negatively impact the performance on the standard evaluation metrics measuring coherence and analogy-solving abilities.
The paper addresses a novel problem which is inherent in the wildly used word embedding model. However, there are indeed something can be improved.           -) Although the idea is quite novel, the technical contributions in this work are limited. I wonder that the proposed debiasing algorithm useing PCA to identify the stereotype subspace and a linear transformation to reduce it could work on more complex stereotype problems.           -) It lacks the systematic analysis to demonstrate the usefulness of the proposed debiasing algorithm. In other words, it lacks the higher level natural language experiment to prove the effectiveness of the proposed method.           -) The analysis for the several hyperparameters such like δ and the threshold of the score S at line 146 and line 151 are missed.           -) The number of analogies for experiments are too small. It may increase the number of analogies from other stereotypes (e.g. race, cultural, employment).           -) There are several typos in the article             line 10 between between -> between             line 31 [17] Consider -> [17]. Consider             line 33 networks.” -> networks.”.             line 163 were judged to -> were judged to.             line 231 a a -> a             line 246 corresponds corresponds -> corresponds             line 278 Table -> Figure2.             The article needs to be proofread before submission.           -) The readability of the section "Step 3: surgery to modify gender-definition words" is too terrible to follow. It needs to be revised and clarified.           I would request the authors to provide response to the questions/explainations as mentioned above.


Blazing the trails before beating the path: Sample-efficient Monte-Carlo planning
You are a robot and you live in a Markov decision process (MDP) with a finite or an infinite number of transitions from state-action to next states. You got brains and so you plan before you act. Luckily, your roboparents equipped you with a generative model to do some Monte-Carlo planning. The world is waiting for you and you have no time to waste. You want your planning to be efficient. Sample-efficient. Indeed, you want to exploit the possible structure of the MDP by exploring only a subset of states reachable by following near-optimal policies. You want guarantees on sample complexity that depend on a measure of the quantity of near-optimal states. You want something, that is an extension of Monte-Carlo sampling (for estimating an expectation) to problems that alternate maximization (over actions) and expectation (over next states). But you do not want to StOP with exponential running time, you want something simple to implement and computationally efficient. You want it all and you want it now. You want TrailBlazer.

This paper introduces a new Monte Carlo-based planning algorithm called TrailBlazer. The algorithm assumes the access to the generative model of the MDP, and its goal is to find the optimal value function of the root node V(s0) with as few call to the generative model as possible. The setting is discounted reward.  The algorithm works for both finite and infinite state spaces. The paper provides several theoretical guarantees: PAC consistency, a high-probability upper bound on the number of calls to the generative model for finite state spaces, and an upper bound on the expected number of calls for infinite state spaces.  Depending on the scenario, the paper either 1) improves the previous worst-case upper bounds (e.g., for finite state space and with stochastic dynamic) or 2) is the same as the previous results (deterministic dynamic; or without control — the same as Monte Carlo), or 3) provides new results (infinite state space).  The TrailBlazer algorithm alternates between two types of nodes: Avg and Max. An Avg node computes the average value of its children, which are generated according to the transition probability. So it is essentially a Monte Carlo estimator. The parameter m controls the variance of this estimator. The Max node tries to find the value of the maximizing node among its children. It does so by eliminating the children that cannot be a child with maximum value (with high probability).
This is a good paper. It introduces a new algorithm, which has a potential to be used in many applications. The algorithm not only has the basic theoretical justification (consistency), but also comes with certain guarantees that are stronger than what is already known (for finite state space with generative model and stochastic system).  The paper is generally well-written. But I think it can be revised to give more intuition as why the proposed algorithm works better than other approaches. The intuition is somehow missing. Also because of the induction on the tree, the proofs are not very straightforward. I don’t know if much can be done about it though.  I have some comments/questions:  - What is the main reason that the guarantee in Theorem 3 is only in expectation?  - The algorithm does not exploit the possible regularities of the value function, e.g., its smoothness. Is it possible to benefit from them, for example, similar to the StoSO algorithm (Valko, Carpentier, Munos, “Stochastic Simultaneous Optimistic Optimization,” ICML 2013)?  - The paper mentions that for non-vanishing action-gaps, the dimension d can be set to zero. More realistically, the action-gaps can follow a distribution over states (as introduced and analyzed by Farahmand, “Action-Gap Phenomenon in Reinforcement Learning,” NIPS 2011).  What can be said about such a case?  - Section 3.1 (Separate bias and variance) is not very clear. In particular, please expand on “In doing so, their algorithms compute … . However in our planning …”.   - Typos (The line numbers refer to the Supplementary material):  L81: worst —> worse L84: an near-optimal —> a near-optimal L220: “and an a term” L229: “an problem-dependent” L472: “i.d.d.” Appendix D: At several places, \Delta is written as Delta L504: “one need” —> “one needs”  ******* Thank you for your response.


Domain Separation Networks
The cost of large scale data collection and annotation often makes the application of machine learning algorithms to new tasks or datasets prohibitively expensive. One approach circumventing this cost is training models on synthetic data where annotations are provided automatically. Despite their appeal, such models often fail to generalize from synthetic to real images, necessitating domain adaptation algorithms to manipulate these models before they can be successfully applied. Existing approaches focus either on mapping representations from one domain to the other, or on learning to extract features that are invariant to the domain from which they were extracted. However, by focusing only on creating a mapping or shared representation between the two domains, they ignore the individual characteristics of each domain. We hypothesize that explicitly modeling what is unique to each domain can improve a model's ability to extract domain-invariant features. Inspired by work on private-shared component analysis, we explicitly learn to extract image representations that are partitioned into two subspaces: one component which is private to each domain and one which is shared across domains. Our model is trained to not only perform the task we care about in the source domain, but also to use the partitioned representation to reconstruct the images from both domains. Our novel architecture results in a model that outperforms the state-of-the-art on a range of unsupervised domain adaptation scenarios and additionally produces visualizations of the private and shared representations enabling interpretation of the domain adaptation process.

This paper proposes to learn both common and domain-specific representations for domain adaptation from source domain to target domain. Experiments on digits and traffic sign dataset show good performance (even though not convincing even fatal flawed, see below).
------Initial Review----- This paper proposes Domain Separation Networks that learn both common and domain-specific representations for domain adaptation. I like the idea of domain separation even though it is not new. However, I do not think the current draft sufficiently validate the proposed approach. First of all, neither the experimental settings nor the datasets used are standard. To have a direct comparison to other methods (e.g., [7], [29], [17], [26], etc.), standard settings and benchmark datasets should be used. For the current draft, it is not clear the better performance is due to better hyper-parameter tuning on the validation set or from the proposed model. What’s more, since the baseline methods in the original papers are NOT tested on these datasets, it might be that with proper tuning of parameters on the validation set (that not available in the standard setting), the baseline methods achieve better performance. Thus, I encourage the authors to experiment on standard benchmark datasets with the standard protocol. Secondly, I do not think pixel-wise reconstruction is widely applicable. It might work for small gray-level digits data, but the space of real images is much larger (e.g., 256^5 for 256*256 RGB image). What’s more, pixel-level reconstruction prohibits invariance of features. Higher level reconstruction (e.g., conv3) might be better. Last but not least, as the model is very complicated, I would expect the author to provide sensitivity analysis of parameters/hyper-parameters and both mean and std of the accuracy by running the experiments multiple times.   ------Additional Review after authors' feedback------ After reading all the reviews and authors' feedback, I think the CURRENT draft is not suitable for publication at NIPS.   My concerns are for both the approach and the evaluation (experimental setting is flawed). I REALLY hope the authors would have taken the advice of three reviewers and test it on some real images of 3D objects. Since the authors did not provide any meaningful feedback (at least to my three points), I just have to delve deeper into the approach. Even though the novelty is limited, I liked the combination before. However, further reading confirms my opinion that the current draft has not sufficiently validated the approach. My personal conclusion now is that the proposed approach is NOT going to work on read image of 3D objects with DEEP networks. For classification of the digits/traffic signs, it is a pretty much solved problem and the experiment setting is also flawed.   For experiment settings, I think it is flawed. All the baseline methods are designed to be purely UNSUPERVISED, NO labelled TARGET data is used anyway (training or validation). In this paper's setting, there are >1000 labeled TARGET data used for VALIDATION. As showed in recently published methods (e.g., ladder networks, virtual adversarial, etc.), 1000 (10/category) labeled images for digits are enough to train a strong classifier, might even stronger than the number reported in this paper with the more complicated adaptation method. I really can NOT see any point of using that large portion of labelled TARGET data to tune parameter for some complicated methods rather than train a much stronger and simpler classifier on it directly. Thus, in the 5 toy experiments designed by the authors, no adaptation (e.g., ladder networks, virtual adversarial, etc.) might achieve better performance than the methods proposed by the authors with adaptation. What's the point of use it as a validation set rather than train a classifier on it?  A not very accurate analogy would be putting a gas powered generator on an electrical car to generate electricity for the car. Why can not I buy a car powered by gas directly? It is more energy efficient.  For the authors' feedback, some of the issues pointed by fellow reviewers are very easy to address. However, the author just provided vague discussion instead of quantitative analysis/results. For example, the mean and std can be provided by running the same experiments many times. For the sensitivity test, the authors claim to be less sensitive than DANN. What's do you mean by less? How about comparisons to other methods mentioned in the paper?   For the datasets, as mentioned by fellow reviewers, the 5 datasets are rather similar and constrained in several cases. The variance of digits or traffic signs is MUCH lower than real images. They are both 2D objects with pretty much the same viewpoints. For real images of 3D objects, due to change of pose and viewpoints, the same object can appear dramatically in two images. The problem of digits/traffic sign classification is pretty much solved with recent advances (e.g., ladder networks, spatial transformer networks, etc.). What's more, as far as I know, NO paper in the domain adaptation literature RECENTLY published in top conferences tested on digits/traffic signs ALONE. These data might good to generate some plots to illustrate the idea.   What's more, Office is NOT the only benchmark real images dataset of 3D objects. I STRONGLY suggested the authors to test it on standard benchmarks. I do not think the authors' claim of the limitation of Office is a solid response. There are several much larger standard benchmarks beyond Office, e.g., 'A Testbed for Cross-Dataset Analysis', 'ImageCLEF Domain Adaptation Dataset'.  The other thing is that the authors REIMPLEMENTed the other baseline methods despite the fact that most of them have released the source code. The reimplementation is also different from the original one (e.g., gaussian kernel vs. RBF kernel). What's more, some baseline methods do NOT have hyper-parameter (e.g., [26]), how do you tune it on the validation set? How do I know whether the comparison is fair or not?  For pixel-wise reconstruction, due to the much larger space and variance of real images of 3D objects, I do NOT think it is going to work. A reasonable analogy would be autoencoder/decoder. The autoencoder/decoder approaches have been shown to be performed well on MNIST digits. However, as far as I know, no one showed pixel-wise autoencoder/decoder work on real images of 3D objects (e.g., ImageNet). For deep methods on real images of 3D objects, the features need to be much more invariant. These invariances make pixel-wise reconstruction impossible. I do not think one can have both at the same time.   For the Domain Separation, the authors claim that 'the shared encoder represents the object and the private encoder represents the background'. Why can not I use some existing segmentation methods to separate the object from the background? It seems much more easier to leverage the existing segmentation methods as the orthogonality constraint might not work for deep networks (see below).  For the orthogonality constraint, I think it is NOT going to work in modern deep frameworks. From LeNet to AlexNet is not just scaling up. There are several techniques that contribute a lot. For example, ReLU, Dropout. I really do not think one can train a deep network without all the techniques. For ReLU, it changes all the negative values to zero. For Dropout, the training and testing phases are quite different. During training, randomly zeroing out some (e.g., 50%) activations. For testing, a weight (e.g., 0.5) is put on ALL activations. In terms of distributions of activations, the training and test are probabilistically the same. However, in terms of orthogonality, they are different. Roughly speaking, there are MORE (since ReLU changes the negative parts to zero) than 50% data are zero after ReLU and Dropout if dropout rate is 0.5. To constrain two nonnegative metrics to be orthogonality, at least one of the two corresponding entries need to be zero. This might happy during training since more than 50% data are zero. However, it should be VERY UNSTABLE since for different iteration Dropout will zero out DIFFERENT dimensions. However, during testing as a weight (e.g., 0.5) is put on every dimension rather than zeroing out 50%, most of the data will be positive. The zeros can only be from ReLU not Dropout. For two nonnegative matrices with ~90% positive data, I think it is VERY UNLIKELY to be orthogonal.   In summary, even though the writing is excellent, I do NOT think the CURRENT draft is suitable for publication at NIPS.  I recommend for a major revision


Finite-Dimensional BFRY Priors and Variational Bayesian Inference for Power Law Models
Bayesian nonparametric methods based on the Dirichlet Process (DP), gamma process and beta process, have proven effective in capturing aspects of various datasets arising in machine learning. However, it is now recognized that such processes have their limitations in terms of the ability to capture power law behavior. As such there is now considerable interest in models based on the Stable Processs (SP), Generalized Gamma process (GGP) and Stable-Beta Process (SBP). These models present new challenges in terms of practical statistical implementation. In analogy to tractable processes such as the finite-dimensional Dirichlet process, we describe a class of random processes, we call iid finite-dimensional BFRY processes, that enables one to begin to develop efficient posterior inference algorithms such as variational Bayes that readily scale to massive datasets. For illustrative purposes, we describe a simple variational Bayes algorithm for normalized SP mixture models, and demonstrate its usefulness with experiments on synthetic and real-world datasets.

Priors for random measures that induces power-law distributions (such as the GGP) has recently captured a great deal of attention in Bayesian non-parametrics. Inference in models based on these measures is often made difficult because of their analytical properties, for instance the difficulty in obtaining a simple representation of the posterior distribution. This makes existing samplers slow and difficult to implement and makes the use of other inference techniques such as VB difficult. The authors propose a finite-dimensional (truncated) representation of the underlying random measure based on the BFRY distribution which allows simpler sampling procedures and easy construction of VB inference. The construction is tested on realistic data sets.
This paper addresses a timely and important problem in modern non-parametrics, efficient inference for an important class of power-law inducing random measures. I was not aware of the BFRY distribution and pointing out tools such as this, and how they can be implemented, is in my view a good contributions to this conference. I must however admit I did not understand all details of the paper and I hope other reviewers will be able to confirm it's technical accuracy.   I understand this is a difficult and technical topic however I think the authors should improve the presentation. Many difficult topics are introduced or mentioned very quickly and it would serve the paper well if any of these could be omitted for brevity. Is it necessary to introduce the sigma-stable random variable, Kingmann processes, and tilting in the *introduction*? A suggestion could be to introduce the representation eq. (14) in the introduction as an equation (perhaps with generic distributions instead of the BFRY) to provide the reader with a clear picture of what we want to end up with and what the concept of "ideal" finite-dimensional distribution refers to. If the authors need more space, eq.13-15 contains some redundancy.  In regards to the simulations, I think the graphs needs to be cleaned up. In my printed version they are hardly readable and more thought should be given to their readability (font/line sizes, data area, etc.). Should the figure on p.4 not be named? I would also recommend giving the references size/spacing some more thought (are you sure this agrees with the NIPS format?).   I would suggest not showing the difference in figure 1 but the absolute value. Later, the error bars overlap in a way that makes the graphic very difficult to read. I think the tables should be cleaned up to either exclude variance or show it in bolded; quite frankly I got lost in terms of what conclusions I should draw from the tables because of the many abbreviations and it would be recommendable to show a subset of the  results and better communicate what the conclusions are. I would also suggest showing a single number and bolding “best” entries.   A thing I missed a lot from the simulations are results showing that the power-law behavior of the finite-dimensional representation is easily preserved. Playing the devils advocate, the prior distribution (DP or something with a power low behavior) is sometimes not that important in data modelling, and therefore a good test log-likelihood in the actual models does not necessarily show the finite-dimensional representation preserves power-law behavior.  It would in my view benefit the paper if it could be shown by simulation for which K (compared to some dataset size simulated from a power-law distribution such as a GGP) power-law behavior in the posterior is preserved by the finite-dimensional representation. One could expect the tail distribution to require a lot of simulated atoms (high K) to accurately preserve the tail and therefore the power-law somewhat limiting the utility of the result and disproving this would strengthen the paper in my view.    Overall I think this is a good paper that deserves to be on NIPS provided some editorial effort is given to the presentation.


Safe and efficient off-policy reinforcement learning
In this work, we take a fresh look at some old and new algorithms for off-policy, return-based reinforcement learning. Expressing these in a common form, we derive a novel algorithm, Retrace(λ), with three desired properties: (1) it has low variance; (2) it safely uses samples collected from any behaviour policy, whatever its degree of "off-policyness"; and (3) it is efficient as it makes the best use of samples collected from near on-policy behaviour policies. We analyze the contractive nature of the related operator under both off-policy policy evaluation and control settings and derive online sample-based algorithms. We believe this is the first return-based off-policy control algorithm converging a.s. to Q * without the GLIE assumption (Greedy in the Limit with Infinite Exploration). As a corollary, we prove the convergence of Watkins' Q(λ), which was an open problem since 1989. We illustrate the benefits of Retrace(λ) on a standard suite of Atari 2600 games.
One fundamental trade-off in reinforcement learning lies in the definition of the update target: should one estimate Monte Carlo returns or bootstrap from an existing Q-function? Return-based methods (where return refers to the sum of discounted rewards � t γ t r t ) offer some advantages over value bootstrap methods: they are better behaved when combined with function approximation, and quickly propagate the fruits of exploration (Sutton, 1996). On the other hand, value bootstrap methods are more readily applied to off-policy data, a common use case. In this paper we show that learning from returns need not be at cross-purposes with off-policy learning.
We start from the recent work of Harutyunyan et al. (2016), who show that naive off-policy policy evaluation, without correcting for the "off-policyness" of a trajectory, still converges to the desired Q π value function provided the behavior µ and target π policies are not too far apart (the maximum allowed distance depends on the λ parameter). Their Q π (λ) algorithm learns from trajectories generated by µ simply by summing discounted off-policy corrected rewards at each time step. Unfortunately, the assumption that µ and π are close is restrictive, as well as difficult to uphold in the control case, where the target policy is greedy with respect to the current Q-function. In that sense this algorithm is not safe: it does not handle the case of arbitrary "off-policyness".
Alternatively, the Tree-backup (TB(λ)) algorithm (Precup et al., 2000) tolerates arbitrary target/behavior discrepancies by scaling information (here called traces) from future temporal differences by the product of target policy probabilities. TB(λ) is not efficient in the "near on-policy" case (similar µ and π), though, as traces may be cut prematurely, blocking learning from full returns.
In this work, we express several off-policy, return-based algorithms in a common form. From this we derive an improved algorithm, Retrace(λ), which is both safe and efficient, enjoying convergence guarantees for off-policy policy evaluation and -more importantly -for the control setting.
Retrace(λ) can learn from full returns retrieved from past policy data, as in the context of experience replay (Lin, 1993), which has returned to favour with advances in deep reinforcement learning (Mnih et al., 2015;Schaul et al., 2016). Off-policy learning is also desirable for exploration, since it allows the agent to deviate from the target policy currently under evaluation.
To the best of our knowledge, this is the first online return-based off-policy control algorithm which does not require the GLIE (Greedy in the Limit with Infinite Exploration) assumption . In addition, we provide as a corollary the first proof of convergence of Watkins' Q(λ) (see, e.g., Watkins, 1989;Sutton and Barto, 1998).
Finally, we illustrate the significance of Retrace(λ) in a deep learning setting by applying it to the suite of Atari 2600 games provided by the Arcade Learning Environment (Bellemare et al., 2013).

The paper introduces a new algorithm called Retrace(\lambda) that is meant to achieve a trade-off between efficiency and stability. In the process, it proves convergence of a large class of off-policy value function algorithms (including Q-learning with eligibility traces) in finite state spaces. The theoretical analysis is underpinned by a new general way of writing off-policy value function updates with eligibility traces (eq. 4).  Experimental results show the Retrace algorithm performing better than DQN and the tree backup algorithm on the Atari domain.
***** UPDATE *****  I have lowered my scores for novelty and impact because I just came across the paper "High Confidence Off-Policy Evaluation" by Thomas, Theocharous and Ghavamzadeh (not referenced paper 602), which deals with some very similar issues. In particular, it bounds the performance of off-policy importance sampling as a function of a truncation coefficient, and discusses how to choose that coefficient based on the bound they propose. The lack of a discussion of the relationship to that work makes paper 602 considerably weaker in my opinion. I would still lean towards acceptance, but only as a poster.  *******************   The paper is a great starting point for a theoretical understanding of current off-policy value-based methods and for developing new ones. Analyzing the convergence of the general-form off-policy updates in Equation 4 is novel and important. The theory is limited to finite state spaces (something that should be stated in the abstract) for discounted MDPs, but the empirical results show that the new Retrace algorithm can perform well in conjunction with value function approximation. The theoretical results are sound as far as I can tell.  Retrace as the combination of truncated importance sampling and eligibility traces is a new method, however the authors should cite existing work on reducing the variance of importance sampling by truncating coefficients, e.g. "Truncated Importance Sampling" (Ionides, 2008).   The paper suffers in clarity to some extent because a lot of content is being crammed within the bounds of a conference paper. In particular, some of the theory takes a while to read through as several steps are condensed into one, and the experimental results at the end feel a bit rushed. Also at several points the text is begging for more discussion (e.g. about the asymptotic commutativity assumption vs GLIE, or why limit the upper bound for c_s at 1 as opposed to some other value). Perhaps a good solution would be for the paper to only include the theory and discussion, and to reference an extended version containing a more in-depth examination of the experimental results - my feeling is that a few years down the road the theory in this paper will be of a lot more interest than the experiments.  Several comments regarding clarity (some of them minor): - in the equation above line 62, the operator definition is missing something (it becomes equal to zero when lambda=1) - line 131: i would say that it is more obvious that Q^\pi is the fixed point from Lemma 1 than from Equation 4 - typo in lines 219-220 - the equation after line 262 doesn't look like a Q-function target, but rather as part of a gradient update - line 270: Figure 1 does not show that Q* diverges for lambda=1


Discriminative Gaifman Models
We present discriminative Gaifman models, a novel family of relational machine learning models. Gaifman models learn feature representations bottom up from representations of locally connected and bounded-size regions of knowledge bases (KBs). Considering local and bounded-size neighborhoods of knowledge bases renders logical inference and learning tractable, mitigates the problem of overfitting, and facilitates weight sharing. Gaifman models sample neighborhoods of knowledge bases so as to make the learned relational models more robust to missing objects and relations which is a common situation in open-world KBs. We present the core ideas of Gaifman models and apply them to large-scale relational learning problems. We also discuss the ways in which Gaifman models relate to some existing relational machine learning approaches.

The paper present Gaifman models that combined Gaifman’s locality theorem with relational machine learning models. It perform learning and inference within and across locally connected regions of relational structures.
The research problem of this paper is interesting and meaningful. The paper present a novel family of relational machine learning models. However, it would be better if some flaws of the paper can be modified. 1. In line 20, the paper write "We aim to advance the .. models which learn efficient and effective cs". What are the shortcomings of existing methods in terms of efficiency and representation? 2.In line 20, the paper write "...learn effective representations from these local neighborhoods". What is " effective representations"? Could you give an example? 3.In line 143, the paper give a theorem 2. Could you give the proof of the theorem? 4. In line 20, the paper write " We obtained ... categorical cross-entropy as loss function ".Why use it? Could you give a reasonable analysis or advantage analysis? 5. There are similarities with PTranE(Modeling Relation Paths for Representation Learning of Knowledge Bases). Could you compare with this method? In addition, TransE and TransR are typical Knowledge Graph embedding model. But ,at present, the best model of Knowledge Graph Embedding is TranSparse (Knowledge Graph Completion with Adaptive Sparse Transfer Matrix). Could you compare with this method in the experiment to illustrate effective representations?


Learning from Rational Behavior: Predicting Solutions to Unknown Linear Programs
We define and study the problem of predicting the solution to a linear program (LP) given only partial information about its objective and constraints. This generalizes the problem of learning to predict the purchasing behavior of a rational agent who has an unknown objective function, that has been studied under the name "Learning from Revealed Preferences". We give mistake bound learning algorithms in two settings: in the first, the objective of the LP is known to the learner but there is an arbitrary, fixed set of constraints which are unknown. Each example is defined by an additional known constraint and the goal of the learner is to predict the optimal solution of the LP given the union of the known and unknown constraints. This models the problem of predicting the behavior of a rational agent whose goals are known, but whose resources are unknown. In the second setting, the objective of the LP is unknown, and changing in a controlled way. The constraints of the LP may also change every day, but are known. An example is given by a set of constraints and partial information about the objective, and the task of the learner is again to predict the optimal solution of the partially known LP.

The paper provides learning algorithm for predicting the solution of linear program with partial observation and provides some mistake bounds for these algorithms. Two different problems are considered: 1) The objective function of LP is known but the constraint sets are not known.  At each time a new constraint is revealed and the goal is to predict the optimal solution of the LP including all known and unknown constraints. 2) The objective function is unknown (it can be a linear combination of different subsets of a collection of fixed but unknown vectors), but the learner observes the changing constraints at each day and again the goal is to predict the optimal solution of the LP.   In the first problem, the mistake bounds are obtained for two cases: when the new constraints are selected adversarially, in which case the authors show that given a uniform bound on the precision of the constraints and some other assumptions such as non-degeneracy of the constraints, there exists an algorithm with mistake bound and running time linear in terms of the number of edges of the underlying unknown polytope and the number of precision bits. It is shown that uniform upper bound on the precision bits is essential for dimensions higher than 2. To relax this issue, the authors consider the known objective LP problem in stochastic scenario where the revealed constraints at each time are independently chosen from an underlying unknown distribution, in which case a learning algorithm by expanding convex hulls is provided which attains in expectation at most linear mistakes in terms of the number of edges and grows only logarithmic in time. Moreover, this algorithm has been derandomized to give a deterministic algorithm with high probability bounds.      In the case of unknown setting, the authors use a variant of ellipsoid method to design a learning algorithm with polynomial bounds on running time and number of mistakes in terms of parameters of the problem (dimension, precision bits, etc).
The proposed bounds scale linearly with the number of edges of the unknown polytope. As is mentioned, only when the number of edges is within an additive constant of the dimension, the results in this paper give polynomial bounds in terms of the dimension of the space, and finding an efficient algorithm with polynomial mistakes in terms of dimension is left as an open question. Could the authors please comment on this issue a bit further that what do they think about the existence of such algorithm based on their observations? Also, line 55 talks about halving technique with some performance guarantee. Is there any specific reference for this result?     Could you please explain why theorem 18 does not work for d=1, or d=2?  The idea of designing learnhull algorithm and its analysis is very interesting. Is there any similar approach which has been used earlier for a similar type of problem?    The definition of set \mathcal{F} and the intuition behind it in line 355 is not very clear. Could the authors please clarify it a bit further and explain how it is related to [5]?


A Communication-Efficient Parallel Algorithm for Decision Tree
Decision tree (and its extensions such as Gradient Boosting Decision Trees and Random Forest) is a widely used machine learning algorithm, due to its practical effectiveness and model interpretability. With the emergence of big data, there is an increasing need to parallelize the training process of decision tree. However, most existing attempts along this line suffer from high communication costs. In this paper, we propose a new algorithm, called Parallel Voting Decision Tree (PV-Tree), to tackle this challenge. After partitioning the training data onto a number of (e.g., M ) machines, this algorithm performs both local voting and global voting in each iteration. For local voting, the top-k attributes are selected from each machine according to its local data. Then, globally top-2k attributes are determined by a majority voting among these local candidates. Finally, the full-grained histograms of the globally top-2k attributes are collected from local machines in order to identify the best (most informative) attribute and its split point. PV-Tree can achieve a very low communication cost (independent of the total number of attributes) and thus can scale out very well. Furthermore, theoretical analysis shows that this algorithm can learn a near optimal decision tree, since it can find the best attribute with a large probability. Our experiments on real-world datasets show that PV-Tree significantly outperforms the existing parallel decision tree algorithms in the trade-off between accuracy and efficiency.

This paper proposes a new parallel algorithm for training decision trees (for classification or regression). The main idea of this algorithm is to distribute the data on different nodes and then at each node, to identify the top-k splits on each data subset, determine globally to top-2k splits by voting individual top-k lists and then identify the best ones among them by combining histograms from individual workers. A theoretical analysis is performed that gives a lower bound on the probability to find the globally optimal split depending on the main method parameters. Several experiments are conducted on two large-scale datasets, where PV-tree are shown to outperform other parallel decision tree implementation.
Given the popularity of decision trees, proposing an efficient parallel implementation of this method is of course very relevant. The proposed parallelization is original with respect to existing methods and it should indeed lead to less communications than other methods. The theoretical analysis is sound and I like the discussion of the impact of the main problem and method parameters that follows from the lower bound provided in theorem 4.1. Experiments are conducted on two very large problems, where, in the limit of the tested settings (see below), PV-tree is clearly shown to outperform other parallel implementations, in terms of both computing times to reach a given accuracy level and communication costs.  I nevertheless have two major concerns with the proposed parallelization. First, given the way it works, the performance of the algorithm is clearly affected by the number of workers and this number thus needs to be tuned for optimal performance. This is a serious drawback as It means that the algorithm can not benefit from large number of workers. From my understanding, this problem is not shared by other parallelizations. Using more machines with these methods might not always give significant improvement in terms of computing times but at least, it will not deteriorate predictive performance. While the experiments show the impact of the number of workers on PV-trees, the impact on other methods should be also studied in comparison. Experiments should show if PV-tree is still the best method on LTR (resp. CTR) when the number of machines grows (far) beyond 8 (resp. 32) machines.  Second, I'm also puzzled by this statement from the authors in Section 5.2.1 (and similar statements are given in Section 4):  "Only when the entire training data are huge (and thus distribution of the training data on each local machine can be similar to that of the entire training data), the full power of PV-Tree can be realized."  If this is indeed true, then I'm wondering why it's actually needed to distribute the data over different machines and why just selecting the best split over a single random subset of N/M examples at each node is not enough. (Or similarly is the tree built using PV-tree really better than a tree grown using only the subset of data available at one of the worker?) Indeed, if the distribution of the training data on each local machine is assumed, for the method to work, to be similar to that of the entire data, then the local top-k splits are expected to be the same as the global top-k splits and there is no need to vote over several workers. I think that this should be checked in the experiments. Do we really need to vote over several workers or is one worker not actually enough, in the conditions (in terms of the number of workers) where PV-tree works best?  The paper is very pleasant to read. Several important details are however missing in the textual description of the algorithms and in the pseudocodes. Some points that need to be clarified are as follows: - How are PV-tree and the competitors implemented? (language, framework, etc) - The pseudocode does not clearly indicate how and when the data is distributed. At each node or only once globally? What about other methods? What is included in their communication cost? - I don’t find the pseudo-codes very clear, probably because of the use of an object oriented formalism. Most functions/methods are not defined and it’s not always easy to guess what they are doing from their names. I think it could be improved. - Which information is contained in the histograms, in particular in the case of continuous attributes? How is this information merged between workers to yield the global histogram and how is the split threshold determined from this histogram? I don't have a clear idea of what kind of information is actually communicated between the machines. - Is the global split found using the call ‘globalHistograms.FindBestSplit()’ at the end of Algorithm 3 exactly the same as the split that would have been obtained with all available data (assuming the optimal variable is among the 2k selected ones).


Data driven estimation of Laplace-Beltrami operator
Approximations of Laplace-Beltrami operators on manifolds through graph Laplacians have become popular tools in data analysis and machine learning. These discretized operators usually depend on bandwidth parameters whose tuning remains a theoretical and practical problem. In this paper, we address this problem for the unnormalized graph Laplacian by establishing an oracle inequality that opens the door to a well-founded data-driven procedure for the bandwidth selection. Our approach relies on recent results by Lacour and Massart [LM15] on the so-called Lepski's method.

This paper provides a method for choosing the bandwidth parameter of a point cloud Laplacian (but introduces other parameters in its place).  Theoretical guarantees and a minimalistic numerical experiment are provided as evidence of usefulness of the technique.
I am unqualified to assess the theoretical significance of this paper but am quite concerned about the practicality.  My key concern about this paper is that it does not seem to improve the state of affairs re: parameters needed for Laplacian approximations.  In particular, the abstract/intro led me to believe the paper was going to alleviate the need to choose a bandwidth parameter for the Laplacian (a very worthy task!).  But in reality, the technique still has parameters a, b, and the paper says "calibration of the constants a and b in practice is beyond the scope" (line 92).  If this is the case, what is the value of the current paper?  The restrictions described at the beginning of sec 5.1 are similarly concerning.  Beyond this, the numerical experiments are far too minimal.  Recovering the Laplacian of a sphere does not represent the noise and geometry typically encountered in machine learning.  Other comments:  - There are quite a few spelling mistakes and other typos that should be fixed; rather than list them here I will encourage the authors to run "spell check"  - You can say "Lepski's method" instead of "the Lepski's method"  - Say early on (e.g. in sec 1) whether the dimension d should be known a priori for this method to be useful.  (Perhaps a relevant citation is "Testing the Manifold Hypothesis" by Fefferman and colleagues)  - l. 32 -- cite papers instead of saying "known results"  - l. 39 -- define "contrast"  - The assumption that H is finite (line 67) seems quite limiting and does not align with e.g. choosing the width of a Gaussian kernel  - I am a little confused about the density function P discussed around eq (2).  In particular, how does P interact with the parameterization chosen in eq (1)?  Is it possible for the isotropic kernel in eq (1) to deal with irregular or unevenly-distributed P's?  - The explanation at the beginning of sec 2.1 of Lepski's method was *very* dense for an inexperienced reader.  Please add equations and/or a concrete example rather than just describing from a high level in words.   - l. 79 "thus seems natural" -- why?  - In eq (3), it seems that h depends on f.  Is this right?  If so, does that make the Laplacian nonlinear?  - In prop 2.1, I am unclear what it means to have a formula for V(h).  It appears above that V(h) is *any* bound satisfying the condition in line 89.  Is this an upper bound for V(h)?  A formula that satisfies the definition?  - The heuristic in lines 170-177 comes out of thin air.  Please provide justification or at least intuition.


On the Recursive Teaching Dimension of VC Classes
The recursive teaching dimension (RTD) of a concept class C ⊆ {0, 1} n , introduced by Zilles et al. [ZLHZ11], is a complexity parameter measured by the worst-case number of labeled examples needed to learn any target concept of C in the recursive teaching model. In this paper, we study the quantitative relation between RTD and the well-known learning complexity measure VC dimension (VCD), and improve the best known upper and (worst-case) lower bounds on the recursive teaching dimension with respect to the VC dimension. Given a concept class C ⊆ {0, 1} n with VCD(C) = d, we first show that RTD(C) is at most d · 2 d+1 . This is the first upper bound for RTD(C) that depends only on VCD(C), independent of the size of the concept class |C| and its domain size n. Before our work, the best known upper bound for RTD(C) is O(d2 d log log |C|), obtained by Moran et al. [MSWY15]. We remove the log log |C| factor. We also improve the lower bound on the worst-case ratio of RTD(C) to VCD(C). We present a family of classes {C k } k≥1 with VCD(C k ) = 3k and RTD(C k ) = 5k,

The paper is the first to provide an upper bound on the recursive teaching dimension (a complexity parameter in co-operative teacher/learner settings) in terms of the VC-dimension that does not depend on the size of the concept class. The upper bound is exponential in the VC-dimension. Further, the paper is the first to show that there are concept classes whose recursive teaching dimension exceeds their VC-dimension by more than a factor of 3/2 (namely, they provide classes that exhibit a factor of 5/3).
The paper is very insightful - the authors quite nicely explain the approach they took for proving their results. The questions addressed, while interesting only for a fairly small subcommunity of the machine learning community, are really important in that subcommunity, and the authors have achieved a substantial breakthrough on an open problem posed in COLT 2015. I quite liked the idea to formulate the problem of finding a concept class with RTD > 3/2 VCD as a SAT problem. In my eyes, the results should definitely be published, and they are important enough to deserve publication in a leading venue like NIPS.  The paper is generally well written and easy to read, but there are a few minor (easy to fix issues (mostly just typos etc). The two things I'd like the authors to fix, since they are rather  misleading, are the following: (1) They keep talking about proving a "lower bound" on RTD in terms of VCD, and even in the abstract they say "RTD(C)\ge (5/3)VCD(C) in general". While I know what they want to say, what they do say is actually incorrect. There are classes of RTD 1 that have arbitrarily large VCD, so there is no lower bound on RTD in terms of VCD, and certainly (5/3)VCD(C) is not a lower bound. But 5/3 is a lower bound on the worst-case factor by which RTD may exceed VCD. The authors need to go through the whole paper and fix this everywhere (I spotted it here: last para of abstract, heading of section 3, first sentence of section 3, line 1 on page 7, line 2 on page 7, but I may have overlooked other places). (2) After Definition 1, the authors say that "RTD(C) measures the worst-case number of labeled examples needed to learn any target concept in C, if the teacher and the learner agree a priori on a specific order of the concepts". While this is not incorrect, is is rather misleading, since no such a priori agreement is needed. If teacher and learner do not agree on an order, they can still use a well-defined process to uniquely derive teaching sets, and the largest teaching set will have a size equal to RTD(C): first, one determines ALL concepts in C that have the smallest teaching dimension in C, and assigns teaching sets accordingly. Then one removes ALL those concepts from the class and proceeds recursively.  Hence, it would create the wrong impression to say that RTD corresponds to some order that the two parties have to agree on a priori. I'd like to urge the authors to modify the writing in this point.  Everything else is just typos and the like: line 9: at most d2^{d+1} -- > in O(d2^{d+1}) line 11: is -- > was line 16: is -- > was line 22: resultS line 83: largest _known_ multiplicative ... line 84: is -- > was lines 116 and 118: I believe it should be VCD(C) = 2 in both cases (not 3) formula display after line 134: VCD should be TS_min same after line 141 line 181: delete "that" at the end of the line line 206: insert a "times" symbol before the last C_0


Community Detection on Evolving Graphs
Clustering is a fundamental step in many information-retrieval and data-mining applications. Detecting clusters in graphs is also a key tool for finding the community structure in social and behavioral networks. In many of these applications, the input graph evolves over time in a continual and decentralized manner, and, to maintain a good clustering, the clustering algorithm needs to repeatedly probe the graph. Furthermore, there are often limitations on the frequency of such probes, either imposed explicitly by the online platform (e.g., in the case of crawling proprietary social networks like twitter) or implicitly because of resource limitations (e.g., in the case of crawling the web). In this paper, we study a model of clustering on evolving graphs that captures this aspect of the problem. Our model is based on the classical stochastic block model, which has been used to assess rigorously the quality of various static clustering methods. In our model, the algorithm is supposed to reconstruct the planted clustering, given the ability to query for small pieces of local information about the graph, at a limited rate. We design and analyze clustering algorithms that work in this model, and show asymptotically tight upper and lower bounds on their accuracy. Finally, we perform simulations, which demonstrate that our main asymptotic results hold true also in practice.

The paper considers community detection in an evolving graph setting where the nodes and edges of the graph are changing over time but are not observed. Instead, observations are gathered using a limited number of probes or queries of the neighbors of individual nodes. The authors consider a simple model for evolving the graph by augmenting a stochastic block model with community changes and provide theoretical results showing that their proposed probing strategy is asymptotically optimal in a special case and also performs well in simulations.
Strengths: + Among the first papers that I know of to provide theoretical results for estimation in dynamic graph models. This is an important first step that I hope will motivate further research in this area. + Considers a partial observation/active sensing setting that has not been explored much in the community detection literature. + The paper is written in a very accessible manner, and a lot of theory has been condensed and explained in a fluid and coherent way.  Weaknesses: - The theoretical results don't have immediate practical implications, although this is certainly understandable given the novelty of the work. As someone who is more of an applied researcher who occasionally dabbles in theory, it would be ideal to see more take-away points for practitioners. The main take-away point that I observed is to query a cluster proportionally to the square root of its size, but it's unclear if this is a novel finding in this paper. - The proposed model produces only 1 node changing cluster per time step on average because the reassignment probability is 1/n. This allows for only very slow dynamics. Furthermore, the proposed evolution model is very simplistic in that no other edges are changed aside from edges with the (on average) 1 node changing cluster. - Motivation by the rate limits of social media APIs is a bit weak. The motivation would suggest that it examines the error given constraints on the number of queries. The paper actually examines the number of probes/queries necessary to achieve a near-optimal error, which is a related problem but not necessarily applicable to the social media API motivation. The resource-constrained sampling motivation is more general and a better fit to the problem actually considered in this paper, in my opinion.  Suggestions: Please comment on optimality in the general case. From the discussion in the last paragraph in Section 4.3, it appears that the proposed queue algorithm would is a multiplicative factor of 1/beta from optimality. Is this indeed the case?  Why not also show experiment results for just using the algorithm of Theorem 4 in addition to the random baselines? This would allow the reader to see how much practical benefit the queue algorithm provides.  Line 308: You state that you show the average and standard deviation, but standard deviation is not visible in Figure 1. Are error bars present but just too small to be visible? If so, state that it is the case.  Line 93: "asymptoticall" -> "asymptotically" Line 109: "the some relevant features" -> Remove "the" or "some" Line 182: "queries per steps" -> "queries per step" Line 196-197: "every neighbor of neighbor of v" -> "neighbor of" repeated Line 263: Reference to Appendix in supplementary material shows ?? Line 269: In the equation for \epsilon, perhaps it would help to put parentheses around log n, i.e. (log n)/n rather than log n/n. Line 276: "issues query" -> I believe this should be "issues 1 query" Line 278: "loosing" -> "losing"  I have read the author rebuttal and other reviews and have decided not to change my scores.


Linear-Memory and Decomposition-Invariant Linearly Convergent Conditional Gradient Algorithm for Structured Polytopes
Recently, several works have shown that natural modifications of the classical conditional gradient method (aka Frank-Wolfe algorithm) for constrained convex optimization, provably converge with a linear rate when: i) the feasible set is a polytope, and ii) the objective is smooth and strongly-convex. However, all of these results suffer from two significant shortcomings:
1. large memory requirement due to the need to store an explicit convex decomposition of the current iterate, and as a consequence, large running-time overhead per iteration 2. the worst case convergence rate depends unfavorably on the dimension In this work we present a new conditional gradient variant and a corresponding analysis that improves on both of the above shortcomings. In particular:
1. both memory and computation overheads are only linear in the dimension 2. in case the optimal solution is sparse, the new convergence rate replaces a factor which is at least linear in the dimension in previous work, with a linear dependence on the number of non-zeros in the optimal solution At the heart of our method and corresponding analysis, is a novel way to compute decomposition-invariant away-steps. While our theoretical guarantees do not apply to any polytope, they apply to several important structured polytopes that capture central concepts such as paths in graphs, perfect matchings in bipartite graphs, marginal distributions that arise in structured prediction tasks, and more. Our theoretical findings are complemented by empirical evidence which shows that our method delivers state-of-the-art performance.

In this paper, the authors provide a new conditional-gradient-type method for minimization of a smooth strongly convex function over a special class of polytopes. The proposed method is less memory consuming and have cheaper iteration than the state-of-the-art methods. Convergence rate of the method is provided and numerical comparison with existing methods is also presented.
Initial review.  Technical quality: 1. Line 138: Actually, this statement holds for large n. 2. In my opinion, the arguments about the drawbacks of the existing methods in the end of Section 3.1 are not so strong. Existing methods can have superlinear memory and running-time overheads in general case. But in the considered in this paper case of structured polytopes, it may not hold. For example, in the case of the standard simplex, the step 5 in Algorithm 2 costs $O(n)$ operations. The same holds for the memory consumption. It seems that one can criticize the decomposition-specific performance argument in the same way. 3. It is not clear how one can use logarithmic-scale search to estimate $M_1/sqrt{M_2}$. 4. It is not clear why, for the numerical experiments, the authors choose the "line-search" version of their algorithm, despite that it doesn't have a proved convergence theorem. 5. It is not clear, why the authors do not compare their method with the method of [6]. It seems that the latter can be efficient on the considered class of structured polytopes.  Novelty/originality: To the best of my knowledge, the method and its analysis are new.  Potential impact or usefulness: 1. A more detailed description of "applications in machine learning, signal processing and other domains" would increase the potential impact of the paper. 2. Potential drawback of the method could be that one needs an information about $card(x^*)$ in order to implement method with provable convergence rate. 3. It seems that the line-search option in Algorithm 3 could be quite hard to implement in practice.  Clarity and presentation: 1. It was quite hard for me to clarify, what is meant by minimization of a linear form which is given by a vector with infinite-valued components. My first guess was that if you minimize such a linear form given by a vector $(\infty, 1)^T$, it is the same as minimization of a linear form given by a vector $(1, 0)^T$. The reason is that you can neglect 1 in comparison with $\infty$.  2. The text should be proof readed. Here are some examples of misprints. Line 54: "a the convex" should be "as the convex" Line 92: "shaper" should be "sharper" Step 6 of Algorithm 3: $R^m$ should be $R^n$   ===================================== Update after author feedback  Technical quality: 1. This comment remains the same.  2. As it is done now, in the end of Section 3.1, it seems a bit unfair to criticize the existing methods and their performance for _general_ polytopes, and, then, propose a method for a _narrow_ class of polytopes. In that sense, the authors provide in their rebuttal a reasonable argument "However for more complex polytopes ..."  It should be included in the text.  At the same time, the authors write in the rebuttal "This difference in memory requirement is well demonstrated in our OCR experiment, showing significant savings for our method." But, in the text, they report only memory consumption of ACG and PCG, but not the memory consumption of their method. 3. After clarification in the authors' feedback, I still have some doubts concerning the logarithmic-scale search for estimation of $M_1/sqrt{M_2}$. As far as I understood from the text, it will work if we estimate M_1/sqrt{M_2} by some M \in [0.5 M_1/sqrt{M_2}, M_1/sqrt{M_2}]. This interval does not seem to be very wide. 4. In the rebuttal, the arguments on choice of "line-search" version for experiments seem to be reasonable and it should be added to the text. 5. After clarification in the authors' feedback, this point is still not clear for me. It could be fair to compare the proposed provable algorithm with [6], since both of them do not use line search.  Novelty/originality: No change in my opinion.  Potential impact or usefulness: 1. This comment remains the same.  2. In their rebuttal, the authors write that there is no need to know $card(x^*)$. I agree with their explanation. Nevertheless, as I think, potential drawback of the method could be that one needs a good approximation for M_1/sqrt{M_2} (i.e. some M \in [0.5 M_1/sqrt{M_2}, M_1/sqrt{M_2}]) in order to implement method with provable convergence rate. 3. I think the remark in authors' feedback "In general, one can apply the line-search to the upper-bound given by Definition 2, which is always easy to compute." should be added to the text with some explanations on why minimizing the upper-bound will work. 4. New item. I think, the description of logarithmic-scale search for estimation of $M_1/sqrt{M_2}$ should be added to the text.  Clarity and presentation: 1. I think, the authors can easily clarify this point in the text and should do it. 2. This comment remains the same.


Stochastic Gradient Geodesic MCMC Methods
We propose two stochastic gradient MCMC methods for sampling from Bayesian posterior distributions defined on Riemann manifolds with a known geodesic flow, e.g. hyperspheres. Our methods are the first scalable sampling methods on these manifolds, with the aid of stochastic gradients. Novel dynamics are conceived and 2nd-order integrators are developed. By adopting embedding techniques and the geodesic integrator, the methods do not require a global coordinate system of the manifold and do not involve inner iterations. Synthetic experiments show the validity of the method, and its application to the challenging inference for spherical topic models indicate practical usability and efficiency.

This paper exploits the geodesic flow to perform stochastic MCMC. The authors leverage the symmetric splitting integrator for the numerical simulation. The two proposed methods are evaluated by synthetic and real data.
1. The extension of SGGMC from previous work (SGRHMC)[1] are in two folds. First, the proposed method use Geodesic flow rather than Riemmannian manifold. Second, the proposed method leverage a symmetric splitting integrator (ABOBA) scheme. However, unfortunately none of extensions have a clear and convincing novelty as far as I can see. The drop-in replacement of D and Q are not surprising. The main difference comparing to SGRHMC is to replace the fisher information $G(q)$ with geodesic metric $M(q)^T C M(q)$, thus eliminating the need for global coordinates, which may induce artificial boundaries in some manifolds [4]. This is well-studied in [4], thus the contribution seems marginal. The SSI scheme is originally proposed in [2], and what they did seems to be a direct application of the standard SSI. In fact as pointed out by [3], it does not lead to correct sampler if SSI is not applied, as the motion is not reversible and symplectic. The gSGNHT is not very appealing either because its an immediate extension from [1]. I am expecting to see some cases where the SGRHMC fails (e.g. due to a lack of global coordinates), and their methods survives. However, in their experiments, they didn't perform any comparison between their proposed method with [1]. Thereby I am not convinced that they did provide something more useful than [1].  2. In GMC[4] the splitting integrator is guaranteed to be reversible and symplectic. However this may not directly applied to cases where stochastic gradient is used. I haven't found any clear theoretical evidences in the paper supporting that their splitting integrator is still reversible and symplectic.  3. The toy and synthetic experiments does not provide insight on why SGGMC is more desirable than GMC. The results of SGGMC seems suboptimal, particularly in Fig. 3(b) and Fig. 4(a). A comparison on running time would help. There is no comparison with SGRHMC.  4. In real data experiment, the compared sampling methods does not seem to reach a convergence. I would provide a better intuition on why the stochastic version gives a very large performance gain on log-perplexity (from 3000 to 1500 in Fig. 5(a)). I would also provide more information about the parameters (e.g. stepsize) and initialization setting, otherwise it is hard to conclude where the benefit comes from.   The paper is well written and pedagogical, though more introductions about Geodesic flow would be helpful.  [1] Yi-An Ma, Tianqi Chen, and Emily Fox. A complete recipe for stochastic gradient mcmc. [2] Changyou Chen, Nan Ding, and Lawrence Carin. On the convergence of stochastic gradient mcmc algorithms with high-order integrators. [3] Mark Girolami and Ben Calderhead. Riemann manifold langevin and hamiltonian monte carlo methods. [4] Simon Byrne and Mark Girolami. Geodesic monte carlo on embedded manifolds.


Coin Betting and Parameter-Free Online Learning
In the recent years, a number of parameter-free algorithms have been developed for online linear optimization over Hilbert spaces and for learning with expert advice. These algorithms achieve optimal regret bounds that depend on the unknown competitors, without having to tune the learning rates with oracle choices. We present a new intuitive framework to design parameter-free algorithms for both online linear optimization over Hilbert spaces and for learning with expert advice, based on reductions to betting on outcomes of adversarial coins. We instantiate it using a betting algorithm based on the Krichevsky-Trofimov estimator. The resulting algorithms are simple, with no parameters to be tuned, and they improve or match previous results in terms of regret guarantee and per-round complexity.

Many online learning algorithms are based on a learning parameter which controls how far the algorithm changes its answer after receiving new data. While many of these algorithms fix this rate in advance, others change it as they progress. The main problem discussed in this paper is trying to construct an algorithm that is comparable to the case where we know in advance all the data and not receive it part by part.  In this paper, the authors construct a strategy for a coin betting game such that in each turn the gambler needs to decide how much to bet and on which outcome. This strategy is based on suitably chosen potential functions. Each such family of functions produce a betting strategy which also gives an upper bound on the regret for the game, i.e. how much money the gambler could have won if he knew all the coin tosses in advance compared to how much he won using this strategy.  The authors then use this coin betting strategy to solve other online learning problems such as the Online Learning Optimization and Learning with Expert Advice problems. This strategy seems to generalize other parameter-free algorithms which already exist.
The coin betting game and both the Online learning optimization and Learning with expert advice are interesting problems in online learning, and the choice of the learning rate is fundamental to many learning algorithms. Thus any framework that provides a way to chose this rate in a smart way is highly interesting and can have many applications.  The main issue with the paper is the definition of Coin betting potential which lead to the choice of the learning rate. While the formula below line 137 explains why the condition in line 130 is useful, and lines 138-140 explain from where the expression for beta_t comes from (though it is a bit confusing explanation), it is not at all clear how to construct such a function. Later on it is shown that the known online learning algorithms have corresponding potential functions. It would be really interesting if the authors can give some intuition on how to produce more such functions, thus saying that not only their algorithm generalizes the known results, but actually can produce new ones (and not just other similar potentials).  some other minor comments:  line 16 - "the Hilbert space" should be "a Hilbert space" unless this space is defined before. line 24-29 - It might be better to add some explanation what is a learning rate (and not just how you use it in the examples of OGD). If the reader didn't know what learning rate is before reading this paper, he would just be more confused after this introduction. Also, it might be a good idea to write that "parameter free" means that we do not set the learning rate in advance. It can also be as simple as writing "the learning parameter" or the "learning rate or leaning parameter" instead of just the "learning rate".  line 49 - what do you mean by dom(f) not empty? Is the domain not all of V? if this is possible, then you should write it explicitly.  line 109 - The inequality in (5) doesn't seem trivial. If it is just some calculations, maybe add another step or say that this is the case.  line 114 - why are the names of Krichevsky and Trofimov in Blue?  line 141 - The coin betting seems to be one dimensional always. What is the meaning of infinite dimensional in this line?  line 196 - is this "peculiar property" has any significance ?


Following the Leader and Fast Rates in Linear Prediction: Curved Constraint Sets and Other Regularities
The follow the leader (FTL) algorithm, perhaps the simplest of all online learning algorithms, is known to perform well when the loss functions it is used on are positively curved. In this paper we ask whether there are other "lucky" settings when FTL achieves sublinear, "small" regret. In particular, we study the fundamental problem of linear prediction over a non-empty convex, compact domain. Amongst other results, we prove that the curvature of the boundary of the domain can act as if the losses were curved: In this case, we prove that as long as the mean of the loss vectors have positive lengths bounded away from zero, FTL enjoys a logarithmic growth rate of regret, while, e.g., for polyhedral domains and stochastic data it enjoys finite expected regret. Building on a previously known meta-algorithm, we also get an algorithm that simultaneously enjoys the worst-case guarantees and the bound available for FTL.

After rebuttal ================  Reviewer 5 brings up a good point about the importance of the condition on the losses in Thm 3.4. It is important that this gets sufficiently emphasized in the final version of the paper, as the authors promise in their rebuttal, because the high-level message that fast rates *only* depend on the curvature of the constraint set is too simplistic.   Original review ================     In online convex optimization, it is known that Follow-the-leader (FTL) achieves logarithmic regret for strongly convex losses, but FTL is generally considered unsuitable for linear losses, because it may then suffer linear regret. The present paper shows that, even for linear losses, FTL may be able to achieve logarithmic regret (for curved domains) or even finite regret (for polyhedral domains), provided that the average of the cost vectors stays away from 0. Technically, the results are based on Proposition 3.1, which rewrites the regret in terms of the differences between consecutive leaders, together with arguments showing that shape of the domain may cause the difference between leaders in consecutive rounds to be small.
The results of this paper are interesting for multiple reasons: 1) FTL is widely considered the most natural algorithm, it often works well in practice, and it is the standard algorithm in batch learning, so it seems strange that the online learning literature should be so dismissive of it.  2) As mentioned by the authors, it raises the question of whether the shape of the domain can also affect the regret rates for other methods. (Although regularized methods like gradient descent do not necessarily hit the boundary of the domain on every round like FTL, in high dimensions it does not seem unlikely that the domain constraints can be active quite frequently.)  Comments:  There is something that confuses me in the proof of Theorem 3.4: at the top of page 6 a plane P is constructed that is spanned by tilde{theta}_1 and w_2 - w_1, and passes through w_1. Under the assumptions of the theorem, I believe that w_i is simply a scaling of tilde{theta}_i. So then P is also spanned by tilde{theta}_1 and tilde{theta}_2 and passes through the origin. But then P also passes through tilde{theta}_2, so why is it necessary to consider the projection of tilde{theta}_2 onto P?  I would suggest mentioning that the bound on the regret in terms of the number of leader changes in Theorem 3.11 is well-known, or at least follows easily from the be-the-leader lemma (for which you cite Lemma 2.1 of Shalev-Shwartz.)  Minor comments:  I wonder if Proposition 3.2 might extend to non-differentiable points Theta_t as well if the FTL choice w_t is suitably defined in case of ties. For instance, for the experts setting, I would guess that setting w_t to the uniform distribution on the set of leaders might do the trick. In general, the right definition might be arrived at by looking at Follow-the-Regularized leader and then letting the learning rate tend to infinity.  Could you perhaps say something about why the lower bound on ||Theta_t|| is necessary for curved domains in Theorem 3.4, but not for polyhedral domains in Theorem 3.11? I guess this lower bound comes in in Theorem 3.4 in order to control the difference between w_{t+1} and w_t, right? So then in a sense the two results are analogous after all.  Instead of requiring that Phi be differentiable in your results, I would suggest requiring that the FTL leader be unique, which seems easier to interpret.  The fonts in Figure 2 are very small, making it hard to read.


Single-Image Depth Perception in the Wild
This paper studies single-image depth perception in the wild, i.e., recovering depth from a single image taken in unconstrained settings. We introduce a new dataset "Depth in the Wild" consisting of images in the wild annotated with relative depth between pairs of random points. We also propose a new algorithm that learns to estimate metric depth using annotations of relative depth. Compared to the state of the art, our algorithm is simpler and performs better. Experiments show that our algorithm, combined with existing RGB-D data and our new relative depth annotations, significantly improves single-image depth perception in the wild. Figure 1: We crowdsource annotations of relative depth and train a deep network to recover depth from a single image taken in unconstrained settings ("in the wild").
Is it possible to collect ground-truth depth for images in the wild? Using depth sensors in unconstrained settings is not yet feasible. Crowdsourcing seems viable, but humans are not good at estimating metric depth, or 3D metric structure in general [13]. In fact, metric depth from a single image is fundamentally ambiguous: a tree behind a house can be slightly bigger but further away, or slightly smaller but closer-the absolute depth difference between the house and the tree cannot be uniquely determined. Furthermore, even in cases where humans can estimate metric depth, it is unclear how to elicit the values from them.
But humans are better at judging relative depth [13]: "Is point A closer than point B?" is often a much easier question for humans. Recent work by Zoran et al. [14] shows that it is possible to learn to estimate metric depth using only annotations of relative depth. Although such metric depth estimates are only accurate up to monotonic transformations, they may well be sufficiently useful for high-level tasks, especially for occlusion reasoning. The seminal results by Zoran et al. point to two fronts for further progress: (1) collecting a large amount of relative depth annotations for images in the wild and (2) improving the algorithms that learn from annotations of relative depth.
In this paper, we make contributions on both fronts. Our first contribution is a new dataset called "Depth in the Wild" (DIW). It consists of 495K diverse images, each annotated with randomly sampled points and their relative depth. We sample one pair of points per image to minimize the redundancy of annotation 1 . To the best of our knowledge this is the first large-scale dataset consisting of images in the wild with relative depth annotations. We demonstrate that this dataset can be used as an evaluation benchmark as well as a training resource 2 .
Our second contribution is a new algorithm for learning to estimate metric depth using only annotations of relative depth. Our algorithm not only significantly outperforms that of Zoran et al. [14], but is also simpler. The algorithm of Zoran et al. [14] first learns a classifier to predict the ordinal relation between two points in an image. Given a new image, this classifier is repeatedly applied to predict the ordinal relations between a sparse set of point pairs (mostly between the centers of neighboring superpixels). The algorithm then reconstructs depth from the predicted ordinal relations by solving a constrained quadratic optimization that enforces additional smoothness constraints and reconciles potentially inconsistent ordinal relations. Finally, the algorithm estimates depth for all pixels assuming a constant depth within each superpixel.
In contrast, our algorithm consists of a single deep network that directly predicts pixel-wise depth (Fig. 1). The network takes an entire image as input, consists of off-the-shelf components, and can be trained entirely with annotations of relative depth. The novelty of our approach lies in the combination of two ingredients: (1) a multi-scale deep network that produces pixel-wise prediction of metric depth and (2) a loss function using relative depth. Experiments show that our method produces pixel-wise depth that is more accurately ordered, outperforming not only the method by Zoran et al. [14] but also the state-of-the-art image-to-depth system by Eigen et al. [8] trained with ground-truth metric depth. Furthermore, combing our new algorithm, our new dataset, and existing RGB-D data significantly improves single-image depth estimation in the wild.

In this paper, the authors collected a new dataset for training 3D reconstruction from a single image. Unlike previous datasets that recorded the true 3D structure of the scenes, this dataset only records the relative depth of a pair of points in each image. It is because that compared to the annotating absolute depth of the environment, it is easier for human beings to judge the relative depths of two certain points in the scene. In addition, the authors designed a new method that used annotations of relative depth to learn a model for single-image 3D reconstruction.
First, since the new dataset is one of the key contributions of this paper, I suggested the authors to provide more statistics of the dataset. For example, among the 495K images in the dataset, how many images describe large scenes? How many images focus on certain small objects? Among all the scene images, how many images describe indoor scenes, outdoor urban scenes, and outdoor wild scenes? Among all the object images, how many images describe man-made objects and animals? Based on such statistics, we can evaluate the generality of the new dataset.  Second, the new dataset can be only applied to the methods that can learn from ordinal relations between two points in an image. Thus, most existing methods cannot use this dataset.  Third, I do not understand why the authors only label a single pair of points in each image. I admit that labeling two pairs of points in two different images may provide more information than labeling two pairs of points in a single image, because the two pairs of points in a single image may share similar relative depth. However, there also are some advantages for annotating more pairs of points in a single image. 1) Labeling more points may greatly decrease the structural uncertainty of the image, thus providing more reliable training images. It is possible that a small number of reliable training images may contribute more to model learning than a large number of unreliable training images. 2) Labeling more points in a single image may make this dataset applicable to more methods, because only using one pair of annotations in an image is a very strict constraint.  Fourth, 50% of the point pairs for annotation were randomly selected, and the other 50% of point pairs had symmetric shapes. Such a design for point annotation is quite arbitrary. I suggest the authors to design a certain loss to achieve an active framework for point annotation, which uses the loss to identify the most informative point pairs in an image. This will be a more efficient way for dataset construction.  Fifth, the proposed method for learning from ordinal point relations has a minor novelty.


What Makes Objects Similar: A Unified Multi-Metric Learning Approach
Linkages are essentially determined by similarity measures that may be derived from multiple perspectives. For example, spatial linkages are usually generated based on localities of heterogeneous data, whereas semantic linkages can come from various properties, such as different physical meanings behind social relations. Many existing metric learning models focus on spatial linkages, but leave the rich semantic factors unconsidered. Similarities based on these models are usually overdetermined on linkages. We propose a Unified Multi-Metric Learning (UM 2 L) framework to exploit multiple types of metrics. In UM 2 L, a type of combination operator is introduced for distance characterization from multiple perspectives, and thus can introduce flexibilities for representing and utilizing both spatial and semantic linkages. Besides, we propose a uniform solver for UM Visualization results also validate its ability on physical meanings discovery.

This paper presents a framework - UM2L - for learning multiple metrics at the same time for then applying a kind of nearest neighbor classification or a majority voting. The framework is tailored to triplet-based constraints and allows to integrate different loss functions, different functions to combine the metrics that can depend be adapted to similar instances or  impostors and various regularizers even though the paper considers mainly the L-2,1 norm. The problem is formalized as a general optimization problem for which a theorem of convergence is given in the context of gradient descent. Some examples illustrating the interest of the framework are given. A large experimental evaluation is presented.
Pros -Generality of the framework for multiple metric learning allowing to deal with different metric combination -Wide range of applicability and good empirical results -Theoretical result on the convergence of the optimization method -paper well written Cons -The framework does not really provide a solution for combining metrics learned in different regions that may suffer from discontinuity between two regions. So there is no particular novelty in the combination of local metrics. -The method requires a specific initialization that is not really discussed or analyzed -No theoretical guarantees in terms of generalization capability  Overall: Interesting paper for combining multiple metrics that offer a wide range of application. The paper has no theoretical guarantees and does not really bring a solution to the problems posed by local metric learning methods.  Comments -I like the proposed framework because it embeds a large set of possible way of combining metrics.  The idea of using different functions for combining metrics could have been used in other papers, for me the real novelty is to provide a sufficiently general framework for combining these metrics.  Even though, it must be noticed that combining multiple metrics with a max function has not really been studied, as far as I know.  -The initialization of the model is evoked quickly in the supplementary material but it is not really discussed in the paper.  This initialization is based on a clustering step which has a probably an important impact on the result and deserves more attention/analysis.  -Some local metric learning approaches try to learn a metric for a local region to capture local regularities. However, at test time, the choice of the metric to use is a critical issue because of the discontinuity between the learned metrics in case where the two instances to compare may belong to two different regions. Indeed, even for two close regions the metric can be rather different and combining them is not really easy. This is a difficult task, but this paper does not seem to provide a solution to this problem, check this reference for example: *V. Zantedeschi, R. Emonet, M. Sebban. Metric Learning as Convex Combinations of Local Models With Generalization Guarantees. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.   At test time, if I understood correctly, the paper uses all the metrics for all the pairs of examples without penalizing the metrics wrt to spatial information, which seems not natural for taking into account local similarities. I understand that each metric used in UM2L can capture different semantic information in the data, but I do not see how local information can be captured if the learning(or testing) step is global to the whole dataset.  -In the experimental comparison - Section 4.2 in particular, UM2L is used with a specific setting: use of the mixed norm and a majority vote over 3*K distances. The other approaches, in particular the local ones, do not use such a setting and it would be interesting to evaluate if this really the framework that is better or the procedure to take the decision.  For example, it could be interesting to evaluate the performance of each individual metric which can help to assess the expressiveness of the model.  The impact of regularization can also be analyzed.  -In the experimental setup, it is mentioned that 3 targets en 10 impostors are built for each dataset, but we do not know how these triplet are chosen (randomly, by a landmark selection method, cluster centers, ...)  -In the multiview setting, an assumption often made is that some views may compensate other views when they are wrong. In this setting, if one can not bring different weights to each view, it not clear if the combination can bring a better information than individual view.  -When considering the average similarity (ACS), in the batch setting, it is not clear for me if the framework is useful in a 1-view setting if one initializes the model to the same metric before optimization. I do not see how the model can learn different metrics unless we force  the triplets to be optimized only on some particular metrics.  -There is no theoretical guarantees in terms of generalization ability to new instances.  I think that this is probably an important line of work to consider. In particular, nothing is said about the importance of maximizing the margin or in other words on the separation ability brought by each metric. However, this seems an important aspect in particular with RGS and OVS settings.  In OVS, for example, trying to maximize the margin can be way to avoid trivial solutions.  On the other hand, using a ||M_k-I|| regularization can be a good idea in problems where the Euclidean distance works well.   -l.90, notation "\kappa s" has not been introduced, is it  \kappa_v ?.


Tight Complexity Bounds for Optimizing Composite Objectives
We provide tight upper and lower bounds on the complexity of minimizing the average of m convex functions using gradient and prox oracles of the component functions. We show a significant gap between the complexity of deterministic vs randomized optimization. For smooth functions, we show that accelerated gradient descent (AGD) and an accelerated variant of SVRG are optimal in the deterministic and randomized settings respectively, and that a gradient oracle is sufficient for the optimal rate. For non-smooth functions, having access to prox oracles reduces the complexity and we present optimal methods based on smoothing that improve over methods using just gradient accesses.

The authors provide some complexity bounds on problems that are interested in optimizing composite functions of the form F = \frac{1}{m} \sum_{i=1}^{n} f_i when one knows 0th and 1st order terms: f_i(x),\nabla f_i(x) and \Prox_{f_i}(x,.).  Then, the paper produces a very long tiresome results on what can be said on the complexity of optimization procedures in several different frameworks (smooth, non smooth, deterministic, stochastic, upper bounds, lower bounds).  The paper then proposes a very (very) long supplementary material that gathers bibliographic comments and proofs of the proposed results.
I am not able, from a theoretical point of view, to produce a reliable judgement of the paper. However, I can comment several drawbacks:  1) If I understand the global motivation of saying something in each situation for a journal paper, for a brief contribution in a conference proceedings, such a work is very painful for the referee. If you imagine I have to read 7 papers like this one in one month, you can easily conclude that it is not possible for me to produce a constructive reading of this work: 28 pages of a very very NOT self-contained paper, with an excessive abuse of acronysms and without any motivation in the beginning of the introduction. It results in a very boring enumeration of a potpourri of results.  2) The paper is not motivated at any moment. It lacks of real focus on a particular motivated example. Please, reduce the size of the supplementary material and just prove one or two result with a clear example, a clear simulation, a clear state of the art and a clear discussion. The paper does not contain at least one of this point above.  3) The way the paper is written is far from being standard for a mathematician: for example, the lines from 101 to 106 are not written rigorously  (even though I believe the results being true).  Minor comments:  Line 115: strange sentence. Line 123: "Using the Lemma allows us to" -> change the sentence Line 123-135: I am not fan of this paragraph, with an enumeration of non horizon free methods:  most of  penalty terms involve highly dependent terms with respect to generally unkwnown parameters. Line 158: othonormal Line 161: the following truncations of EQUATION (6) Supp material: reduce the 20 pages to 3/4 pages!!!