Multiple-Play Bandits in the Position-Based Model
Sequentially learning to place items in multi-position displays or lists is a task that can be cast into the multiple-play semi-bandit setting. However, a major concern in this context is when the system cannot decide whether the user feedback for each item is actually exploitable. Indeed, much of the content may have been simply ignored by the user. The present work proposes to exploit available information regarding the display position bias under the so-called Position-based click model (PBM). We first discuss how this model differs from the Cascade model and its variants considered in several recent works on multiple-play bandits. We then provide a novel regret lower bound for this model as well as computationally efficient algorithms that display good empirical and theoretical performance.

This paper studies a variant of the multi-armed bandit problem in the multi-position setting, that is when the learner is allowed to select a list of L actions - as opposed to a single action - to be displayed to an incoming user. In the variant that is considered, the user is not assumed to scan the list of selected actions entirely, rather it examines item in position l of the list with some unknown probability \kappa_l. Now, rewards are classically defined as Bernoulli random variables with unknown mean parameter.   In this novel setting that extends pervious works on ranking bandits, the authors provide a lower bound (Theorem 6) for this problem. In Section 4, two algorithms are introduced whose regrets are provided in Theorem 9 and Theorem 11, respectively. In Section 5, these algorithms are compared against two other algorithms directly inspired from the bandit literatute, both on simulated and real data sets.
I have read the feedback from the authors, and maintain my score.  This is a strong paper, with a solid motivation, a well-grounded theoretical analysis, algorithmic contribution, and numerical experiments.  The setting is very natural and makes a lot of sense, as it focuses on a key feature of recommender systems little addressed in the literature: namely the examination probability.  The censored Bernoulli reward model does not reduce trivially to previous settings, and the paper is written in a clear way, with intuition given about contribution. It could be perhaps useful to provide similar intuitions in the appendix in the derivations of each result, as it is not always clear what is the role of each step/intermediate result.   Question: Theorem 9:  It may be useful to provide further intuition about the term C(\kappa): why it appears, exampel of scaling etc.  Algorithm 1: The choice of a probability 1/2 of choosing between \cal B() and \cal L_L(t) looks a bit arbitrary. Also, one would naturally consider that if \kappa is known, a more natural choice could depend on \kappa.  In Theorem 11, do you think one could improve on the "\kappa_L" factor that looks not needed when compared with the lower bound? Do you think playing on the 1/2 probability could help remove/reduce this factor?  The most important critic one can make regarding the contribution is that the \kappa vector is assumed to be perfectly known.  At the top of page 3, this assumption is discussed, and is essentially convincing. However, a natural question that is not addressed here is the effect of having only an approximate knowledge of \kappa to the final regret (Say, it is known up to some error epsilon in infinite or L2 norm).  Another point is when \kappa depends on the user and a user is not observed a lot, or when it depends on the user class but the class is unknown and must be estimated. In these cases, one may want to be able to estimate \kappa while solving the bandit problem. This is however not discussed. Can you give some insights about the main difficulties in order to handle such a case (beyond the identification problem of the B(\kappa_l\theta_k)) model ?    Minor comments: P.5, L. 182:  the letter "delta" is very often used as a confidence level in [0,1]. I thus advise you use a different letter here  for clarity. P.5,L.176 : "an simpler" P6., Algo 1: "\cal L_l(t)" is undefined. I guess this is the l-th component of \cal L(t).   ///////   Technical Quality: Good. The assumption that \kappa is perfectly may not always be a desirable assumption. Novelty/Originality: Good. Potential impact or usefulness: Very good. Clarity and presentation: Very good.


Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
Understanding the 3D world is a fundamental problem in computer vision. However, learning a good representation of 3D objects is still an open problem due to the high dimensionality of the data and many factors of variation involved. In this work, we investigate the task of single-view 3D object reconstruction from a learning agent's perspective. We formulate the learning process as an interaction between 3D and 2D representations and propose an encoder-decoder network with a novel projection loss defined by the perspective transformation. More importantly, the projection loss enables the unsupervised learning using 2D observation without explicit 3D supervision. We demonstrate the ability of the model in generating 3D volume from a single 2D image with three sets of experiments: (1) learning from single-class objects; (2) learning from multi-class objects and (3) testing on novel object classes. Results show superior performance and better generalization ability for 3D object reconstruction when the projection loss is involved.

The paper proposes an alternative/additional loss for the task of single image object reconstruction - that the projections of the predicted and ground-truth 3D shape should agree across various views. This is cleanly implemented in a CNN framework using spatial transformers and the paper empirically demonstrates the benefits of using this loss instead of / in addition to a conventional loss.
         I like the motivation of the paper that in case full 3D information is not available, we can still train a volume prediction CNN using an alternative loss which enforces silhouette consistency with available observations (eg. can be used on Ebay/Amazon etc.). The implementation proposed in the paper (using spatial transformers and max-projection on predicted volume to obtain a silhouette) is also clean, intuitive and can be easily adoptable.          My main concerns are regarding the experimental setup and results (explained in detail below)          Experimental Results : A central point in the experiments is that using the projection loss gives better reconstructions as well as more generalizable models - I don't understand why that should be true. Basically, the projection loss is just an alternative loss which instead of directly penalizing distance between predicted and true models, instead penalizes difference in their projections. It is not clear why this a better loss or why it adds any value at all in model-comb (except perhaps as an additional regularizer). However, the experiments show a clear gain over model-vol (chair in Table 1, generalization in Table 2) - one possible explanation for this is the evaluation metric (IU @ 0.5). Since the models are trained independently, the threshold 0.5 is really arbitrary (as different models may be calibrated differently) and I suspect having max-projection and loss with silhouette makes it a better threshold - for eg., the predictions by VO in Fig 3 are not well formed, but they would be if a lower threshold were used. I strongly recommend that the following experiment be run for the rebuttal - instead of measuring IU @ 0.5, the evaluation metric should be IU @ t where the threshold t is selected independently for each model (and category). If the conclusions presented still hold, I would be happy to update my assessment but otherwise I think this is a serious concern as I find it extremely counter-intuitive that the projective loss would help in training better/more general reconstruction models.          Experimental Setup : Despite the concerns above, I agree that the projection-loss based strategy is useful as it allows one to train without having the underlying 3D model available. Unfortunately, the experimental setup in the paper does not present this strongly - as L172 states, all the 24 views of the object are used for training the CNN and while technically the full 3D model is not available, it practically is. Infact recent experiments in the community (eg. Multi-view CNN) show that representing a 3D model by a collection of views instead of voxels performs even better for category classification. The point here being that using a diverse set of views essentially has similar information as using the full 3D model. If the experiments were to use a more natural setting where for each 3D model, only a few views from a narrow range of azimuth were available (which is arguably a better model for a learning agent which moves around objects) and still obtain comparable reconstruction, it would be a much stronger demonstration of the utility of the technique presented here.          Overall, while the central idea is interesting and novel with a clean implementation, there are concerns regarding both the conclusions mentioned - 1) the experimental setup used here is not really representative of a secnario with 'ground-truth 3D volume not available' and 2) It is unclear if the generalization is better die to the the alternative loss or is an artifact of the metric - hopefully the experiment suggested above would clarify (though the question of why it is better would still remain).          Minor Concerns/Suggestions :         - Technically, the encoder+decoder is supposed to learn a 3D view invariant output and that should just emerge as training progresses, and it should not be necessary to enforce anything on the encoder. The pre-training described in L134-144 seems adhoc. Also, it's unclear why using the deep-rotator encoder does leads to view-invariance since it's not explicitly enforced during the deep-rotator training.          - Using L1 loss instead of L2 in Eqn 2 typically leads to better reconstructions.          - While the overall writing of the paper was great, the introduction using the conscious learning agent seems contrived. As mentioned above, an active agent would not explore all diverse views of the car, would not have a common canonical frame across all cars (and so not know the absolute pose of the car - only relative pose of different views) etc.  ----- Update after Rebuttal : The authors addressed the major concerns above by showing the conclusions hold with the suggested evaluation. An additional experiment with a restricted set of training views available for each object was also presented and I feel this better justifies the motivation of the paper.


Normalized Spectral Map Synchronization
Estimating maps among large collections of objects (e.g., dense correspondences across images and 3D shapes) is a fundamental problem across a wide range of domains. In this paper, we provide theoretical justifications of spectral techniques for the map synchronization problem, i.e., it takes as input a collection of objects and noisy maps estimated between pairs of objects along a connected object graph, and outputs clean maps between all pairs of objects. We show that a simple normalized spectral method (or NormSpecSync) that projects the blocks of the top eigenvectors of a data matrix to the map space, exhibits surprisingly good behavior -NormSpecSync is much more efficient than state-of-the-art convex optimization techniques, yet still admitting similar exact recovery conditions. We demonstrate the usefulness of NormSpecSync on both synthetic and real datasets.

This paper concentrates on a normalized spectral technique for solving the joint shape mapping / synchronization problem.  It provides a theory towards understanding the performance guarantees of the spectral technique.  It was claimed by the authors that the spectral technique achieves nearly equivalent performance as SDP,  while enjoying much lower computational complexity.  Extensive numerical simulations have been carried out to verify the theoretical findings.   Overall speaking, spectral technique is definitely a very interesting and efficient paradigm for this joint mapping problem, and hence an attempt to provide rigorous theoretical justification is a very timely and valuable contribution to this field.   One of my concerns is that the key sufficient recovery condition (11) is not very easy to interpret,  and does not seem to match the prior results well (but I could be wrong).  The authors might want to provide more detailed explanation about (11),  ideally specializing it to several concrete examples and then comparing each of them with prior results.
1.  The observation matrix X_obs is approximately low-rank as well without normalization.  What is the advantage of "normalized" spectral method compared to "unnormalized" spectral method?   2.  For the benefits of those not familiar with this subject,  it would be good to provide a simple yet concrete example to explain X_{ij}.    3.  Define \otimes (Kronecker product) in Equation (2).   4. 3rd paragraph of Section 2.1:   {d_i} should be formally defined.  What if some d_i = 0?   5. I think in the initialization stage of Algorithm 1, it suffices to say "compute the first few eigenvectors of M using a block power method" (and perhaps provide some reference for the block power method).  There is no need to write out all details for the power method as this is fairly standard.   6.  For the projection step (i.e.the sentence after Equation (3)):  please either explain in detail how to compute it by solving a block-wise linear assignment or cite an appropriate reference, if any.   7.  Last sentence of the "Complexity" paragraph in Page 3: For concreteness, the authors might want to mention that full SVD takes time O(n^3) in order to make it more concrete.   8.  Equation (5):  P_ij has not been formally defined.   9.  The authors might need to state explicitly that {X_ij} are independently generated.  10.  It is not completely clear to me what Equation (7) means; please define s, V, Lambda more precisely.   11.  The notation seems to be inconsistent from here to there. For instance, In Algorithm 1, it says computing the first n eigenvectors, while in the last paragraph of Section 2.3, it says top-m eigenvectors. Please correct the inconsistency.   12.  Condition (11), which is perhaps the most important condition herein, is not easy to interpret at least to me.  Can the authors specialize it to some simple case (e.g. when the noise is a uniformly random permutation), and then provide a condition in terms of the parameters p, m, and n rather than rho and || N ||?    13.  In fact, I'm not sure I understand (11) correctly.  The author says rho is the spectral norm of \overline{A},  which is the normalized adjacency matrix of G.  Why can we "interpret rho as the noise from the adjacency graph"?  The noise is usually thought of as the variance term; what about the mean component?  14.  On the other hand, suppose G is a complete graph, then it seems that the adjacency graph is very close to an all-one matrix, and hence  \overline{A}  is close to  1/n* ones(n,n), and hence rho is almost 1.  In this case, what happens to the second condition of (11) if p is bounded away from 1?   In prior results, however,  G being a complete graph is actually the easiest case, in the sense it can tolerate the largest possible noise (or smallest possible p which is much smaller than 1).   15.  In Lemma 3.2, the last sentence reads strange, as "Show that" sounds like an exercise rather than a result.  Also, you might want to define "converges linearly" somewhere,  as there could be two different interpretation (i.e. geometric convergence rate vs. a convergence rate of O(1/t) ) depending on whether the readers are from the optimization society or not.   16. Typos and minor things:  (1) Abstract:  "as random permutation matrix" --> "as a random ..." (2) Abstract:  "this algorithm NormSpecSync" --> "the proposed NormSpecSync algorithm" (3) 2nd paragraph:  "leads remarkable denoising ability" --> "leads to ..." (4) 2nd paragraph:  "then find the top eigenvectors" --> "then finds the top eigenvectors";  "project each block" --> "projects each block" (5) 3rd paragraph:  "have been considered" --> "has been considered" (6) Section 2.1:  "need not obey the consistency property" --> "needs not ..."


Feature-distributed sparse regression: a screen-and-clean approach
Most existing approaches to distributed sparse regression assume the data is partitioned by samples. However, for high-dimensional data (D N ), it is more natural to partition the data by features. We propose an algorithm to distributed sparse regression when the data is partitioned by features rather than samples. Our approach allows the user to tailor our general method to various distributed computing platforms by trading-off the total amount of data (in bits) sent over the communication network and the number of rounds of communication. We show that an implementation of our approach is capable of solving 1 -regularized 2 regression problems with millions of features in minutes.

The authors present an approach to distributed sparse regression based on sketching. The proposed method partitions the, high-dimensional, data rather than samples.   The proposed method uses a two-step process called screening and cleaning. In the screening step, the overall number of features is reduced to d=O(N), where N is the number of samples, by removing irrelevant features. By using a screening method that has the sure screening property the authors ensure that relevant features are not removed.  In the cleaning step the reduced regression problem is solved iteratively via a series of regularized quadratic programs (line 96). At each iteration, the current estimate \hat{y} is computed in a distributed manner, each node then sends to the central machine the correlations of the sketched features with current estimate \hat{y}. The central machine then updates the estimate of the regression coefficients \beta. This iterative process terminates after a predetermined number of iterations.  The authors provide a theoretical analysis of the distributed algorithm showing that the prediction error of \beta convergences q-linearly to that of the lasso estimator. The authors further show that the number of iterations of the cleaning stage required to produce an iterate with a prediction error smaller than \epsilon depends on the sketch size n. As the communication costs of the proposed method also depend on n, the authors show that there is a trade-off between "the total rounds of communication with the total amount of data (in bits) sent over the network".
I find the work very interesting and of high quality. I would agree with the motivation put forward by the authors that data split by features rather than samples is an interesting setting to explore in cases where the data is high-dimensional.    What is not entirely clear is the exact contribution of the paper hence my low score on question 6.  Reading Pilanci and Wainwright's JMLR paper it would seem that the cleaning stage of this work is very close to being an application of the algorithm proposed there (Iterative Hessian Sketching) to a distributed setting. The authors point out some points of difference, lines 52-55, but I am not convinced that these really constitute a novel contribution. I hope the authors can elaborate further on this point. What are the practical differences of applying the proposed method as opposed to IHS, are there tangible advantages to using the proposed method? Obviously IHS would incur a much greater communication cost for the same number of iterations. However how is convergence affected in this case?


Refined Lower Bounds for Adversarial Bandits
We provide new lower bounds on the regret that must be suffered by adversarial bandit algorithms. The new results show that recent upper bounds that either (a) hold with high-probability or (b) depend on the total loss of the best arm or (c) depend on the quadratic variation of the losses, are close to tight. Besides this we prove two impossibility results. First, the existence of a single arm that is optimal in every round cannot improve the regret in the worst case. Second, the regret cannot scale with the effective range of the losses. In contrast, both results are possible in the full-information setting.

This paper presents several lower bounds for adversarial bandit setting. In most cases, upper bounds matching these lower bounds  (within logarithmic factors) are known, but it was not known earlier if those upper bounds bounds were tight. In one case (second order bounds in terms quadratic variation) there is a gap between the best known upper bound and the lower bound shown by the authors.
 Merits: It is a good paper to read, well-written and with very few typos. While most results provided in this paper are as one would expect in these problems (the authors also note this), I think it does the necessary job of filling the gaps in this literature and prove tightness of many existing algorithms.  From technical point of view, the construction of stochastic correlated losses used for Theorem 1 and 2 (zero order bounds) and its proof seems particularly interesting, and different from the common constructions for lower bounds in this literature (like the one used in Section 3 for first order bounds). It will be useful to provide some further intuition on why constructions like those used first order bounds don't work for zero order bounds. I can see that Section 3 techniques are good for proving expected bound, while for zero order bounds authors need a construction where they can employ Lemma 2. But, the authors may be able to provide a better high level intuition.   Critique:  As mentioned earlier, the results in these papers are not very surprising. Most of the techniques, barring the construction Section 2, are standard in the literature.  Summary of review: Overall, I think it is a good technical paper, which will be of interest and useful to researchers working in this area.


Active Learning with Oracle Epiphany
We present a theoretical analysis of active learning with more realistic interactions with human oracles. Previous empirical studies have shown oracles abstaining on difficult queries until accumulating enough information to make label decisions. We formalize this phenomenon with an "oracle epiphany model" and analyze active learning query complexity under such oracles for both the realizable and the agnostic cases. Our analysis shows that active learning is possible with oracle epiphany, but incurs an additional cost depending on when the epiphany happens. Our results suggest new, principled active learning approaches with realistic oracles.

This work studies the problem of active learning in a setting where the oracle might not be sure how to answer queries in certain (unknown) regions of the space until asked a few times for samples from those regions.  The problem is well-motivated by behavioral studies of human labelers, and by intuitive descriptions of cases that might be seemingly ambiguous at first (e.g., when asked whether a webpage is about sports, the labeler might initially not make up her mind about how to label webpages about sports memorabilia).  The paper models this phenomenon by supposing there are K unknown regions of the space, and each time a query is made in one of these regions for which an "epiphany" has not yet occurred, there is a beta probability the epiphany occurs then, in which case the oracle returns a label as usual (and will henceforth return a label whenever points in that region are queried), and otherwise the oracle abstains from labeling. They propose simple modifications of known active learning algorithms (CAL in the realizable case, OracularCAL in the agnostic case), and analyze the query complexity in each case, paying particular attention to the increase in query complexity induced by these abstentions, which is generally a function of K and beta.
Overall I enjoyed reading this paper.  It is very well written, and the algorithms and analysis seem to be natural modifications of these existing approaches to active learning.  The theoretical issues that arise in handling these abstentions and quantifying their effect on the query complexity are at times nontrivial, and are handled in elegant and appropriate ways.  I suspect that not many people are aware of this problem, but it is quite well motivated in the paper, and seems to be a good problem to study.  The specific theoretical model proposed for this phenomenon is, however, a bit toy-like.  From the motivation, it seems these epiphanies might have more to do with the labeler needing enough data to get a feel for what the distribution of samples will be, to see where to draw the line, rather than there being some random internal event that could happen at any time upon being queried for a data point of that type.  But I suppose this simplistic model could at least be a reasonable starting place, which can hopefully be made more realistic or general in future work.  I have a few technical comments for the authors:  The halting criterion for EPICAL is that \mu_X(D) \le \epsilon. However, it seems far preferable to halt upon  \sup_{h,h' \in V} \mu_X(h(x)\neq h'(x)) \le \epsilon.   Not only is this more closely related to the error rate, but it  would guarantee that the algorithm actually halts (with query  complexity at most roughly the passive sample complexity);  in contrast, the halting criterion as currently stated might never  actually be satisfied (e.g., interval classifiers with target as  empty interval), so that the algorithm never halts.  In Theorem 1, it seems clear that the terms M_CAL and 1/beta are unavoidable (as argued on page 4).  It would be nice to also have some example(s) illustrating the kinds of scenarios where the \bar{M} term is also unavoidable and nonredundant.  If the region U has large probability, then we'll very quickly get 1/beta abstentions, so \bar{M} seems unnecessary.  But if U has small probability (eg, \epsilon/2), then we don't care about the abstentions anyway.  So it seems the medium-sized U case is where \bar{M} might be needed in the bound.  Is there a good example to illustrate its necessity in bounding the query complexity of EPICAL?  In Corollary 7, in the query complexity bound, the first appearance of \tilde{d} in this expression is redundant (the second term already includes a value greater then \tilde{d}(e*/\epsilon) toward the right-most side of the expression).  In the paragraph after Corollary 7, it is claimed that the leading term in the  agnostic setting is of order \theta (e*/\epsilon)(\tilde{d}+\tilde{K}/\beta). However, the bound also includes a term of order \theta (e*/\epsilon)^2 \tilde{d}. So the correct expression here would be \theta ( (e*/\epsilon)^2 \tilde{d} + (e*/\epsilon)\tilde{K}/\beta ).


Stochastic Three-Composite Convex Minimization
We propose a stochastic optimization method for the minimization of the sum of three convex functions, one of which has Lipschitz continuous gradient as well as restricted strong convexity. Our approach is most suitable in the setting where it is computationally advantageous to process smooth term in the decomposition with its stochastic gradient estimate and the other two functions separately with their proximal operators, such as doubly regularized empirical risk minimization problems. We prove the convergence characterization of the proposed algorithm in expectation under the standard assumptions for the stochastic gradient estimate of the smooth term. Our method operates in the primal space and can be considered as a stochastic extension of the three-operator splitting method. Numerical evidence supports the effectiveness of our method in real-world problems.

In this paper the authors proposed a stochastic optimization algorithm, STCM, for a three composite  convex minimization problems. This problem can be write the sum of two proper, lower semicontinuous convex  functions and a smooth function with restricted strong convexity. This work is based on the deterministic three operator splitting method proposed by Davis and Yin. The author(s). The almost surely convergence and a convergence rate are established.  Major comments:  (1) The main result is quite clear, but lacks the support in details for important aspects. For example, it would be better to show some intuition of the proposed algorithm. If it is an directly extension of Davis and Yin’s work by changing the gradient into the stochastic gradient, authors should point out it. I would like to suggest authors to show more insights and intuition behind of the proposed algorithm, since you have enough space. The proof looks quite rush. Many derivation details are omitted. It is hard to check the correctness in proofs, although the results sound correct.  (2) In the Numerical experiments section, some basic algorithms should be included for comparison, e.g., Wang et al., Random Multi-Constraint Projection: Stochastic Gradient Methods for Convex Optimization with Many Constraints, 2015.  (3) It is important to compare the convergence rates with some other stochastic gradient based method. Basically, we want to know how tight it is? For example, is it consistent the standard SGD, accelerated SGD, and the method above. Otherwise, it is difficult to evaluate the theoretical merit of this work.   Minor comments:  (1) There are lots of typos in this paper, e.g., equation (4), the citation in line 136, line 72, line 95, line 285.  (2) In line 2 of equation (10), the grad(x_g,n - x*) should be (grad(x_g,n) - grad(x*)) instead.  (3) $\mu$ strong convex => $\mu$ strongly convex.
Major comments:  (1) The main result is quite clear, but lacks the support in details for important aspects. For example, it would be better to show some intuition of the proposed algorithm. If it is an directly extension of Davis and Yin’s work by changing the gradient into the stochastic gradient, authors should point out it. I would like to suggest authors to show more insights and intuition behind of the proposed algorithm, since you have enough space. The proof looks quite rush. Many derivation details are omitted. It is hard to check the correctness in proofs, although the results sound correct.  (2) In the Numerical experiments section, some basic algorithms should be included for comparison, e.g., Wang et al., Random Multi-Constraint Projection: Stochastic Gradient Methods for Convex Optimization with Many Constraints, 2015.  (3) It is important to compare the convergence rates with some other stochastic gradient based method. Basically, we want to know how tight it is? For example, is it consistent the standard SGD, accelerated SGD, and the method above. Otherwise, it is difficult to evaluate the theoretical merit of this work.


Bayesian Intermittent Demand Forecasting for Large Inventories
We present a scalable and robust Bayesian method for demand forecasting in the context of a large e-commerce platform, paying special attention to intermittent and bursty target statistics. Inference is approximated by the Newton-Raphson algorithm, reduced to linear-time Kalman smoothing, which allows us to operate on several orders of magnitude larger problems than previous related work. In a study on large real-world sales datasets, our method outperforms competing approaches on fast and medium moving items.

The paper aims to forecast accurate probability distributions of demand, particularly when the data is count or bursty as in an e-commerce setting. To do this, state space models are extended as in the case of GLMs, with a double logistic link function, and algorithms are proposed for maximum likelihood learning of parameters.
The authors should familarize themselves with the work on State Space Models in statistics: a useful starting point would be Durbin, James, and Siem Jan Koopman. Time series analysis by state space methods. No. 38. Oxford University Press, 2012. There is a big section of the book devoted to non gaussian models, and they would also find discussion on adding regression effects, referred to as "modeling of deterministic part" in the paper, line 27-28. In particular, closely related papers would be:        Fahrmeir, Ludwig, and Stefan Wagenpfeil. "Penalized likelihood estimation and iterative Kalman smoothing for non-Gaussian dynamic regression models." Computational Statistics and Data Analysis 24.3 (1997): 295-320.    Durbin, James, and Siem Jan Koopman. "Time series analysis of non‐Gaussian observations based on state space models from both classical and Bayesian perspectives." Journal of the Royal Statistical Society: Series B (Statistical Methodology) 62.1 (2000): 3-56.    The above papers discuss how finding the mode reduces to kalman Smoothing, and simplifying the non-gaussian likelihood by linearising it around its mode.     Its hard for me to judge the paper without any discussion on relation with very closely related papers. However, based on my understanding a lot of the technical contributions of the paper are well known. More detailed comments follow below.     1.) How is the proposed method bayesian ?    2.) The abstract says you pay special attention to "intermittent and burst" items, but then at the end, it is mentioned that improvements are for fast and medium moving items. Isnt this contradictory ? How can an item be both intermittent and fast moving at the same time ?     3.)12-13. "Classical forecasting methods produce gaussian distrbutions only": not true. See the references I cited.    4.)18-19. very crucial point. should have been emphasized and elaborated on more.    5.) line 28: again not true representation of related work.    6.)Discussion in 3.1 and 3.2, are very hard to follow. Since this is a core and crucial technical contribution of the paper, its important to be presented in a clear way.    7.) Eq. (3) is non-standard, and most people with Math/Statistics background would find it puzzling since measurement error is completely skipped. I know this notation is from one of the references, but it would be better to follow standard notation which is more widely followed.    8.)Section 5.1 is fairly obvious and can be skipped.


Probabilistic Linear Multistep Methods
We present a derivation and theoretical investigation of the Adams-Bashforth and Adams-Moulton family of linear multistep methods for solving ordinary differential equations, starting from a Gaussian process (GP) framework. In the limit, this formulation coincides with the classical deterministic methods, which have been used as higher-order initial value problem solvers for over a century. Furthermore, the natural probabilistic framework provided by the GP formulation allows us to derive probabilistic versions of these methods, in the spirit of a number of other probabilistic ODE solvers presented in the recent literature [1,2,3,4]. In contrast to higher-order Runge-Kutta methods, which require multiple intermediate function evaluations per step, Adams family methods make use of previous function evaluations, so that increased accuracy arising from a higher-order multistep approach comes at very little additional computational cost. We show that through a careful choice of covariance function for the GP, the posterior mean and standard deviation over the numerical solution can be made to exactly coincide with the value given by the deterministic method and its local truncation error respectively. We provide a rigorous proof of the convergence of these new methods, as well as an empirical investigation (up to fifth order) demonstrating their convergence rates in practice.

%%% UPDATE %%% Thank you for your detailed response.   In discussion, several points were raised by a meta-reviewer about the connection between this work and the work of Schober et al in NIPS 2014. In particular, it was suggested that the two methods may be near identical, albeit with different derivations. I shall not repeat the discussion here as I have only a limited understanding and I do not wish to make statements that are incorrect. I do hope these points will be made available to the author(s) by the meta reviewer(s).  At the outset this paper excited me, although I am not expert in ODEs. It is clearly well written and technically proficient. However, the nature of the concerns that have been raised make it difficult to argue strongly in favour of the paper. As a result, my confidence score and corresponding novelty scores have now significantly decreased from the initial review stage.  I would like to add that it would have been much better if the issue of equivalence with Schober et al 2014 had been put to the authors during the initial review stage, so that there was an opportunity to provide a rebuttal. (A comparison was requested by one reviewer, but at that stage the argument that the two works may be near identical was not put forward.) The unfortunate situation appears to be due to the structure of the NIPS review process and I do sympathise with the author(s)' predicament.  %%% END %%%   This paper develops a probabilistic version of the classical Adams-Bashforth and Adams-Moulton numerical integrators. The authors propose a distribution over (stochastic) solutions of the ordinary differential equation (ODE) that is intended to provide a probabilistic quantification of epistemic uncertainty due to the presence of numerical error. This paper is in line with other recent research into probabilistic numerical methods.
This is an outstanding paper. The author(s) should be warmly congratulated on a timely, intelligent and thought-provoking contribution to this emerging literature.  The only substantive negative comment that I can make is that there is not a frequentist assessment of the coverage properties of the proposed distributional quantification of epistemic uncertainty (empirical or theoretical). However, that would not be easily achieved within the confines of 8 pages.  In general terms, the fundamentalist Bayesian could reject the idea because information is "thrown away" outside the s-length moving window. They could also object because the prior measure on the solution is not placed on an infinite dimensional function space at the outset, but is instead defined on the finite dimensional distributions in a somewhat ad-hoc manner (relative to the fundamentalist Bayesian approach). The paper presumably aims to strike a balance between being "fully" Bayesian and being a computational pragmatist. Perhaps there is room to raise and address this trade-off in discussion?  Some minor comments:  Please provide a reference for the Picard-Lindelof theorem (page 2).  On page 2, lines 55-56, I find the description "marginal measure given evaluations of the vector field" to be rather odd, since it sounds like you are in fact conditioning on evaluations of the vector field. My guess is that the equation that follows this description is not the one that was intended to be written here.  Again on page 2, rather than \mathbb{Z}^+, do you mean \mathbb{N}? As far as I can tell, zero is excluded from this statement, so \mathbb{N} = {1,2,3,...} would be more standard. (I may well be mistaken - perhaps some clarification could be made.)  I wonder how standard it is to refer to (e.g.) Eqn. 10 as a Gaussian "process" prior? Would not simply a "Gaussian prior" suffice here, given there doesn't seem to be a continuous time variable present at this point?  I believe references [2] and [3] are both now published, so these reference should be updated.  In the appendix a bold-face `f' is used, whereas in the main text it is not bold.


Bayesian Optimization with a Finite Budget: An Approximate Dynamic Programming Approach
We consider the problem of optimizing an expensive objective function when a finite budget of total evaluations is prescribed. In that context, the optimal solution strategy for Bayesian optimization can be formulated as a dynamic programming instance. This results in a complex problem with uncountable, dimension-increasing state space and an uncountable control space. We show how to approximate the solution of this dynamic programming problem using rollout, and propose rollout heuristics specifically designed for the Bayesian optimization setting. We present numerical experiments showing that the resulting algorithm for optimization with a finite budget outperforms several popular Bayesian optimization algorithms.

Bayesian Optimization algorithms for deterministic functions (possibly observed in noise) with continuous input domain have been intensively adressed in recent decades within the framework of Gaussian Process modelling. Here the authors focus on an important issue with most state-of-the-art strategies, namely that the acquisition function is often relying on one-step-lookahead considerations, and does not account for the information brought by candidate points regarding susceptible pay-offs in future iterations. This is known as the finite budget / time horizon set-up, and has inspired several of the papers referenced in the bibliography. The focus here is on transferring know-how and formalism from the world of Dynamic Programming (DP) to Bayesian Optimization (BO). In particular, it is recalled and formalized how the finite time BO problem can be cast as a DP one, and more centrally how an Approximate DP approach called Rollout can be adapted to it.  A finite time BO algorithm based on Rollout is proposed and tested on Gaussian Process realizations and four classical test functions. Empirical results suggest that the Rollout outperforms the considered state-of-the-art one-step-look-ahead criteria on GPs with fixed covariance parameters, while on test functions results are more mitigated and further highlight the importance of adequately choosing the Rollout horizon and discount factor.
I enjoyed this paper where an effort has been made to transfer a relevant formalism and an appropriate technique from the world of Operations Research and Dynamic Programming to Bayesian Optimization. While this was partly done in previous work here it it seems to go one step further, and I am not aware of publications where the Rollout was adapted to this precise problem. The results look promising, especially on the GP realizations, but I really felt the absence of comparison to other strategies recently proposed to adress this very issue; GLASSES of [5] seems a natural competitor here (and maybe also the MCTS of [13]). Also I was wondering if further improvements could be reachable at reasonable research investment regarding the (currently rather simple) base policies. As for the empirical comparisons on functions, the way the models are set appears a bit contrived, and the conclusions would have more weight with some experiments in more realistic conditions.  Overall, I assess this work as good and promising, but I feel that it should be seriously improved before publication. Beyond the points mentioned above here are a few additional (essentially minor) remarks along the lines: * abstract, line 5: about `uncountable', isn't it the case already in most usual sequential settings (without the finite budget issue)? * lines 23-24 after the colon: this formulation sounds unclear to me.  * line 28: `optimal' with respect to which cost? * Equation 1 (and later, e.g. in Equation 5): why should there be unicity of the maximizer? Besides, what guarantees its existence unless assumptions such as continuous f and compact X are made?  * In Equations 3-4: noise appears at a sudden. But it was not really mentioned earlier (unless I oversaw it), nor the consequences for Bayesian optimization.  * Lines 99 and 103: the set Wk first depends on zk and uk then not anymore. As formalisation is an important part of this paper, I guess it should be all the more taken care of.  * In Equation 7: shouldn't wk be upper-cased here as it is a random variable with respect to which expectation is taken? Similar questions for further quantities at stake might be relevant.  * In equation (9): where are u{k+1} and w{k+1} now?  * The beginning of the sentence in line 118 seems somehow redundant.  * In line 212 (and/or around): is h typically assumed much smaller than N?  * In 6.1: how are the GP simulated? What is the discretization? (+The software used?)


Unified Methods for Exploiting Piecewise Linear Structure in Convex Optimization
We develop methods for rapidly identifying important components of a convex optimization problem for the purpose of achieving fast convergence times. By considering a novel problem formulation-the minimization of a sum of piecewise functions-we describe a principled and general mechanism for exploiting piecewise linear structure in convex optimization. This result leads to a theoretically justified working set algorithm and a novel screening test, which generalize and improve upon many prior results on exploiting structure in convex optimization. In empirical comparisons, we study the scalability of our methods. We find that screening scales surprisingly poorly with the size of the problem, while our working set algorithm convincingly outperforms alternative approaches.

The authors propose new algorithms for minimizing composite objectives comprised of strongly convex and piecewise (?) components. Apart from numerous claims about importance of the paper contribution, the authors propose a “working set” algorithm and a piecewise screening test, along with an experimental study which seems to show good performance in some application scenarios.
The authors seem to be unaware of the literature about bundle methods (see [Hiriart-Urruty, Lemaréchal, 1993] for the references). The proposed “working set” algorithm looks like a bundle family method. Note that the “basic” bundle algorithm – the Kelley method – also often shows fast convergence. Nevertheless, it minimax complexity is disastrous (although in the strongly convex situation it exhibits dimension-independent convergence rates, which are suboptimal). Same as a majority of work on bundle algorithms, the authors provide some bounds for the model gap, but do not seem being interested in providing any guaranties. My feeling is that to legitimate this type of methods (which “often” exhibit “fast convergence”), the minimax analysis is necessary (cf. methods of bundle-level family, [Lemarechal et al, 1995, Lan 2015].  Hiriart-Urruty, J. B., & Lemaréchal, C. (1993). Convex analysis and minimization algorithms II: Advanced theory and bundle methods, vol. 306 of Grundlehren der mathematischen Wissenschaften. Lemaréchal, C., Nemirovskii, A., & Nesterov, Y. (1995). New variants of bundle methods. Mathematical programming, 69(1-3), 111-147. Lan, G. (2015). Bundle-level type methods uniformly optimal for smooth and nonsmooth convex optimization. Mathematical Programming, 149(1-2), 1-45.


Finding significant combinations of features in the presence of categorical covariates
In high-dimensional settings, where the number of features p is much larger than the number of samples n, methods that systematically examine arbitrary combinations of features have only recently begun to be explored. However, none of the current methods is able to assess the association between feature combinations and a target variable while conditioning on a categorical covariate. As a result, many false discoveries might occur due to unaccounted confounding effects. We propose the Fast Automatic Conditional Search (FACS) algorithm, a significant discriminative itemset mining method which conditions on categorical covariates and only scales as O(k log k), where k is the number of states of the categorical covariate. Based on the Cochran-Mantel-Haenszel Test, FACS demonstrates superior speed and statistical power on simulated and real-world datasets compared to the state of the art, opening the door to numerous applications in biomedicine.

This paper proposed a discriminative itemset mining algorithm named FACS that deals with categorical co-founding effect in statistical analysis with specific applications in biology.  FACS corrects for a confounding covariate in multiple hypothesis testing. The main contribution of FACS is to allow applying Tarone's testability criterion to the CMH test. At the same time, FACS is very fast and scalable.
The paper is well written and clearly organized, especially where it motivates the problem and the goals of the paper. It is a novel point of view to aim to find all feature subsets for which a statistical association test rejects the null hypothesis, and thus allowing to correct for a confounding categorical covariate. The authors results in the scope of the paper demonstrates that FACS keeps the computational efficiency, statistical power and the ability to correct for multiple hypothesis testing of existing method. The introduction of the branch-and-bound algorithm for conditional statistical association tests is novel and well-explained. Along with the results, the authors provide good intuition behind the methods and the key aspects, such as the appropriate testability criterion for the CMH test and the pruning criterion for the CMH test. So I think this work has provides a lot of new insights to field of machine learning in very high-dimensional settings.   The paper has some minor shortcomings: 1) The proposed technique FACS is limited to applications with binary features with binary labels and cofounding factors. This setting is restricting its wide applications, although the extension of continuous setting is not trivial. 2) The experimental part is weak. In simulations, I would not agree with the authors' conclusion that Bonf-CMH is worse than FACS. Bonf-CMH actually performs well in terms of speed and false positive detection with good precision (Figure 1C). More insights could be put into the distinction between the two methods. Further, in real-world applications, FACS is only tested on small size GWAS dataset. I would strongly recommend authors to apply FACS on large scale GWAS data, where we have many cases and controls and large number of genes and report the effectiveness. On such a small datasets, it is really hard to validate the proposed algorithm.


Reward Augmented Maximum Likelihood for Neural Structured Prediction
A key problem in structured output prediction is direct optimization of the task reward function that matters for test evaluation. This paper presents a simple and computationally efficient approach to incorporate task reward into a maximum likelihood framework. By establishing a link between the log-likelihood and expected reward objectives, we show that an optimal regularized expected reward is achieved when the conditional distribution of the outputs given the inputs is proportional to their exponentiated scaled rewards. Accordingly, we present a framework to smooth the predictive probability of the outputs using their corresponding rewards. We optimize the conditional log-probability of augmented outputs that are sampled proportionally to their exponentiated scaled rewards. Experiments on neural sequence to sequence models for speech recognition and machine translation show notable improvements over a maximum likelihood baseline by using reward augmented maximum likelihood (RML), where the rewards are defined as the negative edit distance between the outputs and the ground truth labels.

The submitted paper presents an alternative to expected reward optimization (eq. 3) for neural structured prediction. The new objective (eq. 8) requires to work with a normalized reward score that defines a proper probability distribution. Sampling from this stationary distribution has the great advantage of reduced variance (which is furthered by annealing with a temperature hyperparameter). The paper is evaluated on two sequence-to-sequence learning tasks, with promising results.
The paper addresses an important problem in optimizing expected reward for structured prediction, namely the high variance of estimation procedures that have to sample from a non-stationary model distribution. In the natural language community, especially in statistical machine translation, "expected reward optimization" is known under the name of "minimum risk optimization" or "maximum expected BLEU" training, and has been successfully applied for linear discriminative models (Och: Minimum error rate training in statistical machine translation, ACL 2003; Li & Eisner: First- and second-order expectation semirings with applications to minimum-risk training on translation forests, EMNLP 2009; He & Deng: Maximum expected BLEU training of phrase and lexicon translation models; inter alia) and non-linear neural models (Shen et al.: Minimum risk training for neural machine translation, ACL 2016). It is also known in vision as "expected loss optimization" (Yuille & He: Probabilistic models of vision and max-margin methods. Frontiers of Electrical and Electronic Engineering. 2012). In all these approaches, instead of sampling, expected reward is calculated exactly on n-best lists of outputs or via dynamic programming over the output graph. These approaches should be mentioned, especially the application to neural MT by Shen et al. 2016, and it should be explained why a direct calculation of expected reward over n-best lists or translation graphs is not an option in your approach. Additionally, since these approaches are well-known in machine translation, they should be considered as baseline in your experimental comparison.  The issue of sampling becomes important if reward feedback on output is only available for a single sampled output. This is the case in Ranzato et al.'s reinforcement learning approach or in bandit structure prediction approaches (Sokolov et al.: Stochastic Structured Prediction under Bandit Feedback. http://arxiv.org/abs/1606.00739). The first approach solves the variance problem by initializing the model with a word-level model trained in batch-mode. The second approach explores different objectives among which a cross-entropy objective is used which is identical to (7) (or (8) without the maximum entropy regularizer). Since the approach of Solokov et al. works in a bandit setting, normalizing reward in order to sample from it is not possible. Thus one drawback of the presented work is that is applicable only in cases where rewards are available to all output structures, otherwise the normalization constant cannot be calculated properly. Speaking of which, what are the guarantees on the quality of approximating the normalization constant based on samples? Firstly, a normalization constant is not an expectation that can be easily be approximated by sampling. Secondly, an approximate per-sentence BLEU score needs to be used in the practical application to machine translation. What is it in your experiment?  Concerning the analysis given in the paper, it is nice to see how objective (8) and (3) differ. However, a simple application of Jensen's inequality (mentioned in Sokolov et al. 2016) shows that (8) is a convex upper bound on (3):  Let q and p be defined as in Section 3, then E_q[ -log p] >= -log E_p[q].  It is not entirely clear what the lengthy derivation given in the presented paper adds to this. Furthermore, for unbounded SMT metrics such as Translation Error Rate (Snover et al.: A Study of Translation Error Rate with Targeted Human Annotation, AMTA 2006) the term ||s-r|| can be very large. Thus r and s need to be specified more clearly in the submitted paper.  What is also not mentioned in the presented paper is the aspect of convexity which could give rise to faster convergence, which is again not mentioned or measured in the presented paper. Since variance reduction is claimed as a key contribution, the relation of variance reduction to convergence speed might be worth exploring.  Lastly, the experiments presented in the paper seem to show promising improvements, however, many details are missing: What was the per-sentence BLEU approximation used as reward signal? Why did you not optimize the temperature constant on a heldout set - the presented evaluation basically selects the temperature constant on the test set. Are the result differences statistically significant?  Finally, possible citations for using annealing schedules and sampling in expected loss training for structured prediction are:  Smith & Eisner: Minimum risk annealing for training log-linear models. ACL 2006. Arun et al.: A unified approach to minimum risk training and decoding. WMT 2010.


Communication-Optimal Distributed Clustering *
Clustering large datasets is a fundamental problem with a number of applications in machine learning. Data is often collected on different sites and clustering needs to be performed in a distributed manner with low communication. We would like the quality of the clustering in the distributed setting to match that in the centralized setting for which all the data resides on a single site. In this work, we study both graph and geometric clustering problems in two distributed models: (1) a point-to-point model, and (2) a model with a broadcast channel. We give protocols in both models which we show are nearly optimal by proving almost matching communication lower bounds. Our work highlights the surprising power of a broadcast channel for clustering problems; roughly speaking, to spectrally cluster n points or n vertices in a graph distributed across s servers, for a worst-case partitioning the communication complexity in a point-to-point model is n · s, while in the broadcast model it is n + s. A similar phenomenon holds for the geometric setting as well. We implement our algorithms and demonstrate this phenomenon on real life datasets, showing that our algorithms are also very efficient in practice.

The paper proposes several distributed clustering algorithms and study the communication complexity in two models (point to point and blackboard). The authors prove lower and upper bounds for graph spectral clustering, and geometric clustering (i.e. k-means, k-median,...). The algorithms are implemented and evaluated on large graphs (the edge size is larger than 70 million).
  In this paper, a large dataset is maintained in s sites that communicate in a point to point protocol (private message passing between sites and one coordinator) or in a blackboard protocol (broadcast messages to a centralized site).   The proposed algorithm for spectral graph clustering within the message passing model is based on spectral sparsification.   The contribution is to show the the sparsification can be computed  locally. The proof  that the clustering the union of  the sparse subgraphs is close to the clustering on the whole graph highly relies on [19, Peng et al, COLT 2015]. The lower bound is based on a reduction to the multiparty set disjointness problem.  The algorithm in the Blackboard model relies on the iterative construction of a sequence of matrices of length O(log n) whose last element is a spectral sparsifier. The presentation of this algorithm is less clear than the previous one. In particular it is hard to follow what can be done locally and when the communication with the coordinator is needed (I think that a communication is needed at each step to compute \tau_i).  In both cases, the proofs in appendix seem to be necessary to understand the algorithms. So the presentation could be improved.     The experiments confirm the theoretical results.    I think that the paper is interesting and deserves publication.    Typos and technical remarks.  - Lines 469/471: \gamma_u is \lambda_u. Line 475: I don't understand the append operator.  - I think that a concluding Theorem could be added to Section 3.2.  - It should be more clear if you announce sooner that the reduction will encode X_i^j=1 if item j *is not* in site i.


Robust k-means: a Theoretical Revisit
Over the last years, many variations of the quadratic k-means clustering procedure have been proposed, all aiming to robustify the performance of the algorithm in the presence of outliers. In general terms, two main approaches have been developed: one based on penalized regularization methods, and one based on trimming functions. In this work, we present a theoretical analysis of the robustness and consistency properties of a variant of the classical quadratic k-means algorithm, the robust k-means, which borrows ideas from outlier detection in regression. We show that two outliers in a dataset are enough to breakdown this clustering procedure. However, if we focus on "well-structured" datasets, then robust k-means can recover the underlying cluster structure in spite of the outliers. Finally, we show that, with slight modifications, the most general non-asymptotic results for consistency of quadratic k-means remain valid for this robust variant.

This paper examines the statistical property of the “robust k-means clustering” procedure. Specifically, they look at its robustness in terms of the finite sample breakdown point and its consistency.  The idea is to use techniques from variational analysis to study the functional properties of the RKM. In particular, the paper studies the “breakdown point” of a family of robust k-means objective, with different regularization terms (f_lambda). Their main result is that this family of k-means objective is actually not robust at the presence of at most two outliers (Thm 1).  This result shows that “robust” k-means objectives are actually not robust on general datasets. However, they explained that when the dataset has a clusterable structure, the objective is robust to outliers. Finally, they proved consistency of the objective.
Technical quality:  I want to mainly point out one thing: the experiment section seems to be a little detached from the rest of the paper. The paper aims to study the property of the robust k-means objective, while the experiments compare two different robust methods for clustering, one is based on a heuristic (coordinate descent) that locally decreases the robust k-means objective, and the other is not related to any specific objective, if I understand it correctly.    Novelty/Originality:  I think the novelty of this paper lies in the fact that they apply techniques from optimization to study the properties of robust k-means objective.   Potential impact & Usefulness:  I think it is important to know that in general, the robust k-means is not robust to outliers, and that one needs to focus on dataset with more structure to obtain robustness. This discovery is inline with other recent work on the importance of clusterability for studying clustering procedures. I think the fact that they introduce the method of variational analysis to study clustering objectives could also inspire more work on understanding clustering objectives via this perspective.   Clarity and presentation  The interpretability of their result could be improved: in Section 3.3, it’s not clear to me what does Corollary 2 have to do with the conclusion at the end of this section. Also, it is also not clear to me what is the contribution of this paper regarding the discussion here: Is it already addressed in [2]? If not, how does the paper add insight to robustness of k-means objective with an underlying clusterable structure?  In general, it'd be nice if the authors can put the work in context of related work.


Supervised Word Mover's Distance
Recently, a new document metric called the word mover's distance (WMD) has been proposed with unprecedented results on kNN-based document classification. The WMD elevates high-quality word embeddings to a document metric by formulating the distance between two documents as an optimal transport problem between the embedded words. However, the document distances are entirely unsupervised and lack a mechanism to incorporate supervision when available. In this paper we propose an efficient technique to learn a supervised metric, which we call the Supervised-WMD (S-WMD) metric. The supervised training minimizes the stochastic leave-one-out nearest neighbor classification error on a perdocument level by updating an affine transformation of the underlying word embedding space and a word-imporance weight vector. As the gradient of the original WMD distance would result in an inefficient nested optimization problem, we provide an arbitrarily close approximation that results in a practical and efficient update rule. We evaluate S-WMD on eight real-world text classification tasks on which it consistently outperforms almost all of our 26 competitive baselines.

This paper proposes a supervised metric learning approach built on top of the recent “word-movers distance” [19] (a transport distance applied to word2vec word embeddings). The proposed approach can be applied to any transport distance based on squared Euclidian distances after linear projection. On top of learning the linear projection to maximise performance, the proposed approach also learns a weighting of words.  The latter is specific to the document application, it consists in re-weighting the frequency of each word by a learned weight, and re-normalising the word frequencies. The paper presents an efficient optimisation method to learn the proposed parameterised metric. A thorough experimental evaluation shows the effectiveness of the proposed approach by comparing it to a large number of reasonable baselines on a number of document classification datasets using a kNN classifier based on the proposed learned metric.
+ This work extends the recent word-movers distance, which is an interesting technically and useful extension (shown to be effective on a number of kNN text classification tasks).   - The potential impact is perhaps limited by the fact that earth-movers distance with a metric learning via a “word centroid distance” initialisation performs similar to the full learning of the proposed distance. See the difference of the last two rows in Table 3. In other words, the main technical contributions of this paper do not seem to contribute very much over this initialisation method based on existing techniques. - One thing that I missed in table 3 is an evaluation of the “word centroid embedding” (weighted mean of word embeddings for the document) with all tested existing metric learning approaches (ITML, LMNN, NCA). - Another point missing in the evaluation is an assessment of the relative importance of the word-specific weights, and the projection matrix A. It would be useful to see performance with each one of these components being active in isolation.  o The originality is mostly related to the way to optimise the metric (Line 156–175), the idea of a linear metric learning within the earth-movers distance is very interesting but a limited contribution in itself.   + The paper is very well written and a pleasure to read. There are a few minor issues that could be clarified: - Line 152: “The authors”: of which paper? [1,7,8] ? - Equation 11: shouldn’t alpha and beta carry stars here ? - It would be useful to explicitly recall around line 170 that this is the Sinkhorn scaling that is referred to in the introduction in line 56.


Mixed vine copulas as joint models of spike counts and local field potentials
Concurrent measurements of neural activity at multiple scales, sometimes performed with multimodal techniques, become increasingly important for studying brain function. However, statistical methods for their concurrent analysis are currently lacking. Here we introduce such techniques in a framework based on vine copulas with mixed margins to construct multivariate stochastic models. These models can describe detailed mixed interactions between discrete variables such as neural spike counts, and continuous variables such as local field potentials. We propose efficient methods for likelihood calculation, inference, sampling and mutual information estimation within this framework. We test our methods on simulated data and demonstrate applicability on mixed data generated by a biologically realistic neural network. Our methods hold the promise to considerably improve statistical analysis of neural data recorded simultaneously at different scales.

The authors develop tractable means of sampling from and calculating the likelihood of C-vine copulas which permit both discrete and continuous marginal distributions.  The proposed algorithms for sampling and likelihood calculation are extension of previous work (that of Aas et al. for likelihood calculation and  Panagiotelis et al. for sampling).  The benefit of the methods proposed is that while naive sampling strategies for calculating likelihoods and sampling would scale exponentially with the number of discrete random variables in a model, the proposed methods scaled quadratically.    The authors propose to apply such copulas to the study of neural data which consists of both continuous measurements (such as local field potentials) and discrete measurements (spike counts).  The authors demonstrate that data generated from such mixed copulas is better fit by these models than by models which assume independence between the marginal variables or use continuous distributions to model variables which are in fact discrete.  They also simulate LFP and spike data from a network driven by two different stimulus conditions and show that the mutual information between the stimulus and recorded neural signals differs substantially when it is calculated with the mixed copulas proposed in this work vs. a model which assume independence between marginal variables or a copula model which assume all marginal variables as continuous.
The development of flexible methods to model the joint distribution between continuous and random variables is a important problem with many application areas, one of which, as the authors note, is neuroscience.  Copula which allow for both discrete and continuous random variables are one means of approaching this problem, and the development of general and computationally tractable methods for fitting and performing inference with such models is of broad interest.    The paper makes multiple methodological contributions, which I find valuable.  The proposed family of models seems flexible and likely useful in practice. While others have previously proposed pair copula constructions as well as efficient algorithms for sampling from discrete copulas, the development of pair copula constructions and associated efficient algorithms for sampling and inference for mixed discrete and continuous data is valuable.  However, I feel that other aspects of the paper could be improved.   I believe the paper would benefit from more clarity.  There were multiple parts of the paper that I found difficult to follow.  For example, equation 2 provides a general formula for the likelihood of data under a mixed copula model, but it was not until after reading some of the cited work that I was able to understand it was derived by taking differences of the discrete random variables.  While I was previously familiar with copula in general, this was my first exposure to C-vine copula.  The description of C-vines required additional reading for me to follow.  It is certainly understandable that it may be hard to fully explain all concepts in an 8 page paper, but I believe a general NIPS audience would benefit from the provision of a little more explanation and background.  There were also symbols which were used without introduction (the super script +/- signs which I believe indicate differences, "c" for the pdf of a copula, etc.).  While it was possible for me to follow the gist of paper, it was difficult to fully follow all the details.    It would be helpful to explain why a copula approach might be beneficial over existing approaches which have been applied in the neuroscience literature.  While it is not referenced in the present paper, there is established body of work in the neuroscience community which has examined the relationship between neural spiking and LFP (e.g., the first three references below and references therein).    Finally, I believe the results section would benefit from more depth.  If I understood correctly, in the first analysis data was generated from a particular mixed copula model.  The fit of mixed copula models, models which assumed independent random variable and copula models which modeled all marginal variables as continuous to this data was then compared.   Mixed copula models were shown to best fit the data.  While this is reassuring, this result also seems expected, as models which are subject to mismatch would not be expected to fit data as well as a model which is of the same family the data was generated from.  To show the utility of proposed methods, an application to more realistic data is beneficial, and the author's provide this by applying their methods to simulated spike and LFP data and show that mutual information (MI) between a stimulus and the simulated LFP and spike data is different under mixed copula models vs. models which assume independent random variables or require all random variables to be continuous.  I find the author's claim that a mixed copula model vs. the other two options should best fit the data believable, and seeing the difference in the values of MI between the calculated models is interesting.  However, if an actual application scenario, I believe experimentalists would not be primarily concerned with the value of MI returned - but by differences in MI returned by the same model in different experimental contexts.  For example, one might ask if MI between LFP and neurons in two areas changes when a subject is attending vs. not attending to a stimulus.  In such a scenario the primary question of interest is if a statistical method is sensitive enough to reveal differences between MI between the two conditions, and thus it would perhaps be more interesting to see an analysis showing the benefit of the proposed methods in such a setting.    References:   Lepage, K. Q., Gregoriou, G. G., Kramer, M. A., Aoi, M., Gotts, S. J., Eden, U. T., & Desimone, R. (2013). A procedure for testing across-condition rhythmic spike-field association change. Journal of neuroscience methods, 213(1), 43-62.  Kelly, R. C., Smith, M. A., Kass, R. E., & Lee, T. S. (2010). Local field potentials indicate network state and account for neuronal response variability. Journal of computational neuroscience, 29(3), 567-579.  Carlson, D. E., Borg, J. S., Dzirasa, K., & Carin, L. (2014). On the relations of LFPs & Neural Spike Trains. In Advances in Neural Information Processing Systems (pp. 2060-2068).  K. Aas, C. Czado, A. Frigessi, and H. Bakken. Pair-copula constructions of multiple dependence. Insurance: Mathematics and Economics, 44(2):182–198, 2009.  A. Panagiotelis, C. Czado, and H. Joe. Pair copula constructions for multivariate discrete data. Journal of the American Statistical Association, 107(499):1063–1072, 2012.


Learned Region Sparsity and Diversity Also Predict Visual Attention
Learned region sparsity has achieved state-of-the-art performance in classification tasks by exploiting and integrating a sparse set of local information into global decisions. The underlying mechanism resembles how people sample information from an image with their eye movements when making similar decisions. In this paper we incorporate the biologically plausible mechanism of Inhibition of Return into the learned region sparsity model, thereby imposing diversity on the selected regions. We investigate how these mechanisms of sparsity and diversity relate to visual attention by testing our model on three different types of visual search tasks. We report state-of-the-art results in predicting the locations of human gaze fixations, even though our model is trained only on image-level labels without object location annotations. Notably, the classification performance of the extended model remains the same as the original. This work suggests a new computational perspective on visual attention mechanisms, and shows how the inclusion of attention-based mechanisms can improve computer vision techniques.

The paper proposes a Sparse Diverse Region Classifier (SDR), a method based on Region Ranking SVM (RRSVM) [29], which imposes diversity on the regions selected by RRSVM when training an image classifier. Diversity in the selected regions is introduced by a mechanism called "Inhibition of Return", which is in charge of only selecting regions with low spatial overlap. This overlap is computed using the traditional intersection over union criterion used in non-maximun suppresion (NMS).   Experiments are conducted on the POET[26], MIT900 [8], PET [13] datasets, using a RRSVM and SDR models trained on the Pascal VOC 2007 dataset. The evaluation covers different scenarios: a) Single-target is present, b) the target is absent, and c) when multiple-targets are present. The Judd variation of the Area Under Curve (AUC-Judd) is used as a performance metric, where the ability of the method to recover visual fixation points is measured. Experimental results show that the introduced Inhibition Return mechanism introduced via NMS, brings a boost in performance of ~4 performance.
- The content of the paper is well presented and accompanied by clear illustrations which make the content of the paper easy to follow. Likewise, the experimental protocol followed in the paper is quite clear, which eases reproducibility of the results obtained in this work.  Having said this, I consider there are two major flaws in the paper:  - I appreciate the comparison wrt. to methods that use the combination of the output of object detectors to create the priority maps. However, in my opinion, despite the fact that R-CNN takes into account object annotations during training, it is not a strong baseline as is claimed in the manuscript due to the following reasons. First, R-CNN uses which has the weakness of using Selective Search (Uijlings et al., IJCV'13) which is a generic method for generating object proposals. R-CNN is not anymore the state-of-the-art, I recommend repeating the experiment with Faster-RCNN, Ren et al. CVPR'15 which addresses that weakness. Second, and more critical, R-CNN is trained to optimizing the object detection which is measured by the area under the curve where matching between predicted object regions and ground-truth is computed by taking into account object bounding boxes (not the fixation points). Since these are two different tasks, ie. object detection (localizing the full extent of the object)  VS. human visual attention (localizing the specific regions that define the object), in my opinion it is not surprising that R-CNN has sub-standard performance.  - Regarding the AnnoBoxes baselines, I agree that it is exploiting more information (annotated bounding boxes on test images). However, I believe that the way in which this information is used, reduces significantly the advantage that it can bring to the task of visual attention prediction. As stated in Sec. 4.2: "for this method, the priority map is created by applying a Gaussian filter to a binary map where the center of the bounding box over the target(s) is set to 1 and everywhere else 0", this implies that the points in the binary map with values==1 (of high priority) are mostly located at the center of the bounding box of each object, and, as can be observed in the figures showing animal-related classes, this is not the same as the fixation points. This is further confirmed in Table 1, where it can be noted that is in the animal-related classes (cat, cow, dog and horse) where the proposed SDR method has a significant advantage over AnnoBoxes (with boosts of 5, 4, 8 and 6 percentage points, respectively).   In my opinion, a comparison against the two methods listed above (R-CNN and AnnoBoxes) is not a strong empirical evidence that the proposed method is good at predicting visual attention (fixation points). Perhaps, from the results of the previous two comparisons, it is better to say that solely focusing on learning how to localize objects does not imply learning how to predict visual attention (fixation points). This could serve as a motivation for the diversity introduced by SDR via NMS.  I believe the paper can be strengthened by performing experiments to evaluate the effect that some of the parameters (e.g., overlap considered when doing NMS for SDR, width of the Gaussian blur kernel) have on performance. In addition, I suggest making more concrete the conclusions made during this work (At this point they were not so clear to me).   In addition, regarding to the comparisons wrt. to existing work, I suggest considering the method from Li et al. arXiv:1506.06343 which tries to detected visual patterns on image regions that can be informative for image classification. In my opinion this method has some similarities wrt. the method proposed in the paper.   In my opinion the paper is still at a premature stage for publication at this point .  Given the promising potential and links that the proposed method has in different fields, I encourage the authors to properly position their work wrt. existing work, revise the baselines considered during their evaluation, and, through an ablation study, provide an insight on the effect of different parameters that are part of the proposed method.


Homotopy Smoothing for Non-Smooth Problems with Lower Complexity than O(1/ )
In this paper, we develop a novel homotopy smoothing (HOPS) algorithm for solving a family of non-smooth problems that is composed of a non-smooth term with an explicit max-structure and a smooth term or a simple non-smooth term whose proximal mapping is easy to compute. The best known iteration complexity for solving such non-smooth optimization problems is O(1/ ) without any assumption on the strong convexity. In this work, we will show that the proposed HOPS achieved a lower iteration complexity ofÕ(1/ 1−θ ) 1 with θ ∈ (0, 1] capturing the local sharpness of the objective function around the optimal solutions. To the best of our knowledge, this is the lowest iteration complexity achieved so far for the considered non-smooth optimization problems without strong convexity assumption. The HOPS algorithm employs Nesterov's smoothing technique and Nesterov's accelerated gradient method and runs in stages, which gradually decreases the smoothing parameter in a stage-wise manner until it yields a sufficiently good approximation of the original function. We show that HOPS enjoys a linear convergence for many well-known non-smooth problems (e.g., empirical risk minimization with a piece-wise linear loss function and 1 norm regularizer, finding a point in a polyhedron, cone programming, etc). Experimental results verify the effectiveness of HOPS in comparison with Nesterov's smoothing algorithm and the primal-dual style of first-order methods.

The submission considers algorithms for solving a specific class of optimization problems, namely min_{x in Omega_1} F(x), where F(x) = max_{u in Omega_2} \langle Ax, u \rangle - phi(u) + g(x). Here, g is convex, Omega_1 is closed and convex, Omega_2 is closed, convex, and bounded, and the set of optimal solutions Omega_* \subset Omega_1 is convex, compact, and non-empty. The submission also assumes a proximal mapping for g can be computed efficiently.   The above framework is apparently general enough to capture a number of applications, including various natural regularized empirical loss minimization problems that arise in machine learning.   Classic work of Nesterov combined a smooth approximation technique with accelerated proximal gradient descent to converge to a solution with epsilon of optimal in O(1/epsilon) iterations. Roughly, the smoothing parameter mu controls both the quality of the smooth approximation to the original problem (smaller mu means a better approximation), and the speed of convergence (smaller mu means slower convergence). Choosing mu to balance these costs yields an eps-accurate solution in O(1/eps) iterations.  The submission shows that if F satisfies a "Local Error Bound" (LEB), then one can obtain faster convergence. Roughly, an LEB guarantees that the distance of any x from an optimal solution falls like a constant power of the distance of the value F(x) from the optimal value F(x^*). The key idea is to use Nesterov's approach, but the start with a large mu, and gradually decrease it over the course of the execution. The point is that one does not need a good approximation to the original problem at the start (since one is far from an optimal solution then anyway). But as a good solution is approached, one can afford to drop mu to get a better smooth approximation, without killing the convergence rate. A primal-dual version of the algorithm that does not require hand-tuning of parameters is also described (mostly in the supplement).  Applications to specific classes of problems are discussed, and some experimental results from 3 different application domains are provided. The experiments show significantly improved iteration complexity for small values of epsilon, compared to basic Accelerated Proximal Gradient Descent, and a first-order primal-dual method.
This seems like a nice contribution, both in terms of the analysis (which is clean, intuitive, and apparently the first for this kind of smoothing algorithm), and the concrete efficiency as demonstrated in the experiments. (However, I am not knowledgeable enough about the area to know for certain that the algorithms experimentally compared against truly state of the art).   The class of problems considered capture a variety of important applications, and the theoretical results do seem to generalize or asymptotically improve on several directly comparable prior works.   For applications where theta=1, such as regularized empirical loss minimization with L_1-norm or L_{infty}-norm as the regularizer and a non-smooth loss function like hinge or absolute loss, the submission's algorithm will converge in O(log(epsilon_0/epsilon)) iterations, while the submission suggests that prior work would require O(1/epsilon) iterations to converge. Is this accurate? If so, then this is an exponential improvement on prior work, which seems rather impressive, and perhaps the submission should emphasize this point more.   It would be nice to see how the proposed algorithm fares against prior art for larger values of epsilon. My understanding from the experiments is that the advantage of the present algorithm is seen only at small values of epsilon. But if this is the case, why not use the other methods to quickly find a reasonably good solution, and then run the new algorithm starting from said solution?  Smaller questions: *The submission states that if L_mu is difficult to compute, one can use the "backtracking trick". Was this necessary in any of the applications considered in the experiments?   *Since the proposed primal-dual method requires operating on the dual as well as the primal, it is perhaps surprising that the experimental wall time for this version of the algorithm was best overall. Did the reported wall time account for both the time to update the primal solution and the dual?


The Sound of APALM Clapping: Faster Nonsmooth Nonconvex Optimization with Stochastic Asynchronous PALM
We introduce the Stochastic Asynchronous Proximal Alternating Linearized Minimization (SAPALM) method, a block coordinate stochastic proximal-gradient method for solving nonconvex, nonsmooth optimization problems. SAPALM is the first asynchronous parallel optimization method that provably converges on a large class of nonconvex, nonsmooth problems. We prove that SAPALM matches the best known rates of convergence -among synchronous or asynchronous methods -on this problem class. We provide upper bounds on the number of workers for which we can expect to see a linear speedup, which match the best bounds known for less complex problems, and show that in practice SAPALM achieves this linear speedup. We demonstrate state-of-the-art performance on several matrix factorization problems.

The paper proposes an asynchronous proximal gradient descent algorithm and provides a proof of its guaranteed convergence to a local optimal solution. The main contribution is the finding that the prox function for nonconvex problems defines a set which in turn induces a measurable space and to interpret the unchecked asynchronous updates of variables as a source of noise when computing the gradient. In doing so, the distributed computation can be seen as an instance of stochastically altered gradient descent algorithm. The authors report convergence behavior under two different noise regimes resulting in constant and decreasing step sizes respectively.
I find the approach rather interesting, especially the broad and general definition of the problem makes the approach applicable to a wide range of problems. However, I was surprised by the absence of any reference to the seminal Robbins/Munro paper and also to the recent developments in stochastic gradient descent based sampling (see below). The authors do local gradient descent updates of coordinate blocks by computing partial gradients and adding noise in each asynchronous step. I was wondering, how this relates to the "usual" stochastic gradient descent update, i.e., given that the locally computed partial gradient will be based on delayed (noisy) variable states, a sequence of these noisy partial gradients would converge to the true partial gradient as well.  Further, recent SGD based sampling has shown that adding noise to the variable states obtained by noisy gradient updates (as the authors do as well) provides good samples of the distribution underlying the optimal variable setting also in a non-convex setting.  That being said, the work handed in remains valid, but it would have been interesting to compare the proposed approach to well established stochastic gradient methods. The overall procedure is laid out well and comprehensible. The chosen examples in the experiments section are well suited to demonstrate the scalability benefits of the algorithm.  Minor to that, I have a few remarks on style and the overall rationale: - line 60: "each" is unnecessary here when m = 1 - line 69: k is not yet defined as are g, and \nu - line 106: the notation using y is probably wrong, shouldn't it read argmin_{x_{j_k}^k} r_{j_k} (x_{j_k}^k) + ... ? - Algorithms 1 and 2 lack a break condition and output - Table 1: I assume the timing is in seconds?  Literature: Robbins, H., & Monro, S. (1951). A stochastic approximation method. The Annals of Mathematical Statistics, 400–407. Welling, M., & Teh, Y.-W. (2011). Bayesian learning via stochastic gradient Langevin dynamics (pp. 681–688). Presented at the Proceedings of the 28th International Conference on Machine Learning.


Exact Recovery of Hard Thresholding Pursuit
The Hard Thresholding Pursuit (HTP) is a class of truncated gradient descent methods for finding sparse solutions of ℓ 0 -constrained loss minimization problems. The HTP-style methods have been shown to have strong approximation guarantee and impressive numerical performance in high dimensional statistical learning applications. However, the current theoretical treatment of these methods has traditionally been restricted to the analysis of parameter estimation consistency. It remains an open problem to analyze the support recovery performance (a.k.a., sparsistency) of this type of methods for recovering the global minimizer of the original NP-hard problem. In this paper, we bridge this gap by showing, for the first time, that exact recovery of the global sparse minimizer is possible for HTP-style methods under restricted strong condition number bounding conditions. We further show that HTP-style methods are able to recover the support of certain relaxed sparse solutions without assuming bounded restricted strong condition number. Numerical results on simulated data confirms our theoretical predictions.

This paper analyses the Hard Thresholding Pursuit (HTP) Algorithm which aims at solving problem (1) for a  smooth and convex loss function f. The contributions are Theorems 1, 2 and 3.  * Theorem 1 guarantees that under a RIP-like condition on the loss function (it must be strongly convex and strongly smooth on the set of s-sparse signals) and a condition on the minimal nonzero entry of x compared to the maximal entry of the gradient of f at x, the solution to (1) is unique and HTP recovers it in a finite number of iterations. * Theorem 2 guarantees that (under similar but slightly more restrictive conditions) HTP recovers perfectly the support of an arbitrary k-sparse solution. * Theorem 3, finally, without requiring the RIP-like conditions, guarantees that HTP will recover a support containing the support of a \bar{k}-sparse vector (where \bar{k} < k) as soon as the "rooted value gap" is small enough compared to the minimal entry of that vector. These theorems are deterministic sufficient conditions. They show that "HTP is powerful for exact recovery of signals even in noisy settings".
This paper is quite technical but is very interesting and promising as it provides sufficient conditions for exact recovery or sparsistency of the solutions of an quite broad class sparsity constrained smooth optimization problems using a very simple, yet powerful, class of greedy algorithms. Despite the technicality of the results, they are very clearly synthesized and nicely presented. I particularly like the summarizing tables.  I have only a few (but important) concerns: Line 26 : Problem (1) with a squared regression loss is IMO more general than compressed sensing. Depending on the forward model, it can tackle any noisy linear inverse problem such as optimal sparse representation in a dictionary, denoising, deblurring (or deconvolution),... Line 32 : There is also that paper : [1]A. Beck and Y. C. Eldar, “Sparsity Constrained Nonlinear Optimization: Optimality Conditions and Algorithms,” arXiv:1203.4580 [cs, math], Mar. 2012. Line 81: least squared -> least squares Line 121: In the definition of x^{-*} the constraint with strict inequality leads to something that is ill-posed. For example take f(x)=x^2 and k=n=1. Then we can take x as close to 0 as we want without ever reaching it. It should be something like an infimum rather than minimum which would allow a gap of 0. You should review this definition in any case. I guess that you meant "such that x has a different support than x^*" ? Please correct or justify as this has huge implications in the rest of the paper (proof of uniqueness for example). Line 129: same remark for \Delta^{-*}  In the proof of theorem 1 Part (b), supplementary material, line 416. As you wrote, the entries of the gradient and of x^(t-1) can but NOT HAVE TO be swapped. Therefore I do not agree with the next statement F^(t) \neq F^(t-1). I can provide a simple counter example where the above inequality would hold but the support would remain unchanged: eta Grad^(t-1) = -(1,2,3,4,5) x^(t-1) = (4.1,4.1,3,0,0) F^(t-1) = {1,2,3} F^* = {3,4,5} We have norm(4.1,4.1) < norm(4,5) but nevertheless \tilde{x}^(t) = (5.1,6.1,6,4,5) and therefore F^(t) = {1,2,3} = F^(t-1) Please correct me if I'm wrong or explain your development or correct the proof.  Line 215: You can also give other references for the Irrepresentability Condition such as Fuchs or Candès and Fernandes-granda, or others