Symbolic Opportunistic Policy Iteration for Factored-Action MDPs
This paper addresses the scalability of symbolic planning under uncertainty with factored states and actions. Our first contribution is a symbolic implementation of Modified Policy Iteration (MPI) for factored actions that views policy evaluation as policy-constrained value iteration (VI). Unfortunately, a naïve approach to enforce policy constraints can lead to large memory requirements, sometimes making symbolic MPI worse than VI. We address this through our second and main contribution, symbolic Opportunistic Policy Iteration (OPI), which is a novel convergent algorithm lying between VI and MPI, that applies policy constraints if it does not increase the size of the value function representation, and otherwise performs VI backups. We also give a memory bounded version of this algorithm allowing a space-time tradeoff. Empirical results show significantly improved scalability over state-of-the-art symbolic planners.

This paper presents a novel policy iteration algorithm
      for symbolic MDPs with factored-action (in addition to factored-state)
      dynamics. The algorithm, MB-OPI, yields a way to trade
      representational complexity between value and policy iteration for the
      class of MDPs defined over algebraic decision diagrams, just as MPI
      gives a way to smoothly trade computational complexity. In doing so,
      the authors generalize several existing algorithms which consider
      factored actions and memory constraints independently. The
      main technical challenge is that ADD policy iteration requires
      multiplying an explicit policy representation into the current value
      function, which can significantly increase its size. The solution is
      to control this increase in size by defining a procedure to
      conservatively combine policy and value diagrams using a pruning
      procedure, rather than naively multiplying them. Results are
      presented in terms of solution time, and show a ~2-6x improvement over
      existing approaches. This paper is technically sound and well
      written. The authors make a theoretical contribution to the literature
      on symbolic MDP planning by introducing the concept of pruning as an
      alternative to ADD products, and proving that this satisfies the
      guarantees of MPI. Also couched as generalization to existing work in
      symbolic dynamic programming, and appears to be state of the art for
      planning with factored actions Empirical results support the idea
      that pruning offers an MPI approach to SDP planning that avoids
      representational bloat, and offers a several factor speed up
      The paper is also generally well written and easy to follow.
      If possible, I would suggest adding more background on SDP solving
      using ADDs for representing value and DBNs, the basic policy iteration
      approach using ADD product, and the difference between multiplying pi
      into V vs. pruning V with pi. It would also be nice to have
      (a) discussion of practical problems with many parallel actions for
      which factoring actions is critical, and (b) a toy test case with
      large parallel actions that highlights the best-case improvement over
      SPUDD and FAR. Some notes on clarity that might be helpful:
      053, "enforcement of policy constraint": 'constraint' hasn't been
      defined yet, and only makes sense if you remember to view policy iteration
      as policy-constrained value iteration 060, Figure 1: ordering of
      state and action nodes would be more readable if they were interleaved or
      stacked (something consistent) 060, Figure 1: as state variables
      these are propositions, not predicates, so might be better to use
      underscores (e.g. reboot_c1) 095, "marginalization operators":
      examples include marginalization but also max. should reword for
      correctness 110, equation 1: this was confusing without referring
      back to SPUDD paper. I suggest adding 2 things: (a) explanation of how
      expectation for factored models turns into a product of DBNs, and that
      sums can be pushed in, and (b) simple explanation that "primed" literally
      adds a prime to each state variable, and is necessary to make the ADD
      operations well defined (saying "swaps the state variables X in the
      diagram V with next state variables X′" can be confused with more
      complicated mapping issues) 152, policy ADD: Would be helpful to
      have a sentence like "Intuitively, this representation makes it possible
      to express 1-step policy backup as the ADD product of a policy with a
      value function". 179, Figure 4: caption uses C to denote policy,
      but figures use \pi. other places too 206, "size of a Bellman
      backup": usually think of backups in terms of computational complexity, so
      should clarify that "size" refers to the data structure that results from
      applying eq1. also would be helpful to intuitively explain what this looks
      like, as opposed to pi (both have action variables, so why is pi generally
      bigger??) 206, (due to…): for clarity, would help to clarify that
      pi is bigger because it represents joint-actions, whereas backup only
      represents value and (product of) factored actions 212, "only
      those paths in D that": add "in D" to make clear that policy is being
      applied to value function. otherwise can be confused with original
      definition of policy 247, eq3: two extra close parens 252,
      "sandwiched": intuitive, but perhaps too informal (though who am I to say
      such a thing?) 278, "parameterized by k, …": missing comma
      327, Figure 6: colors for OPI 2 and 5 are reversed in b... I
      think.


Conditional Random Fields via Univariate Exponential Families
Conditional random fields, which model the distribution of a multivariate response conditioned on a set of covariates using undirected graphs, are widely used in a variety of multivariate prediction applications. Popular instances of this class of models, such as categorical-discrete CRFs, Ising CRFs, and conditional Gaussian based CRFs, are not well suited to the varied types of response variables in many applications, including count-valued responses. We thus introduce a novel subclass of CRFs, derived by imposing node-wise conditional distributions of response variables conditioned on the rest of the responses and the covariates as arising from univariate exponential families. This allows us to derive novel multivariate CRFs given any univariate exponential distribution, including the Poisson, negative binomial, and exponential distributions. Also in particular, it addresses the common CRF problem of specifying "feature" functions determining the interactions between response variables and covariates. We develop a class of tractable penalized M -estimators to learn these CRF distributions from data, as well as a unified sparsistency analysis for this general class of CRFs showing exact structure recovery can be achieved with high probability.

This interesting paper contributes to the literature
      on CRFs with covariate-dependent structure, making three separate and
      interesting contributions: (a) (Theorem 1) generalizes the
      classical and well-known correspondence between autoregressive Gaussian
      models and multivariate Gaussians to arbitrary. The interesting and
      subtle bit here is the dependence of equations (2) and (3) on X. We
      started in (1) by allowing the autoregressive weights to depend
      arbitrarily on X and Y\s, but by making an additional CRF assumption on
      P(Y|X) we find that these weights factorize into a terms that depends
      on X and the sufficient statistics of Y_s's neighbours. (b)
      Section 2.1: If we assume an exponential family and graphical model over X
      in addition to our CRF P(Y|X), this implies additional structure in the
      model parameters of the CRF. This is a neat idea. (c) The
      structure from (b) allows one to use a particular l1 regularizer for
      structure discovery, for which we can provide guarantees about learning
      error and structure recovery. Although (a) and (c) do appear to
      follow relatively closely the previous results from Yang et al about MRFs,
      (b) certainly seems more new, as it is specific to conditional models.
      One (perhaps too ambitious) question that I had was regarding the
      marginal distribution P(Y_s | X). If the node conditionals are Gaussian,
      then of course these marginals are Gaussian; similarly if the node
      conditions are fully-parameterized discrete distributions, then the node
      marginals are also in the same exponential family. I don't suppose that
      this property would hold for other exponential families? I have
      only minor suggestions for improvement: * I'm not sure that the
      Introduction sells the contribution of the paper as clearly as it could,
      as it seems to focus on having different exponential families in the
      autoregressive structure, where it seems to me the real modelling
      contribution is the interplay among the autoregressive weights and the
      sufficient statistics of X. * The real data example in Section 4
      seems like it is probably a nice example of the modelling benefits of
      having covariate-dependent structure, although this is almost impossible
      to understand for readers who are not already familiar with the biology in
      this example. It would be nice to expand the discussion of these results
      for readers coming from a more general ML background, althought I
      understand that this could be difficult given space constraints. *
      Theorem (1): The statement of the theorem is fairly long, and the phrase
      "These assertions are consistent" is a bit less precise than I'd perhaps
      like. I wonder if it's possible to introduce some terminology that
      makes clauses (a) and (b) shorter and therefore the theorem easier to
      understand. * Equation (2): I am assuming that if the functions
      B_s return a vector, then the product should be interpreted elementwise?
      * line 157: B_s(Y_s) = Y_s. Surely Y_s^2 needs to be a sufficient
      statistic somehow? * line 298: This is a *pseudo*-likelihood, yes?
      * line 316: Are the constants M_1 and M_2 computable? Is there a
      simple expression for them? Are these constraints always satisfiable?


Phase Retrieval using Alternating Minimization
Phase retrieval problems involve solving linear equations, but with missing sign (or phase, for complex numbers). Over the last two decades, a popular generic empirical approach to the many variants of this problem has been one of alternating minimization; i.e. alternating between estimating the missing phase information, and the candidate solution. In this paper, we show that a simple alternating minimization algorithm geometrically converges to the solution of one such problem -finding a vector x from y, A, where y = |A T x| and |z| denotes a vector of element-wise magnitudes of z -under the assumption that A is Gaussian. Empirically, our algorithm performs similar to recently proposed convex techniques for this variant (which are based on "lifting" to a convex matrix problem) in sample complexity and robustness to noise. However, our algorithm is much more efficient and can scale to large problems. Analytically, we show geometric convergence to the solution, and sample complexity that is off by log factors from obvious lower bounds. We also establish close to optimal scaling for the case when the unknown vector is sparse. Our work represents the only known theoretical guarantee for alternating minimization for any variant of phase retrieval problems in the non-convex setting.

This paper presents a simple modification of the
      alternating minimization algorithm for phase retrieval and analysis to
      prove its convergence. In phase retrieval the measurement model is based
      on a linear system y = A x where the common term y (the measurements)
      is known up to the sign or phase, A is given and one needs to find x.
      The modification lies in the initialization. Prior alternating
      minimization algorithms used a random guess for the reconstructed
      signal x. The proposed modification is to use the main singular vector
      of a matrix built with a subset of columns of A and the corresponding
      entries of the vector y. Then, at each iteration of the alternating
      minimization a new subset of the columns of A and the corresponding
      entries of y is used until no columns are left. The analysis shows
      that this algorithm can converge to the correct solution within a given
      error with high probability when the measurements y_i are iid standard
      complex Gaussians random variables. A second result applies to the
      case of phase retrieval problems where the unknown signal is sparse.
      The authors propose to first identify the nonzero entries by a greedy
      algorithm and then suggest to run the previous phase algorithm only on
      the nonzero entries. They also characterize the probability that the
      greedy algorithm finds the correct support of the signal. Quality
      -- I have tried to verify the proofs in the paper, but it is not
      feasible to review 17 pages of proofs in the supplementary material...
      this is more than 3 papers in one. I have looked at some proofs, but I
      can't see how the authors do some steps. For example, in lines 308-9 I
      could not verify the inequalities. Besides, the sentence in line 311
      does not make much sense. I also believe that the solution in C at lines
      190-1 is missing y and the expression for S at line 199 does not match
      that in Algorithm 1 at line 1. Probably a square of y is missing.
      Even if all proofs were correct, what is missing to me is how
      practical this strategy is. Essentially we have a linear system where
      the phase/sign of the measurements is lost. The idea is to split this
      (non)linear system into a collection of (non)linear systems such that
      each system has enough equations to yield a unique solution and then
      one can use each system sequentially to arrive at the final solution.
      The question is then: How many equations does one need to achieve a
      reasonable convergence with reasonably high probability? To guarantee
      convergence with high probability one needs a very small \eta in Thm
      4.4 and this constraints the constant c which in turn provides a lower
      bound for the number of measurements m. The bound is m > c n (log^3
      n + k) where k depends on the convergence error and n is the number of
      entries in the unknown signal. Even with modest c and k values the
      number of equations m might grow very quickly compared to n. In
      practice this might make the algorithm quite impractical. We can see
      in sec 6 at lines 428-9 that a modest m = 6n is used. However, do the
      constant vary much with real data? I would have liked to see a
      discussion of the authors on this aspect. Clarity --
      The paper is generally well-written and makes an effort in explaining
      the general idea either before or after a result has been presented.
      However, I have still found it quite heavy. Also, please provide
      all the information needed to replicate your experiments.
      Originality -- The proposed analysis is novel in
      my opinion. The algorithm perhaps not. Significance --
      The fact that this simple modification of the alternating minimization
      algorithm converges might be useful also to similar problems. I
      suppose that other researchers might try this algorithm on their
      problems.


Real-Time Inference for a Gamma Process Model of Neural Spiking 1
With simultaneous measurements from ever increasing populations of neurons, there is a growing need for sophisticated tools to recover signals from individual neurons. In electrophysiology experiments, this classically proceeds in a two-step process: (i) threshold the waveforms to detect putative spikes and (ii) cluster the waveforms into single units (neurons). We extend previous Bayesian nonparametric models of neural spiking to jointly detect and cluster neurons using a Gamma process model. Importantly, we develop an online approximate inference scheme enabling real-time analysis, with performance exceeding the previous state-of-theart. Via exploratory data analysis-using data with partial ground truth as well as two novel data sets-we find several features of our model collectively contribute to our improved performance including: (i) accounting for colored noise, (ii) detecting overlapping spikes, (iii) tracking waveform dynamics, and (iv) using multiple channels. We hope to enable novel experiments simultaneously measuring many thousands of neurons and possibly adapting stimuli dynamically to probe ever deeper into the mysteries of the brain.

This paper introduces a Bayesian nonparametric method
      to address the problem of spike detection and sorting (i.e. for jointly
      detecting neural spikes and assigning them to neurons). These steps are
      naturally of critical importance to many computational models of neural
      spiking data as they essentially create the 'ground truth' data on which
      many models operate. Improving these methods thus is of great interest to
      the wider computational neuroscience community. The authors develop an
      efficient online method based on the gamma process that models both the
      waveforms of spikes for each neuron and the assignment of spikes to
      neurons. The use of a Chinese restaurant process elegantly allows the
      model to add new neurons (i.e. the number of neurons from which spikes are
      detected does not need to be specified a priori). There are some neat
      features to the model, such as the ability to model non-stationary
      waveforms (i.e. varying over time). The authors perform an empirical
      comparison to previous state of the art approaches that provides a
      compelling case for their method and provide an interesting discussion.
      Quality: The authors derive a model that is conceptually quite
      complex although neat and appears correct. The novel concepts, methods and
      empirical analysis are of high quality. The writing should be improved on
      (this perhaps falls under the clarity category) particularly to clarify
      the model, although this can be done for a camera ready. Clarity:
      As stated above, the model is rather complex. The authors try to
      explain the model by starting with simpler models (i.e. a homogenous
      Poisson process) and then in a step by step fashion add complexity and
      draw connections to various distributions until they arrive to what their
      model actually is. I believe this is an attempt to explain their approach
      to the reader in an accessible way, but I feel it is somewhat confusing.
      Rather than draw parallels to the entire Bayesian nonparametrics toolbox,
      it would be better in my opinion to state what the actual model is and
      then explain the assumptions taken to arrive at this model. I think this
      is also an opportunity to reclaim a significant amount of space (which is
      clearly scarce here) which can be used to put in a diagram (graphical
      model) of the model(s) introduced in this paper. Originality:
      This appears to be original work and addresses some shortcomings of
      previous approaches. There are similar approaches to spike sorting (e.g.
      the Dirichlet process mixture models for spike sorting of Gasthaus et
      al.). However, this paper makes significantly more than delta changes to
      those works in the model, in how it adapts the waveforms and performs real
      time spike sorting using variational Bayesian methods.
      Significance: As stated above, spike sorting and detecting are
      two critical early steps in the computational neuroscience modeling
      pipeline. Thus improving accuracy would be very useful to many
      computational models which analyse the data resulting from these steps and
      improving speed can significantly help achieve real time decoding and move
      towards larger populations of neurons. Detailed comments: 48:
      inference -> infer 52: develop -> developing 58: that that
      77: It sounds a bit strange to say that you 'relax' simplifying
      assumptions - i.e. sounds like you make them simpler 79: as -> are
      90: Rather that -> Rather than 163: The mean and covariance are
      drawn from a Wishart distribution, of which the parameters are drawn from
      the CRP correct? Perhaps you should add another line in the equation array
      to make this clear. 183: Twiddles is not a word I am familiar with.
      Tildes? 184: Associate -> Associated 185: in -> is 197:
      the subvector 209: I really appreciate Algorithm 1 - I think this
      should be in the main text if you can find space 224: case a ->
      case of a 424: being estimates? 428: "embarassingly" parallel
      sounds rather colloquial and sarcastic The paper states that code
      is provided in the supplementary materials to run the models introduced in
      the paper. I would like to run this code to verify but I can't find it in
      the supplementaries. A reference that motivates the importance of
      waveform non-stationarity. G. Santhanam, M. D. Linderman, V. Gilja, A.
      Afshar, S. I. Ryu, T. H. Meng, and K. V. Shenoy. HermesB: A continuous
      neural recording system for freely behaving primates. IEEE Transactions on
      Biomedical Engineering, 54(11):2037–2050, 2007. One concern of
      this reviewer in the nature of the empirical evaluation is that the ground
      truth is from only on a single neuron from a single data set. Naturally, I
      sympathize that obtaining such data is rather difficult and we are lucky
      that even this ground truth data exists and is provided online. However,
      it does raise a number of concerns - is it possible that we can overfit as
      a community if we all attempt to achieve state of the art on this single
      neuron. Furthermore, what is the guarantee that the assumptions made in
      this work hold for this particular neuron (or a particular class) but
      don't for others (e.g. we know different classes of neuron exhibit very
      different patterns of behavior). It may be useful to state that
      [23] compares empirically to a number of models that your method doesn't
      directly compare to such as [27]. So although not comparing directly to
      that work you are more or less comparing by
      proxy.


B-tests: Low Variance Kernel Two-Sample Tests
A family of maximum mean discrepancy (MMD) kernel two-sample tests is introduced. Members of the test family are called Block-tests or B-tests, since the test statistic is an average over MMDs computed on subsets of the samples. The choice of block size allows control over the tradeoff between test power and computation time. In this respect, the B-test family combines favorable properties of previously proposed MMD two-sample tests: B-tests are more powerful than a linear time test where blocks are just pairs of samples, yet they are more computationally efficient than a quadratic time test where a single large block incorporating all the samples is used to compute a U-statistic. A further important advantage of the B-tests is their asymptotically Normal null distribution: this is by contrast with the U-statistic, which is degenerate under the null hypothesis, and for which estimates of the null distribution are computationally demanding. Recent results on kernel selection for hypothesis testing transfer seamlessly to the B-tests, yielding a means to optimize test power via kernel choice.

The authors present an interesting contribution of a
      block splitted MMD based two-sample test. The authors give a very good and
      clear motivation in the introduction. It is proposed and described with
      respect to the two extremes of MMD_u and MMD_l. A recommendation for the
      block size B is given. Besides the theoretical contribution, the method is
      convincingly illustrated on several data sets. The work can be promising
      e.g. towards big data. On the other hand, with respect to the
      bootstrap, also block based methods have been proposed: A scalable
      bootstrap for massive data. A. Kleiner, A. Talwalkar, P. Sarkar and M. I.
      Jordan. arXiv:1112.5016, 2012. It would be good to position the
      work also with respect to these results. When reading the abstract
      it is not immediately clear what the authors mean by B-test. I would
      suggest to already explain in this early stage that B stands for block
      splitting. Currently, one expects from the name B-test in the title and
      abstract that is is a completely new approach, which is not really the
      case. typo: Komogorov should be Kolmogorov When taking
      sigma = 1, isn't a suitable scaling of the data assumed here? please
      explain. Eq 13: shouldn't this be rounded to the nearest integer?
      The choice sigma \in {-15, ..., 10} is unclear to me, what do you
      mean and why negative values? Is this choice crucial or not?
      Please explain in more detail what you mean by multiple kernels.
      The authors have carefully answered to the questions. I have
      updated my score.


Geometric optimisation on positive definite matrices with application to elliptically contoured distributions
Hermitian positive definite (hpd) matrices recur throughout machine learning, statistics, and optimisation. This paper develops (conic) geometric optimisation on the cone of hpd matrices, which allows us to globally optimise a large class of nonconvex functions of hpd matrices. Specifically, we first use the Riemannian manifold structure of the hpd cone for studying functions that are nonconvex in the Euclidean sense but are geodesically convex (g-convex), hence globally optimisable. We then go beyond g-convexity, and exploit the conic geometry of hpd matrices to identify another class of functions that remain amenable to global optimisation without requiring g-convexity. We present key results that help recognise g-convexity and also the additional structure alluded to above. We illustrate our ideas by applying them to likelihood maximisation for a broad family of elliptically contoured distributions: for this maximisation, we derive novel, parameter free fixed-point algorithms. To our knowledge, ours are the most general results on geometric optimisation of hpd matrices known so far. Experiments show that advantages of using our fixed-point algorithms.

Summary of the paper The paper proposes a
      framework for optimizing (potentially non-convex) functions on the space
      of hermititian positive definite matrices. This framework makes use of
      convexity, with respect to geodesics, to empirically estimate the
      parameters of elliptically contoured distributions based on the
      minimization of the negative log-likelihood. For this sake, a fixed point
      algorithm is derived as well as its convergence guarantees. Theoretical
      tools that help to recognize such geodesical convex functions are given.
      Finally, numerical experiments illustrate proposed algorithm.
      Comments This paper propose promising results for the
      optimization of functions subject to Hermitian Positive Definite (HPD)
      constraints. However the paper has a strong optimization and discrete
      machine learning flavor and is a bit tedious to read. I would recommend to
      modify the paper such that the machine learning aspect of this work is
      pushed forward. Seemingly, the work fits more the topics of a pure
      optimization conference. Some nice results about geodesic
      convexity and log-nonexpansive functions are stated. Those results are
      used to build a fixed-point algorithm. Even if the derivation seems
      trivial to the authors, it would be helpful to make explicit the
      optimization algorithms derived from the theoretical results.
      Numerical experiments illustrating the effectiveness of proposed
      algorithms need to be improved. Indeed, comparison with the matlab
      function "fmincon" looks impressive, but it would be more interesting to
      consider state of the art methods such as the gradient (or hessian) based
      methods developed in the context of optimization over manifolds (see for
      instance the work "Optimization algorithm on matrix manifolds" by Absil et
      al.). Comparison to such competitive approaches in terms of convergence
      speed of the objective function or of the iterates would be appreciable.
      Besides these experimental evaluations the paper lacks an analysis of the
      algorithm convergence rate. Finally, there are some comments about
      the notations. It is somewhat confusing to refer to HPD matrices both as a
      cone or as a manifold. Indeed, in those two cases, the implied geometrical
      structures are different.


Reconciling "priors" & "priors" without prejudice?
There are two major routes to address linear inverse problems. Whereas regularization-based approaches build estimators as solutions of penalized regression optimization problems, Bayesian estimators rely on the posterior distribution of the unknown, given some assumed family of priors. While these may seem radically different approaches, recent results have shown that, in the context of additive white Gaussian denoising, the Bayesian conditional mean estimator is always the solution of a penalized regression problem. The contribution of this paper is twofold. First, we extend the additive white Gaussian denoising results to general linear inverse problems with colored Gaussian noise. Second, we characterize conditions under which the penalty function associated to the conditional mean estimator can satisfy certain popular properties such as convexity, separability, and smoothness. This sheds light on some tradeoff between computational efficiency and estimation accuracy in sparse regularization, and draws some connections between Bayesian estimation and proximal optimization.

This paper establishes that all Bayes estimates of z
      under squared error loss (i.e., posterior mean) in the model y ~
      N(Az, \Sigma) z ~ P correspond to MAP estimates (i.e.,
      penalized regression) which optimize ||y - Az||_2^2 + \phi(z)
      for some \phi depending on A, P, and \Sigma. This is an extension
      of Gribonval (2012), which established this result in the case of A =
      \Sigma = Id. The author also characterizes the Bayesian problems for which
      \phi ends up being a convex regularizer. Although the proofs are
      entirely straightforward given Gribonval (2012), the result is
      interesting, and the extension to A non-orthogonal is significant (the
      extension to correlated errors is less interesting). Furthermore, the 2012
      paper was published in the IEEE Trans. on Signal Processing, and I think
      that this result would be of interest to the machine learning community, a
      large segment of which does not follow the signal processing literature.
      However, I would note a few things: the fact that the prior \phi
      depends on A is somewhat unsatisfactory, as it suggests the prior somehow
      depends on the likelihood. Also, there seem to be many typos, and I hope
      the author will carefully proofread the manuscript before the final
      submission. I also think that the author should be careful in saying that
      "sparsity inducing penalties are necessarily suboptimal" in Section 3.1.
      They are only suboptimal if you assume the truth has to be generated from
      some prior! Given the wealth of literature around the optimality of L1
      regularization over linear estimates (which do correspond to an MMSE
      estimator) over certain function classes, I think this is a dangerous
      statement to make, even if it is qualified by "in a mean squared error
      sense". Editorial suggestions: - The term is "quid pro quo,"
      not "qui pro quo." - Line 76: "A qui pro quo been these..."? -
      Line 88: "ping" should be "pinv" - Line 303-4: should be "in terms of"
      - Line 312: should be "one of the reasons" - Line 414:
      "investigating other choices"


New Subsampling Algorithms for Fast Least Squares Regression
We address the problem of fast estimation of ordinary least squares (OLS) from large amounts of data (n p). We propose three methods which solve the big data problem by subsampling the covariance matrix using either a single or two stage estimation. All three run in the order of size of input i.e. O(np) and our best method, Uluru, gives an error bound of O( p/n) which is independent of the amount of subsampling as long as it is above a threshold. We provide theoretical bounds for our algorithms in the fixed design (with Randomized Hadamard preconditioning) as well as sub-Gaussian random design setting. We also compare the performance of our methods on synthetic and real-world datasets and show that if observations are i.i.d., sub-Gaussian then one can directly subsample without the expensive Randomized Hadamard preconditioning without loss of accuracy.

The paper compares three very simple algorithms for
      approximating OLS using less than O(np^2) computation, "full subsampling",
      "covariance subsampling" and "uluru". Each method uses the same input
      covariance matrix estimate based on an r-subsample, but handles the
      input-output co-variance slightly differently. Both fixed and random
      designs are considered. The approximate algorithms can be made to run in
      O(np), ie p times faster the OLS. The paper concludes with an experimental
      section with simulated and realistic data. The main thrust of the
      paper is the analysis, showing how the error depends on various
      quantities, and I think is is a pity, or misjudgement not to include the
      analysis in the paper (three theorems a stated in the paper, but all
      analysis is only in the supplementary material). The motivation is
      a bit unclear. What types of data are these methods suitable for? All the
      data sets shown are so small, that exact OLS can be performed without
      difficulty. If there indeed are problems of practical interest out there
      with sufficiently large n and p, that exact OLS cannot be done
      conveniently, then why don't you present a few of these? Also, it is not
      quite clear to me that the data sets presented really do lie solidly on
      the n and p ranges needed by the theory. The main result for uluru
      requires O(p^3) < n << exp(p), is this really fulfilled in the
      data presented? The plots in figure 1 (main experimental results)
      are poor. The logarithmic y-axis spans from 1e-29 to > 1e6. Most of
      this scale extending over 35 orders of magnitude is not of practical
      interest; chosing such a large scale compresses the relevant region into
      such a small space that it becomes illegible. It is


Dropout Training as Adaptive Regularization
Dropout and other feature noising schemes control overfitting by artificially corrupting the training data. For generalized linear models, dropout performs a form of adaptive regularization. Using this viewpoint, we show that the dropout regularizer is first-order equivalent to an L 2 regularizer applied after scaling the features by an estimate of the inverse diagonal Fisher information matrix. We also establish a connection to AdaGrad, an online learning algorithm, and find that a close relative of AdaGrad operates by repeatedly solving linear dropout-regularized problems. By casting dropout as regularization, we develop a natural semi-supervised algorithm that uses unlabeled data to create a better adaptive regularizer. We apply this idea to document classification tasks, and show that it consistently boosts the performance of dropout training, improving on state-of-the-art results on the IMDB reviews dataset.

*Summary of the paper: This paper studies
      "dropout training" in the framework of feature noising as a
      regularization. It is know that adding an additive gaussian noise
      to the feature is equivalent to an l_2 regularization in a least square
      problem (Bishop). This paper studies multiplicative Bernoulli
      feature noising, in a shallow learning architecture, with a general loss
      function and shows that it has the effect of adapting the geometry through
      an "l_2 regularizer " that rescales the feature (beta^{\top} D(beta,X)
      beta). The Matrix D(beta,X) is a estimate of the inverse diagonal
      fisher information. It is worth noting that D does not depend on the
      labels. The equivalent regularizer of dropout is non convex in general.
      A connexion to AdaGrad in online learning introduced in Duchi et
      al is made, as both approaches reduces to adapting the geometry towards
      rare but predictive features. The Matrix D(beta,X) could be
      estimated using unlabeled data, authors devise a semi supervised variant
      of the equivalent dropout regularizer. Comments: -
      Dropout as originally introduced by hinton et al considers multiplicative
      Bernoulli noising for deep learning architectures (neural nets). An
      average neural net is obtained, by training independently many neural nets
      where some hidden layers were dropped out at random. Hinton's dropout
      seems to be close to ensemble methods (boosting, bagging etc.), and a
      different analysis is required to understand this paradigm even with one
      layer. Referring to Hinton's dropout in the introduction is a bit
      misleading as there is no models averaging in the current paper. It
      would be more precise to mention that this work consider another form of
      dropout, where we learn one model robust to Bernoulli multiplicative
      perturbation. - The regularizer is not an l_2 regularizer as D
      depends on beta also in a non linear way through the hessian of the
      likelihood. one should be careful with this appellation. - It is
      interesting to derive the approximate equivalent regularizer, to
      understand the effect of dropout. The non convexity of the regularizer
      poses computational issues. Authors mention in a footnote that they use
      lbfgs, more explanation and discussion are needed to understand how to
      avoid local minimas. The non convexity of the equivalent regularizer
      is not discussed in section 6. - The parameter (delta/1-delta)
      corresponds to a regularization parameter. any insight on how to choose
      delta? according to Hinton et al , delta=1/2 seems to have a good
      performance. - The log partition function is sometimes hard to get
      in closed from any insight on how to go around that? Authors
      answered promptly questions raised by the reviewer.


The Fast Convergence of Incremental PCA
We consider a situation in which we see samples X n ∈ R d drawn i.i.d. from some distribution with mean zero and unknown covariance A. We wish to compute the top eigenvector of A in an incremental fashion -with an algorithm that maintains an estimate of the top eigenvector in O(d) space, and incrementally adjusts the estimate with each new data point that arrives. Two classical such schemes are due to Krasulina (1969) and Oja (1983). We give finite-sample convergence rates for both.

This paper theoretically analyse the Oja’s algorithm
      for estimating the first principal component. Convergence properties and
      relation to stochastic gradient descent have been discussed. Overall, this
      paper is not well written and should not be accepted in the current form.
      Detail comments are: 1. All theoretical analyses are based
      on the potential functions defined in line 109-111. There is no citation
      for this potential function and it is not explained why this function is
      defined in this form. Using the norm of the difference between v_n and the
      optimal first principal component is a more standard way to analyse the
      convergence rate. The author should explain why it is not adopted in this
      paper. 2. The assumption of the example in section 3 is too
      strong. It is inappropriate to assume all data points are drawn from
      eigenvectors of the covariance matrix. It is only a special case which
      cannot be used as a fact to analyse the initialization situation and the
      further theoretical proofs in Section 4 and Section 5. 3. In line
      153-155, “if v_n is orthogonal to some e_i, it will remain so forever” is
      not true, because x_n^Tv_{n-1} is not always 0 and v_n is not always
      equals to v_{n-1}. x_n^Tv_{n-1} = 0 only when x_n is e_i. However, x_n
      could be e_j, where j\neq i, according to the construction of the data
      points. 4. The submitted version is not compiled properly which
      has many question marks when citing Appendix. The supplementary file has
      the right citation. However, the correctness of proofs is still hard
      to be verified because of the extremely poor organization of the
      structure. For example: The proof of Theorem 4.1 ->A1->Theorem
      5.2->E.1->C.1->F.1-F.3, where “->” means “refers to”. It
      is impossible to follow unless using a clear structure. 5. This
      paper cites too many arxiv papers. Most of these papers have been
      published on peer reviewed journals or conferences. Please cite the peer
      reviewed version. 6. There are many grammar mistakes which make
      the paper difficult to read. For example, the section paragraph in
      Introduction is not readable.


Aggregating Optimistic Planning Trees for Solving Markov Decision Processes
This paper addresses the problem of online planning in Markov decision processes using a randomized simulator, under a budget constraint. We propose a new algorithm which is based on the construction of a forest of planning trees, where each tree corresponds to a random realization of the stochastic environment. The trees are constructed using a "safe" optimistic planning strategy combining the optimistic principle (in order to explore the most promising part of the search space first) with a safety principle (which guarantees a certain amount of uniform exploration). In the decision-making step of the algorithm, the individual trees are aggregated and an immediate action is recommended. We provide a finite-sample analysis and discuss the trade-off between the principles of optimism and safety. We also report numerical results on a benchmark problem. Our algorithm performs as well as state-of-the-art optimistic planning algorithms, and better than a related algorithm which additionally assumes the knowledge of all transition distributions.

---Response to Author Response--- Thank you
      for the clarifications. I hope the authors will consider the following
      points: - I'm glad the authors plan to add more comparative
      results, as they are much needed, but I still see no reason not to compare
      to UCT. I appreciate the value of comparing to algorithms with similar
      theoretical properties, but that does not preclude also comparing to the
      current go-to algorithm in this problem setting. If ASOP does better than
      UCT in one or more examples, then that is compelling evidence for ASOP's
      practical applicability as well as its theoretical desirability. If UCT
      does better then we get a sense of the magnitude of the practical cost of
      ASOP's desirable guarantees (and can compare that to other algorithms that
      make similar promises). Neither outcome diminishes ASOP's contribution and
      either way the paper creates more knowledge for very little extra effort
      or page space. - I appreciate the clarification of the
      relationship between the regret bounds for ASOP and UCT; I hope future
      versions of this paper will make a similar direct comparison. - I
      may be missing something, but I'm not sure I see why it would be difficult
      to apply Coquelin and Munos' algorithms to your planning problem.
      "Modified UCT" is essentially just UCT except the exploration bonus is
      smaller in deeper nodes and it enjoys a singly exponential regret bound
      with high probability. In any case, the point is just that the paper makes
      prominent mention of UCT's unfortunate worst case bound, but does not
      clearly situate ASOP amongst other algorithms in this setting that also
      improve upon UCT's worst case behavior (even those that are explicitly
      cited!). - I now understand better what was meant by "good
      approximation to the MDP." For me personally, the S3 tree being a good
      approximation to the MDP suggests that the tree is a close approximation
      to the MDP model. It sounds like the real point is that the *value
      function* if the tree is a close approximation to the value function of
      the MDP. I think it would not be difficult to make this wording more
      clear/specific in the paper. - The clarification regarding the
      best-first expansions helps, and I hope similar clarity will be added to
      the paper. I personally would find it informative to see an empirical
      comparison to ASOP with only breadth-first expansions as an illustration
      of the relative importance of the "optimism" in the algorithm, but if the
      role of the best-first expansions is clarified in the text, I don't think
      it would be critical to add this. ---Summary of paper---
      The paper presents an online planning algorithm for large MDPs.
      The main idea is to sample several determinations of the system in the
      form of roll-out trees where each state/action pair has only one sampled
      successor. A combination of breadth-first and best-first search is used to
      explore the deterministic trees, and then they are recombined to create a
      stochastic model from which a policy can be calculated. The algorithm is
      proven to be consistent (as the number of trees and number of nodes in
      each tree both approach infinity, the value at the root can be arbitrarily
      approximated with high probability). The algorithm is empirically compared
      to an planning algorithm that requires a full transition model and
      performs well in comparison. ---Quality--- The analysis
      provided is, as far as I can tell, correct. I am intrigued by the overall
      approach, and particularly like the intuition that the S3 trees are
      "off-policy Monte Carlo" samples. My main concern is with the evaluation
      of the algorithm. The strengths and weaknesses of this algorithm in
      comparison to existing approaches to this problem are not very well
      explored. The shortcomings of UCT are given as primary motivators
      for this work, and yet ASOP is not empirically compared to UCT (or to
      "Modified UCT," the algorithm given in [4] with a singly exponential
      regret bound with high probability). Also, the regret analysis given here
      is in terms of the simple regret, but the worst case analysis in [4] is
      not, making it difficult to compare the two to get a sense of the extent
      of the improvement (if any) in the bounds. The regret in [4] is related to
      the number of visits to sub-optimal leaves, and, because it is performing
      breadth-first search, it seems straightforward to force ASOP to visit
      exponentially many sub-optimal leaves. So, it is not clear whether this
      algorithm's theoretical performance guarantees represent an improvement
      over existing algorithms. The empirical comparison to OP-MDP seems
      strange, since it is not even intended for the same problem setting. The
      fact that OP-MDP requires a full transition model seems to limit the
      experiment to smallish MDPs, which is counter to the point of ASOP, isn't
      it? The argument seems to be implied that, because OP-MDP requires access
      to the transition dynamics, it should have an advantage but, having never
      heard of it before, I have no idea what kind of performance to expect from
      OP-MDP nor what to make of the fact that ASOP does better. I can't even
      tell if either algorithm is doing objectively well, since I don't know
      what optimal performance would look like in this problem. In all, I did
      not feel this was an effective or informative empirical demonstration.
      The authors should correct me if I am wrong, but it didn't seem
      like the optimistic part of the algorithm actually played any role in the
      regret bound. Presumably, then, this part of the algorithm is there to
      improve empirical performance? If that is the case, it seems like it
      should be better evaluated. How much do the best-first expansions really
      help performance? Is it possible for them to hurt performance? Are there
      another expansion strategy one could consider combining with breadth-first
      search? In the conclusions section it is suggested that ASOP
      should perform best when single-successor trees are good approximations to
      the original MDP, but I don't see much indication of this in the theory or
      the experiments. Conceptually, when I think about the extreme case of a
      deterministic environment, it seems like ASOP would either waste its time
      creating multiple identical trees or, if it determinism is known a priori,
      only create one tree but simply perform simultaneous breadth-first and
      best-first search, which in a lot of cases has done much worse than Monte
      Carlo tree search algorithms. So I think it would help to have some more
      justification of this statement. ---Clarity--- For the
      most part I found the paper to be clear, and the ideas accessible. I
      appreciate that the high level idea was well-laid out before diving into
      the details, and that each component of the framework was placed clearly
      in context in the larger algorithm. I also found the proofs to be mostly
      easy to follow with a good mix of intuitive and formal argument (though
      there were some parts that tripped me up, listed below). - Eq. 1
      would be more clear if it was "d = …" - The proof of Lemma 2 ends
      in "< \epsilon", but should be "= \epsilon", so the lemma statement
      should be "<= \epsilon" (adjustments also needed in later statements).
      - Shouldn't the exponent in Eq. 3 be ceil[log(\epsilon/4(1 -
      \gamma))(log \gamma)^{-1}] + 1? I'm not sure where some of those -1s came
      from. - Figure 3 is never discussed. What are we meant to learn
      from it? There are also several type-os and grammatical mistakes
      throughout, so the authors should be sure to take some more
      editing/polishing passes. ---Originality--- The
      algorithm's conceptual connections to existing work are clearly laid out.
      I believe the algorithm and its analysis to be novel, and the general
      approach seems interesting. ---Significance--- The
      presented algorithm does seem like a new approach to online planning in
      large domains. It's possible that this approach will turn out to yield
      significant advances in the state of the art. However, at the end of this
      paper, I did not feel I had a sense of where this algorithm fits in with
      existing techniques. It's not that I think every paper needs to be a horse
      race ending with "Our algorithm is better than that other algorithm." I do
      think that a new approach, even if it does not immediately compete with
      existing methods, might still be interesting and valuable to have in the
      literature. However, there has to be some indication of what that approach
      brings to the table that is not already there, some indication of what
      promise it may hold. In this paper, ASOP is presented largely as a novel
      algorithm in a vacuum, without any comparative analysis (whether
      theoretical, empirical, or even intuitive). In my opinion that
      substantially hurts the potential impact of the
      work.


A New Convex Relaxation for Tensor Completion
We study the problem of learning a tensor from a set of linear measurements. A prominent methodology for this problem is based on a generalization of trace norm regularization, which has been used extensively for learning low rank matrices, to the tensor setting. In this paper, we highlight some limitations of this approach and propose an alternative convex relaxation on the Euclidean ball. We then describe a technique to solve the associated regularization problem, which builds upon the alternating direction method of multipliers. Experiments on one synthetic dataset and two real datasets indicate that the proposed method improves significantly over tensor trace norm regularization in terms of estimation error, while remaining computationally tractable.

**** Updated after authors' feedback **** My main
      concerns are still that - the improvement compared to L1 (or trace
      norm) comes from the discontinuousness at the boundary of the domain,
      where the solution almost never lies (as the authors point out). - on
      the other hand, probably the proposed regularizer agrees with the L1 if
      ||w||_1 ≤ alpha, because it agrees with L1 on all the points on the L2
      ball that have cardinality one, and the convex envelope of those points
      defines the ball ||w||_1 ≤ alpha.
      ***************************************** This paper
      proposes a new convex relaxation of the tensor multi-linear rank based on
      the convex envelope of the cardinality function intersected with an
      L2-norm ball of some radius alpha. The proposed convex relaxation
      is in the line of previous studies [7,15,19,20,22] that studied trace norm
      of the unfoldings (matricizations) of a tensor along different modes. The
      current paper points out that the tensor trace norm used in previous
      studies is not the tightest convex relaxation with respect to the
      Frobenius norm (L2 norm of the spectrum). To fix this, the authors apply
      the convex envelope of the cardinality function intersected with an
      L2-norm ball to the spectra of the unfoldings of a given tensor.
      The authors show that the proposed convex envelope (for
      cardinality) is tight for any vector having the prespecified radius alpha.
      The trace norm is the tightest convex relaxation of the rank in terms of
      the spectral norm. The authors argue however that the spectral norm is not
      invariant to the choice of the mode to unfold a given tensor, nor is the
      tightness of trace norm. On the other hand, the Frobenius norm is
      invariant, so is the tightness of the proposed convex relaxation.
      Some numerical experiments show that the proposed convex
      relaxation outperforms tensor trace norm in tensor completion.
      Strength: - The proposed convex relaxation is new and it seems
      to work. - The idea might be used for inducing sparsity in general.
      Weakness: - The result of Lemma 1 seems applicable to other
      sprasity problems (sparsity, group sparsity, low-rank matrices). The
      authors should clarify how much of the contribution is specific to
      low-rank tensors. - The authors do not prove that the proposed
      regularizer is the tightest convex relaxation for the combinatorial
      regularizer Omega(W) (they only show the tightness of the convex envelope
      function used to construct the regularizer in Lemma 1 and it is *tighter*
      than the trace norm in Prop 2). - The proposed convex relaxation seems
      rather hard to compute. It is infinity for ||x|| > alpha and it has
      many discontinuities for ||x||=alpha. This probably makes the optimization
      and also proving something about the estimator hard. - It would be
      nice if the authors could discuss any connection between this work and the
      k-support norm proposed in Argyriou et al. NIPS 2012) Minor
      comments: - The statement "if F_alpha(W) ≤ Omega(W) for every W\in G_2
      then F_alpha(W) ≤ Omega(W) for every G_infty." (line 138) seems false,
      because G_2 is a smaller set than G_infty. - The choice of alpha as in
      line 321 may not be the best, because the proposed regularizer is infinity
      for ||W||_F > alpha. It should be chosen more carefully. - It was
      not clear how tight the proposed regularizer is when ||W||_F < alpha. I
      guess it equals the trace norm when ||W||_tr ≤ alpha (Correct me if I am
      wrong). - What kind of performance guarantee can you prove for this
      regularizer? How does that compare to Tomioka et al. "Statistical
      Performance of Convex Tensor Decomposition" (NIPS
      2011)?


Learning the Local Statistics of Optical Flow
Motivated by recent progress in natural image statistics, we use newly available datasets with ground truth optical flow to learn the local statistics of optical flow and compare the learned models to prior models assumed by computer vision researchers. We find that a Gaussian mixture model (GMM) with 64 components provides a significantly better model for local flow statistics when compared to commonly used models. We investigate the source of the GMM's success and show it is related to an explicit representation of flow boundaries. We also learn a model that jointly models the local intensity pattern and the local optical flow. In accordance with the assumptions often made in computer vision, the model learns that flow boundaries are more likely at intensity boundaries. However, when evaluated on a large dataset, this dependency is very weak and the benefit of conditioning flow estimation on the local intensity pattern is marginal.

The paper proposes using recent large-scale optical
      flow datasets to learn priors on optical flow. Two types of priors are
      learned: a prior on flow vectors, and a joint prior on intensity and flow.
      GMMs are used for both priors. It is shown that these rich data-driven
      priors outperform previous ad-hoc priors on flow restoration tasks. On the
      other hand, the advantages of these priors for actual optical flow
      estimation are not demonstrated. Overall, this is a nice paper,
      albeit a bit narrow. I am positive on it and support acceptance, in part
      because I work in the area and find some of the observations made in this
      paper to be useful. Pros: - Natural idea, it's good that
      somebody did it, nice to know what the results are. - It's nice that
      the likelihood estimation results translate across datasets. - The
      observation that flow boundaries are not strongly correlated with
      intensity boundaries is very nice and is the highlight of the paper. It's
      not completely clear that the evidence in the paper actually supports
      making this claim broadly and strongly, but the evidence is certainly
      suggestive. This observation is a stimulating addition to the literature.
      I would like to see it published. Cons: - Benefits to
      actual optical flow estimation are not demonstrated, and are potentially
      marginal. The paper's contribution seems to be mostly conceptual rather
      than practical. - With regards to the intriguing results described in
      line 373 and onwards: could this be due to the suboptimal learning
      algorithm? Can we really draw general conclusions from these experiments,
      or should the conclusions be qualified a bit due to the limitations of the
      learning procedure? Or the data? Specific/minor comments:
      - lines 73, 74: "(figure 1b)", "city [5]" - line 82: "GMM's",
      "show that it is" - equation (1): fix parentheses and brackets in
      multiple places - line 355: "of of" -> "of" - lines 300-323 and
      figure 6: I found it hard to follow the text and to see precisely the
      phenomena that are being described in Figure 6. The figure should be
      annotated much better. The phenomena should be visually apparent. I just
      don't see some of the things that are being described in the figure. -
      lines 357-358: what intensity weighting did you use for this model and
      why? - line 424: "assumed priors"


Multiclass Total Variation Clustering
Ideas from the image processing literature have recently motivated a new set of clustering algorithms that rely on the concept of total variation. While these algorithms perform well for bi-partitioning tasks, their recursive extensions yield unimpressive results for multiclass clustering tasks. This paper presents a general framework for multiclass total variation clustering that does not rely on recursion. The results greatly outperform previous total variation algorithms and compare well with state-of-the-art NMF approaches.

The authors study a variant of ratio cut with R
      clusters where the balancing function is biased towards partitions
      where each cluster has the same size. The main contribution of the
      paper is a continuous formulation and an algorithm to optimize the
      criterion directly, whereas previous algorithms are mostly limited to
      recursive splitting. The direct solution of multi-cut problems
      instead of using recursive splitting is an important problem given the
      new developments in finding balanced graph cuts [3,4,5,11,12,18].
      The authors first describe the discrete problem (P) and then
      derive a relaxation of the problem (P-rlx). Opposite to previous
      work [11,12,17,18] for the two cluster case, they cannot show that
      this relaxation is exact, that is the discrete and continuous problems
      are equivalent. While they argue that the relaxation is very good,
      this is unfortunately not necessarily true. If there exists a
      minimizer f=ones_A of the *single* ratio problem which is far better
      than all other ones (in the extreme case A and A^c are disconnected),
      then the solution of (P-rlx) is just f_r=c_r 1_A or f_r=c_r 1_A^c and
      the c_r are chosen such that they sum up to one for every vertex
      (note that rescaling of f_r does not change the objective). Thus
      the condition that F generates a partition is *not* enforced by the
      simplex constraint. In the extreme case where there exists a set A
      which is disconnected, then the optimal value of the relaxation (P-rlx)
      is zero whereas the optimal value of the discrete problem (P) clearly
      need not be zero - thus the relaxation can be arbitrarily bad. A
      more careful discussion is needed rather than just arguing that the
      total variation enforces indicator functions in Section 3.1. - this is
      well-known already in image processing and can be made precise using
      the relation between submodular set functions and their Lovasz
      extensions and was the basis of the previous work [3,4,5,11,12,18] to
      prove that the relaxation is exact. The new claim that the simplex
      constraint enforces that the sets form a partition is not shown and it
      is impossible to show as the above counterexample suggests. The
      algorithm in 4.2. looks similar to previous work and the integration of
      the simplex constraint is rather standard. I cannot see how the estimate
      in Theorem 3 is related to descent of the total energy - the authors argue
      that the ratio B_r^{k+1}/B_r^k tends to one and thus the total energy
      decreases. However, this is not proven anywhere. It is not even clear
      if the algorithm does converge. The experimental results are good
      even though [19] seems to perform better. However, the actual point of
      the paper that MTV leads to better normalized cuts is not documented,
      as just clustering errors are reported - at least for the previous
      methods using recursive splitting [3,11] and MTV one could have shown
      also the normalized cut values. Pro: - direct tackling
      multi-normalized-cut is an important problem - new algorithm -
      good experimental results Con: - relaxation is not tight as
      argued by the authors - it can be arbitrarily bad - no convergence
      proof for the algorithm or even descent is guaranteed
      ------------------------------------------------------------------------------
      Added after author response: Tightness: The suggested
      relaxation is not tight as the example in the review shows - and this
      problem will remain even if the graph is connected and there is one
      dominating cut. In the case where the graph is disconnected the
      suggested relaxation has value zero whereas the original discrete
      problem need not. Also the spectral relaxation would yield a better
      relaxation as it sums up the first k eigenvalues (where k is the number of
      clusters) which would be non-zero (say k=10 and the graph has two
      connected components). This argument holds even for criterion (P) as
      one can easily lower- and upper bound it in terms of (1). The
      argument that the algorithm anyway just finds a solution close to an
      initial starting point is very weak. If this is the case then the
      suggested algorithm is very bad. Moreover, then the problem is
      transferred to finding a good initial solution which is as difficult
      as the original problem and also not discussed in the paper.
      Monotonicity and Convergence: If these results are
      available then it is unclear why they were not put into the
      paper/supplementary material. Anyway even if the algorithm has better
      properties as reported in the paper, the problems with the continuous
      relaxation itself remain. Purity: To report purity is
      definitely fine. However, a comparison to recursive splitting techniques
      [3,11] or standard spectral clustering (with k-means in the embedding
      space) could have been done both with respect to criterion (1) and the
      suggested criterion (P), in order to judge if the suggested algorithm
      really produces better cuts - as this is the only criterion one has in
      practice to judge a given partition. It would even be interesting to
      see the performance of the NMF methods there.


Scoring Workers in Crowdsourcing: How Many Control Questions are Enough?
We study the problem of estimating continuous quantities, such as prices, probabilities, and point spreads, using a crowdsourcing approach. A challenging aspect of combining the crowd's answers is that workers' reliabilities and biases are usually unknown and highly diverse. Control items with known answers can be used to evaluate workers' performance, and hence improve the combined results on the target items with unknown answers. This raises the problem of how many control items to use when the total number of items each workers can answer is limited: more control items evaluates the workers better, but leaves fewer resources for the target items that are of direct interest, and vice versa. We give theoretical results for this problem under different scenarios, and provide a simple rule of thumb for crowdsourcing practitioners. As a byproduct, we also provide theoretical analysis of the accuracy of different consensus methods.

This paper examines the problem of determining
      what fraction of control questions (with known answers) vs. target
      answers (unknown) to use when using crowdsourcing to estimate
      continuous quantities (i.e., not categorical judgments). The authors
      describe two models for using control questions to estimate worker
      parameters (bias and variance from true answers); a two-stage
      estimator which estimates worker parameters from the control items
      alone, and a joint estimator which comes up with an ML estimate using
      both control and target items. The authors derive expressions for the
      optimal number of target items (k) for each case, beginning with a
      clear statement of the results and then going through detailed but
      clear derivations. They then show their results empirically on both
      synthetic and real data, showing how the estimates align with the true
      optimal k in cases where model is a perfect match to the data vs.
      misspecified (for synthetic) and then show how the practical effects
      of misspecification when dealing with real data. They close with
      valuable recommendations for practitioners in terms of choosing
      between the models. First of all, this is a very important and
      highly practical setting - as someone who has run many crowdsourced
      tasks as well as read/heard many accounts from others, using control
      questions is a tried and true method of estimating bias and variance
      in judges; much of the past theoretical work has ignored this and
      assumed no such control questions are available. While control
      questions could be used in these other methods in principle, I
      know of no previous paper that has examined the *value* of control
      items and how many should be used in a given setting. I have often
      wondered about this question in my own experiments, and have
      considered working on the problem myself; as such I was delighted to
      read this thorough treatment by the authors. This paper is excellent
      in so many ways: it is unusually clear in its writing, from the
      motivation and setup to the explanation of their the strategy and
      purpose of their approach before diving into the derivations, to the
      setup and explanation of the experiments and their implications. The
      past literature is well-covered, the figures are clear, the notation
      and development are easy to follow. The estimation algorithms and
      the optimal k are clear, and the discussion of the effects of
      model mismatch and recommendations for real settings/applications
      are insightful and valuable. I would recommend this paper not only to
      my colleagues who work on the analysis of crowdsourcing, but also to
      many others who are users of crowdsourcing: an excllent paper all
      around, that I expect will be well-regarded at the conference and
      well-cited in future years.


Machine Teaching for Bayesian Learners in the Exponential Family
What if there is a teacher who knows the learning goal and wants to design good training data for a machine learner? We propose an optimal teaching framework aimed at learners who employ Bayesian models. Our framework is expressed as an optimization problem over teaching examples that balance the future loss of the learner and the effort of the teacher. This optimization problem is in general hard. In the case where the learner employs conjugate exponential family models, we present an approximate algorithm for finding the optimal teaching set. Our algorithm optimizes the aggregate sufficient statistics, then unpacks them into actual teaching examples. We give several examples to illustrate our framework.

The authors generalize the teaching dimension theory
      [Goldman & Kearns, 95] to the family of Bayesian Learners (and
      hypothetically to noisy teachers) -- although they restrict any actual
      analysis to exponential families of distributions (or at least
      distributions with conjugate priors -- which admittedly much of machine
      learning does). The basic model minimizes the combination of loss from the
      resulting learner hypothesis and effort of the teacher. Based of the basic
      idea of selecting a dataset that has the correct sufficient statistics,
      they first provide a solution to the 1D threshold function (under two
      different scenarios), a restricted model selection problem (one of two
      Gaussian distributions), and eventually generalize this to teaching
      Bayesian Learners of the family of Exponential Distributions.
      Overall, the paper is very clear, although quite a bit of space is
      spent on trivial examples while the exponential family section is by far
      the most interesting and practical. My biggest concern with this paper is
      that while it does improve the teaching dimension literature, it seems to
      think it is making a bigger jump than it is (and sort of overselling its
      claims a bit to be honest) as it is missing some reference to
      statistics/cognitive science literature that have very similar results (I
      am not even sure if “mostly studied from th query learning angle” is
      entirely correct -- although it certainly has been studied from this
      angle). For example, [Tenenbaum & Griffiths, 2001; Shafto &
      Goodman, 2008; Rafferty & Griffiths, 2010]. There are clearly some
      very strong connections here and the work should at least be acknowledged
      (and I actually think could save some space for discussion on the
      exponential families model) -- although I do admit that I like the
      presentation better in this paper (but this is likely since I am not a
      cognitive scientist). While I am not a cognitive scientist, I am
      also somewhat concerned about the claims regarding the application to
      human learning. Given the authors knowledge of query learning, I am
      actually somewhat surprised by this claim. My understanding is that the
      current theory is that humans are not passive Bayesian learners -- not
      only is the presentation of the examples by the teacher not iid, but it is
      influenced by feedback from the students (which I suppose could
      hypothetically be modeled as some sampling from the distribution of the
      student) and if nothing else, the prior of the student are certainly not
      known (although, again, I suppose they could be probed). That
      being said, I think it is a nice work overall and would be interesting to
      the NIPS community (in particular relative to the machine learning
      community in general). It is a very clean presentation, made me think a
      bit, and would likely influence my work -- which means it would likely be
      of interest to the NIPS community (even if I think it is a bit “in
      progress” with respect to contextualizing within related work).


Embed and Project: Discrete Sampling with Universal Hashing
We consider the problem of sampling from a probability distribution defined over a high-dimensional discrete set, specified for instance by a graphical model. We propose a sampling algorithm, called PAWS, based on embedding the set into a higher-dimensional space which is then randomly projected using universal hash functions to a lower-dimensional subspace and explored using combinatorial search methods. Our scheme can leverage fast combinatorial optimization tools as a blackbox and, unlike MCMC methods, samples produced are guaranteed to be within an (arbitrarily small) constant factor of the true probability distribution. We demonstrate that by using state-of-the-art combinatorial search tools, PAWS can efficiently sample from Ising grids with strong interactions and from software verification instances, while MCMC and variational methods fail in both cases.

The paper presents and evaluates a new method to
      sample from complex high-dimensional distributions by reducing the problem
      to a small number of related optimization and enumeration problems. It is
      shown that with high probability the sampling distribution is within a
      constant factor from the true distribution. Experimental results
      demonstrate cases where the proposed method outperforms standard MCMC and
      variational methods. The work continues on a great series of
      earlier studies on how sampling and counting problems can be solved by
      using solvers for SAT and other hard combinatorial problems. The
      approach is well motivated by the fact that existing solvers tend to be
      very fast on typical instances encountered in practice, even if the
      problems are NP-hard and the solvers' worst-case runtime guarantees look
      rather hopeless. Prior works in this research direction (after the early
      works of Valiant, Vazirani, and Jerrum) are chiefly by Gomes, Sabharwal,
      Selman et al.: 1. Carla P. Gomes, Ashish Sabharwal, Bart
      Selman: Near-Uniform Sampling of Combinatorial Spaces Using XOR
      Constraints. NIPS 2006: 481-488 2. Carla P. Gomes, Ashish
      Sabharwal, Bart Selman: Model Counting: A New Strategy for Obtaining
      Good Bounds. AAAI 2006: 54-61 3. Carla P. Gomes, Jörg Hoffmann,
      Ashish Sabharwal, Bart Selman: Short XORs for Model Counting: From
      Theory to Practice. SAT 2007: 100-106 4. Carla P. Gomes, Jörg
      Hoffmann, Ashish Sabharwal, Bart Selman: From Sampling to Model
      Counting. IJCAI 2007: 2293-2299 5. Carla P. Gomes, Willem Jan van
      Hoeve, Ashish Sabharwal, Bart Selman: Counting CSP Solutions Using
      Generalized XOR Constraints. AAAI 2007: 204-209 6. Stefano Ermon,
      Carla P. Gomes, Bart Selman: Uniform Solution Sampling Using a
      Constraint Solver As an Oracle. UAI 2012: 255-264 7. Stefano
      Ermon, Carla P. Gomes, Ashish Sabharwal, Bart Selman: Taming the Curse
      of Dimensionality: Discrete Integration by Hashing and Optimization. ICML
      2013 8. Stefano Ermon, Carla P. Gomes, Ashish Sabharwal, Bart
      Selman: Optimization With Parity Constraints: From Binary Codes to
      Discrete Integration. UAI 2013 While the series as a whole
      constitutes a major innovation in the counting and sampling methodology,
      the present contribution is rather incremental. The main message of the
      paper, namely that combinatorial optimization and search tools enable
      sampling from complex distributions with controllable accuracy guarantees,
      follows already from the WISH algorithm given in the ICML'13 paper (7),
      since it is well known (e.g., due to Jerrum, Valiant, Vazirani) that
      accurate counting enables accurate sampling (supposing the problem is
      self-reducible in a way they often are). The proposed algorithm, PAWS,
      however does not implement that reduction: PAWS takes, in a sense, a more
      direct route to turn optimization and solution enumeration tools into a
      sampling algorithm. Only a single optimization instance needs to be
      solved, under the original combinatorial domain, which seems like a
      significant advantage. The downside is that one has to solve several
      enumeration instances in a much larger and more complicated domain. Under
      what conditions is PAWS more efficient than WISH (whether for sampling or
      integration)? The paper does not give even a preliminary answer to this
      question. In particular, the experiments reported in Figure 1 concern a
      variant of WISH that faster but potentially much less accurate than the
      original WISH. The presentation is mostly excellent. The title and
      the abstract are particularly well written and informative, close to
      perfect. In Sections 3.2 and 3.3 there is, however, room for improvement:
      Def2 is difficult to follow at first glance. The description of the
      algorithm around Lemmas 3 and 4 gets dense and almost impossible to follow
      for a reader that is not familiar with the cited works [3,5,6]. While I
      appreciate the rigorous statements of Lemmas 3 and 4, I'd like to see an
      expanded description of the key ideas. Why is it so interesting whether
      the hashed space contains at least P solutions? What is the role of k? To
      get more space, I'd consider removing Corollary 2 and the Conclusions
      section. - I'd use a bold font for emphasizing purposes only
      when defining new concepts. - Just before Def1 I'd avoid
      "log-likelihood" because of the association to the likelihood function (of
      model parameters) that assumes some data are around. - Sentences that
      end with a displayed equation should end with a ".". - In Def2 I'd
      write write M/r^\ell < w(x) \leq M/r^i . - In Section 3.3 use the
      reference numbers, e.g. [6], as parenthetical information, removing of
      which leaves an intelligent sentence. - Consider placing the caption
      of Figure 2 below the figure.


Fisher-Optimal Neural Population Codes for High-Dimensional Diffeomorphic Stimulus Representations
In many neural systems, information about stimulus variables is often represented in a distributed manner by means of a population code. It is generally assumed that the responses of the neural population are tuned to the stimulus statistics, and most prior work has investigated the optimal tuning characteristics of one or a small number of stimulus variables. In this work, we investigate the optimal tuning for diffeomorphic representations of high-dimensional stimuli. We analytically derive the solution that minimizes the L 2 reconstruction loss. We compared our solution with other well-known criteria such as maximal mutual information. Our solution suggests that the optimal weights do not necessarily decorrelate the inputs, and the optimal nonlinearity differs from the conventional equalization solution. Results illustrating these optimal representations are shown for some input distributions that may be relevant for understanding the coding of perceptual pathways.

Summary Finding the objective functions that
      regions of the nervous system are optimized for is a central question in
      neuroscience, providing a central computational principle behind neural
      representation in a given region. One common objective is to maximize the
      Shannon Information the neural response encodes about the input (infomax).
      This is supported by some experimental. Another is to minimize the
      decoding error when the neural population is decoded for a particular
      variable or variable. This has also been found to have some experimental
      evidence. These two different objectives are similar in some
      circumstances, giving similar predictions, in other cases they differ
      more. Studies finding model optimal distributions of neural
      population tuning that minimizes decoding error (L2-min) have mostly
      considered 1-dimensional stimuli. In this paper the authors extend
      substantially on this, by developing analytical methods for finding the
      optimal distributions of neural tuning for higher dimensional stimuli.
      Their methods apply under certain limited conditions , such as when there
      is an equal number of neurons as stimulus dimensions (diffeomorphic). The
      authors compare their results to the infomax solution (in most detail for
      the 2D case), and find fairly similar results in some respects, but with
      two key differences. That the L2-min basis functions are more orthogonal
      than the infomax, and that the L2-min has discrete solutions rather than
      the continuum found for infomax. A consequence of these differences is
      that L2-min representations encode more correlated signals. On
      the quality score, impact score and confidence score. I chose accept
      (top 50%) as I thought it was a good and interesting paper. I chose 2
      (although this is a rather course grain measure, I probably would go for
      1.7). I chose 3 (fairly confident) but if I could chose 2.5 I would. I
      think I got the gist of the paper, and Sections 1-2 were clear. I got lost
      in several place in Sections 3-6 but I think I followed the general
      argument to a varying degree. Section 7 was clear. Quality
      Is the paper technically sound? As far as I can tell, the
      paper is technically sound. The areas of the paper that I could
      confidently follow held up in my opinion. Are the claims well
      supported? I would say that the claims are well supported as well as I
      could follow them. A little more explanation as to why the more orthogonal
      solution of the L2-min encodes correlated signals would be useful.
      Is this a complete piece of work? For a NIPS paper I would say
      yes. Are the authors careful (and honest) about evaluating both
      the strengths and weaknesses of the paper? Yes, the authors are very
      careful and honest in evaluating the paper’s strength and weaknesses. In
      fact, I think they are a little too circumspect, and I would have liked
      just a little more discussion of its strengths and possibilities. However,
      I recognize there are space limits. Clarity Is the
      paper clearly written? Where I understood the paper is seems
      moderately clearly written. Where I didn’t understand I don’t know if it
      was the authors lack of clarity or my lack of familiarity with what they
      were doing. The clarity and consistency of the notation could be
      improved. The notation varies somewhat across the paper and terms are
      sometimes not defined. For example the don’t define E[ |s] and <> as
      expectation. They use n for the number of neurons and the number of
      spikes. They use ‘ for differentiation but also as simply a marker (as s’
      in 2.4). Is the paper well organized? Yes. Does the
      paper adequately inform the reader? I would have liked some areas
      spelt out more. See above. Originality Are the
      problems or approaches new? Yes, analytical methods for high
      dimensional L2-min representation are new as far as I know. Is it
      clear how this work differs from previous contributions? Yes.
      Is related work adequately referenced? Yes.
      Significance Are these results important? This
      work could help form the basis of a more general research program that
      examines what general objective functions particular neural systems use.
      Thus I think the results are important. Are other people likely to
      use these ideas? Yes, I think they could be useful as the basis of
      future more specific and empirical analyses, and also expanded into a more
      general framework. Does the paper address a difficult problem in a
      better way than previous research? Yes, the analytical results are
      more flexible and secure and can be used to speed up numerical methods.
      Does it advance the state of the art? Yes Does it
      provide a unique theoretical approach? Yes. Strengths
      High-dimensional L2-min is novel. Novel analytical approach.
      Comparison with Infomax. Clear differences in solutions given.
      Weaknesses Restricted circumstances (diffeomorphic). A
      little unclear in places (may be due to reviewer). Detailed
      review Sections 1 and 2 are clear to me. Only minor points
      070: As opposed to the encoding, the 070: function called
      080: E[|s] is the expectation over r, but might be good to define
      it. 098: Perhaps you are using a notation I am not familiar with,
      but by analogy with the L2 norm, should it be the L22 loss? 101:
      In this equation I am more used to a ≥. Please define the meaning of the
      curved ≥ in this context. 113: Define s’. Later ‘ is used for
      derivative, please clear up this ambiguity. In section 3 I get
      somewhat confused. Please give more detail on how Holder’s inequality is
      used to find the minimum? 163: …, the right side of Equation 10
      is… 178: Use n for number of spikes in the appendix, need to
      change one of these variables. Section 4. Also found this somewhat
      confusing. 189: double integral sign for a multi-dimensional
      integral is confusing. I suggest using placing … between them, or just
      having one integral sign. 193: as 189. Sections 5 and 6.
      Found this somewhat confusing, but less than 3 and 4. 229: What
      happens to |wk| in the relationship between Equations 17 and 18? Confused
      here. 241: “Minimizing the numerator Var[wkTs] makes…” The
      numerator of which equation? I assume you are referring to equation 17, if
      so the relationship between Var[wkTs] and the numerator needs to be made
      more explicit. 321: Might want to briefly discuss the similarity
      and differences between Equation 16 and Equation 28. 371:
      Apologies if I am totally missing the point. I would imagine that a more
      orthogonal basis would results in less correlated signals. Where am I
      wrong? Please explain. A little more elaboration on this point in the text
      would be useful. Section 7. Generally clear. Section A.
      Clear.


Solving inverse problem of Markov chain with partial observations
The Markov chain is a convenient tool to represent the dynamics of complex systems such as traffic and social systems, where probabilistic transition takes place between internal states. A Markov chain is characterized by initial-state probabilities and a state-transition probability matrix. In the traditional setting, a major goal is to study properties of a Markov chain when those probabilities are known. This paper tackles an inverse version of the problem: we find those probabilities from partial observations at a limited number of states. The observations include the frequency of visiting a state and the rate of reaching a state from another. Practical examples of this task include traffic monitoring systems in cities, where we need to infer the traffic volume on single link on a road network from a limited number of observation points. We formulate this task as a regularized optimization problem, which is efficiently solved using the notion of natural gradient. Using synthetic and real-world data sets including city traffic monitoring data, we demonstrate the effectiveness of our method.

The paper proposes an estimation strategy for
      recovering the parameters of a finite state Markov chain given observed
      stationary frequencies of some states. (The paper also considers using
      additional hitting frequencies for a few state pairs, but this is not used
      in the final evaluation.) Although the problem proposed is
      potentially very interesting, the paper does not appear to be in a mature
      state. Some fundamental issues are not adequately addressed, while the
      evaluation is limited, and the writing quality is not strong. Note
      that there is an uncountable set of ergodic transition models that can
      exactly match a given subset of stationary frequencies when the number of
      observed stationary state frequencies is small relative to the total
      number of states. In short, the problem is ill posed. This means that
      additional prior knowledge is required to recover a unique transition
      model, whether via regularization or imposing a compact parametric form.
      Unfortunately, the paper does not even discuss the necessity of
      imposing prior knowledge to achieve non-vacuous results in this setting.
      That is, even though the paper proposes a parametric representation and
      considers regularization, it does not observe that the problem is ill
      posed, hence that *zero* loss is obtainable for an uncountable set of
      transition models. Instead, the paper merely proposes a few heuristic loss
      functions and an arbitrary regularizer. This seems to be a somewhat
      superficial approach to the issue. Even the paper's main motivating
      example of a traffic network possesses clear prior structure: the
      transition probability must be zero between unconnected links. There will
      be some subsets of stationary state probabilities that can identify the
      transition model in such a structured setting, but this possibility is not
      even discussed. The proposed loss functions (8) and (9) are
      clearly heuristic, and I found the attempted justification below (8) to be
      unconvincing. A distributional assumption can always be reverse engineered
      from a loss formulation like (8), but that does not mean the
      log-frequencies will have this distributional form. The discussion
      of "bias" in Section 3.2 appears to be too informal: on the context of
      estimation, unbiasedness normally refers to a property that the expected
      value of an estimator is the true value of the underlying parameter.
      Although the loss formulation (8) is claimed to be "unbiased", no argument
      is given that the expected value of the minimizer of (8) has the correct
      value. If one accepts the regularized loss formulation, the
      natural gradient approach developed in Section 4 appears to be reasonable.
      The technical development in this section is sound. The
      experimental evaluation appears to be limited. One synthetic and one
      "real" experiment are conducted. In the synthetic experiment, the
      model exactly matches that used by the paper's proposed estimator, which
      is an ideal scenario that cannot be guaranteed in practice. Here the
      proposed method's results are compared to a baseline estimator (NWKR) that
      does not exploit the fact that the target stationary frequencies are
      associated with a Markov chain. This seems to be a weak straw-man
      competitor, and it is not too surprising that it performs poorly by
      comparison. In the real experiment, it is somewhat disappointing
      that hitting rate observations are not included in the observations, which
      means that the initial distribution is irrelevant to the estimation
      problem (since only stationary frequencies are being used). Nevertheless,
      this is an interesting experimental domain. The results look reasonable,
      although again, the baseline competitor is too weak to take very
      seriously. A more credible competitor would identify a transition model
      that exactly matches the observed stationary frequencies (and the sparsity
      pattern implied by the road network) while maximizing entropy. The
      experimental evaluation would also be more convincing if results could be
      obtained for a larger number of synthetic and real
      domains.


An Approximate, Efficient Solver for LP Rounding
Many problems in machine learning can be solved by rounding the solution of an appropriate linear program (LP). This paper shows that we can recover solutions of comparable quality by rounding an approximate LP solution instead of the exact one. These approximate LP solutions can be computed efficiently by applying a parallel stochastic-coordinate-descent method to a quadratic-penalty formulation of the LP. We derive worst-case runtime and solution quality guarantees of this scheme using novel perturbation and convergence analysis. Our experiments demonstrate that on such combinatorial problems as vertex cover, independent set and multiway-cut, our approximate rounding scheme is up to an order of magnitude faster than Cplex (a commercial LP solver) while producing solutions of similar quality.

Summary: The paper considers LP relaxations in
      machine learning and studies the problem of solving the LP relaxation
      only approximately and derives bounds on the overall approximation
      rate when solving the integer program (by rounding the LP solution).
      Pros: - very interesting problem (solve LP relaxation, knowing
      that in the end we will round the solution). This is a relevant
      problem in many machine learning fields. - to the best of my
      knowledge the first work that gets approximation guarantees for the
      overall algorithm (LP approximation by smoothing, only solve smoothed
      LP to a some threshold, integer rounding). Cons: - I would
      skip the SCD algorithm in the paper and rather try to give more
      intuition and proof sketches in the main part of the paper. I do not
      think a reader can appreciate section 3.3 too much. It looks too me
      that the running time is not very exciting, both n and 1/epsilon^2
      appear (SGD does for example not have the dependence on n in SVMs),
      why not solve the problem with a more standard solver? It seems to me
      SCD is not the main contribution of the paper. - the speed
      improvements seen in the experiments are not hugely impressive
      (roughly one order of magnitude). Also there is probably some
      precision option in CPLEX, what happens if you lower the precision in
      CPLEX LP? - few language problems, nothing major. - long appendix
      Quality: Experiments could be improved, some questions about
      the main theorems remain. Clarity: Language problems and
      suboptimal choices in the presentation (e.g. get rid of 3.3 and
      include more material from the appendix). Originality: To the
      best of my knowledge the main section (3.2) is original.
      Significance: LP relaxations are important tools for
      combinatorial optimization in machine learning. Comments:
      39: Despite ? 40: .. Gurobi cannot scale .., the order 10 of
      speedup of your method is probably not going to change this. 43:
      -- but often much more efficiently. Rewrite. 52: Cplex do no scale to
      52: The Cplex LP solver was slower but it ran through in the time
      limit on almost all problems. 55: Obviously message passing is
      related in that it also solves LP relaxations, but I fail to see the
      connection to your method. The smoothing? 69: in which to articulate
      74: would appreciate some citations 98: is a subset of V each edge
      (rephrase!) 102: vertex cover as a 104: for each vertex v \forall
      v \in V, rephrase! 130: one can easily 132: and an example such
      scheme (rephrase) 144: Figure caption needs more details about the
      parameters. (3): min_x (4): max_u 167: is the definition
      standard? if so, cite 177: \prod_[0,1]^n was not really introduced
      176: to the linear program in 193: how is d defined, it is a
      concatenation of vectors and a matrix, what's the norm on d? 191:
      the paragraph here seems to apply to general LPs, it is a bit confusing,
      as it's introduced after (5) 236: \beta depends on optimal
      solution, how would we estimate this? 280: algorithm 2 is described
      (rephrase) 304: what does thetis stand for? 310: have rarely been
      used? for 315: Boolean 317: the resulting graphs 320: Gibbs
      sampling is not really state of the art. How exactly, annealing? 321:
      why didn't you also use an LP MAP solver such as the one in [14]? 331:
      have placed if (rephrase) Figure 2: what about energies (apart of P,
      R, F1)? 357: in this paragraph you mention the appendix twice 360:
      the second family of 369: this experiment is to compare 418:
      Cplex-IP on large scale 960: describe difference between Table 3 + 4
      in more detail