Improved and Generalized Upper Bounds on the Complexity of Policy Iteration
Given a Markov Decision Process (MDP) with n states and m actions per state, we study the number of iterations needed by Policy Iteration (PI) algorithms to converge to the optimal "-discounted optimal policy. We consider two variations of PI: Howard's PI that changes the actions in all states with a positive advantage, and Simplex-PI that only changes the action in the state with maximal advantage. We show that Howard's PI terminates after at most n(m ≠ 1)
improving by a factor O(log n) a result by [3], while Simplex-PI terminates after at most n 2 (m ≠ 1)
iterations, improving by a factor O(log n) a result by [11]. Under some structural assumptions of the MDP, we then consider bounds that are independent of the discount factor ": given a measure of the maximal transient time · t and the maximal time · r to revisit states in recurrent classes under all policies, we show that Simplex-PI terminates after at most n 2 (m≠ 1) (Á· r log(n· r )Ë + Á· r log(n· t )Ë)
iterations. This generalizes a recent result for deterministic MDPs by [8], in which · t AE n and · r AE n. We explain why similar results seem hard to derive for Howard's PI. Finally, under the additional (restrictive) assumption that the state space is partitioned in two sets, respectively states that are transient and recurrent for all policies, we show that Howard's PI terminates after at most n(m ≠ 1) (Á· t log n· t Ë + Á· r log n· r Ë) =Õ(nm(· t + · r )) iterations while Simplex-PI terminates after n(m ≠ 1) (Án· t log n· t Ë + Á· r log n· r Ë) = O(n 2 m(· t + · r )) iterations.
improving by a factor O(log n) a result by [3], while Simplex-PI terminates after at most n 2 (m ≠ 1)
iterations, improving by a factor O(log n) a result by [11]. Under some structural assumptions of the MDP, we then consider bounds that are independent of the discount factor ": given a measure of the maximal transient time · t and the maximal time · r to revisit states in recurrent classes under all policies, we show that Simplex-PI terminates after at most n 2 (m≠ 1) (Á· r log(n· r )Ë + Á· r log(n· t )Ë) # (m ≠ 1)Án· t log(n· t )Ë + Án· t log(n 2 · t )Ë
iterations. This generalizes a recent result for deterministic MDPs by [8], in which · t AE n and · r AE n. We explain why similar results seem hard to derive for Howard's PI. Finally, under the additional (restrictive) assumption that the state space is partitioned in two sets, respectively states that are transient and recurrent for all policies, we show that Howard's PI terminates after at most n(m ≠ 1) (Á· t log n· t Ë + Á· r log n· r Ë) =Õ(nm(· t + · r )) iterations while Simplex-PI terminates after n(m ≠ 1) (Án· t log n· t Ë + Á· r log n· r Ë) = O(n 2 m(· t + · r )) iterations.

Summary: The paper considers the complexity of
      policy iteration (PI) for finite MDPs with n states and m actions per
      state in a number of settings. The following results are derived:
      For Howard's PI, a previous bound is improved by removing a log(n)
      factor. For Simplex PI, a previous bound is improved by halving its
      multiplicative constant. When the MDP is deterministic, a previous
      bound for Simplex PI is improved by decreasing the power of
      dependence on n by one. This result is generalized to stochastic
      MDPs under additional restrictions on the MDP. Originality and
      significance: The complexity of PI is of major interest to the RL
      community. Even though the finite setting is not practical, it is
      important to have a good understanding of this simple case before
      moving to more complicated settings. This paper essentially picks up
      the previous line of research that started with a paper by Yinyu Ye in
      2011 and continued by Post and Ye, as well as by Hansen, Miltersen and
      Zwick. Some of the improvements are marginal and the core ideas of the
      proofs go back to the 2011 work of Ye. However, the proofs are much
      more streamlined and are way more accessible to RL researchers than
      those of Ye (who prefers to use the language of linear programming).
      The extension of strong polynomial results to stochastic MDPs under
      structural assumptions are the most important novel parts of the paper.
      Clarity: The paper is relatively well-written, though the
      authors should check it carefully as it contains a few typos (lines
      285, 292). Also, at times fluency could be improved (e.g., "However,
      on the other hand" on line 303 is awkward). Soundness: I
      believe that the results are sound (I did not have time to check all the
      details). Minor issues: - line 367: specify the lemma used


Context-sensitive active sensing in humans
Humans and animals readily utilize active sensing, or the use of self-motion, to focus sensory and cognitive resources on the behaviorally most relevant stimuli and events in the environment. Understanding the computational basis of natural active sensing is important both for advancing brain sciences and for developing more powerful artificial systems. Recently, we proposed a goal-directed, context-sensitive, Bayesian control strategy for active sensing, C-DAC (ContextDependent Active Controller) (Ahmad & Yu, 2013). In contrast to previously proposed algorithms for human active vision, which tend to optimize abstract statistical objectives and therefore cannot adapt to changing behavioral context or task goals, C-DAC directly minimizes behavioral costs and thus, automatically adapts itself to different task conditions. However, C-DAC is limited as a model of human active sensing, given its computational/representational requirements, especially for more complex, real-world situations. Here, we propose a myopic approximation to C-DAC, which also takes behavioral costs into account, but achieves a significant reduction in complexity by looking only one step ahead. We also present data from a human active visual search experiment, and compare the performance of the various models against human behavior. We find that C-DAC and its myopic variant both achieve better fit to human data than Infomax (Butko & Movellan, 2010), which maximizes expected cumulative future information gain. In summary, this work provides novel experimental results that differentiate theoretical models for human active sensing, as well as a novel active sensing algorithm that retains the context-sensitivity of the optimal controller while achieving significant computational savings.

The paper explores theoretical issues in active
      sensing -- situations where people much actively gather information to
      achieve some goal. A extension to a previous model (C-DAC) is proposed
      which myopically gathers information in order to maximize a task-specific
      cost function. The model is compared against an alternative that favors
      information without respect to cost (Infomax). The overall goal of the
      paper is interesting and important. The novelty of the paper is somewhat
      reduced simply because it reports an extension to Ahmad & Yu (2013) to
      deal myopically with action selection. This is fine but I wondered how
      novel or important that aspect of the modeling actually was (e.g., it
      didn't seem to figure centrally into explanations of the experiment
      itself). There is some prior work on this issue. In particular,
      the authors might check out Markant & Gureckis, 2012 "Does the utility
      of information influence sampling behavior?" which explores a similar
      issue: whether people optimize information or some task-specific cost
      function. They found that information optimizing provided a better fit to
      people's behavior. The paper was poorly written overall. The
      abstract was much too long, the paper exceeded the 8 page limit at NIPS,
      and the paper cites Ahmad & Yu, 2013 in almost every sentence
      including twice in the abstract alone! I had a really hard time
      understanding the experiment design. Terms like "patch" are introduced
      without definition (is a patch a set of trials with a particular
      distribution of target probabilities, or possible target locations? I
      couldn't really tell). Also, while the overall theoretical issue seemed
      well motivated, the paper itself focused on this "confirmatory search"
      phenomena which the authors seemed to ad-hoc identify in their data. Why
      was this phenomena selected or critical for differentiating the models?
      Also, the model result for section 3 were not that compelling. The
      full C-DAC hits all the qualitative patterns as do the myopic one. The
      authors don't discuss what this means. Is it the case that the measures
      the authors selected here just don't tell any of the models apart very
      well? No fit statistics or quantification of the advantage for the more
      limited model is offered other than to describe is as "more
      psychologically plausible." Section 4 was also very opaque. I didn't
      understand the policy-space graphs in Figure 3 and I suspect that most
      readers will have significant difficulty with this as well. It is unclear
      what "Greedy MAP" is as well as this model is never mentioned in the
      paper. The analysis presented in Figure 4 is odd because if this
      is meant to be a model of humans, no mention is made of human performance.
      It is fine to say that the myopic C-DAC model performs well on a number of
      metrics, but as a model it should match human observer as well (unless
      this is meant as a engineering analysis of a computer vision based visual
      search algorithm?). *Note* My initial evaluation was a little more
      negative, but after discussion with the other reviewers I have given a 7.
      I think some of the critical comments above still apply, but there are
      also strengths including the emphasis on multiple model comparison, the
      empirical data, and the novelty of deriving theorems about the optimal
      stopping rule. I thought the author's response was helpful and it would be
      nice if the paper was edited somewhat in accordance with the reviewer
      questions to be more clear if it is accepted.


Heterogeneous-Neighborhood-based Multi-Task Local Learning Algorithms
All the existing multi-task local learning methods are defined on homogeneous neighborhood which consists of all data points from only one task. In this paper, different from existing methods, we propose local learning methods for multitask classification and regression problems based on heterogeneous neighborhood which is defined on data points from all tasks. Specifically, we extend the knearest-neighbor classifier by formulating the decision function for each data point as a weighted voting among the neighbors from all tasks where the weights are task-specific. By defining a regularizer to enforce the task-specific weight matrix to approach a symmetric one, a regularized objective function is proposed and an efficient coordinate descent method is developed to solve it. For regression problems, we extend the kernel regression to multi-task setting in a similar way to the classification case. Experiments on some toy data and real-world datasets demonstrate the effectiveness of our proposed methods.

The paper presents a set of novel Multi-Task
      binary-classification and regression algorithms based on a weighted K-NN
      prediction rule. The main idea is to make use of the labels of the nearest
      neighbors from all the tasks, in order to predict the score for a target
      sample (instead of using only the labels of the neighbors from the target
      task). The prediction provided by each of the nearest neighbors is
      weighted by a scalar which encodes the task-relationship between the task
      of the target sample and the task of the neighbor considered. The
      task-to-task weights are collected in a matrix and a global objective
      function is devised to minimize the prediction error on the training set.
      The objective function penalizes non-symmetric matrices and includes
      constraints to force the magnitude of the task-to-task weights to be
      smaller whenever the source and the target tasks differ. The penalties
      used for the classification tasks include the hinge-loss and the
      square-loss. For regression tasks the authors employ the square loss and
      replace the K-NN classification rule with a kernel-regression rule. In
      all the settings the optimization is performed using a (block)
      coordinate-descent approach, where each sub-problem is solved using
      efficient ad-hoc algorithms. For classification, the training time
      complexity is reported to be linear in the number of tasks and log-linear
      (linearithmic) in the number of training samples from each task, however
      this complexity does not include the time for the K-NN search for each
      sample. Moreover, the training complexity of the regression algorithm does
      not seem to be provided. The algorithm is tested on two
      classification tasks and one regression task and is compared with two
      other classical multi-task learning approaches. The results show that the
      proposed methods compare favorably w.r.t. the selected baselines, while
      the proposed training procedures result in a significant speedup w.r.t.
      the use of standard solvers (CVX). The main novelty of the
      approach is the use of the nearest neighbors from all tasks, combined with
      the learned task-to-task relatedness matrix. For classification tasks this
      results in the possibility of learning positive, neutral and negative
      correlations between tasks, which in turn result in positive, zero, or
      negative weights assigned to each neighbor label. For regression task, the
      weighting allows to learn a linear relationship between the different
      tasks. A concern is about the convergence of the
      coordinate-descent approach when used to minimize non-smooth functions
      (e.g. hinge loss). The authors claim that the algorithm: "can converge to
      a global optimum" since "all the subproblems are strictly convex", citing
      the results in [1]. However, the hypothesis for local convergence in [1]
      require the objective function to be smooth (C^2), while for global
      convergence the solutions in the optimization procedure are required to
      lie in a compact set. A more in-depth discussion and explanation seems
      thus to be necessary. [1] Bezdek, James C., and Richard J.
      Hathaway. "Convergence of alternating optimization." Neural, Parallel
      & Scientific Computations 11.4 (2003): 351-368. The
      experimental evaluation seems rather limited: - the classification
      datasets considered (USPS, Letter) do not seem to match the datasets used
      in the papers of the considered baselines - the baseline selected only
      include multi-task methods which assumes that all the tasks are related,
      while more recent multi-task approaches (e.g. [2] and [3]) are moving away
      from this assumption; since the proposed method is also able to learn how
      the tasks are related to each other, a comparison should be made with such
      methods - while using nearest neighbors from all the tasks seem to
      improve the performances (w.r.t using only the K-NNs of the target task),
      it is also supposed to increase the K-NN search time; a comparison of
      training-complexity or training-times w.r.t. the baseline methods would be
      appreciated. The exposition is not always clear. Examples:
      - in lines 40-42 the authors claim that "When the number of
      labeled data is not very large, the performance of the local learning
      methods is limited due to sparse local density". This is an important
      point for the proposed methodology, as it is one of the motivations for
      using the samples from all the tasks. It should thus definitely discussed
      better, or a citation should be provided. - in lines 53-55 the authors
      write about "low-confidence" and high-confidence estimations. However, the
      notion of confidence of the estimations is never formalized for the
      methods in question, so that the claims result to be quite vague - in
      lines 61-64 the authors write: "Since multi-task learning usually
      considers that the contribution of one task to another one equals that in
      the reverse direction, we define a regularizer to enforce the
      task-specific weight matrix to approach a symmetric matrix"; the
      requirement of symmetry in the task-to-task relationships is also an
      important point of the proposed approach and should thus be discussed more
      thoroughly, or a citation should be provided - in lines 358-361 of the
      experimental section two baseline algorithm (mtLMNN and MTFL) are
      apparently introduced without any citation; while it is possible to infer
      the related publications from the acronyms the citations should be
      reported - the URL reported in line 377 for the Letter dataset
      (http://multitask.cs.berkeley.edu/) does not seem to exist anymore
      Minor comments While the classification function is local, the
      task-to-task relationship matrix W is global and the learning of W is done
      in a global-fashion, rather than in a local, as it figures in the title.


Efficient Algorithm for Privately Releasing Smooth Queries
We study differentially private mechanisms for answering smooth queries on databases consisting of data points in R d . A K-smooth query is specified by a function whose partial derivatives up to order K are all bounded. We develop an -differentially private mechanism which for the class of K-smooth queries has accuracy O(n
The mechanism first outputs a summary of the database. To obtain an answer of a query, the user runs a public evaluation algorithm which contains no information of the database. Outputting the summary runs in time O(n 1+ d 2d+K ), and the evaluation algorithm for answering a query runs in timẽ
Our mechanism is based on L ∞ -approximation of (transformed) smooth functions by low degree even trigonometric polynomials with small and efficiently computable coefficients.

NOTE: Due to the number of papers to review and the
      short reviewing time, to maintain consistency I did not seek to verify the
      correctness of the results. As a result, I generally ignored any proofs or
      additional details in the supplementary material. This review should be
      considered as a review of the main paper alone. SUMMARY: The
      authors propose a new method for approximating a database in a
      differentially private manner when the database will be used to answer
      queries with certain smoothness properties. The method appears to be
      correct but is unclear if it is primarily theoretical interest or if it is
      practically relevant. STRENGTHS: - This is a new approach to
      database approximation for real data, a topic which has not received as
      much attention in the differential privacy literature. - The
      application of polynomial approximation is novel with respect to this
      problem (although similar ideas for related problems have been proposed
      before). WEAKNESSES: - It is unclear if the proposed method
      actually "works" in any meaningful sense. The authors are not very clear
      on this point. Since there is no experimental verification, it should be
      more clear that this is a primarily theoretical investigation. - It is
      not clear from the statement of the results if bounds are surprising, to
      be expected, or if there is a hope of matching lower bounds. This makes it
      hard to evaluate the "impact" of the result. COMMENTS: -
      Does it make sense that the smoothness of the function should depend on
      the data dimension? In particular, the fact that $K/d$ is the relevant
      quantity is not very intuitive. - The authors seem to want to make a
      big deal out of the fact that their approach allows "infinite number of
      queries," but this is pretty obvious because they are not really looking
      at a query model but instead a differentially private approximation of the
      data. In the discrete setting, if one makes a differentially private
      histogram, one gets the same thing. - I was hoping for more of a
      comparison with [18], which (as I recall) also looks at real-valued data
      and derives estimators based on a quantization approach. The authors cite
      this paper but do not provide any context. - Indeed, the whole "name
      of the game" here is density estimation, which is a well-studied area in
      statistics and machine learning. What does this literature have to say
      about the approach suggested by the authors? - Experiments,
      evaluation? This seems like a nice recipe for approximation but it is a
      little hard to see if, e.g. it will work at all for d of "practical"
      interest. Indeed some examples of useful smooth queries could be nice. Is
      this an interesting sub-class of problems that could be used as an
      example? UPDATE AFTER REBUTTAL: * After the response and
      discussion I remain sanguine about this paper. However, I would like many
      of the details from the author's rebuttal to appear in the manuscript --
      this will help clarify the presentation and might actually facilitate some
      discussion at NIPS. * The authors did not really address many of my
      comments, which is somewhat disappointing. I think an example of
      interesting smooth queries (e.g. in the introduction) will help ground the
      theory. Otherwise many of the more applied folks interested in privacy
      will say "oh, the authors show an algorithm for some version of smoothness
      for which they give no interesting examples except references to [23] and
      [28]." A concrete example of an interesting smooth query that is relevant
      to the NIPS audience will make this paper more accessible.


Online Learning with Switching Costs and Other Adaptive Adversaries
We study the power of different types of adaptive (nonoblivious) adversaries in the setting of prediction with expert advice, under both full-information and bandit feedback. We measure the player's performance using a new notion of regret, also known as policy regret, which better captures the adversary's adaptiveness to the player's behavior. In a setting where losses are allowed to drift, we characterize -in a nearly complete manner-the power of adaptive adversaries with bounded memories and switching costs. In particular, we show that with switching costs, the attainable rate with bandit feedback is � Θ(T 2/3 ). Interestingly, this rate is significantly worse than the Θ( √ T ) rate attainable with switching costs in the full-information case. Via a novel reduction from experts to bandits, we also show that a bounded memory adversary can force � Θ(T 2/3 ) regret even in the full information case, proving that switching costs are easier to control than bounded memory adversaries. Our lower bounds rely on a new stochastic adversary strategy that generates loss processes with strong dependencies.

This paper studies the problem of prediction with
      expert advice and finds matching upper and lower bounds on regret in terms
      of T (of T^{2/3}) in a number of settings: bandit feedback with
      adversaries of the form (bounded memory, size-1 memory, and imposing
      switching costs) and full-information feedback with adversaries with
      bounded memory. This is a fundamental question, and it's nice to
      see progress here. There are a couple of unusual things to note
      about the setting: - The definition of expected regret is not the
      usual one. Traditionally, one looks at the expected difference between the
      costs of the selected actions and the performance of the best fixed action
      in hindsight (meaning given the actual history of play). Instead, they
      look at the expected value of the difference between the actual
      performance and that of the best fixed action (not taking the actual
      history into account, but rather assuming that the fixed action had been
      played all along). This difference is salient because they consider
      adaptive adversaries who e.g. take into account a limited history when
      selecting their cost functions. The regret definition used here ("policy
      regret") seems to be the right one for these settings, and the paper
      mostly does a good job of making this distinction. But since this is a
      potential source of confusion, I would have preferred if the paper had
      been more explicit with references to the cases where similar results are
      known under the standard expected regret. Also, it's strange that Table 1
      summarizing results doesn't provide standard references for the results in
      prior work. - The results require a weakening of the usual assumption
      of bounded loss values (and hence the reproving of the corresponding upper
      bounds, which the paper does do). I still think the results are
      interesting, but of course it would be nicer to have results in the
      standard model. The paper gives a thorough treatment, with nice
      proof sketches of the main results in the main body of the paper, and a
      nice discussion of open questions.


Robust Low Rank Kernel Embeddings of Multivariate Distributions
Kernel embedding of distributions has led to many recent advances in machine learning. However, latent and low rank structures prevalent in real world distributions have rarely been taken into account in this setting. Furthermore, no prior work in kernel embedding literature has addressed the issue of robust embedding when the latent and low rank information are misspecified. In this paper, we propose a hierarchical low rank decomposition of kernels embeddings which can exploit such low rank structures in data while being robust to model misspecification. We also illustrate with empirical evidence that the estimated low rank embeddings lead to improved performance in density estimation.

The authors make a kernel extension of paper [1]. That
      is they give an svd based algorithm for low rank approximations of
      embeddings with an underlying latent structure. Weak guarantees are given
      under the assumption that low rank approximations of certain key objects
      have a low error. Experimental results are promising. Section 4 is
      cryptic. This is Section 3.1 in [1] in which the ideas seem more clear due
      to the simpler setting. Eq 7 (and the follow ups) are the point of concern
      ( I'm actually not sure how they define C_X|Z; before sec 3 they write
      mu_X|z = C_X|Z phi(z) so C_X|Z : H -> F; but in the sentence above it
      is said that C_X|Z acts on z. I think for 7 the phi definition is wrong
      and they mean the second (?) let's go with this one). They want to
      condition to extract the conditioning variable and show a low rank of the
      covariance matrix. I guess the argument is 1) condition
      with Z and average over Z. 2) Fubini theorem (?) for vector
      integrals(?) -- I guess technical details, e.g. when can we apply this for
      vector integrals, are not really of interest to the community and I'm
      happy to follow the heuristic argument of the authors 3) The next
      step is then really cryptic. I think what they do is to denote the vector
      (E_X1|Z=(1 0) [phi(X1)] E_X1|Z=(0 1) [phi(X1)] ) with E_X1|Z [phi(X1)] and
      then the multiple this with Z itself to regain the conditional
      expectation. The notation really needs to be cleaned up a lot.
      Actually it might be better to first go through a short version of Sec 3.1
      from [1] and just show what changes in the kernel setting. 4) The
      next step is then to substitute the cond exp vector with C_X1|Z . The
      whole thing is just super confusing since the notation is not explained
      and it is up to the reader to guess what the objects are. #
      I must admit that I found it hard to follow the algorithm
      discussion in Sec 5/6. E.g. in Sec 5 the svd of A_1 is understandable, but
      I guess ,this is the trivial part. The main algorithm seems to be then in
      the sentence: "This leads to the first intermediate tensor G_1 = U_r and
      we reshape S_r V_r and recursively decompose it" which I admit does not
      give me much of a picture of what is going on. As far as I get it they
      split (say for their example Fig 2 c) the problem into the first Z1- X1
      pair and all the rest and perform an svd on the covariance between the
      first X1 and this potentially really long tensor. I'm wondering
      here how this can lead to anything good. This sounds really like a
      formidable challenge if you have an averaged length HMM, say with d=100,
      to get anything of relevance from the svd?? # The bounds
      in sec 7 are a bit odd. First, the statement "More specifically, we will
      bound, in terms of generalized Frobenius norm || ... ||, the difference
      between the true kernel embeddings and the low rank kernel embeddings
      estimated from a set of n iid sample {..} " is misleading, since they make
      the assumption that they have already low rank approximations of different
      covariance operators with error at max epsilon. So what they really seem
      to do here is just to say if the individual estimates are very good then
      the chained together is not so bad either. But obviously getting these
      guarantees on the low rank approximations will be a real difficulty.
      The next thing with the bound is the 1/lambda^(d-2). lambda is the
      smallest singular value of a family of covariance matrices and d the
      dimension of the latent variable. I would expect that this lambda can get
      really small as you go over quite a number of matrices. The authors also
      argue in a similar line and say that lambda might be misleading and that
      in experiments the behaviour differs significantly from the exponential
      dependence of performance on lambda. So this seems to say that
      under the rather strong conditions they found a bound that is far from
      tight. Also, in the introduction they motivate with a mixture of
      Gaussians and argue that their estimate outperforms kernel density
      estimators. But there seems to be no statements to support this like
      asymptotic results or bounds. # In general there is
      significant overlap with the paper [1]. The authors say at one point that
      it is a kernel generalization of [1]. Some of the arguments look extremely
      similar like the low rank k thing in Sec 4 and Sec 3.1 in [1]. A clear
      discussion of what are the significant new findings compared to [1] must
      be given. # A spell-checker would also be much
      appreciated, so would be checking the math parts for typos and an overall
      polishing of the presentation. # The experimental results
      are the positive aspect of the paper where the method outperforms a number
      of competitors. # In summary I think that the paper has
      one main idea which is to kernelize the approach in [1]. The novelty
      itself does not feel ground breaking due to this. The paper is also
      lacking in presentation. I can't see people outside a small community to
      follow this paper through without significant difficulties. I think the
      main ideas could be nicely summarised in one or two paragraphs but it is
      currently a pain to extract these; the notation is guesswork and will
      frustrate a reader who is not from the field or wants to go into details.
      There is no theory to support the density estimation point they make and
      the covariance approximation bounds are also not super significant since
      they make strong assumptions and the bounds seem to be not very tight.
      Also the link to paper [1] needs to be pointed out clearly with a proper
      discussion. On the plus side is Table 1 with the experimental results
      which seem promising. [1] Hierarchical Tensor Decomposition of
      Latent Tree Graphical Models. Le Song et al 2013 ICML.


Minimax Theory for High-dimensional Gaussian Mixtures with Sparse Mean Separation
While several papers have investigated computationally and statistically efficient methods for learning Gaussian mixtures, precise minimax bounds for their statistical performance as well as fundamental limits in high-dimensional settings are not well-understood. In this paper, we provide precise information theoretic bounds on the clustering accuracy and sample complexity of learning a mixture of two isotropic Gaussians in high dimensions under small mean separation. If there is a sparse subset of relevant dimensions that determine the mean separation, then the sample complexity only depends on the number of relevant dimensions and mean separation, and can be achieved by a simple computationally efficient procedure. Our results provide the first step of a theoretical basis for recent methods that combine feature selection and clustering.

The paper studies the problem of identifying Gaussians
      in a mixture in high dimensions when the separation between the Gaussians
      is small. The assumption is that the Gaussians are separated along few
      dimensions and hence by identifying these dimensions, that is, feature
      selection, the curse of dimensionality can be bitten and the Gaussians can
      be found. Clustering in high dimension is an open problem that
      well deserve a study. The theoretical approach taken by the authors is
      good step in the path towards better understanding the problem. However, I
      have two reservations about this work. In Line 36, the authors say “there
      appears to be no theoretical results justifying the advantage of variable
      selection in high dimension setting”. As far as I understand, K. Chaudhuri
      and S. Rao, ("Learning Mixtures of Product Distributions using
      Correlations and Independence", In Proc. of Conf. on Learning Theory,
      2008) tried to address similar problem of learning mixture models in high
      dimension under the assumption that the separation between the components
      is in a small, axis aligned space. While the results there are different
      then the results here, I was missing a comparison between the results.
      In their response to the original review, the authors have
      addressed the differences between the current work and the work of
      Chaudhuri and Rao. I am confident that the work here contains novel
      results that should be published. However, I think that more careful
      thought needs to be given to presenting the novelty and contrasting it
      with the state of the art. The main message of the authors in their
      response is “these methods [the methods of Chaudhuri and Rao] do not
      explicitly provide variable selection guarantees, i.e. the methods are not
      guaranteed to correctly identify the relevant features.” However, in the
      paper, the authors present as the main result “In this paper, we provide
      precise information theoretic bounds on the clustering accuracy and sample
      complexity of learning a mixture of two isotropic Gaussians”. Therefore,
      the main goal is the accuracy of the separation as opposed to
      identification of relevant features. Therefore, I find the main
      contribution of this paper to be vague. Since I believe that this paper
      contains significant results, I think that the authors should reconsider
      the presentation to make a clear statement about the objectives and the
      contribution. I also had some problems following the proof of
      Theorem 2 but the authors clarified it in their response.
      Comments: 1. Theorem 4: The term \frac{d-1}{4}+1 could be
      simplified to \frac{d+3}{4} 2. Lemma 1: I believe you could improve
      the bound on M to $2^{7m/16}$ by a simple covering argument: $\Omega$ is
      of size $2^m$. Every $\omega_i$ “covers” a ball of size $m/8$. A ball of
      size $m/8$ contains $\sum_{i\leq m/8} \choose{m}{i}$ points from $\Omega$
      which can be bounded (Sauer’s Lemma) by (8e)^{m/8} which is bounded by
      2^{9m/16}. Therefore, if we choose $M=2^{7m/16}$ we have that M balls of
      radius $m/8$ do not cover $\Omega$ and hence there is a $m/8$ packing
      (which is what you are looking for) of size $M$. (This is not a major
      issue and you may decide to ignore it in favor of the simplicity of the
      statement). 3. Lemma 2: The bound on M here is stated with
      logarithms while the bound in Lemma 1 does not use logarithms which makes
      it harder to follow. 4. Proposition 3: what are $\phi$ and $\Phi$?
      5. Proof of Theorem 2, line 275: What is $\theta_\nu$, where is it
      introduced? 6. Proof of Theorem 2, line 279: It is not clear why on
      the r.h.s. there is $\frac{\sqrt{d-1}\espilom}{\lambda}$. Using the term
      for the cosine of \beta and since $sin(\beta) = \sqrt{1-cos^2(\beta)$ I
      get that $sin(\beta) = \frac{2\rho\epsilon^2}{\lambda^2} \geq
      \frac{2(d-1)\epsilon^2}{\lambda^2}$ and this $tan{\beta} \leq
      \frac{4(d-1)\epsilon^2}{\lambda^2}$ 8. Line 466: you use full
      names for all references but Tsybakov’s paper.


Learning a Deep Compact Image Representation for Visual Tracking
In this paper, we study the challenging problem of tracking the trajectory of a moving object in a video with possibly very complex background. In contrast to most existing trackers which only learn the appearance of the tracked object online, we take a different approach, inspired by recent advances in deep learning architectures, by putting more emphasis on the (unsupervised) feature learning problem. Specifically, by using auxiliary natural images, we train a stacked denoising autoencoder offline to learn generic image features that are more robust against variations. This is then followed by knowledge transfer from offline training to the online tracking process. Online tracking involves a classification neural network which is constructed from the encoder part of the trained autoencoder as a feature extractor and an additional classification layer. Both the feature extractor and the classifier can be further tuned to adapt to appearance changes of the moving object. Comparison with the state-of-the-art trackers on some challenging benchmark video sequences shows that our deep learning tracker is more accurate while maintaining low computational cost with real-time performance when our MATLAB implementation of the tracker is used with a modest graphics processing unit (GPU).

This paper presents an application of deep learning to
      visual tracking. Under the framework of particle filter, this paper builds
      a likelihood model based on learned image representation from stacked
      denoising auto-encoder (SDAE), and achieves good results on 11 selected
      videos compared to previous trackers. The reviewer will evaluate this
      paper from four aspects below. 1. Quality The main algorithms
      (particle filter and SDAE) used in this paper are well known and should be
      correct. Unsupervised feature learning is also well established for image
      representation. There is no surprise when it works reasonably well for
      representing tracking targets. It will be good that the authors can
      present some insights that how the deep learning here help solve any
      intrinsic problems of visual tracking? For example, is the learned
      representation robust to occlusions and motion blur? Do the learned
      features make the visual matching easier, or tracking targets more
      salient? In addition, the SDAE is online updated while tracking. The
      reviewer would like to know if this will cause over-fitting and thus
      tracking drift. 2. Clarity This paper is clearly written and
      easy to read. Since the source codes of SDAE and particle filter are
      public available, the results should be easily reproduced. 3.
      Originality This paper is an incremental work. It shares similar ideas
      and algorithm framework with [22] of combining feature learning with
      particle filter. The only difference is that this paper use SDAE for
      feature learning while [22] uses sparse coding for feature extraction.
      Although this paper learns features from pixels while [22] learns features
      from SIFT, no experimental results show the advantages of SDAE in this
      paper over [22]. The reviewer suggests the authors present thorough
      comparisons with [22] so as to make explicit the merits of the proposed
      algorithm. 4. Significance Since the SDAE and particle filter
      are well known in the community, the direct combination of them bring
      little new insights towards both deep learning and visual tracking.


Data-driven Distributionally Robust Polynomial Optimization
We consider robust optimization for polynomial optimization problems where the uncertainty set is a set of candidate probability density functions. This set is a ball around a density function estimated from data samples, i.e., it is data-driven and random. Polynomial optimization problems are inherently hard due to nonconvex objectives and constraints. However, we show that by employing polynomial and histogram density estimates, we can introduce robustness with respect to distributional uncertainty sets without making the problem harder. We show that the optimum to the distributionally robust problem is the limit of a sequence of tractable semidefinite programming relaxations. We also give finite-sample consistency guarantees for the data-driven uncertainty sets. Finally, we apply our model and solution method in a water network optimization problem.

The authors considered robust optimization for
      polynomial optimization problems where the uncertainty set is a set of
      possible distributions of the parameter. In specific, this set is a ball
      around a density function estimated from data samples. The authors showed
      that this distributionally robust optimization formulation can be reduced
      to a polynomial optimization problem, hence computationally the robust
      counterpart is of the same hardness as the nominal (non-robust) problem,
      and can be solved using a tower of SDP known in literature. The authors
      also provide finite-sample guarantees for estimating the uncertaity set
      from data. Finally, they applied their methods to a water network problem.
      This seems to be an interesting paper. Distributionally robust
      optimization is now topical in operations research. Most previous
      literature is restricted to the case that the distribution is
      characterized by (first and second) moments, which has the apparent
      disadvantage that consistency is impossible regardless of the number of
      samples obtained. In contrast, the proposed approach overcomes this
      difficulty, as when more samples are obtained, we can shrink the
      uncertainty set until it becomes a singleton, and hence achieving
      consistency. It is also nice to see that, at least for the polynomial
      optimization problems, the resulting robust counterpart does not incur
      additional computational difficulty. Some minor points: 1.
      Assumption 4.1 is a bit confusing. In particular, I am not sure how the
      sentence "and there exist u\in R[x]... compact." is related to the set X
      and S. 2. In line 194, "\hat{S} satisfies Assumption 4.1". Why? 3.
      L236, "We consider the multi-dimensional case in the next section" appears
      to be wrong. 4. L375 "Table 6". There is no table 6 in the submission.
      5. L379 "Un-biased". I am not sure why the uncertainty set are
      unbiased (in fact I am even sure what does it mean that a set is
      unbiased). 6. L148, "A notion of distributional uncertainty set has
      also been studied in the setting of Markov decision problems [14]". I
      think [14] does not consider distributional uncertainty. To my knowledge,
      distributional uncertainty in MDP is investigated in Xu and Mannor (2012).
      Ref: H. Xu, S. Mannor, "Distributionally Robust Markov
      Decision Processes", Mathematics of Operations Research, 37(2), pp288-300,
      2012.


Compressive Feature Learning
This paper addresses the problem of unsupervised feature learning for text data. Our method is grounded in the principle of minimum description length and uses a dictionary-based compression scheme to extract a succinct feature set. Specifically, our method finds a set of word k-grams that minimizes the cost of reconstructing the text losslessly. We formulate document compression as a binary optimization task and show how to solve it approximately via a sequence of reweighted linear programs that are efficient to solve and parallelizable. As our method is unsupervised, features may be extracted once and subsequently used in a variety of tasks. We demonstrate the performance of these features over a range of scenarios including unsupervised exploratory analysis and supervised text categorization. Our compressed feature space is two orders of magnitude smaller than the full k-gram space and matches the text categorization accuracy achieved in the full feature space. This dimensionality reduction not only results in faster training times, but it can also help elucidate structure in unsupervised learning tasks and reduce the amount of training data necessary for supervised learning.

The method consists of compressing documents for
      accelerating the document processing in other tasks, such as
      classification. The coding scheme is related to the one used in the
      Lempel-Ziv algorithm, storing pointers of substrings appearing at
      several locations in the document. The proposed approach is formulated
      as a combinatorial optimization problem, whose solution is
      approximated by a sequence of convex problems solved by ADMM. The
      experiments are carried on text classification problems, where
      compressing the documents leads to some gains in memory and
      computational efficiency, at a minor loss in terms of precision.
      I found the approach interesting, even though I am not familiar
      enough with the NLP literature to exactly judge the novelty of the
      approach. I have only a few minor remarks to make - the sentence
      ``an optimal lossless compression of D...'' requires some
      clarifiation. Is the coding scheme optimal in terms of minimum
      entropy? - it is not clear that the reweighting scheme can be
      interpreted here as a majorization-minimization procedure. Is it
      really the case here? - minimizing (4) with respect to w amounts to
      computing the Fenchel conjugate of the (weighted) l_infty-norm, which
      involves a projection on a weighted l1-norm (the dual norm of the
      l_infty-norm). When adding the non-negativity constraint, this
      involves a projection on the simplex. Algorithms for projecting on the
      simplex have been known for a while, and are similar to the approach
      described in the paper. See Brucker. An O(n) algorithm for quadratic
      knapsack problems. 1984. see also Bach et al. Optimization with
      Sparsity-Inducing Penalties. 2012, for the computation of Fenchel
      conjugates for norms.


Correlated random features for fast semi-supervised learning
This paper presents Correlated Nyström Views (XNV), a fast semi-supervised algorithm for regression and classification. The algorithm draws on two main ideas. First, it generates two views consisting of computationally inexpensive random features. Second, multiview regression, using Canonical Correlation Analysis (CCA) on unlabeled data, biases the regression towards useful features. It has been shown that CCA regression can substantially reduce variance with a minimal increase in bias if the views contains accurate estimators. Recent theoretical and empirical work shows that regression with random features closely approximates kernel regression, implying that the accuracy requirement holds for random views. We show that XNV consistently outperforms a state-of-the-art algorithm for semi-supervised learning: substantially improving predictive performance and reducing the variability of performance on a wide variety of real-world datasets, whilst also reducing runtime by orders of magnitude.

Summary: This paper brings together two recently
      popular trends of research, namely random features and multiview
      regression. It provides a two step algorithm for semi-supervised learning.
      In the first stage, they generate random features corresponding to two
      views using the Nystrom method and then in the second stage, they use CCA
      to bias the optimization towards features correlated across the views via
      the canonical norm which penalizes less correlated features across the two
      views more. The experimental results show the superior performance of
      their approach over a state-of-the-art competitor Laplacian Regularized
      Least Squares (SSSL) Comments: The paper is well
      motivated and addresses an important problem. The paper operationalizes
      the ideas proposed in (Kakade and Foster 2007) and presents a Nystrom
      method to generate two sets of views which follow the multiview assumption
      (also detailed in (Kakade and Foster 2007)). In short, the paper
      harnesses the CCA results from (Kakade and Foster 2007) and Nystrom Method
      results from (Bach 2013) to come up with an efficient and scalable semi
      supervised learning algorithm. That said, I found the paper to
      have limited mathematical novelty for a venue like NIPS and is mostly an
      engineering paper. The authors could have considered joint
      learning of the random features and CCA bases, which would have been more
      novel and could possibly lead to better error bounds than the two methods
      separately. The experimental results are detailed and the proposed
      approach (XNV) significantly beats SSSL. The authors should
      consider adding an additional tuning parameter (\lambda) for the canonical
      norm in Algorithm 1 (9), which can hopefully improve accuracy further as I
      can see the term to be on a different scale than the other two terms in
      the objective.


A Novel Two-Step Method for Cross Language Representation Learning
Cross language text classification is an important learning task in natural language processing. A critical challenge of cross language learning arises from the fact that words of different languages are in disjoint feature spaces. In this paper, we propose a two-step representation learning method to bridge the feature spaces of different languages by exploiting a set of parallel bilingual documents. Specifically, we first formulate a matrix completion problem to produce a complete parallel document-term matrix for all documents in two languages, and then induce a low dimensional cross-lingual document representation by applying latent semantic indexing on the obtained matrix. We use a projected gradient descent algorithm to solve the formulated matrix completion problem with convergence guarantees. The proposed method is evaluated by conducting a set of experiments with cross language sentiment classification tasks on Amazon product reviews. The experimental results demonstrate that the proposed learning method outperforms a number of other cross language representation learning methods, especially when the number of parallel bilingual documents is small.

The paper proposes a matrix completion approach to the
      cross domain classification task, which is capable of exploiting labeled
      data in an auxiliary domain and also unlabeled parellel data. The approach
      involves two steps. First, it constructs a single incomplete matrix that
      includes all documents and domains, and then completes this matrix while
      enforcing low-rank and sparsity conditions using the projected gradient
      descent algorithm. Second, it reduces the feature dimension of this
      completed matrix using LSI, and then trains a standard classifier on this
      new representation. The paper also presents a convergence guarantee on the
      projected gradient descent algorithm, and favorable experimental results.
      Here are some comments and questions: - After estimating
      the completed matrix M*, you reduce the feature dimension from d to k with
      LSI because M* "lacks sufficient capacity of handling feature sparseness".
      Can you elaborate on this? Is it basically that you want small dimensions
      for computational convenience, since M* will be relatively dense even with
      the sparsity enforcement? Also, in that case, is there a reason you chose
      LSI over other dimensionality reduction techniques (e.g., PCA)? -
      There seem to be two disjoint aspects of exploiting external data in this
      paper: 1. leveraging the labeled source language data to help learn the
      labeled target language data, and 2. leveraging the unlabeled parallel
      data. If this is correct, it would be helpful to clearly distinguish the
      two and report results separately. - On a related note, the
      experiment settings are a bit unclear (section 5.2). So you were using (a)
      4k labeled source language documents, (b) 100 labeled target language
      documents, and (c) 2k unlabeled parallel documents for your training data?
      And you are reporting the classification results on the remaining 1900
      labeled target language documents? Then the only difference in setion 5.3
      is that you are varying the number of unlabeled parallel documents from
      200 to 2k? In that case, why is the performance of CL-KCCA better than TSL
      in Figure 1 for EFM and EGM at 2k but it is not in Table 1? - Can
      you briefly clarify why a fully observed document-term matrix would be
      low-rank? It is sparse, but probably no document is a linear combination
      of other documents in vocabulary, meaning the rank in that case will be
      min(d, k). - Can you give an intuition on why TSL is able to
      perform better in general than other methods, especially CCA? A multiview
      approach like CCA feels like a natural approach to this cross language
      representation learning task. In contrast, the matrix completion approach
      is not as natural. It seems the main driving force is enforcing the
      low-rank condition on M*. How does this compare with other techniques?
      - What is the running time of the algorithm? Does the projected
      gradient descent algorithm take more, less, or about the same time as
      other methods like CL-LSI, CL-KCCA, and CL-OPCA? Here are some
      suggested changes: - Fix (ij) to (i,j) two sentences before
      equation (2). - Remove "such as" in the sentences before equation (4)
      and (5).


Latent Structured Active Learning
In this paper we present active learning algorithms in the context of structured prediction problems. To reduce the amount of labeling necessary to learn good models, our algorithms operate with weakly labeled data and we query additional examples based on entropies of local marginals, which are a good surrogate for uncertainty. We demonstrate the effectiveness of our approach in the task of 3D layout prediction from single images, and show that good models are learned when labeling only a handful of random variables. In particular, the same performance as using the full training set can be obtained while only labeling ∼10% of the random variables.

The authors extend the work of [Schwing et al.,
      Efficient Structured Prediction with Latent Variables for General
      Graphical Models; ICML 2012] to include active learning protocols. In
      essence, they use the entropy of the local variable marginals to estimate
      a value of uncertainty -- so this can be viewed as local variable
      uncertainty sampling for structured predictions with partially observed
      variables. To this end, they propose two active learning variants, namely
      (1) separate active: each active learning round requires a "separate"
      inference step over the unlabeled and partially labeled variables after
      learning on the observed variables and (2) joint active: each active
      learning round follows a joint learning procedure over the labeled,
      partially labeled, and unlabeled instances. They also explore batch sizes
      and warm-starting of the learning procedure between rounds in this
      setting. The empirical evaluation is 3D layout of rooms (a computer vision
      problem explored in [Scwing et al., ICML 2012], demonstrating that (1)
      "joint" active learning works the best, achieving an annotation savings of
      ~90%, (2) batch mode works for reasonable small batches, (3) querying
      "full" vs. "partial" labels [unsurprisingly, partial labels works better]
      (4) sensitivity to \epsilon and (5) computational reuse saves time
      [unsuprisingly]. Overall, it is a nice paper that shows active
      learning works in a challenging setting with this particular formulation.
      I will address the {quality, clarity, originality, significance}
      components separately below. Quality: The paper is certainly of
      high quality. The model for structured learning with latent variables is
      state of the art and the active learning method achieves dramatic savings
      over just using random sampling (which actually makes quite a bit of sense
      in the latent variable case when you think about it). My biggest complaint
      from an active learning perspective is that the saving are somewhat
      disingenuous (even if not intentionally so) -- while the "$1 for 1 label"
      simulations were very popular from ~1995-2008ish, I think it has been
      accepted that these sorts of simulations aren't very realistic since
      active learning often picks the most difficult examples for labeling
      (e.g., [Kapoor et al., Selective supervision: Guiding supervised learning
      with decision-theoretic active learning; IJCAI 2007],[Settles et al.,
      Active learning with real annotation costs; NIPS WS 2008], [Ringger et
      al., Assessing the costs of machine-assisted corpus annotation through a
      user study; LREC 2008], [Haertel et al., Return on investment for active
      learning; NIPS WS 2008], [Arora et al., Estimating annotation cost for
      active learning in a multi-annotator environment; NAACL WS 2009],
      [Baldridge and Palmer, How well does active learning actually work?
      Time-based evaluation of cost-reduction strategies for language
      documentation; EMNLP 2009], [Wallace et al., Modeling Annotation Time to
      Reduce Workload in Comparative Effectiveness Reviews; IHI 2010]...amongst
      others. I think there should at least be some discussion along these lines
      (although the full vs. partial labels experiments do at least hint at this
      a bit). My second "active learning" concern is a bit on the semantics
      side, but both [Culotta and McCallum, AAAI 2005] and [Roth and Small, ECML
      2006] consider partial labels during querying time -- which is somewhat in
      contrast to the last paragraph of page 1. I think what you mean to say is
      that the output space is fully observable or something along these lines,
      as they certainly do not require "full" labels from the querying function
      (and I actually don't think this is what you meant) Clarity: For
      the most part, the paper is clear -- although I had to go read [Scwing et
      al., ICML 2012] to understand some of the details. The biggest thing I
      noticed in this case is (1) I didn't completely understand the "task-loss"
      function until I read the previous work; some example/discussion here
      would be nice (2) I think the notation for this loss function may have
      suffered from some cutting/pasting from the ICML paper as there appears to
      be some confusion between s and y here and there (assuming I am
      understanding everything correctly) and (3) you never say anything about
      $\ell^c$ -- which may be difficult to digest for those without explicit
      graphical models expertise. Originality: While the step is
      straightforward (and based on uncertainty sampling), it hasn't been done
      before in this setting. However, I also think anybody with knowledge of
      these areas could have put these elements together. Significance:
      I would guess that this paper will not have the same sort of impact as
      [Scwing et al., ICML 2012] as it is almost a "companion piece". The ideas
      are relatively straightforward, but done well. Probably having the code
      would be more useful than reading the paper as there are no theoretical
      results or anything along these lines and the setting of $\epsilon$ seems
      non-trivial in deployed settings. Therefore, I would say it is
      interesting, but not groundbreaking in this regard. A few small
      comments: abstract: -> $\sim 10\%$ (get rid of the weird spacing)
      pg 1: "have to deal with exponentially sized output variables" ->
      "have to deal with output spaces which are exponentially sized in the
      number of local variables" (or something along these lines) pg 2:
      "that employ both fully labeled training sets" As I previously stated, I
      think there is a subtle distinction between "labeled" and "observed" --
      but this is obviously your call if you agree pg 5: "as well as the
      messages $\lambda$ and believes" -> "... beliefs"


Understanding Dropout
Dropout is a relatively new algorithm for training neural networks which relies on stochastically "dropping out" neurons during training in order to avoid the co-adaptation of feature detectors. We introduce a general formalism for studying dropout on either units or connections, with arbitrary probability values, and use it to analyze the averaging and regularizing properties of dropout in both linear and non-linear networks. For deep neural networks, the averaging properties of dropout are characterized by three recursive equations, including the approximation of expectations by normalized weighted geometric means. We provide estimates and bounds for these approximations and corroborate the results with simulations. Among other results, we also show how dropout performs stochastic gradient descent on a regularized error function.

This work aims to shed light to the novel neural
      network algorithm of dropout, but grounding it to theory and through this
      provide guidance of how relevant parameters should be chosen to achieve a
      good performance. Quality: The quality of the work is high, the
      quality of the manuscript can be however, improved: there are several
      typos, or missing symbol definitions. A summary of the findings at the end
      would also enhance the readability of the manuscript. Clarity: The
      paper is overall clearly written subject to typos, definitions and summary
      missing (see above). A non-exhaustive list of suggested corrections is
      given below: -Introduction, line 7, there may be a typo/ ambiguous
      -Eq (1): l needs to be defined -3.1, n needs to be defined. Is
      stochasticity is S due to dropout? Please explain m=2^n (sems to be the
      number of different inputs in the network). Typo in equation 7 -5.1
      typo in eq 25, typo right after eq 28 (an extra fullstop) -5.2 first
      line: in the definition of the sigmoidal a + is missing. -the caption
      layout of the figures is non consistent -references in the text start
      from number [6] -A summary /conclusions section is missing
      Originality: To the best of my knowledge, the paper is
      original, providing answers to some key questions regarding the setup of a
      network with dropout. Significance: In my view the article is
      significant, and I was wondering if parallels could be drawn between the
      dropout algorithm, applied to the connecting weights rather than the
      nodes, with the release probability of biological synapses, as those may
      be unreliable, effectively implementing a “dropout”
      mechanism.


Parallel Sampling of DP Mixture Models using Sub-Clusters Splits
We present an MCMC sampler for Dirichlet process mixture models that can be parallelized to achieve significant computational gains. We combine a nonergodic, restricted Gibbs iteration with split/merge proposals in a manner that produces an ergodic Markov chain. Each cluster is augmented with two subclusters to construct likely split moves. Unlike some previous parallel samplers, the proposed sampler enforces the correct stationary distribution of the Markov chain without the need for finite approximations. Empirical results illustrate that the new sampler exhibits better convergence properties than current methods.

The authors propose a parallel algorithm for the DPMM
      that parallelizes a RJMCMC sampler that jumps between finite models. While
      the parallelization and the RJMCMC sampler are proposed together, I will
      separate them for the purpose of this review, in order to ask questions
      about each part separately. First, the RJMCMC algorithm (by which
      I mean, the algorithm we would have on a single cluster). Here, we use a
      reversible-jump MCMC algorithm to jump between finite-dimensional
      Dirichlet distributions. As an aside, since $\bar{\pi}_{K+1}$ is not used
      in the mixture model (the mixture model is defined on the renormalized
      occupied K components), it would seem to make more sense to define a
      K-dimensional, rather than a K-1 - dimensional, Dirichlet distribution;
      this is valid under marginalization properties of the Dirichlet
      distribution, since equation 10 samples from a distribution proportional
      to $\pi_1 ... \pi_K$ To jump between model dimensionalities, the
      authors propose a split/merge RJMCMC step that is reminiscent of that of
      Green and Richardson. In fact, the “naive” choice is very similar to the
      Green and Richardson method, and I am not sure why it is not valid in this
      case.. I would like to see a more detailed comparison of the Green and
      Richardson method (and other split/merge methods) and the proposed method,
      and the reasons for the different design choices. The
      parallelization method splits the clusters onto processors with
      probability related to the similarity between clusters. Each data point
      can only be assigned to processors within its own cluster. The split/merge
      method of creating new clusters, and the fact that atoms are explicitly
      instantiated, ensures that there is no alignment problem of new clusters
      across processors. Update of cluster parameters can be done in parallel;
      however contrary to line 194 I do not see how sampling the $\pi_k$ can be
      done in parallel. Further, sampling the superclusters must be done in
      parallel. The experiments suggest improved performance; however I
      would be interested in seeing further experimental evaluation and
      information. What is the number of superclusters? How often are $\pi$ and
      the super-clusters updates? How does performance scale with number of
      clusters? Also, it would be interesting to discuss why your split/merge
      sampler performs so much better than the other split/merge sampler.
      Additional points: Line 107: The Dirichlet process can
      be sampled by normalizing an infinite sequence of gamma random variables,
      not beta random variables (or more accurately, by normalizing a sample
      from a gamma process). The Sethuraman paper cited describes an iterative
      algorithm for sampling size-biased samples from a DP, but it is based on
      the fact that the DP has Dirichlet marginals rather than normalization of
      an unnormalized random measure. Line ~079, line ~131: In addition
      to the approximate truncated and finite methods mentioned, there exist
      instantiated-weight samplers that are asymptotically correct -- for
      example slice sampling and retrospective sampling approaches, or the
      recent paper by Favaro & Teh


Approximate inference in latent Gaussian-Markov models from continuous time observations
We propose an approximate inference algorithm for continuous time Gaussian Markov process models with both discrete and continuous time likelihoods. We show that the continuous time limit of the expectation propagation algorithm exists and results in a hybrid fixed point iteration consisting of (1) expectation propagation updates for discrete time terms and (2) variational updates for the continuous time term. We introduce postinference corrections methods that improve on the marginals of the approximation. This approach extends the classical Kalman-Bucy smoothing procedure to non-Gaussian observations, enabling continuous-time inference in a variety of models, including spiking neuronal models (state-space models with point process observations) and box likelihood models. Experimental results on real and simulated data demonstrate high distributional accuracy and significant computational savings compared to discrete-time approaches in a neural application.

-- Update Just a few other thoughts. Simo Saarka
      has more recent work on continuous-discrete time systems (as you
      referenced) that might be interesting to contrast against - there, using
      Gaussian cubature that provides a nice alternative deterministic
      approximation method. This might be useful for additional discussion and
      future work. In addition I also now wondered if it is possible to derive
      an algorithm directly using the variational Gaussian approach. This would
      be more appealing from the point of having a well defined objective
      function with which to optimise, potentially fewer numerical issues and
      interpretation directly in terms of the marginal likelihood. We could
      afterwards add low-order marginal corrections using cumulant perturbations
      (like those of Opper for EP) - the only place I think shows this is Barber
      and van de Laar (http://arxiv.org/pdf/1105.5455.pdf). I do look forward to
      reading the final version of the paper. -- Original The paper
      presents an algorithm for approximate Bayesian inference in models
      with continuous and discrete time observations. The model can be
      cast in the framework of latent Gaussian models and a parallel
      expectation propagation algorithm can be used to derive a principled
      approach for inference and learning dealing with both continuous and
      discrete time. This EP inference algorithm is embedded within an EM
      algorithm to both learn parameters of the model as well as marginal
      distributions. The algorithm is shown to be effective in the number of
      experimental settings. Overall I enjoyed the paper and thoughts
      that it extends the applicability of approximately message passing to
      a wider class of models. In particular I thought it was
      interesting that EP updates for a continuous time limit collapsed to
      the variational Gaussian updates. This is related to the latent
      Gaussian structure but I wondered if there is a deeper reason
      underlying this connection. The algorithm seems robust due to be
      implied fractional updating but I wondered if you could comment on any
      experienced difficulties in implementation, such as issues of slow
      convergence of parameter learning, numerical stability, etc.
      The algorithm is still cubic due to the inverses is in the
      inference as well as the M-step updating - could comment on approaches
      to scaling up such algorithms. In the experimental section it
      would be nice to see plots giving insight into the convergence of the
      algorithm. Can we also demonstrate the advantages obtained by of
      having an estimate of the marginal likelihood. For example, it could
      be possible in figure 3C to plot each of the individual points with a
      size proportional the marginal likelihood value.


Linear Convergence with Condition Number Independent Access of Full Gradients
For smooth and strongly convex optimizations, the optimal iteration complexity of the gradient-based algorithm is O( √ κ log 1/ǫ), where κ is the condition number.
In the case that the optimization problem is ill-conditioned, we need to evaluate a large number of full gradients, which could be computationally expensive. In this paper, we propose to remove the dependence on the condition number by allowing the algorithm to access stochastic gradients of the objective function. To this end, we present a novel algorithm named Epoch Mixed Gradient Descent (EMGD) that is able to utilize two kinds of gradients. A distinctive step in EMGD is the mixed gradient descent, where we use a combination of the full and stochastic gradients to update the intermediate solution. Theoretical analysis shows that EMGD is able to find an ǫ-optimal solution by computing O(log 1/ǫ) full gradients and O(κ 2 log 1/ǫ) stochastic gradients.

quality: 7 (out of 10) clarity: 8 originality:
      8 significance: 7 SUMMARY: The authors consider the problem of
      optimizing a smooth and strongly convex function over a convex constraint
      set such that the gradient mapping update can be computed efficiently. The
      optimal first-order algorithm of Nesterov has linear convergence for such
      problem but the constant depends on the square root of the condition
      number k. The authors consider the situation where one has access to the
      expensive full gradient of the objective as well as a cheap stochastic
      gradient oracle. They propose a hybrid algorithm which only requires O(log
      1/eps) calls to the full gradient oracle (independent of the condition
      number) and O(k^2 log(1/eps)) calls to the cheaper stochastic gradient
      oracle -- as long as the condition number is not too big, this could be
      faster in theory. The main idea behind their algorithm(called Epoch Mixed
      Gradient Descent - EMGD) is to replace a full gradient step (called an
      epoch) with a fixed number O(k^2) of mixed gradient steps which use a
      combination of the full gradient (computed once for the epoch) and
      stochastic gradients (which vary within an epoch). By taking the average
      of the O(k^2) iterates within an epoch, they can show a constant decrease
      of the suboptimality *independent* of the condition number, which is why
      the number of required full gradient step computations (the number of
      epochs) is independent from the condition number. They provide a simple
      and complete self-contained proof of their convergence rate, but no
      experiment. I enjoyed reading this paper and I think that the idea
      of mixing a few full gradient computations with a large number of cheap
      stochastic gradient steps is novel and interesting. This work situates
      itself in a string of recent papers which attempt to use the cheaper
      stochastic oracle while maintaining a linear convergence rate. A recent
      theoretical AND practical breakthrough was made with the SAG algorithm
      [16] which works for a smooth strongly convex objective which is the sum
      of n simple functions (such as in regularized empirical loss minimization
      - where n is the number of training examples). In this case, a reasonable
      assumption is that the full gradient oracle is n times more expensive to
      compute than the stochastic gradient one. Then SAG is faster in theory to
      Nesterov' algorithm as long as the condition number k < = n/8. In
      contrast, EMGD in this case is faster to Nesterov' algorithm in theory as
      long as the condition number k < = n^(2/3), and EMGD is slower than SAG
      in all regimes (it has the same big O when k < = n^1/2). To get a sense
      of these speed-ups, if k = n^1/2, then both SAG & EMGD are O(n
      log(1/eps)) whereas Nesterov' algorithm O(n^(5/4) log(1/eps)). The two
      main advantages that I see for EMGD over SAG are that 1) as mentioned by
      the authors, EMGD works for constrained optimization [supposing that the
      gradient mapping update can be computed efficiently] whereas SAG is only
      defined so far for unconstrained problems; and 2) the convergence proof
      for EMGD is much simpler than the one for SAG and so could yield more
      insights as well as make the modifications to EMGD more amenable to
      provable guarantees [their result is also stronger as it holds with high
      probability vs. in expectation for SAG]. The authors also mention a
      possible memory / parallelization advantage, though this is less clear as
      SAG can also be parallelized using mini-batches (which also reduces the
      memory requirement by the size of the mini-batches). EVALUATION
      SUMMARY: Pros: - Gives a novel and interesting algorithmic idea
      with a clean, simple and solid theory. - Like SAG, get a linear
      convergence rate for regularized empirical loss minimization where the
      condition number k is not multiplying the number of training examples n in
      the constant; but their algorithm is more general (works for constrained
      optimization) and the proof is much simpler. - The paper is clearly
      written and the proof is self-contained. Cons: - There is no
      experiment which could show that this algorithm could actually make a
      difference in practice (and doesn't provide any concrete example to
      illustrate why its suggested theoretical advantages could be relevant in
      practice). - There is no discussion of the limitations / drawbacks of
      the algorithm (especially, in comparison to the existing algorithms --
      section 3.4 should be improved! I make several suggestions in this
      review). - The proof is lacking some high-level comments which could
      justify the essential insights used to its construction. QUALITY:
      The paper is technically sound. Some experiments would have been
      appreciated, though I think that the theoretical contribution could stand
      on its own. The authors should definitively extend section 3.4 with the
      limitations and drawbacks of their algorithm though. They should also add
      a more concrete discussion of the sum of n functions example which
      highlights nicely the differences with SAG and Nesterov (as I mentioned
      above -- e.g. EMGD is worse than Nesterov for (roughly) k > n^(2/3);
      SAG is same as EMGD for 1 < = k < = n^(1/2); better than EMGD (and
      Nesterov) for n^(1/2) < k < = n/8; and worse than Nesterov for k
      > n/8). A major drawback of the EMGD for the practitioner is that the
      number of steps within an epoch needs to be fixed in advance (with the
      knowledge of k) -- in contrast, both SCDA [17] and SAG don't have to fix
      the number of steps in advance and so can benefit for having a faster
      practical convergence than the bound would predict (or benefit from better
      local condition number e.g.). Moreover, SCDA has automatic step-size
      selection; whereas SAG has an adaptive step-size heuristic which seems to
      work quite well in practice. The authors should add this discussion in the
      paper. ** [ADDENDUM after discussion with other reviewers: a) As
      another reviewer mentioned, often in practice in ML the regularization
      parameter is C/n and so the condition number is ~= n/C', which is a regime
      in which SAG still does better than Nesterov, but in which EMGD *doesn't*.
      This should be pointed out (and perhaps another practical setting where k
      < n^(2/3) should mentioned). b) The authors should cite [Hybrid
      Deterministic-Stochastic Methods for Data Fitting. M. Friedlander, M.
      Schmidt. SISC, 2012] which also presents a hybrid deterministic-stochastic
      algorithm with a *linear* rate of convergence. This latter algorithm still
      has the condition number appearing in the rate though, so the ultimate
      rate is not faster than standard gradient descent (just the beginning is
      faster because of using cheaper steps) -- so the current submission can
      still improve theoretically over this rate when k is not too big with
      respect to n.] ** CLARITY: The paper is fairly clear. I have
      appreciated the summary from Table 1 which I haven't seen in the
      literature in such a clear manner. The proof can be followed tightly, but
      would be more useful to the reader if the authors could add a few
      high-level comments motivating some of the defined quantities which are
      fitting together a bit too magically in its current form.
      ORIGINALITY: This is a novel combination of known techniques.
      SIGNIFICANCE: The practical relevance of the algorithm is not
      demonstrated yet. But the theoretical contribution could have impact. In
      the context of the difficult proof of convergence of SAG (and the simple
      proof of SDCA, but which only applies to a restricted setup), the simple
      proof of EMGD is a major contribution. == More detailed
      suggestions == - line 050: I suggest to say "are summarized in
      Table 1 and detailed in the related work section". One can wonder at first
      what are the citations for these rates. - line 118-119: I would
      specify here that SCDA is only defined for a specific form of f_i, unlike
      SAG which handles any smooth convex f_i. - line 132: I would
      explicitly state that this condition is "for all w" -- it was a bit
      ambiguous whether the condition was only for a fixed w (the 'given input
      point w' mentioned in 1), which of course would not be sufficient to have
      the gradients match. - line 139: I would mention explicitly that
      we also have \grad F(w) = E[\grad f(w) ]. - line 224: The claim
      that SAG (or SCDA) cannot take advantage of distributed computing is
      false: they both can use mini-batches (this can also reduce the memory
      requirement). - line 288: I would write "(7) on (10) with x=w*
      (feasible by (5)) and x*=w_{t+1}" to be more explicit -- it is not that
      obvious how to obtain the line otherwise... - line 304 (and all
      other places): Add parenthesis around (F(w_t)-F(w*)) to be explicit that
      F(w*) is also summed T times... - line 366: Add that the 2nd
      inequality is only valid for all T > = 1 if delta < = exp(-1/2)
      [this also explains a condition stated in (4) which appeared nowhere
      explicitly]. Some typos: -042: 'an convex' - > 'a convex'
      -056: *strong* convexity, *condition* number -088 (and other
      places): please correct the notation of your domain to b consistent -- you
      use D in 039 and the first few pages; here, you use \Omega -- choose one
      and use it everywhere in the paper! - 307: use norm symbol instead of
      absolute value -366: for consistency, replace log with ln in the
      middle equation === Update after rebuttal === I am
      happy with the response by the authors, but note that I will carefully
      check that their updated version is implementing the changes that we have
      suggested!


On Algorithms for Sparse Multi-factor NMF
Nonnegative matrix factorization (NMF) is a popular data analysis method, the objective of which is to approximate a matrix with all nonnegative components into the product of two nonnegative matrices. In this work, we describe a new simple and efficient algorithm for multi-factor nonnegative matrix factorization (mfNMF) problem that generalizes the original NMF problem to more than two factors. Furthermore, we extend the mfNMF algorithm to incorporate a regularizer based on the Dirichlet distribution to encourage the sparsity of the components of the obtained factors. Our sparse mfNMF algorithm affords a closed form and an intuitive interpretation, and is more efficient in comparison with previous works that use fix point iterations. We demonstrate the effectiveness and efficiency of our algorithms on both synthetic and real data sets.

The paper proposes a new multiplicative
      algorithm for multi-factor nonnegative matrix factorization (mfNMF).
      The closest previous research is [3] (in Section 4.4), where the
      target matrix is approximated based on the general Bregman
      divergenceminimized by a multiplicative algorithm.The current
      paper proposes an efficient algorithm specialized for the general
      Kullback-Leibler (KL) divergence, a special case of Bregman
      divergence.Experiments show that the proposed algorithm outperforms
      the algorithm proposed in [3] for the general KL divergence, as well
      as another type of mfNMF called multilayer NMF [7-9].The paper
      also proposes a sparse variant of mfNMF with a Dirichlet
      regularizer.In the proposed framework, the Dirichlet regularizer
      behaves like a conjugate prior,and the sparse variant is built with a
      simple modification (Eq.(8) to Eq.(12)) of the objective.The
      obtained algorithm (Lemma 4) seems like ones for Bayesian automatic
      relevance determination.I think this is an interesting paper. The
      main idea for deriving a new algorithm is to decompose the target
      matrix into the product of a stochastic matrix and a diagonal scale
      matrix, and then simplify the objective based on the decomposition (2)
      of the generalized KL divergence.This alleviates the ill-posedness
      of mfNMF, and results in better performance (better quality of the
      found local minima), and efficient computation, as shown in the
      experiments. The authors also elaborate on making the algorithm fully
      multiplicative by introducing an auxiliary function (Lemma 2) for the
      partial problem (4).The obtained algorithm seems simple to
      implement, and work fine in the experiments.Pros:- The
      proposed algorithm is simple and experimentally shown to outperform
      the previous methods.- Unique ideas are found: focusing on
      stochastic matrices, and using decomposition (2) of the generalized KL
      divergence. These techniques might be used for other
      problems.Cons:- Experiments on the MNIST data set do not
      appeal the usefulness of mfNMF and sparse mfNMF. I do not understand
      what mfNMF found in Fig.3other than what two-factor NMF can find. In
      other words,why do you want to decompose W2W3 into W2 and W3?
      Quality:The paper is technically sound, and the claims are
      supported by the experiments.Clarity:The paper is clearly
      written. But I would suggest the authors to focus onthe closest
      research [3] in Section 2, and make readers understand what the
      differences between [3] and the current paper without need of checking
      the reference.Originality:The idea to focus on the stochastic
      matrices (in a problem where the purpose is not to find stochastic
      matrices) seems new.Significance:Since the proposed algorithm
      is simple and experimentally shown to outperform the previous methods,
      it would be used by practitioners.The idea of focusing on the
      stochastic part of the target matrixmight be used by other researchers
      when they develop new algorithms.Minor comments:- In Lemma 1,
      S = D^-1 V should be S = V D^-1. Lemma 1 seems trivial becauseits just
      a column-wise normalization, so just stating that 'we decompose V as V
      = SD' might be ok.- I suppose that Eq.(3) is the maximization of the
      similarity between the stochastic matrices, S^{V} and S^{W}, weighted
      by D^{V}. If so, noting this would make it clear that you are focusing
      on approximating the stochastic part.


Multi-Task Bayesian Optimization
Bayesian optimization has recently been proposed as a framework for automatically tuning the hyperparameters of machine learning models and has been shown to yield state-of-the-art performance with impressive ease and efficiency. In this paper, we explore whether it is possible to transfer the knowledge gained from previous optimizations to new tasks in order to find optimal hyperparameter settings more efficiently. Our approach is based on extending multi-task Gaussian processes to the framework of Bayesian optimization. We show that this method significantly speeds up the optimization process when compared to the standard single-task approach. We further propose a straightforward extension of our algorithm in order to jointly minimize the average error across multiple tasks and demonstrate how this can be used to greatly speed up k-fold cross-validation. Lastly, we propose an adaptation of a recently developed acquisition function, entropy search, to the cost-sensitive, multi-task setting. We demonstrate the utility of this new acquisition function by leveraging a small dataset to explore hyperparameter settings for a large dataset. Our algorithm dynamically chooses which dataset to query in order to yield the most information per unit cost.

The paper addresses the problem of automatically
      selecting hyperparameters for machine learning architectures. The authors
      consider Bayesian optimization (BO) with Gaussian processes (GP) and adapt
      this framework to a multi-task setup. This setup allows the sharing of
      information across multiple optimization problems, and can thus, for
      instance, exploit knowledge acquired while optimizing the hyperparameters
      of a learning architecture for one dataset during the optimization of
      these parameters for a second dataset. They explore three variants of
      their scheme: (1) transferring knowledge from previously completed
      optimizations; (2) speeding-up cross-validation by cleverly selecting
      folds instead of evaluating all parameter settings on all folds; and (3) a
      cost-sensitive version of hyperparameter optimization that can take
      advantage of cheaper surrogate tasks (e.g. optimizing the parameters on
      subsets of the data) for solving a computationally more expensive primary
      problem by actively selecting the task that is likely to yield the largest
      improvement per unit cost (e.g. compute time). Optimizing
      the hyper-parameters of machine learning architectures and algorithms is
      an important practical problem. The paper falls in line with several
      recent works that apply sequential model-based optimization (SMBO)
      techniques, including BO with GPs, to this end (see e.g. refs in the
      paper). The idea of applying ideas from multi-task learning to BO for
      hyper-parameter optimization is not completely novel (see the very recent
      paper by Bardenet et al.; ref [18]), but appears promising and has, to my
      knowledge, not been explored widely. The paper is generally well
      written, providing a good explanation of GP BO, and of the proposed
      extensions. There may be a few places where clarity could be improved (see
      below). The contribution of the paper is primarily that it brings
      together, in a sensible manner, various existing tools from the machine
      learning literature to adapt BO to a multi-task setup (I don't think the
      "ingredients" themselves are really novel). Furthermore, it proposes --
      and empirically explores -- three scenarios for knowledge sharing across
      tasks in the context of automatic hyper-parameter optimization that are
      likely to be useful in practice. Specifically, it seems to be
      building on the work of Snoek et al. (ref [6]), integrating the multi-task
      kernel of Bonilla et al. (ref [13]), and the recently proposed
      entropy-search (ES) acquisition function of Hennig & Schuler ([7]). It
      further finds a new use for the idea of cost-sensitive BO that was
      previously proposed in [6]. These "architectural" choices seem generally
      sensible (see below for some additional comments). Of the three
      application scenarios considered I find especially scenario (3)
      interesting, and it appears to be a good potential use-case for
      cost-sensitive BO. I find the experimental evaluation is generally
      convincing: For each scenario, the authors provide at least one real-world
      example demonstrating that the respective multi-task approach leads to
      equally good or better settings of the hyper-parameters in a shorter
      amount of time than performing BO without taking into account the
      information from the related task. In some cases additional
      control experiments could have been performed to provide additional
      baselines, test the limits of the proposed approach, and to justify and
      tease apart the effects of the various architectural choices (see below
      for details). Overall, I think this is a well written paper that
      is likely to be of practical value for the ML community.
      Some additional comments: - entropy
      search criterion (eq. 6): I might well be mistaken but I am a bit confused
      by the notation, specifically the conditioning of p(f | x, \theta, {x_n,
      y_n}) on x. If I'm understanding correctly, p(f|...) represents the
      K-dimensional Gaussian distribution arising from evaluating the posterior
      over the Gaussian process at points in \tilde{X}. So why do you condition
      on x, but not on the remaining points of \tilde{X}? Couldn't you just drop
      x here since you are indexing f in the product term? Also, this
      seems to be slightly inconsistent with eq. (7) where p(f|x) indicates that
      the GP is evaluated at a candidate location for an observation (which, if
      I'm understanding correctly, doesn't necessarily have to coincide with any
      of the points in \tilde{X})? - I think [7] express your equation
      (7) in terms of the KL between the P^y_{min} and a uniform distribution.
      They comment that this is a better choice than the KL you are considering
      (their page 14). Maybe it'd be worth commenting on this? -
      two-step heuristic in CV experiments (lines 204ff): Why is this two step
      approach necessary? Given that the number of folds is probably not huge
      couldn't you optimize the average objective with respect to an x for each
      task, and then choose the best x from that set? (Although maybe not much
      will be gained by this.) - EI vs entropy search (ES; line 240f):
      You say that EI does not generalize directly to the multi-task setup while
      ES does. Maybe you could explain why? Couldn't you use the EI on the
      primary task (per unit cost) as acquisition function? After all, you seem
      to be using EI on the primary task anyway for the generation of candidate
      points at which to evaluate the KL-objective (line 261f). - eq. 9:
      Is there a particular reason why you express (9) in terms of a difference
      in entropies and not in terms of KL as in (7)? - Experiments in
      sections 4.1 / 4.2: Am I right to assume that in these experiments you
      always use the EI criterion as acquisition function? Would you expect ES
      to perform any differently in these experiments? What about the comparison
      between STBO and MTBO in section 4.3: how much of the difference is
      attributable to multi-task vs. single-task and how much to the different
      acquisition functions? - I assume that for the experiments in
      4.1/4.2 single-task and multi-task function evaluations take roughly the
      same amount of time so that there is no need to plot performance as a
      function of compute time? - Experiments in section 4.1: How do the
      presented warm-start results compare to very simple baseline strategies
      for transferring knowledge from previously completed optimizations such as
      > just using the best-performing hyperparameter setting found in
      the previous optimization > fixing some the parameters to good
      values found in the previous optimization and searching only over the
      remaining ones > searching only over a smaller range for all or
      some of the parameters based on the results of the previous optimization
      - Experiments in section 4.1: Is there anything to be gained by
      considering the results of more than a single completed previous
      optimization? (E.g. for CNN on SVHN: combining information from CIFAR as
      well as from the subset of SVHN?) Also, how does the usefulness of
      information sharing depend on the number of iterations performed in the
      previous optimization? - Have you considered using alternative
      multi-task kernels, e.g. one in which task similarity is not modeled
      "free-form" but depends on certain task features? - Section 4.3:
      Your experiments seem to be considering a rather simple case of
      cost-sensitive BO in that the cost exclusively depends on the task s.t.
      c_t(x) = C_t (or does the cost actually depend on x?), and there are only
      two tasks. Have you considered other, more challenging scenarios, e.g. the
      possibility to vary the number of datapoints in the subsets at a finer
      granularity? - y-axis labels in Fig. 3e,f seem wrong - in
      eq. (8) shouldn't there be a task label for each past observation e.g.
      {x_n, y_n, t_n}? - additional reference: Bergstra et al., ICML
      2013: "Making a Science of Model Search: Hyperparameter OPtimization in
      Hundreds of Dimensions for Vision Architectures"