Robust Sparse Principal Component Regression under the High Dimensional Elliptical Model
In this paper we focus on the principal component regression and its application to high dimension non-Gaussian data. The major contributions are two folds. First, in low dimensions and under the Gaussian model, by borrowing the strength from recent development in minimax optimal principal component estimation, we first time sharply characterize the potential advantage of classical principal component regression over least square estimation. Secondly, we propose and analyze a new robust sparse principal component regression on high dimensional elliptically distributed data. The elliptical distribution is a semiparametric generalization of the Gaussian, including many well known distributions such as multivariate Gaussian, rank-deficient Gaussian, t, Cauchy, and logistic. It allows the random vector to be heavy tailed and have tail dependence. These extra flexibilities make it very suitable for modeling finance and biomedical imaging data. Under the elliptical model, we prove that our method can estimate the regression coefficients in the optimal parametric rate and therefore is a good alternative to the Gaussian based methods. Experiments on synthetic and real world data are conducted to illustrate the empirical usefulness of the proposed method.

Response to author feedback: Thank you for
      clarifying the novelty of the robust PCA approached; it is a good idea to
      also describe the novelty saliently in the paper. I suggest also
      explicitly mentioning the possibility of the generalization to the case
      with more than one PCA component, even if the full proof would not fit in
      this paper. Simply saying in the Introduction that you consider here the
      special case of one component, since it is already an interesting case
      backed up with positive empirical experiments, would already help a lot.
      Summary: The paper generalizes principal component
      regression (PCR) by providing a robust variant of the technique based
      on multivariate Kendall's tau in place of the covariance matrix. The
      authors then provide a simple algorithm for learning the model and,
      more importantly, analyze its theoretical properties, providing the rate
      of convergence. Finally, the model is demonstrated on simulated and
      equity data. Quality: The method presented in the paper is
      fairly straightforward, consisting merely of estimating the population
      Kendall's tau, computing its first eigenvector, and then performing
      linear regression to the observations. The method is very intuitive;
      it is easy to understand why this should be preferred to classical
      PCR when working with data that has outliers. The use of Kendall's
      tau in place of the covariance matrix in PCA is a nice idea and also
      well justified based on Oja's theorem. However, it remains unclear
      whether this should be considered as a novel contribution of the
      paper; no citations for earlier works are given, but the authors do
      not seem to describe it as a key contribution of the paper either. I
      believe it has potential for wider impact than the particular
      example considered in this paper, and one could imagine already a
      paper studying such a robust PCA estimator as a valuable contribution.
      To my knowledge, the closest work here would be the very recent paper
      by Han and Liu in ICML'13. Can the authors clarify the relationship
      with that paper? The analysis of the theoretical properties of PCR
      is valuable. However, the whole discussion is limited to the
      special case of simple PCR where the outcome is assumed to be
      related only to the first principal component. Do any of the
      results generalize to the more general case where the outcome is
      related to the first K components? At least the authors should
      explicitly mention that they limit the theoretical analysis to this
      special case; now the paper never even mentions the PCR setup I would
      consider as the standard one. The experiments seem to be conducted
      properly and they clearly illustrate the advantage of the robust
      variant; such a set is sufficient for a theoretical paper. I like the
      fact that the authors show how the equity data is not normally
      distributed, motivating the example. Clarity: The
      paper is reasonably well written, to the degree that a theory-heavy
      paper can be. However, the exact scope and contributions are a bit
      vague; it is unclear whether the use of Kendall's tau for robust PCA
      is novel, the authors do not mention they limit the analysis to a
      special case of PCR, and some of the earlier results are listed as
      "well known" without citations. One problem is that the proofs
      for the perhaps main contributions of the paper, Theorems 3.2 and 3.3,
      are left for Supplementary material, without even mentioning it in the
      paper itself. It is understandable that writing out the full
      proofs would take too much space here, but some sort of an outline
      would be useful. Originality: The paper has two novelties:
      it presents a novel robust PCR technique and it provides new
      theoretical analysis of PCR in general. The significance of the first
      relies fully on whether the use of Kendall's tau in PCA should be
      considered as a novel contribution; if not, the algorithm for robust
      PCR is trivial. The theoretical analysis provides new useful
      results and is based on very recent works by Oja, Vu and Lei, and
      Ravikumar et al. Significance: The robust method is
      clearly useful for anyone applying PCR, especially in light of the
      theoretical analysis for the convergence rate. However, the
      significance may be limited due to the fact that both the theoretical
      analysis and the simulation experiment rely on the simple model where
      the output only depends on the first principal component. The
      application to the equity data suggests the method works well in
      practice, but is merely one example. Detailed comments:
      - The 2nd sentence of Abstract is very complex and tries to lay
      out all the dimensions of the paper at once. You should consider
      splitting it into at least two sentences. - Page 3: "As a
      well-known result..." would need a citation. In general, "well known"
      is not good use of scientific language; you use it twice in the same
      paragraph for results that are not obvious for most readers. -
      Section 3.1: It is good to remind the readers about the elliptical
      distribution, but listing all the equivalent formulations is not be
      necessary when the rest of the analysis only uses one of them.


Adaptive dropout for training deep neural networks
Recently, it was shown that deep neural networks can perform very well if the activities of hidden units are regularized during learning, e.g, by randomly dropping out 50% of their activities. We describe a method called 'standout' in which a binary belief network is overlaid on a neural network and is used to regularize of its hidden units by selectively setting activities to zero. This 'adaptive dropout network' can be trained jointly with the neural network by approximately computing local expectations of binary dropout variables, computing derivatives using back-propagation, and using stochastic gradient descent. Interestingly, experiments show that the learnt dropout network parameters recapitulate the neural network parameters, suggesting that a good dropout network regularizes activities according to magnitude. When evaluated on the MNIST and NORB datasets, we found that our method achieves lower classification error rates than other feature learning methods, including standard dropout, denoising auto-encoders, and restricted Boltzmann machines. For example, our method achieves 0.80% and 5.8% errors on the MNIST and NORB test sets, which is better than state-of-the-art results obtained using feature learning methods, including those that use convolutional architectures.

Summary: "Adaptive dropout for training deep
      neural networks" describes and evaluates a new neural network
      architecture that combines a sigmoid belief network and a conventional
      feedforward network. Pro: * very strong results on MNIST and
      NORB * novel model that generalizes dropout to full neural networks
      Con: * little theoretical insight * empirical results
      somewhat scattered * training algorithm not perfectly clear
      Quality: The strongest element of this work, to me, is
      the empirical results. The performance of this non-convolutional model
      on MNIST and NORB is impressive. The intuition that the stochastic
      units implement some Bayesian sampling over models doesn't inspire me
      personally, but I don't object to the hypothesizing. The learning
      algorithm appears to require more computation per iteration than
      standard backprop and SGD, the authors should add some discussion of
      training speed to the results / discussion. The experimental
      results are not obviously focussed on any particular claim. For
      example, the proposed method is a stochastic neural network, but it is
      never simply trained as a supervised model. Why is it trained first as
      an autoencoder? How much autoencoder training was necessary? Several
      hyperparamters governing the optimization process appear to have been
      tuned, were these tuned separately for the different model
      architectures? If they weren't, then what can the reader draw from the
      model comparison scores? For me, the experiment section mainly says
      that using your model, Nesterov momentum, AE pre-training, and careful
      hyperparameter optimization, you got a tweaked our fully connected
      model to get .8 on MNIST and 5.8 on NORB. Using \alpha and \beta
      to tie \pi to 'w' and then also using ReLU units for the
      non-stochastic parts of the model makes it feel like there's a much
      simpler way to explain what you've done: namely, you have used a new
      activaction function that is a ReLU with some stochastic noise added
      around the inflection point. This may well be a good way (like
      maxout?) to get the learning benefits of linear units without losing
      so many units because they just turn "off" and can't come back on
      again. Clarity: The paper is well-organized, following
      the standard presentation for papers of this type. "principal"
      -> principle "each experts" -> each expert Figures 1 and
      2 are not clear to me, what am I supposed to see? Are the images in
      Figure 1 sorted by class label for example? Why does the leftmost one
      look nothing like a 0 or a 1, and the rightmost one look nothing like
      a 9 or 0. In describing the learning algorithm, could the
      authors explain the procedure for training \pi in more detail? It
      isn't clear to me. What is the free energy here, and how is it
      possible to estimate each m_j's contribution fast enough for learning?
      Section 4 is confusing, because after sketching one procedure for
      training \pi in Section 3, this Section throws it away and makes \pi
      an affine transformation of 'w'. It's not very good to say "we found
      this technique works very well in practice", I'd like to see the
      evidence for myself (that's what this paper's for). If it doesn't make
      any difference in terms of end-of-training accuracy, then why did you
      tie them? Why are not showing your CIFAR-10 results? In
      line 190, you say that \alpha and \beta are "learnt" but then in line 194
      these are described as hyper-parameters. Which is it? If
      \pi and w are tied by \alpha and \beta, then wouldn't a single "activation
      function" plot be a clearer way to explain the effect of \alpha and
      \beta than Figure 3? Figure 3 is tough to interpret for me, I don't
      know what to make of activation histograms. Originality:
      The proposed model is novel. Significance: At
      least some members of the NIPS community love new stochastic/neural
      architectures that work well and no one really understands why, that
      faction will be intrigued by this paper.


Regret based Robust Solutions for Uncertain Markov Decision Processes
In this paper, we seek robust policies for uncertain Markov Decision Processes (MDPs). Most robust optimization approaches for these problems have focussed on the computation of maximin policies which maximize the value corresponding to the worst realization of the uncertainty. Recent work has proposed minimax regret as a suitable alternative to the maximin objective for robust optimization. However, existing algorithms for handling minimax regret are restricted to models with uncertainty over rewards only. We provide algorithms that employ sampling to improve across multiple dimensions: (a) Handle uncertainties over both transition and reward models; (b) Dependence of model uncertainties across state, action pairs and decision epochs; (c) Scalability and quality bounds. Finally, to demonstrate the empirical effectiveness of our sampling approaches, we provide comparisons against benchmark algorithms on two domains from literature. We also provide a Sample Average Approximation (SAA) analysis to compute a posteriori error bounds.

This paper makes several good contributions to the
      literature on robust MDPs. In contrast to previous work, this paper
      handles both reward and transition uncertainty and also breaks away from
      making strong assumptions on the underlying uncertainty distributions.
      One innovation is to sample MDPs from the underlying uncertainty
      distribution to avoid having to represent the distributions explicitly (as
      say, a Gaussian reward distribution). They employ MILP to exactly solve
      for a single deterministic (history-dependent?) policy that minimizes the
      max regret attainable in a sample of |\xi| MDPs. Or they can approximate a
      randomized policy for the same problem and the approximation is a bounded
      distance away from the actual solution [Propositions 2 and 3].
      Another innovation is to adapt Cumulative Expected Regret (CER)
      from the multi-armed bandits literature and apply it specifically for
      MDPs, which if they also assume independence of uncertainty distributions
      across states / actions / epoch, then they can scale up to solve uncertain
      MDP problems with a longish horizon (H = 20) [Proposition 4]. But if one
      can not assume the independence of uncertainty distributions, then the
      scale up isn't possible? These contributions are interesting and
      worthy of publication. I'm slightly unclear on a few things to do
      with the MDP problem formulation. On line 063, the way that policy is
      written looks like we're potentially exploring the space of
      history-dependent policies. But possibly it could mean that it's just
      dependent on the time horizon we're in. If it's the former, then is
      'state' not simply the current state, but rather entire rolling histories
      of state (e.g. state at time 2 = {low in time 0, high in time 1, low in
      time 2} and not state in time 2= {low in time 2})? And if it is rolling
      histories of state, what implications does that have for proposition and
      corollary 1? There's another paper that also takes a
      sampling-based approach to handling uncertain MDPs from NIPS 2012 called
      "Tractable Objectives for Robust Policy Optimization". It may be
      worthwhile to think about how the two papers are related even if that
      paper isn't looking at a minimax regret objetive. Minor Comments:
      - Hyphenate 'sampling-based' (e.g. line 050), 'regret-based' (e.g.
      line 051), state-action' (e.g. line 109). - In a few places, there
      are parens where it might be cleaner to omit. For example: - (line
      064) R*(s) (=\max_a R(s,a)) could just be R*(s)=\max_a R(s,a) -
      (line 098) s (\in S) could just be s \in S - (line 099 s' (\in S)
      and a (\in A) can be s' \in S and a \in A. - (line 206) \epsilon
      (= \frac{d^2 - c^2}{r}) can be \epsilon = \frac{d^2 - c^2}{r} -
      There are some places where I'm not sure the italicizing maximin, reg,
      creg, SOS2 and br is necessary. Italicize when introducing a term for the
      first time but not anywhere else (e.g. for maximin). And in an equation,
      an italicized 'reg' (or 'SOS' or 'br') looks like r*e*g instead of like
      \text{reg}. - The recursive equations need base-cases: -
      (p. 2 Definition 1 and 2) - (Supplemental proof of proposition 1)
      - (p. 2 Preliminaries) you define t and H down on line 099 and 102
      but you refer to them already in line 061. - (p. 2 Preliminaries)
      Both the vanilla MDP and uncertain MDP are finite state, finite action,
      finite horizon problems. But only finite horizon is mentioned on line 096.
      Point this out by the vanilla MDP problem statement. - (line 087)
      The '#' comes a bit out of nowhere as it is defined in the supplemental
      section and even there I'm not sure what it denotes. - (p. 4
      Equation 5) needs a qualifier that this for all s,a, t, \xi_q? -
      (p 4, footnote 3) Footnote -> footnote - (line 305) policy will
      have least regret if it includes *the* CER minimizing policy from t
      - (line 362) and finally, we refer to the maximin value algorithm
      as 'Maximin' : if you're looking for stuff to cut to make room, I'd cut
      this. - (Supplemental 025) *Summing* a geometric progression over
      the time steps yields - (Supplemental line 134) belong->
      belongs - (Supplemental line 134) i = {i_1 … i_H} instead of i =
      {i_1 … i_{|T|}}?


Online Learning in Markov Decision Processes with Adversarially Chosen Transition Probability Distributions
We study the problem of online learning Markov Decision Processes (MDPs) when both the transition distributions and loss functions are chosen by an adversary. We present an algorithm that, under a mixing assumption, achieves O( T log |Π| + log |Π|) regret with respect to a comparison set of policies Π. The regret is independent of the size of the state and action spaces. When expectations over sample paths can be computed efficiently and the comparison set Π has polynomial size, this algorithm is efficient. We also consider the episodic adversarial online shortest path problem. Here, in each episode an adversary may choose a weighted directed acyclic graph with an identified start and finish node. The goal of the learning algorithm is to choose a path that minimizes the loss while traversing from the start to finish node. At the end of each episode the loss function (given by weights on the edges) is revealed to the learning algorithm. The goal is to minimize regret with respect to a fixed policy for selecting paths. This problem is a special case of the online MDP problem. It was shown that for randomly chosen graphs and adversarial losses, the problem can be efficiently solved. We show that it also can be efficiently solved for adversarial graphs and randomly chosen losses. When both graphs and losses are adversarially chosen, we show that designing efficient algorithms for the adversarial online shortest path problem (and hence for the adversarial MDP problem) is as hard as learning parity with noise, a notoriously difficult problem that has been used to design efficient cryptographic schemes. Finally, we present an efficient algorithm whose regret scales linearly with the number of distinct graphs.

Summary: On its first part the paper deals with
      online learning of Markov Decision Processes. Given an MDP and a set of
      policies (=distributions over actions) how should one Summary: The
      first part of the paper addreses the problem of Online Learning in Markov
      Decision Process. In this problem at each iteration: 1) adversary
      chooses a loss function and transition kernel (probabilistic function that
      chooses the next state based on the currenone and the selected action)
      2) the leaner knows the current state x_t and chooses an action a_t.
      3) the leaner sees the adversary choices and suffers the loss 4)
      the next state is sampled. The goal of the learner is to minimize the
      regret w.r.t. PI - a set of action selction policies The authors
      propose an algorithm which is a variant of the Shrinking Dartboard
      algorithm and show a bound of O((Tlog|PI|)^{1/2}). The second part
      of the paper addreses the Adersarial Online Shortest Path Problem. Here at
      each iteration the learner is presented with a graph with source,
      destinations and all other nodes in layers structure. The Learner
      chooses a path from source to dest. and then the weights of the edges is
      revealed and the loss of the learner is defined as the sum of weights
      along the chosen path. The goal of the learner is to minimize the
      regret w.r.t. a set of path choosing policies. The main contribution
      on this part is by showing that if both the graph and weights are chosen
      by an adversary at each stage this problem is as hard as the online
      agnostic parity problem (which is assumed to be a hard problem).
      Quality & Clarity The results presented in the paper
      seem new and interesting to me. However, 2 points bothered me during
      the reading: 1) It would be helpful if the explanation of the
      difference between the online MDP problem and standard online problems
      (lines 200-204) will appear much earlier and would be a bit longer than 2
      sentences. 2) The connection between the 2 parts of the paper is not
      obvious to the learner. The authors state that Online Shortest Path is a
      special case of MDP. I assume that this is correct but expect that such
      claim will be accompanied by a short explanation (States are:..., Actions
      are:... Loss function:...) (I haven't read the Supplementary material)
      Originaluty & Significance I'm not familiar with previous
      results on this problem. The hardness proof might be useful for other
      works in the area Comments: 1) In Remark 3 the authors
      claim that a bound for determintic policies won't apply a bound w.r.t. the
      optimal policy. I'm not sure that I understand this claim: as far as I
      know a regret of an algorithm is always measured w.r.t. a set of competing
      policies. Hence, a bound on the regret of every deterministic policy will
      imply an optimal regret w.r.t. any deterministic policy. I undrstand that
      there might be some stochastic policy which is better but this policy is
      not part of the competing group of policies. 2) In the hardness
      results the authors mention several times "noisy parity" however, I didn't
      find where the noise is mentioned. (Did you mean that the labeling y is
      sometime flipped? Is it matters how is is flipped?) Typos:
      Line 64 defines policy as mapping from actions intead of a mapping
      from states (as in line 135) Figure1:OMDP algo: W_0 is undefined,
      w_{pi_0},-1 is undefined. Line 178: This notion of regret w.r.t.
      specific policy wasn't defined previously. (and appears a lot in the
      paper) Line 367: A denotes an algorithm while earlier it was used for
      the set of possible actions. Line 412: Did you define earlier N as the
      set of all nodes? Line 412 at the end: shouldn't the l in Q_{pi, g l}
      be changed to c?


Top-Down Regularization of Deep Belief Networks
Designing a principled and effective algorithm for learning deep architectures is a challenging problem. The current approach involves two training phases: a fully unsupervised learning followed by a strongly discriminative optimization. We suggest a deep learning strategy that bridges the gap between the two phases, resulting in a three-phase learning procedure. We propose to implement the scheme using a method to regularize deep belief networks with top-down information. The network is constructed from building blocks of restricted Boltzmann machines learned by combining bottom-up and top-down sampled signals. A global optimization procedure that merges samples from a forward bottom-up pass and a top-down pass is used. Experiments on the MNIST dataset show improvements over the existing algorithms for deep belief networks. Object recognition results on the Caltech-101 dataset also yield competitive results.

The key idea of this paper is to use the latent vector
      generated from upper layers of a Deep Belief Network as target vectors, by
      adding a regularization cross-entropy part to the standard unsupervised
      training loss, so as to encourage reconstructions from bottom and top to
      match. Instead of the standard two-stage training of deep architectures
      (unsupervised layer-by-layer pretraining, then full network supervised
      training), training is conducted in three stages with an intermediate
      stage that uses this hybrid loss. Experimental results show that the
      intermediate stage substantially improves results on MNIST and Caltech101.
      This regularization of intermediate layers by top-down generative
      weights shows good results; the paper is clear and shows how the proposed
      intuitive way of bridging unsupervised and supervised training indeed
      improves performance. The paper is overall rather clear although
      there are some problems (see below). Organization: the paper flows
      well. One minor comment: Section 3: it would be easier for the reader
      to get the key idea if the intuition was given briefly at the very
      beginning of section 3, instead of waiting until line 192 -- in the
      current form, the reader is first left wondering what will be used as
      target vector. E.g., before the paragraph "Generic Cross-Entropy
      Regularization", insert a one-sentence summary saying that the main
      intuition is to use latent vector generated from upper layers as target
      vectors. Questions/comments: Line 044: Error backpropagation is a
      way to propagate the error across the lower layers of the networks; it
      does not say anything about being supervised or discriminative. Indeed one
      can very well backpropagate unsupervised reconstruction errors across
      layers. What makes the last training phase supervised is merely *the
      choice of the criterion* used for training, whose error is then
      backpropagated. Same goes for Line 290. L.104: using the softmax in
      Eq. 8 results in the units *not* being independent, and does not
      correspond to the energy given in Eq.6: this is not what is depicted in
      Fig.2 but corresponds to a top logistic regression layer. Fig.2 and Eq.6
      correspond to a logistic just as in Eq.4, with v and d instead of w
      and c, NOT the softmax equation given in Eq.8. Eq. 6 has exactly the same
      form for y and x so the resulting probability forms should not differ.
      L. 262: I am a bit confused: if y is obtained by softmax, as stated
      l.261, is the label never used? It seems that instead the last layer z_L
      should be obtained by softmax (or independent binary activation, see
      comment above -- not sure what was used), and mixed with plugged in ground
      truth labels y during training. Is this what was done? Minor comments:
      writing: use more articles (e.g. "A RBM is *a* bipartite Markov random
      field," "This enables *the* model to be sampled") L.44: "Given a set
      of trainig data, the unsupervised learning phase initializes the
      parameters in only one way, regardless of the ultimate classification
      task." This is confusing (besides, it unnecessarily narrows the context to
      classification as the final task). If what is meant is that the
      unsupervised criterion is agnostic to the final task of interest, then a
      straightforward formulation is better, e.g.: "The unsupervised learning
      phase does not take into account the ultimate task of interest, e.g.
      classification." Line 052- "same gradient descent convention":
      gradient descent is not a convention. "procedure" better. Line 090:
      missing J: V \in \R^{J \times C}. Line 112: Bengio, not Benjio.
      Line 270-278: confusing. Change presentation. Line 344: the best
      score is obtained with setting 3, not 2.


Non-strongly-convex smooth stochastic approximation with convergence rate O(1/n)
We consider the stochastic approximation problem where a convex function has to be minimized, given only the knowledge of unbiased estimates of its gradients at certain points, a framework which includes machine learning methods based on the minimization of the empirical risk. We focus on problems without strong convexity, for which all previously known algorithms achieve a convergence rate for function values of O(1/ √ n) after n iterations. We consider and analyze two algorithms that achieve a rate of O(1/n) for classical supervised learning problems. For least-squares regression, we show that averaged stochastic gradient descent with constant step-size achieves the desired rate. For logistic regression, this is achieved by a simple novel stochastic gradient algorithm that (a) constructs successive local quadratic approximations of the loss functions, while (b) preserving the same running-time complexity as stochastic gradient descent. For these algorithms, we provide a non-asymptotic analysis of the generalization error (in expectation, and also in high probability for least-squares), and run extensive experiments showing that they often outperform existing approaches.

The paper presents novel results on how to achieve
      fast convergence rates O(1/n) in on-line learning via stochastic
      approximation with requiring strong convexity of the loss function and
      without having to rely on well-conditioned data matrices. This addresses
      one of the most interesting open questions in on-line learning, namely how
      to achieve convergence rates closer to the theoretical limit for the
      practically most relevant settings (least squares and logistic
      regression). In my opinion, the paper makes a number of remarkable
      contributions towards this goal: (i) Theorem 1: a fast convergence result
      for constant-step size least squares methods with averaging iterates, and
      an analysis of convergence of higher order moments and (ii) a clever way
      on how to extend this from the linear case to the more general case (of
      M-estimators) via the notion of "support points". While the fully
      satisfactory exploitation of the latter idea seems to require future work
      (the two phase method seems somewhat less appealing from a practical point
      of view and the the algorithm who choses support points on the fly is not
      known to converge), I feel this is already worth publishing in order to
      make these ideas accessible to a broader audience.


Online Learning of Nonparametric Mixture Models via Sequential Variational Approximation
Reliance on computationally expensive algorithms for inference has been limiting the use of Bayesian nonparametric models in large scale applications. To tackle this problem, we propose a Bayesian learning algorithm for DP mixture models. Instead of following the conventional paradigm -random initialization plus iterative update, we take an progressive approach. Starting with a given prior, our method recursively transforms it into an approximate posterior through sequential variational approximation. In this process, new components will be incorporated on the fly when needed. The algorithm can reliably estimate a DP mixture model in one pass, making it particularly suited for applications with massive data. Experiments on both synthetic data and real datasets demonstrate remarkable improvement on efficiency -orders of magnitude speed-up compared to the state-of-the-art.

In this paper the authors presented a sequential
      variational inference algorithm for Dirichlet process mixture models. The
      authors used a posterior characterization of normalized random measures
      with independent increments as the basis for a variational distribution
      that was then used on a sequential decomposition of the posterior. The
      algorithm was demonstrated on a Gaussian mixture model applied to real and
      synthetic data, and a non-conjugate DP mixture of Dirichlet distributions
      to cluster text documents. This is a nice paper that aims to make
      a particular Bayesian nonparametric model useful to analyze massive data
      sets. The overall presentation is clear and well-motivated. The
      experiments are explained clearly and demonstrate advantages of the
      proposed algorithm. I think that the impact of the paper in
      expanding the usefulness of Bayesian nonparametric models could be greatly
      improved by including a supplementary materials section with some proofs
      and derivations so that a wider audience can understand how the algorithm
      works and adapt it for their problem. Specifically, including the
      following would make the paper more self-contained and much clearer:
      1. A proof of Lemma 1. The theorem that the proof is a result from is
      in a pretty dense paper. Restating the theorem and showing how Lemma 1
      follows would be very helpful. 2. Derive Equation 12. The variational
      distribution is non-standard and it's pretty tedious to derive Equation
      12. 3. Derive Equations 15 and 16 from Equation 14. The variational
      distribution is non-standard, the notation for the variational
      distribution changed from q(D) to q(z,phi), and the sequential variational
      framework is different enough from the standard setup that it would save
      some mathematical headaches for the reader to see how Equations 15 and 16
      arise. A few minor comments on the paper that should be addressed:
      This work should situate itself with respect to the work on sequential
      Monte Carlo algorithms for Dirichlet process mixture models which use the
      same sequential decomposition of the posterior. I think Equation 14 is
      missing the \phi's in the variational posterior. There are a few
      parameters of the algorithm, namely \epsilon_d and \epsilon_r, what were
      these set to for the experiments, or were they found with
      cross-validation? Lastly, the abbreviation CRF is not standard, why not
      CRP?


Information-theoretic lower bounds for distributed statistical estimation with communication constraints
We establish lower bounds on minimax risks for distributed statistical estimation under a communication budget. Such lower bounds reveal the minimum amount of communication required by any procedure to achieve the centralized minimax-optimal rates for statistical estimation. We study two classes of protocols: one in which machines send messages independently, and a second allowing for interactive communication. We establish lower bounds for several problems, including various types of location models, as well as for parameter estimation in regression models.

This paper considers minimax lower bounds on the mean
      squared error for statistical estimation problems, in a setting where
      available data is distributed i.i.d. across several machines and there is
      a constraint on the number of bits that may be communicated. The paper
      considers both interactive and non-interactive protocols, where the final
      estimate is a function of the messages sent by the machines. For
      interactive protocols, these include lower bounds for estimating the mean
      of a uniform distribution in a unit cube, and the mean of a univariate
      Gaussian distribution. For non-interactive protocols, the lower bounds are
      for estimating the mean of a multivariate Gaussian distribution, and
      applications for linear and probit regression in a fixed (known) design
      setting. The issue of distributed learning and estimation has
      received plenty of attention in recent years, mostly in the form of
      algorithms and upper bounds. This paper is among the first (see remarks
      below) to explicitly consider lower bounds. Since lower bounds are crucial
      to understanding the limits of achievable performance, I consider this a
      very interesting line of research. The results presented here seem to be
      interesting first steps in this direction. Also, I found the paper clear
      and quite easy to follow. Finally, the results seem overall correct,
      although I haven't checked the proofs in the supplementary carefully.
      My main criticism is that the results focus on somewhat limited
      regimes, which may not correspond to realistic applications of distributed
      learning. To be more specific: 1) The setting considered here is
      only harder than the standard (non-distributed) setting of statistical
      estimation. Thus, all the standard (non-distributed) lower bounds
      trivially hold. The lower bounds presented here are stronger than those
      standard bounds, only in the regime where the average amount of
      communication B_i per machine is smaller than the data dimension d.
      However, this is a very harsh constraint. For example, in standard linear
      prediction, it means that each machine can't even output a single
      predictor. For most applications of distributed learning, one is very
      happy with protocols where B_i is order of d, and for such a regime, the
      results here do not improve on the existing standard lower bounds. For
      fairness, it should be said that it's not known whether the standard lower
      bounds are improvable when B_i >= d, except in some cases, such as
      statistical estimation with sufficiently smooth loss functions (see Zhang
      et al. 2012 - reference [18] in the paper). These results do not apply to
      non-smooth losses, e.g. when we try to distribute support vector machines.
      Moreover, all these results are in terms of mean-squared-error, and do not
      give tight results when converting them to optimization error / excess
      risk (e.g. in terms of dependence on the strong-convexity parameter, which
      can be small in regularized statistical learning problems). Anyway, the
      results in this paper do not shed light on these issues. 2) Most
      of the results here are for non-interactive protocols, where each machine
      can only send a single message based on its data, and the final estimator
      is a function of these messages. This is restrictive, since one can easily
      imagine algorithms which require several communication rounds between the
      machines. The concrete results for interactive protocols (proposition 1
      and theorem 2) seem much more limited, focusing either on a uniform
      distribution in a unit cube, or a 1-dimensional setting. As to the more
      generic theorem 1, I may be wrong, but I think the same proof would apply
      to a non-distributed setting, where the only constraint it that the output
      \hat{\theta} is limited to B bits. So in a certain sense, it may not
      capture an interesting aspect of the distributed setting. 3) The
      regression lower bounds focus on a fixed design setting, where the
      design/instances are known by all machines in advance, and the uncertainty
      is in the target values. In a distributed setting, it's much more
      realistic to assume unknown instances partitioned between the machines.
      Technical Comments ------------------ - Line
      41: "Currently there is little theoretical understanding of the role of
      communication complexity in statistical inference, and thus the design
      and analysis of distributed inference algorithms is more art than
      science". I think this is exaggerated. Principled approaches to design and
      analyze distributed inference and learning have been well-studied in the
      literature. It is true that lower bounds have not been as carefully
      studied. - It would be good to explain why the dependence on B is
      so different between the two settings of proposition 1 and theorem 2.
      - Line 111: "an protocol" - Line 119, definition of
      minimax risk - why is the protocol Pi separated from the estimator
      function \Theta? Pehaps it would make more sense to include the final
      estimate computation as part of the protocol. - Line 148: If I
      understood correctly, example 1 shows the tightness of Theorem 1 only for
      one particular choice of B. It would be better if one can come up with an
      example which shows the tightness of the bound for *any* choice of B.
      - Line 1120 in supplementary: "achieves minimax" -> "which
      achieves the"(?) - In discussing related work, the authors should
      also mention the COLT 2012 paper "Distributed Learning, Communication
      Complexity and Privacy" by Balcan et al., which also discuss lower bounds
      for distributed learning (although in a different setting). -
      Section 2: When defining A_ind and A_inter, do any of the lower bounds
      become better if we require L<=B always rather just in expectation?
      Assuming that not, it might be worthwhile to point this out.


Convex Calibrated Surrogates for Low-Rank Loss Matrices with Applications to Subset Ranking Losses
The design of convex, calibrated surrogate losses, whose minimization entails consistency with respect to a desired target loss, is an important concept to have emerged in the theory of machine learning in recent years. We give an explicit construction of a convex least-squares type surrogate loss that can be designed to be calibrated for any multiclass learning problem for which the target loss matrix has a low-rank structure; the surrogate loss operates on a surrogate target space of dimension at most the rank of the target loss. We use this result to design convex calibrated surrogates for a variety of subset ranking problems, with target losses including the precision@q, expected rank utility, mean average precision, and pairwise disagreement.

The paper demonstrates calibrated convex surrogate
      losses for multiclass classification. If the n by k loss matrix (where the
      label set has size n and the prediction set has size k) has rank d,
      minimizing this surrogate loss corresponds to a d-dimensional least
      squares problem, and converting the solution to a prediction corresponds
      to a d-dimensional linear maximization over a set of size k. The paper
      describes two examples (precision at q and expected rank utility) for
      which n and k are exponentially large, but predictions can be calculated
      efficiently. It describes two more examples (mean average precision and
      pairwise disagreement) with exponential n and k for which the calibrated
      predictions apparently cannot be calculated efficiently, but efficiently
      computable predictions are calibrated for restricted sets of probability
      distributions. This is an exciting contribution that seems to be
      important for a wide variety of multiclass losses of great practical
      significance. Technically, it is a combination of two results from ref
      [16] - the important low rank observation appeared already in [16]. The
      paper is clearly written. The main criticism is of the results on
      calibration with respect to restricted sets of probability distributions.
      For instance, Theorem 4 gives a result that the efficiently computable
      prediction rule is calibrated for a certain family P_reinforce. Why is
      this family interesting? What is the intuition behind it? Are there
      interesting examples of distributions that satisfy these conditions?
      Similar questions apply to the set of distributions considered in Theorem
      7. Minor comments: line 227: 1(\sigma^{-1}(i)\leq q) should be
      1(\sigma(i)\le q). (Similarly at line 236) 242: 1 and 0 interchanged.
      254: Having max(y_i-v,0) in place of simply y_i seems silly. Why not
      just redefine the set of labels as {0,1,...,s-v}^r? 320: u^p_{ij} is
      only defined for i\geq j. The appendix mentions that it is made symmetric,
      but not the paper.


A memory frontier for complex synapses
An incredible gulf separates theoretical models of synapses, often described solely by a single scalar value denoting the size of a postsynaptic potential, from the immense complexity of molecular signaling pathways underlying real synapses. To understand the functional contribution of such molecular complexity to learning and memory, it is essential to expand our theoretical conception of a synapse from a single scalar to an entire dynamical system with many internal molecular functional states. Moreover, theoretical considerations alone demand such an expansion; network models with scalar synapses assuming finite numbers of distinguishable synaptic strengths have strikingly limited memory capacity. This raises the fundamental question, how does synaptic complexity give rise to memory? To address this, we develop new mathematical theorems elucidating the relationship between the structural organization and memory properties of complex synapses that are themselves molecular networks. Moreover, in proving such theorems, we uncover a framework, based on first passage time theory, to impose an order on the internal states of complex synaptic models, thereby simplifying the relationship between synaptic structure and function.

The paper studies the problem of memory storage with
      discrete (digital) synapses. Previous work established that memory
      capacity can be increased by adding a cascade of (latent) states but the
      optimal state transition dynamics was unknown and the actual dynamics was
      usually hand-picked using some heuristic rules. In this paper the authors
      aim to derive the optimal transition dynamics for synaptic cascades. They
      first derive an upper bound on achievable memory capacity and show that
      simple models with linear chain structures can approach (achieve) this
      bound. The paper is clear, high quality, generally well written
      and has a clear contribution to the field. Minor comments:
      line 195: Different systems in the brain might be optimised for
      storage at different timescale. Are there any evidence for that? line
      220: When looking at the molecular network of a synapse it is not
      immediately obvious what are the "states" of the system and what is the
      associated state transition dynamics. A strong prediction of the paper is
      that the molecular network in the synapse has a meaningful structure for
      information storage, not like the one at Fig 2a-b. Is there a simple way
      to check this prediction? line 294: The authors may refer to the
      supplementary material around equation 14. line 299: Isn't it
      problematic, than nu_i depends on M - that we try to optimize? line
      304: An intuitive description of the process depicted on Fig. 3a-b is that
      a potentiation has to move all synaptic states to more potentiated states,
      while a depression event must depress it. line 326: In all the
      examples provided in the paper, half of the states are associated with
      w=-1 the other half has w=+1. How the memory capacity changes if the
      states are more asymmetric, e.g., in the most extreme case there is only a
      single state with w=-1 and M-1 state with w=1? line 415: Linear chains
      maximize area - but not necessarily achieve the best performance (touches
      the envelope) at t_0? Is it true that for high performance at a certain
      time a chain other than linear may be optimal? Is the chain shown on Fig5b
      a linear chain? I appreciate the detailed rebuttal from the
      authors. A short definition of the linear chain would be more useful than
      repeating Fig 4. A possible definition would be that a chain is linear if
      there is a unique ordering of the nodes from depressed to potentiated -
      but this is not what the authors used.


Online Learning in Episodic Markovian Decision Processes by Relative Entropy Policy Search
We study the problem of online learning in finite episodic Markov decision processes (MDPs) where the loss function is allowed to change between episodes. The natural performance measure in this learning problem is the regret defined as the difference between the total loss of the best stationary policy and the total loss suffered by the learner. We assume that the learner is given access to a finite action space A and the state space X has a layered structure with L layers, so that state transitions are only possible between consecutive layers. We describe a variant of the recently proposed Relative Entropy Policy Search algorithm and show that its regret after T episodes is 2 L|X ||A|T log(|X ||A|/L) in the bandit setting and 2L T log(|X ||A|/L) in the full information setting, given that the learner has perfect knowledge of the transition probabilities of the underlying MDP. These guarantees largely improve previously known results under much milder assumptions and cannot be significantly improved under general assumptions.

Summary The paper is about online learning in
      non-stationary episodic loop-free Markov Decision Processes, where the
      loss function is allowed to change between episodes. The proposed
      algorithm O-REPS is an online variant of the Relative Entropy Policy
      Search algorithm, that searches for better policies by bounding the
      related information loss, so that small policies updates are performed.
      The authors describe the proposed algorithm and produce upper bounds
      to the total expected regret both for the bandit and for the full feedback
      cases, concluding that, in many common settings, O-REPS has better
      guarantees than previous results from literature and requires less
      assumptions on the MDP and puts no restriction to the class of policies
      that must be considered. Quality The paper is technically
      sound. The problem is well introduced and the main results are presented
      with proofs (I have checked them and they seem correct). In the last
      section, the authors compare the properties of the proposed approach
      w.r.t. the ones of state-of-the-art algorithms, highlighting the
      improvements introduced. Although no limitation of the proposed
      approach is discussed, some ideas for future research directions are
      suggested. Clarity The paper is well written and clearly
      organized. The proofs are quite detailed, so that they can be easily
      followed in most cases. I suggest only to add some justification to
      inequalities in line 353. The only part that I found not very clear is
      Section 5.3 where the authors tries to compare the proposed approach with
      the original REPS algorithm. Overall, although the paper is quite
      technical, I think that it can be followed and mostly understood also by a
      non-expert audience. Originality The paper builds upon on a
      state-of-the-art algorithm, but it considers a different setting and
      produce a theoretical analysis of the algorithm properties that, as far as
      I know, constitutes an original contribution. The authors compare the
      obtained results with the ones obtained by other approaches, giving a
      quite comprehensive overview of the related literature. The paper is
      not groundbreaking, but represents a solid contribution that advances
      knowledge in a relevant area of machine learning. Significance
      The theoretical results reported in this paper represent a clear
      improvement over the state of the art. The impact of such theoretical
      results on future research can be hardly assessed due to the specific
      class of MDPs addressed (episodic and loop-free). On the other hand, it
      provides interesting results that may be useful to produce new studies in
      more general settings. *** Comments after the rebuttal ***
      Clarifications provided by author rebuttals will further improve
      paper quality.


Cluster Trees on Manifolds
In this paper we investigate the problem of estimating the cluster tree for a density f supported on or near a smooth
We analyze a modified version of a k-nearest neighbor based algorithm recently proposed by Chaudhuri and Dasgupta (2010). The main results of this paper show that under mild assumptions on f and M , we obtain rates of convergence that depend on d only but not on the ambient dimension D. Finally, we sketch a construction of a sample complexity lower bound instance for a natural class of manifold oblivious clustering algorithms.

The paper discusses the estimation of the cluster tree
      when the probability density function is supported on a d dimension
      manifold in a D dimensional space. They show that the algorithm RSL
      proposed in (1) is consistent and the convergence rate, very roughly,
      depends on d-the dimension of the manifold and not on D - the dimension of
      the ambient space (but then the convergence rate also depends on tau-the
      conditional number and an epsilon^{d+2} factor instead of an epsilon^2
      factor). The main result is achieved by repeating the technique in
      (1). To do that, the authors had to show: First,a bound on a size of an
      s-net in the manifold setting. Second, bounds on the deformation of the
      volume (i.e. that B(x,r)cap M has roughly the volume of a d-dimensional
      ball of radius r where d is the dimension of M). The authors are able to
      show both under the assumption of small conditional number. I
      think it is interesting to know how convergence rate changes under this
      assumption (i.e. manifold assumption) and the paper give both lower bounds
      and upper bounds that are not trivial. So even though the convergence rate
      depends on sizes that are not available (the dimension of the manifold and
      the conditional number), still the results are interesting. I
      found the writing very unclear and certain definitions are even confusing:
      ******************************* 1) The statement in thm.4 is
      wrong. A much stronger statement is proved in thm.6 of [1] than
      def.3-consistency. (see also, remark after thm.6 in [1]). Theorem
      6 states that with high probability: uniformly, every A A' that satisfy
      (\sigma,\epsilon)-separation: we get separation and conectedness.
      Theorem 4 states that for every A A' that satisfy (\sigma,
      \epsilon)-seperation, with high probability we get separation and
      connectedness. These are not equivalent statements. Please correct
      this. ***************************** 2) Still in definition 3: what
      is n? who is C_n (is it a random variable? how is it defined? is it
      different than hat{C}) I had to rely on the definition in (1) to
      understand what is meant here. 3) When using Lemma 16: It is worth
      indexing and mentioning which inequality is used and how at each step. Not
      all steps are clear, it seems that at last step you use
      (1+4r/t(1+4r/t))(1+4r/t) < (1+6r/t) but that's not even hinted. The
      steps should be clearer. 4) Lemma 18: I think a 1/2 is missing
      from the definition of v_cap. Worth mentioning that
      Gamma(1/2)=sqrt(pi) otherwise it's not clear where it went.
      Further suggestions: The lower bound you produce
      depends on the conditional number, it might be worth mentioning the lower
      bound you produce are not an improvement over the lower bound in (1), but
      are different (e.g. in a linear subspace that has 1/tau=0 your lower bound
      is meaningless while (1) gives a sensible lower bound). Regarding
      the parameter \rho, does it really make sense to choose salience parameter
      2\sigma > tau? won't it be easier to simply assume (3\sigma/16) <
      (\tau/16)?


Probabilistic Low-Rank Matrix Completion with Adaptive Spectral Regularization Algorithms
We propose a novel class of algorithms for low rank matrix completion. Our approach builds on novel penalty functions on the singular values of the low rank matrix. By exploiting a mixture model representation of this penalty, we show that a suitably chosen set of latent variables enables to derive an ExpectationMaximization algorithm to obtain a Maximum A Posteriori estimate of the completed low rank matrix. The resulting algorithm is an iterative soft-thresholded algorithm which iteratively adapts the shrinkage coefficients associated to the singular values. The algorithm is simple to implement and can scale to large matrices. We provide numerical comparisons between our approach and recent alternatives showing the interest of the proposed approach for low rank matrix completion.

SUMMARY This paper studies the problem of low
      rank matrix completion which exists in many real-world applications such
      as collaborative filtering for recommender systems. A previous work (ref
      [4]) proposed a scalable algorithm called Soft-Impute for solving a convex
      optimization problem involving the nuclear norm as a regularizer. Like
      previous work such as probabilistic matrix factorization (PMF), this paper
      gives the problem a probabilistic interpretation by relating the
      (non-probabilistic) optimization problem to a MAP estimation problem.
      Different (concave) penalty functions of the nuclear norm are proposed and
      then an EM algorithm is proposed to solve the MAP estimation problem. The
      algorithms proposed in this paper are more general than the Soft-Impute
      algorithm proposed in [4] in that the latter comes as a particular case.
      QUALITY The main contribution of this paper is that it
      proposes new penalty functions based on the nuclear norm for low rank
      matrix completion. Although experimental validation has shown that the
      method (HASI) that makes use of the proposed penalty functions generally
      outperforms other methods compared, its technical soundness could be more
      strongly articulated via theoretical analysis. This is especially the case
      when the proposed regularizers are non-convex and hence make the
      optimization problem more challenging than solving a convex optimization
      problem. This price would be worth paying only if it had advantages
      preferably supported well by theoretical analysis, besides experimental
      validation. Even on the experimental validation, besides showing the
      results as in Table 1, the reader could gain a deeper understanding of the
      strengths and weaknesses of different methods if more detailed analysis of
      the results could also be provided. For example, we only know that HASI
      usually gives the lowest NMAE except for the MovieLens 1M dataset (which
      comes second), but we have no clue on what makes the performance degrade
      in this case. Is it because of the large dataset? Does the proposed method
      perform less satisfactorily for large datasets due to non-convexity of the
      optimization problem? Also, there is relatively large variation in the
      rank among different methods. It would help if the authors could comment
      on this aspect as well. Since the proposed method is based on
      point estimation, calling it “Bayesian” may be controversial. It is more
      appropriate to refer to it as “probabilistic” instead. Related previous
      work also used this convention. For example, for matrix
      factorization/completion, a point estimation method is called
      probabilistic matrix factorization: R. Salakhutdinov and A. Mnih.
      Probabilistic matrix factorization. NIPS, 2008. while its full
      Bayesian extension is called Bayesian probabilistic matrix factorization:
      R. Salakhutdinov and A. Mnih. Bayesian probabilistic matrix
      factorization using Markov chain Monte Carlo. ICML, 2008.
      Nevertheless, this is a well-written paper which is a pleasure to
      read. I also like the supplementary material which includes discussions on
      generalization to other mixing distributions as well as the binary case
      which does occur in practice. CLARITY The paper is quite
      clearly written, making it easy for the reader to follow. I particularly
      like the organization which presents the complete matrix case first and
      then considers the more general, incomplete matrix case for matrix
      completion which is the focus of this paper. Nevertheless, I have
      a few minor comments. Addressing them will further improve the clarity of
      the paper. Line 37: “... the low rank assumption is sensitive
      ...” Do you actually mean “sensible”? I don’t understand why it is
      “sensitive”. In Eq (7), strictly speaking some constant terms are
      missing even though they have no effect on the penalty function for
      optimization. In Figure 3, the labels on the axes are too small to
      be visible without zooming in the PDF file. Line 184: When you
      refer to \gamma as regularization parameters, it would help to point to
      the specific regularized objective function in which they play the role of
      regularization parameters. It may not be clear to the reader. In
      Eq (13), it should be the sum of two terms, not their difference.
      ORIGINALITY Needless to say, matrix completion is not a
      new problem. It has already aroused a lot of research interests over the
      past decade. Also, quite a few models have been given a probabilistic
      interpretation. So, this paper is not novel in this aspect either. The
      main contribution of this paper is on the introduction of some penalty
      functions based on the nuclear norm. As far as I know they have not been
      introduced in the context of matrix factorization/completion and related
      problems. In fact, many of the techniques used in the paper are
      existing methods and results too. For example, the global optimality of
      the M-step of the EM algorithm in Eq. (9) is guaranteed by previous
      results reported in [11, 12], not this work; the scalability is due to the
      PROPACK algorithm which is from previous work [4]. If one has to comment
      on the novelty of this paper, it would be on the combination of these
      techniques and findings from previous work in a coherent way.
      SIGNIFICANCE The experimental results reported seem quite
      good, but it is premature to draw conclusions on the practical
      significance of the proposed method based solely on these relatively
      small-scale experiments. Its significance could be articulated more
      convincingly by making extensions at least along the following directions:
      - Using larger datasets, e.g., Netflix dataset - Empirical
      comparison with more state-of-the-art methods, e.g., Bayesian methods,
      nonlinear (kernel) methods, deep learning methods - Theoretical
      analysis of new penalty functions


Gaussian Process Conditional Copulas with Applications to Financial Time Series
The estimation of dependencies between multiple variables is a central problem in the analysis of financial time series. A common approach is to express these dependencies in terms of a copula function. Typically the copula function is assumed to be constant but this may be inaccurate when there are covariates that could have a large influence on the dependence structure of the data. To account for this, a Bayesian framework for the estimation of conditional copulas is proposed. In this framework the parameters of a copula are non-linearly related to some arbitrary conditioning variables. We evaluate the ability of our method to predict time-varying dependencies on several equities and currencies and observe consistent performance gains compared to static copula models and other timevarying copula methods.

Paper summary: The authors propose a Bayesian
      inference procedure for conditional bivariate copula models, where the
      copula models can have multiple parameters. The copula parameters are
      related to conditioning variables by a composition of parameter mapping
      functions and arbitrary latent functions with Gaussian process priors. The
      inference procedure for this model is based on expectation propagation.
      The method is evaluated on synthetic data and on financial time series
      data and compared to alternative copula-based models. The paper also
      includes an overview of related methods. Quality: The
      proposed method is sound. The authors include a description of related
      methods and list strengths and weaknesses of their method.
      Clarity: The paper is written clearly and concisely.
      It makes a very mature impression. I could not find any inconsistencies or
      typos. The figures and tables complement the text seamlessly, equations
      are clear and well explained and the text is flawless.
      Originality: The model and its inference procedure is
      an extension of a similar conditional copula model that was restricted to
      copulas with only one parameter. The present work generalizes this
      approach to allow more than one copula parameter. To my knowledge, this
      extension is novel. Significance: The flexibility of
      copula families with only one parameter is limited and the applicability
      of more flexible families is a huge advantage. Performance gains compared
      to the alternative methods are remarkable. This result is very promising.


Action is in the Eye of the Beholder: Eye-gaze Driven Model for Spatio-Temporal Action Localization
We propose a weakly-supervised structured learning approach for recognition and spatio-temporal localization of actions in video. As part of the proposed approach, we develop a generalization of the Max-Path search algorithm which allows us to efficiently search over a structured space of multiple spatio-temporal paths while also incorporating context information into the model. Instead of using spatial annotations in the form of bounding boxes to guide the latent model during training, we utilize human gaze data in the form of a weak supervisory signal. This is achieved by incorporating eye gaze, along with the classification, into the structured loss within the latent SVM learning framework. Experiments on a challenging benchmark dataset, UCF-Sports, show that our model is more accurate, in terms of classification, and achieves state-of-the-art results in localization. In addition, our model can produce top-down saliency maps conditioned on the classification label and localized latent paths.

This paper proposes the use of eye-gaze data for
      spatio-temporal action localization in videos, showing improved action
      classification accuracy and state-of-the-art action localization in
      videos. The method of incorporating this information is a latent SVM
      framework with modified constraints. The authors conduct various
      experiments to show that their method performs better than existing
      methods for localization and using eye gaze data could serve as a better
      prior than bounding boxes as done by previous works. Overall, I
      found the paper to be well written but the experimental results a little
      lacking. The concerns are explained below. Pros: - Interesting
      problem formulation of using eye-gaze data instead of bounding box data
      for learning - Experimental results are thorough and various problem
      settings are evaluated such as action classification, localization and
      gaze prediction. The experimental results suggest that the method proposed
      in the paper outperforms existing methods on the given tasks.
      Cons: - The cost and difficulty of collecting eye-gaze data is
      not well justified by the observed improvements. While eye-gaze data may
      be more natural in some ways, it is significantly harder to collect as
      compared to bounding box annotations which can be easily collected via
      crowdsourcing as no specialized equipment is required. This makes it
      significantly harder to scale this approach to other datasets. - How
      important is it to model gaze in a category specific manner? If gaze could
      be used for general saliency prediction, then the same method could be
      applied to other datasets more easily. - It is not clear why the
      authors do not compare their results to other existing methods for action
      classification such as [Z1] and [Z2] (mentioned below). The performance
      achieved by them is 85.6% and 86.5% as compared to the reported
      performance of 82.1%. It should be noted that [Z1] and [Z2] do not report
      results on action localization but the same can be said for [16] and [12]
      which are reported here. While [Z1] and [Z2] use chi-square kernels for
      learning, it is an advantage of their methods and it is not clear if the
      same set of kernels can be applied in the formulation presented here.
      - What is the corresponding localization result of the proposed method
      on the 3 categories used by Tran and Yuan[18]? It would be nice to include
      this in the figure caption. - There are no baselines provided for gaze
      localization and prediction. It is possible to use some simple methods
      such as exemplar-based detectors[Z3] for gaze prediction to show the
      necessity/strength of your method. References (I use Z to be
      distinct from paper references) [Z1] Evaluation of local
      spatio-temporal features for action recognition. In BMVC, 2010. [Z2]
      Learning hierarchical invariant spatio-temporal features for action
      recognition with independent subspace analysis. In CVPR, 2011.
      [Z3] Ensemble of Exemplar-SVMs for Object Detection and Beyond. In
      ICCV, 2011. The paper is generally well-written but here are some
      minor language errors: - L51: this comes at expense --> add "the"
      - L69-70: miss --> mis - L82: proved --> have proven -
      L134: extend --> extended


Better Approximation and Faster Algorithm Using the Proximal Average
It is a common practice to approximate "complicated" functions with more friendly ones.
In large-scale machine learning applications, nonsmooth losses/regularizers that entail great computational challenges are usually approximated by smooth functions. We re-examine this powerful methodology and point out a nonsmooth approximation which simply pretends the linearity of the proximal map. The new approximation is justified using a recent convex analysis toolproximal average, and yields a novel proximal gradient algorithm that is strictly better than the one based on smoothing, without incurring any extra overhead. Numerical experiments conducted on two important applications, overlapping group lasso and graph-guided fused lasso, corroborate the theoretical claims.

This paper describes a new proximal-gradient based
      procedure for the minimization of an objective function, composed by the
      sum of a convex smooth term and several nonsmooth convex (regularization)
      terms. The motivation comes from the fact that often, and in significant
      cases, proximal gradient schemes [e.g. FISTA] are not directly applicable
      to that scenario, since the proximity operator of the sum of several
      convex functions is not explicitly computable from the proximity operator
      of each individual term. The proposed algorithm deals directly with these
      individual prox operators. The method is actually a mix of
      approximation techniques (smoothing) as developed by Nesterov [5] and
      proximal gradient algorithms. Indeed they propose to replace the whole set
      of nonsmooth terms, with a unique nonsmooth (close) approximation for
      which the proximity operator is computable; and this approximation turns
      out to be the proximal average of the regularization terms --- which has
      bees studied in detail in [11]. Solutions of the approximate problem are
      then related to the solutions of the original one and a complexity of
      $O(1/\varepsilon)$ is shown. The authors claim this procedure to be faster
      then the one based on the smoothing technique (S-APG algorithm [13]).
      Indeed, although the asymptotic behavior remains the same, an improvement
      of the constants is proved. This improvement is finally shown in the
      experimental section. I find the proposed method quite interesting
      and the main idea, of approximate with a nicer nonsmooth term instead of
      smoothing, valuable and original. A weak aspect is that the nonsmooth
      terms are required to be (globally) Lipschitz continuous, hence finite
      everywhere. This means that constraints cannot be treated. The theoretical
      part is generally sound and precise. Unfortunately I am afraid that the
      proof of the main theorem is not fully correct. Nevertheless the statement
      is correct and the problem in the proof can be overcome without much
      effort as I show below. As regards the experimental section, perhaps other
      comparisons with more recent approaches (like the two loop-algorithm
      proposed in [10]) could have been useful. Anyway it is clearly stated that
      the algorithm is meant to improve S-APG and the section of experiments
      addresses this comparison. TYPOS, SMALL CORRECTIONS AND
      SUGGESTIONS: -- In the properties listed in Prop. 1, I would add
      the differentiability of the Moreau envelope and the explicit expression
      of the gradient $\nabla M_f^\mu = \mu^{-1} (Id - P_f^\mu)$. -- If
      $SS_\mu$ is the class of $\mu$-Lipschitz continuous functions, then $f \in
      SC_\mu \iff f^* \in SS_{1/\mu}$ (not $f^* \in SS_\mu$) and $M^\mu :
      \Gamma_0 \to SS_{1/\mu}$. Moreover $M^\mu : \Gamma_0 \to SS_{1/\mu}$ is
      concave only on finite valued functions, not on the whole space
      $\Gamma_0$, since the convex combination of two functions in $\Gamma_0$
      might be even not proper. Thus, Prop. 2 should be accordingly corrected.
      -- All the results between Def 1 and Prop 3 are in [11]. This
      should be clearly stated. A CRITICAL POINT: I think
      that the proof of Theorem 1 is not quite correct and should be pursued
      with more care. The criticism relies on the fact that generally in prox
      gradient algorithms, the bound on the precision (after a given number of
      iterations) depends on the distance between the starting point and the
      solution set of the problem. This means that the notation $O(\sqrt{\frac 1
      {\mu \varepsilon}})$, recalled in the proof, actually hides a further
      dependence on $\mu$ in the constant --- since it is the approximate
      function that is being minimized. To overcome this problem, I suggest
      the authors to follow the path of [13] (see Proof of Theorem 2 in the
      appendix). Indeed if one chooses $\mu \leq 1/L_0$ and minimizes the
      approximation $F_\mu(x) = \gamma \ell(x) + A^\mu(x)$ with FISTA, then for
      every $x \in H$ it holds $$ F_\mu(x_t) - F_\mu(x) \leq 2 \frac 1
      \mu \frac {\Vert x_0 - x \Vert^2}{t^2}\,. $$ Note that this
      inequality is valid for every $x \in H$, not only for the minimizers of
      $F_\mu$ (just look at the proof of FISTA in [2] or [10]). Then if we now
      take $x_*$ as a minimizer of the original function $F$ and plug it into
      the previous inequality we end up with $$ F(x_t) - F(x_*) \leq
      \frac{\mu \overline{L^2}}{2} + 2\frac{1}{\mu} \frac {\Vert x_0 - x_*
      \Vert^2}{t^2}. $$ This way the constant term $\Vert x_0 - x_*
      \Vert^2$ does not depend anymore from the approximate problem (hence from
      $\mu$), but only on the original problem (see also the discussion in
      section 3.6 in [13]). The statement of the theorem remains unchanged.


Buy-in-Bulk Active Learning
In many practical applications of active learning, it is more cost-effective to request labels in large batches, rather than one-at-a-time. This is because the cost of labeling a large batch of examples at once is often sublinear in the number of examples in the batch. In this work, we study the label complexity of active learning algorithms that request labels in a given number of batches, as well as the tradeoff between the total number of queries and the number of rounds allowed. We additionally study the total cost sufficient for learning, for an abstract notion of the cost of requesting the labels of a given number of examples at once. In particular, we find that for sublinear cost functions, it is often desirable to request labels in large batches (i.e., buying in bulk); although this may increase the total number of labels requested, it reduces the total cost required for learning.

*Summary* This paper discusses
      (disagreement-based) batch-mode active learning, and puts forward a
      theoretical framework bounding the sample complexity and the total cost of
      buy-in-bulk active learning (i.e., batch-mode active learning with
      sublinear cost functions w.r.t. the size of a batch). In particular, they
      study batch-mode active learning from two aspects: (1) Suppose the batch
      sizes are equal. Fix the number of rounds examined, how many labels one
      has to request in total to achieve some certain error rate; (2) given that
      the cost to obtain a batch of labels is sublinear in the size of the batch
      ( as referred to as ``buy-in-bulk discount''), how is the total cost of
      the proposed batch algorithms compared with that of fully-sequential
      active learning methods. For the first aspect, the authors propose
      batch-based variants of the well-known CAL algorithm for sequential active
      learning, and provide upper bounds on label complexity of k-batch active
      learning, for both the realizable case and the non-realizable case (with
      Tsybakov noise). For the second aspect, they provide a cost-adaptive
      modification of the CAL algorithm, and find that the total cost by this
      algorithm may often be significantly smaller than that of the analogous
      methods in the fully sequential setting. *Quality* This
      paper nicely extends the previous work (Hanneke ’10) to the batch-mode
      setting. The first part (section 3, 4) derives upper bounds (depending the
      size of a batch) of label complexity of the proposed batch algorithms. It
      is technically sound, and thoroughly explained. The second part (section
      5), however, is less clear. From line 376 to 380, the authors directly
      compare two upper bounds that are obtained separately (potentially with
      different low-order terms hidden in the big O notation), and conclude that
      buy-in-bulk active learning are usually more cost-efficient - which seems
      not fair to me. Besides, there is no empirical evaluation of the
      proposed algorithm. Given the various motivating applications mentioned
      before, it would be interesting to see how the Cost-Adaptive CAL performs
      empirically with different cost models. *Clarity* The
      second result (Section 5) for buy-in-bulk active learning, as appears in
      the paper, is striking, yet unconvincing. According to Theorem 5.1, if the
      cost function is linear in batch sizes, then the proposed batch-mode
      algorithm (Cost-Adaptive CAL) will exhibit the same form of upper bound as
      that of the fully-sequential CAL algorithm. Intuitively, under such (unit)
      cost setting, the cost of Cost-Adaptive CAL should be higher than fully
      sequential CAL. However, according to the two upper bounded provided in
      line 343 and 367, one can not tell the difference. Further analysis on the
      difference (hidden constant or low-order terms) between the cost bounds
      would be helpful. Originality and Significance Active learning
      problems under the batch-mode setting are quite natural and interesting,
      but so far only a few theoretical work have been done (e.g., Chen &
      Krause, ICML’13 formulate it as an adaptive optimization problem). This
      paper explores the theoretical aspect of the mellow schemes for batch-mode
      active learning. Although the proof technique relies heavily on the
      previous work on (sequential) active learning, it is one of the few
      attempts made along this line of research. Editorial issue(s):
      1. line 171, second term in the max bracket: $er(h)$ is not a
      subscript. 2. the paper exceeds the submission page limit by 3 lines.
      Update: I've considered the authors' rebuttal, and decide not to
      change my score. Specifically, I’m still not convinced that the cost of
      the batch algorithm is only constant factor away from the sequential CAL
      (according to the rebuttal) -- there might be some lower order terms in
      the bound of the batch algorithm, which is important when comparing with
      the sequential bound. This is one of the main reasons that I would like to
      see some empirical evidence of the statement.


Bellman Error Based Feature Generation using Random Projections on Sparse Spaces
This paper addresses the problem of automatic generation of features for value function approximation in reinforcement learning. Bellman Error Basis Functions (BEBFs) have been shown to improve policy evaluation, with a convergence rate similar to that of value iteration. We propose a simple, fast and robust algorithm based on random projections, which generates BEBFs for sparse feature spaces. We provide a finite sample analysis of the proposed method, and prove that projections logarithmic in the dimension of the original space guarantee a contraction in the error. Empirical results demonstrate the strength of this method in domains in which choosing a good state representation is challenging.

This paper proposes to use random projections as a
      proxy to learn BEBFs (Bellman Error Basis Functions). Given a (high
      dimensional) set of features and the currently estimated value function,
      the features are (randomly) projected on a smaller space, and the temporal
      differences errors (related to the currently estimated value function) are
      regressed on these projected features. The (scalar) regressed function is
      then added to the set of features used to estimate the value function. A
      finite sample analysis is conducted, the main result showing that if the
      Bellman residual is linear in the (high dimensional) features, then the
      Bellman error can be well regressed on the compressed space (depending
      notably on the size of this space and on the number of samples). The
      authors also use this result to provide some guarantee on the estimated
      value function. The proposed algorithm is compared to state-of-the-art
      approaches on a high dimensional problem. This paper is very
      clearly written and presents some quite interesting contributions, summed
      up above. I have mainly two comments. 1) First, I found that lemma 4
      was not so clear, which is quite a pity as it summarizes the preceding
      results and quantifies the efficiency of the proposed approach. Notably,
      how strong is the assumption of the existence of constants c1 to c3? Also,
      is it straightforward that the universal (?) constants c4 and c5 exist (or
      is this an assumption?). Considering the provided bound, we should get
      $\gamma_0$ as small as possible, which seems actually a good thing also
      for the choice of $d$ and $n$... this seems quite surprising. What did I
      miss? I think that a proof (in the appendix) may help (even if considered
      as straightforward) 2) Second, I have some comments regarding the
      experimental section: * what is the reward? * it would be
      interesting to consider also CBEBF (the not simple version) in the
      experiments, even if it has a higher computational cost, to see what can
      be gained in estimation quality (above l2-LSTD?) * the results of
      l2-LSTD are quite surprising... a validation set is mentioned, but
      cross-validation for value estimation does not seem to be straightforward.
      Do you use the NRPE to cross-validate? * it would have been
      interesting to compare your algorithm to iLSTD [1], which pursue a similar
      goal (sample efficiency and low computational cost) * why do you
      change the schedule for dimensions between experiments (as it is the same
      problem)? * what is the $\mu$ distribution in the experiment (used for
      NRPE)? * that is true that the gradient-based approaches should be
      less sample-efficient (and implies the choice of some possibly sensitive
      meta-parameters), but it would have been intersting to show it
      (empirically) * regarding the last experiment, it would have been
      interesting to see if the efficiency of l2-LSTD degrades when the feature
      are not sparse (e.g. features transformed through a linear mapping). This
      is actually mentioned in the text (no change), but fig.2 suggests that the
      feature vector is quite sparse (400 non-zero over 65536). [1]
      Alborz Geramifard, Michael Bowling, and Richard S. Sutton. Incremental
      Least-Squares Temporal Difference Learning. In AAAI.
      2006


Wavelets on Graphs via Deep Learning
An increasing number of applications require processing of signals defined on weighted graphs. While wavelets provide a flexible tool for signal processing in the classical setting of regular domains, the existing graph wavelet constructions are less flexible -they are guided solely by the structure of the underlying graph and do not take directly into consideration the particular class of signals to be processed. This paper introduces a machine learning framework for constructing graph wavelets that can sparsely represent a given class of signals. Our construction uses the lifting scheme, and is based on the observation that the recurrent nature of the lifting scheme gives rise to a structure resembling a deep auto-encoder network. Particular properties that the resulting wavelets must satisfy determine the training objective and the structure of the involved neural networks. The training is unsupervised, and is conducted similarly to the greedy pre-training of a stack of auto-encoders. After training is completed, we obtain a linear wavelet transform that can be applied to any graph signal in time and memory linear in the size of the graph. Improved sparsity of our wavelet transform for the test signals is confirmed via experiments both on synthetic and real data.

Summary: The authors present a method for constructing
      wavelets on weighted graphs that can adapt to a class of signals. The
      approach uses the lifting scheme and by connecting this scheme to a deep
      auto-encoder network, the authors are able to perform unsupervised
      training similar to the pre-training of a stack of auto-encoders. The
      approach is linear in the size of the graph. The authors explore the
      performance of the wavelet construction approach through application to
      several data sets. Quality: The authors provide an elegant
      approach for taking into account the class of signals when forming
      wavelets on a graph. Most constructions are based solely on the graph
      structure, and those few methods that do allow adaptivity based on the
      signals [19,21] have significant limitations. The technical
      development is clearly described, novel, and leads to an efficient
      algorithm that produces wavelets with desirable properties. The
      partitioning approach is described very briefly and there is almost no
      discussion of how sensitive the approach is to the partitioning method.
      The spectral clustering algorithm of [20] has some limitations
      (performance being poor for some kinds of graphs) and it would be nice to
      see more information about how sensitive the overall wavelet construct is
      to the partitioning scheme. For their results on irregular graphs,
      where I think the construction is of most interest, the authors do not
      make a comparison to compression or reconstruction using any other type of
      graph wavelet. Instead, the comparison is to (somewhat dated) learning
      techniques that are suited to manifold analysis. This constitutes one
      significant weakness of the paper. Clarity: The paper is very well
      written and the development is easy to follow. As discussed above, more
      detail on some of the Originality: The authors provide a new
      method for constructing wavelets on graphs that has the important ability
      to adapt to the class of functions that the wavelets will be used to
      represent. The method is highly original. Significance: The paper
      represents a useful contribution in the field of wavelets and
      multiresolution analysis on graphs, a field of growing interest due to the
      numerous potential applications. I consider that the signal adaptivity,
      while preserving linearity and efficiency of construction, represents a
      significant advance.


Decision Jungles: Compact and Rich Models for Classification
Randomized decision trees and forests have a rich history in machine learning and have seen considerable success in application, perhaps particularly so for computer vision. However, they face a fundamental limitation: given enough data, the number of nodes in decision trees will grow exponentially with depth. For certain applications, for example on mobile or embedded processors, memory is a limited resource, and so the exponential growth of trees limits their depth, and thus their potential accuracy. This paper proposes decision jungles, revisiting the idea of ensembles of rooted decision directed acyclic graphs (DAGs), and shows these to be compact and powerful discriminative models for classification. Unlike conventional decision trees that only allow one path to every node, a DAG in a decision jungle allows multiple paths from the root to each leaf. We present and compare two new node merging algorithms that jointly optimize both the features and the structure of the DAGs efficiently. During training, node splitting and node merging are driven by the minimization of exactly the same objective function, here the weighted sum of entropies at the leaves. Results on varied datasets show that, compared to decision forests and several other baselines, decision jungles require dramatically less memory while considerably improving generalization.

Summary of the paper: This paper revisits the idea
      of decision DAGs for classification. Unlike a decision tree, a decision
      DAG is able to merge nodes at each layer, preventing the tree from growing
      exponentially with depth. This represents an alternative to decision-trees
      utilizing pruning methods as a means of controlling model size and
      preventing overfitting. The paper casts learning with this model as an
      empirical risk minimization problem, where the idea is to learn both the
      DAG structure along with the split parameters of each node. Two algorithms
      are presented to learn the structure and parameters in a greedy layer-wise
      manner using an information-gain based objective. Compared to several
      baseline approaches using ensembles of fixed-size decision trees,
      ensembles of decision DAGs seem to provide improved generalization
      performance for a given model size (as measured by the total number of
      nodes in the ensemble). Quality: This paper is of generally
      good quality. The related work seems to have been throughly investigated,
      the model and algorithms make intuitive sense, and the experiments are
      fairly compelling and do a good job of investigating the effects of
      varying different design parameters. One aspect that seems to be missing
      is a comparison of the training and evaluation times of different
      approaches. This is an important consideration in these models that I
      think should be addressed. Clarity: The paper is quite clear
      for the most part. The writing is good, the model is well-presented, and
      the experiments are fairly easy to comprehend. One minor detail that I am
      not quite clear on is whether the ensemble is trained with bagging in
      addition to the other randomized elements of the learning algorithm. Also,
      the authors suddenly refer to energy in section 3.1, is this the same as
      the empirical risk? Finally, there is a minor grammatical error at the end
      of the LSearch description: "for considerably less compute."
      Originality: The ideas underlying this paper are fairly well
      established (decision DAGs, ensembles, empirical risk minimization and
      information gain). However, the novelty of this paper is in combining
      these ideas into a cohesive model, and providing intuitive algorithms to
      learn it. Significance: Random forests are a heavy favorite
      among machine learning practitioners, and therefore any related method
      that improves upon them without significantly increasing their
      computational or implementation overhead could have a large
      impact.