Bayes-Adaptive Simulation-based Search with Value Function Approximation
Bayes-adaptive planning offers a principled solution to the explorationexploitation trade-off under model uncertainty. It finds the optimal policy in belief space, which explicitly accounts for the expected effect on future rewards of reductions in uncertainty. However, the Bayes-adaptive solution is typically intractable in domains with large or continuous state spaces. We present a tractable method for approximating the Bayes-adaptive solution by combining simulationbased search with a novel value function approximation technique that generalises appropriately over belief space. Our method outperforms prior approaches in both discrete bandit tasks and simple continuous navigation and control tasks.

This paper deals with Bayes-adaptive model-based reinforcement learning. The underlying principle of this type of approach is to maintain a Bayesian posterior over dynamics (conditioned on past experienced transitions) and to seek at each time step for the action optimizing the related augmented MDP (on state-history meta-states and related meta-dynamics), which is generally an intractable problem (for exact solving). This contribution relies on two previous ideas, simulation-based search (with "root sampling", which avoids updating the belief over dynamics during planning) and value function approximation (which requires introducing proper features for handling histories), combining them to provide a new approach. In addition to this general approach, called BAFA, the authors provide an alternative and more general proof for the validity of root sampling and provide some experimental results.Overall, the paper is well written and clear, it proposes a sound approach, based on known ideas but combining them smartly (especially for the history features, which seems to be the newest part/the core contribution). My major comment is that experiences, if convincing, are not detailed enough to be reproducible (for example, some meta-parameters are not provided, nor discussed). There might also be some problems with corollary 1 (is the policy really stationary?).Detailed (mostly minor) comments:* Line 106, "belief" can be understood as a synonym of "history" in this sentence, which is misleading* In the pseudo-code, Simulate(...,d) should be Simulate(...,t), the "+" in the parameters update (Q-value) should be a "-"* Line 103, a weighted squared loss is mentioned, what are the weights? (do not appear in the next equation, nor in the update rule)* Line 153, it is said that it is necessary to show that root sampling is valid to exploit general convergence results of classical RL. That's right, but is this sufficient?* Lines 190-192 (and Corollary 1). I agree with this remark, stating that using a fixed policy implies the same distribution of parameters update, in light of Lemma 1. However, this is not what does the provided algorithm (at least as stated in the pseudo-code): the considered policy is epsilon-greedy respectively to Q_w, but the parameters are updated during planning, so the policy is not stationary. The policy should remain fixed for at least each drawn dynamics (if this is actually the case, the pseudo-code should be more clear).* For the Bernoulli Bandit, has gamma any importance? How does this compare to standard bandit algorithms?* Generally, for the experiments, more details could be provided (learning rates for the function estimation, more details about parameterizations, etc.). Ideally, they should be reproducible.* Fig. 2, K already denotes the kernel of the GP, choose another notation?* In the height map problem, it is rather strange to consider the reward z as a state component. Why is this done?* When the dynamics distribution is modeled as a GP, cannot the value function be modeled as a GP too (not necessarily straightforward, due the bilinear representation, but maybe an interesting perspective)?* Some uppercases are missing in the bib (eg, pomdps in ref 20)


Generalized Dantzig Selector: Application to the k-support norm
We propose a Generalized Dantzig Selector (GDS) for linear models, in which any norm encoding the parameter structure can be leveraged for estimation. We investigate both computational and statistical aspects of the GDS. Based on conjugate proximal operator, a flexible inexact ADMM framework is designed for solving GDS. Thereafter, non-asymptotic high-probability bounds are established on the estimation error, which rely on Gaussian widths of the unit norm ball and the error set. Further, we consider a non-trivial example of the GDS using k-support norm. We derive an efficient method to compute the proximal operator for k-support norm since existing methods are inapplicable in this setting. For statistical analysis, we provide upper bounds for the Gaussian widths needed in the GDS analysis, yielding the first statistical recovery guarantee for estimation with the k-support norm. The experimental results confirm our theoretical analysis.

The paper introduces a generalization of the Dantzig selector, where the L1/Linfinity norm pair are replaced with an arbitrary norm R and its dual, along with supporting statistical theory and an inexact ADMM.  The authors also provide detailed derivations of both proximal operators and statistical theory for the special case where R is the k-support norm.This is very nicely written paper on an interesting subject.  My only critical comment is that its motivation, beyond generalization for its own sake, is unclear.  Why do we need a generalized Dantzig selector when there are already many papers on penalized loss methods?  This is an important question, but I admit not completely fair to the authors---they're not alone in avoiding this question.  Some ways to address this question include comparisons on real data and simulations, and/or commentary on the computational complexity and practical differences between applying this method and some other penalized regression method.If we ignore the question of motivation and accept that a generalization of the Dantzig selector is worthwhile, then I believe that this work presents a significant advance in a very clearly written paper.  Aside from the Dantzig selector, the auxiliary results on implementing the ADMM algorithm, including derivation of the proximal operator for the k-support norm, are worthwhile on their own.Minor comment:Inequality (27) of Theorem 4 is is contained in Lemma 3.3 of Gordon et al. (2007) which is an even stronger result.  I believe that a bound similar to inequality (28) could also be derived using the results of Gordon et al. (2007).Gordon, Y., Litvak, A. E., Mendelson, S., & Pajor, A. (2007). Gaussian averages of interpolated bodies and applications to approximate reconstruction. Journal of Approximation Theory, 149(1), 59–73. doi:10.1016/j.jat.2007.04.007


Distributed Estimation, Information Loss and Exponential Families
Distributed learning of probabilistic models from multiple data repositories with minimum communication is increasingly important. We study a simple communication-efficient learning framework that first calculates the local maximum likelihood estimates (MLE) based on the data subsets, and then combines the local MLEs to achieve the best possible approximation to the global MLE given the whole dataset. We study this framework's statistical properties, showing that the efficiency loss compared to the global setting relates to how much the underlying distribution families deviate from full exponential families, drawing connection to the theory of information loss by Fisher, Rao and Efron. We show that the "full-exponential-family-ness" represents the lower bound of the error rate of arbitrary combinations of local MLEs, and is achieved by a KL-divergence-based combination method but not by a more common linear combination method. We also study the empirical properties of both methods, showing that the KL method significantly outperforms linear combination in practical settings with issues such as model misspecification, non-convexity, and heterogeneous data partitions.

The main message of this paper is: IF you want to perform density/parameter estimation for an exponential family by dividing the data then combining the locally optimal estimators,  THEN use KL as the combining function BECAUSE  1) if the distribution is a full exponential family then you get the globally optimal solution and 2) if the distribution is a curved exponential family then this the best you can do. The rest of the paper proves these results and showed some empirically evidence on toy datasets.The main message of this paper is theoretical however it still didn't answer a few questions:1- Why do I have to follow the divide and combine approach?  Granted KL is the best if this is all what I can do, but what if I can use asynchronous techniques (or synchronous algorithms) to perform distributed inference? what are the trade-offs?2- At least I would have liked to see point 1 addressed experimentally. 3-  It would be great to link the results in this paper with the following paper:Parallelized Stochastic Gradient Descenthttp://martin.zinkevich.org/publications/nips2010.pdfIt addresses the same setup from an optimization point of view and showed the optimality of the linear averaging method under certain conditions.  Can you relate these two views? Take for instance logistic regression that is also a member of the exponential family, what can you say here?4 - From a practical point of view, even using the method in the paper cited above, there is still a gap between theory and practice. From a hands-on experience on very large dataset (that is hundreds of millions of examples and tens of millions of features), performing divide and combine (via linear average) in a simple SGD setting for even linear/logistic regression results in a solution that is equivalent to the global solution over  5-15% less data (i.e. there is a loss of information). It is not clear to me if this is because of the linear averaging or something more sophisticated can be said using the KL method instead.5 - I understand that this is a theory paper but the experimental section could be stronger. What happens when you vary the number of features, number of groups, Number of model parameters (# of mixtures)?- please mention in the paper under what family (curved or full) is the mixture of Gaussian example?


Decomposing Parameter Estimation Problems
We propose a technique for decomposing the parameter learning problem in Bayesian networks into independent learning problems. Our technique applies to incomplete datasets and exploits variables that are either hidden or observed in the given dataset. We show empirically that the proposed technique can lead to orders-of-magnitude savings in learning time. We explain, analytically and empirically, the reasons behind our reported savings, and compare the proposed technique to related ones that are sometimes used by inference algorithms.

This paper proposes a method for finding a max likelihood estimate of the parameters of a Bayesian Network (BN) with hidden variables by decomposing the global optimization problem into a set of independent sub-problems.  The main motivation for this decomposition is that in a data set the variables are either always hidden, or always observed - i.e. variables aren’t missing at random. By assuming the learning problem has this property, the authors propose a partitioning of the BN’s parameters in the same way that one decomposes the BN parameter estimation problem in the fully observed case.  They then demonstrate the benefit of this decomposition empirically by comparing their method to off-the-shelf implementations of the EM algorithm - perhaps the default approach for BN parameter estimation in missing data problems.Overall, I found this paper to be very well written and easy to follow, with clear and concise definitions and plenty of well placed examples.  My two main criticisms of this paper are with its novelty and significance.  The two main ideas appear to be: 1) that a BN with fixed structure can be decomposed into smaller, easier to learn sub-problems; and 2) that the dataset can be compressed by maintaining counts for each of these subproblems. The idea of exploiting a network’s structure to expedite inference/learning has been studied in various hidden-variable architectures, such as RBMs and HMMs. The proposed decomposition falls out naturally, for example, in an RBM architecture.  In contrast, the proposed method provides no-benefit in the case of an HMM architecture as there is no decomposition.  Along these lines, I felt that the claims made in the experimental suggestion were a bit exaggerated.  It seems a bit unfair to take on off-the-shelf EM method, which has no knowledge of the fixed pattern of missingness, and comparing to a method designed explicitly to exploit such a missingness pattern. The reason these general purpose methods can’t take advantage of the “data projection” operator is because they do not assume that the missingness pattern is fixed!  Rather than just reporting the speed-up, it would have been better to show how the speed-up degrades as the decomposition becomes less prominent. Increasing the percentage of observed variables does not tell us if the number of sub-networks decreases or increases, or anything about the size of the sub-networks relative to the entire network. In addition, how does the method perform as the size of the data set is changed. Finally, I think the authors should have addressed the role of regularization. What type of priors on the hidden parameters are supported by the proposed method and which are not.Detailed Comments:- Line 261: what is meant by classical EM?  EM runs inference on each training instance, while D-EM runs on a sub-network using summarized/compressed data.- Line 267: Why does making all unobserved variables hidden make the problem harder?- Line 270: The caption of Figure 3 suggests that I should see three differently bars, one for each chain length.  However, I only see one set of bars.- Line 284: In Table 1, provide some summary information of the UAI networks, e.g. number of variables, tree-width, variable arity, etc…- Line 324: In Figure 4, the center and right charts were too small to reveal anything.  Perhaps a log-log plot would be better.


Coresets for k-Segmentation of Streaming Data
Life-logging video streams, financial time series, and Twitter tweets are a few examples of high-dimensional signals over practically unbounded time. We consider the problem of computing optimal segmentation of such signals by a k-piecewise linear function, using only one pass over the data by maintaining a coreset for the signal. The coreset enables fast further analysis such as automatic summarization and analysis of such signals. A coreset (core-set) is a compact representation of the data seen so far, which approximates the data well for a specific task -in our case, segmentation of the stream. We show that, perhaps surprisingly, the segmentation problem admits coresets of cardinality only linear in the number of segments k, independently of both the dimension d of the signal, and its number n of points. More precisely, we construct a representation of size O(k log n/ε 2 ) that provides a (1 + ε)-approximation for the sum of squared distances to any given k-piecewise linear function. Moreover, such coresets can be constructed in a parallel streaming approach. Our results rely on a novel reduction of statistical estimations to problems in computational geometry. We empirically evaluate our algorithms on very large synthetic and real data sets from GPS, video and financial domains, using 255 machines in Amazon cloud.

The authors study the problem of k-segmentation of data and provide a compact coreset that is linear in size of the dimensions and number of segments. Strengths:1. Compact coreset that is computed in time linear in n, d, k.2. The paper is well written and provides a good summarization of existing techniques.3. The experiments include a collection of very interesting data sets. Concerns:(Note: A few concerns raised are directly in light of the work in [12], especially since the authors address [12] as the most recent paper worthy of comparison.)1. In the abstract, the authors claim that the size of the coreset is independant of d and n, but in Page 3, the size of the coreset is given as O(dk/\eps^2) and in Page 5, the size of the coreset is given as O(klog(n)/\eps^2). It is therefore very unclear what the size of the coreset is.2. The authors claim that the the size of their coreset is an improvement over [12] whose runtime is cited to be cubic in both d and k. But, the size of the coreset as claimed by [12] is linear in k.3. The cost term involving the coreset is a little convoluted in the paper involing sum of two terms, whereas [12] provides a cleaner cost function very similar to the original cost function used on the entire data. This makes it difficult to evaluate the effectiveness of the coreset.4. While the deterministic nature of the coreset computation is appealing, it is unclear how to run parallel and streaming versions of the algorithm since the effect of an adversary is scrambling the inputs each time is never discussed. 5. In its current form, the meat of the paper only has brief proof sketches and no analysis of run times making it contribution limited. The technical contribution of the paper is limited to a page since a flavor of BICRITERIA algorithm is already discussed in [12] (and not mentioned in this paper).6. In page 7, the authors claim that they significantly outperform other schemes (Figure 2), while the other schemes discussed are merely uniform coreset and a random coreset (which take constant time to compute and are therefore much faster). The authors fail to discuss and compare their approach to existing state of the art methods including [12]. The methods that they compare to are from 40 years ago.


Two-Stream Convolutional Networks for Action Recognition in Videos
We investigate architectures of discriminatively trained deep Convolutional Networks (ConvNets) for action recognition in video. The challenge is to capture the complementary information on appearance from still frames and motion between frames. We also aim to generalise the best performing hand-crafted features within a data-driven learning framework. Our contribution is three-fold. First, we propose a two-stream ConvNet architecture which incorporates spatial and temporal networks. Second, we demonstrate that a ConvNet trained on multi-frame dense optical flow is able to achieve very good performance in spite of limited training data. Finally, we show that multitask learning, applied to two different action classification datasets, can be used to increase the amount of training data and improve the performance on both. Our architecture is trained and evaluated on the standard video actions benchmarks of UCF-101 and HMDB-51, where it is competitive with the state of the art. It also exceeds by a large margin previous attempts to use deep nets for video classification.

Summary:This paper proposes a model for solving discriminative tasks with video inputs.  Themodel consists of two convolutional nets. The input to one net is an appearanceframe. The input to the second net is a stack of densely computed optical flowfeatures. Each pathway is trained separately to classify its input.  Theprediction for a video is obtained by taking a (weighted) average of thepredictions made by each net.The model is evaluated on two datasets.  The results are impressive and match(or come close to) the state-of-the-art methods which use intensivelyhand-crafted features.Strengths:- This model is a simple application of convolutional nets that gets good results.- Previous deep learning models have not tried to use optical flow (or otherhand-crafted features) for vision tasks, preferring to learn all features frompixels directly. This paper shows that at least for action recognition, opticalflow fields are a useful input representation. This is an important contribution.Weaknesses:- The model does not take into account the overall sequence ofactions over the entire video but only models fixed length (L=10) adjacent frames.Quality:The experiments are well-designed. It would be more insightful if the authors also include some analysis of the error modes, for example, are their some classes or group of classes that the model is unable to classify well ? Is it possible to characterize the kind of videos on which the model does not work well ?Clarity:The paper is clearly written. The model is well explained.Originality:The application of a convolutional net to optical flow features is novel.Significance:This approach could have a significant impact on the research in using videos and motion for various vision problems.


Rounding-based Moves for Metric Labeling
Metric labeling is a special case of energy minimization for pairwise Markov random fields. The energy function consists of arbitrary unary potentials, and pairwise potentials that are proportional to a given metric distance function over the label set. Popular methods for solving metric labeling include (i) move-making algorithms, which iteratively solve a minimum st-cut problem; and (ii) the linear programming (LP) relaxation based approach. In order to convert the fractional solution of the LP relaxation to an integer solution, several randomized rounding procedures have been developed in the literature. We consider a large class of parallel rounding procedures, and design move-making algorithms that closely mimic them. We prove that the multiplicative bound of a move-making algorithm exactly matches the approximation factor of the corresponding rounding procedure for any arbitrary distance function. Our analysis includes all known results for move-making algorithms as special cases.

The authors analyze the relationship between randomized roundingalgorithms and move-making algorithms for metriclabeling, which is the problem of MAP inference in a MRF where thebinary potentials are proportional to a distance metric on the labelspace. They demonstrate connections between the two classes ofalgorithms. For each of three rounding schemes---complete, interval,and hierarchical---they present companion move-making algorithms thatachieve the same worst-case approximation ratio (over all possibleunary potentials and edge weights) for any particular distance metricon the label space.The primary contribution of the paper is the analysis, which showsthat the move-making algorithms have the same approximation guaranteesas the rounding-based ones. This is significant because therounding-based algorithms have historically come with the strongestapproximation guarantees, but move-making algorithms are faster. Thus,the guarantees are exported to the faster algorithms.The paper can be situated slightly better in terms of prior work. Fromwhat I can tell:* Previous papers [14,15] have presented move-making algorithms  similar to those given here and proved approximation factors for  those algorithms for certain classes of distance functions. * This work, on the other hand, applies to *any* distance function,  and shows that the approximation ratio of the rounding scheme  (for that distance function) is matched by the companion move-making  algorithm. The difference between this and previous results is  subtle and could be emphasized more.* I believe the move-making algorithms themselves are not  contributions of this paper. It would help a great deal if the  authors could clearly attribute each algorithm to a particular  source and describe which parts are their original contributions. The paper is well-written and generally well executed. I felt that toomuch of the paper was used describing the algorithms in detailgiven that the main intellectual contribution seems to be in theanalysis.A significant question that is left unanswered in the main paper is:what is the general analysis technique for converting a roundingscheme to a move-making algorithm? Is there a clear recipe? If not,what are the significant elements of the analysis? The paper would bestrengthened considerably by providing some discussion of this.


RAAM: The Benefits of Robustness in Approximating Aggregated MDPs in Reinforcement Learning
We describe how to use robust Markov decision processes for value function approximation with state aggregation. The robustness serves to reduce the sensitivity to the approximation error of sub-optimal policies in comparison to classical methods such as fitted value iteration. This results in reducing the bounds on the γ-discounted infinite horizon performance loss by a factor of 1/(1 − γ) while preserving polynomial-time computational complexity. Our experimental results show that using the robust representation can significantly improve the solution quality with minimal additional computational cost.

The paper proposes to use robust MDPs in computing value function approximation for MDPs. The idea is that robustness can compensate for the approximation error when aggregating states.I am generally very positive about the idea. It makes sense and is inline with tying robustness and generalization in other fields of machine learning, esp. classification and regression. A reference to one of the works is probably due.Algorithm 1 is not very well explained. Since the aggregation is given, this can be much simplified (no need for lines 5-9). I think that the role of w and \omega needs to be made much clearer. Perhaps, continuing with Example 3.1 to explain how the robust model is created would do.Section 3.1 is a bit out of the blue. This looks very useful, but can probably be made into a comment.Section 4:The condition that there is a concentration coefficient implies (very) strong mixing. I can't think of many MDPs that would satisfy it (certainly not large MDPs like the ones you'd care about). Theorem 4.1 is a bit strange. I would expect to see \overline{\rho} (\overline{\pi}^*) bounded (i.e., the true optimal policy). But instead you bound \pi^*, the policy where all the actions are forced to be the same in the aggregate. This is what a good aggregation should lead to--what you have now may be trivial if we choose a bad aggregation. So the main issue of finding a good aggregation (as in Bertsekas and Casanton and many other papers) is ignored.After the aggregation we don't have an MDP anymore. This is the case, since the Markovian nature is lost. How is that reflected in the results and proof?I got lost a bit in the proof of Theorem 4.1: I understand why it should work but a better mapping of the proof (overview/steps) would make it much clearer.Theorem 4.4 is the main theorem of the paper, as far as I am concerned. I find the definition of a correlation coefficient interesting, but not very clear. In particular, I don't really understand when $D$ is finite, or even "small." Showing an example where the bound of Theorem 4.4 is computed would be very helpful. RE Section 5: I would have liked to see the aggregation and understand how things play out in terms of the value of the optimal policy (i.e., the true optimal one). At any case, the comparison seems a bit unfair: the (initial) claim is that RAAM can handle the approximation error, not that RAAM is robust to anything. This is a nice bonus. A comparison with the results of Tamar et al from the recent ICML (jmlr.org/proceedings/papers/v32/tamar14.pdf). In particular, a discussion of function approximation for robust MDPs appears there too. I think that the results are sufficiently different, but still, a comparison is due.The references are sloppy. You should double-check them.Update after reading the authors response: Thank you for clarifying. Adding an example with $D$ is small would be good. I like the  paper and the idea.


A Statistical Decision-Theoretic Framework for Social Choice
In this paper, we take a statistical decision-theoretic viewpoint on social choice, putting a focus on the decision to be made on behalf of a system of agents. In our framework, we are given a statistical ranking model, a decision space, and a loss function defined on (parameter, decision) pairs, and formulate social choice mechanisms as decision rules that minimize expected loss. This suggests a general framework for the design and analysis of new social choice mechanisms. We compare Bayesian estimators, which minimize Bayesian expected loss, for the Mallows model and the Condorcet model respectively, and the Kemeny rule. We consider various normative properties, in addition to computational complexity and asymptotic behavior. In particular, we show that the Bayesian estimator for the Condorcet model satisfies some desired properties such as anonymity, neutrality, and monotonicity, can be computed in polynomial time, and is asymptotically different from the other two rules when the data are generated from the Condorcet model for some ground truth parameter.

The author(s) consider social choice mechanisms from the point of view of statistical decision theory. They use this viewpoint to derive two new voting mechanisms based on the principle of Bayesian risk minimization. The rest of the paper studies the properties of the ranking produced by the new voting mechanisms and also an in-depth study of the computational complexity of realizing these mechanisms. The decision problems arising from finding good winning candidates for one of their schemes turns out to be NP-hard while the other is in P. The paper concludes with an asymptotic comparison of the decisions produced by their mechanisms and the Kemeny voting rule and an experimental evaluation of the asymptotic results.The results are original and the paper is well written (albeit dense with results). Minor comments:* Definition 4, the definition of the loss function L_{0-1}(\theta,c) is not provided this was a bit confusing since it is different from the function in Example 1. It seems that the definition should be L_{0-1}(\theta, c) = 0 if \theta \in \mathcal{L}_c(C) (resp. \mathcal{B}_c(C)) and 1 otherwise.* Lemma 2: I found it slightly confusing that the quantities outside the statement of the lemma (lines 249-251) were used implicitly in the lemma. It would be good to clean this up so that the statement is self-contained.* I'm not sure why this is bothering me so much, but it seems to me that the definition of P_||^NP-hard problem should use truth-table reductions rather than many-one reductions. In any case, feel free to ignore this comment since it does not affect the results of the paper.* Typo on Page 7, line 343: "where d is not ranked" ...->... "where b is not ranked".* Typo on Page 7, line 346: "BETTER DECISION" ...->... "OPTIMAL BAYESIAN DECISION".


Causal Strategic Inference in Networked Microfinance Economies
Performing interventions is a major challenge in economic policy-making. We propose causal strategic inference as a framework for conducting interventions and apply it to large, networked microfinance economies. The basic solution platform consists of modeling a microfinance market as a networked economy, learning the parameters of the model from the real-world microfinance data, and designing algorithms for various causal questions. For a special case of our model, we show that an equilibrium point always exists and that the equilibrium interest rates are unique. For the general case, we give a constructive proof of the existence of an equilibrium point. Our empirical study is based on the microfinance data from Bangladesh and Bolivia, which we use to first learn our models. We show that causal strategic inference can assist policy-makers by evaluating the outcomes of various types of interventions, such as removing a loss-making bank from the market, imposing an interest rate cap, and subsidizing banks.

SUMMARY:  This paper presents a model for analyzing policy decisions, particularly ones that require some causal knowledge, in microfinance markets.  The model is justified by several studies, and analyzed to show the existence of equilibrium prices.  A computational framework is then presented to compute an equilibrium and learn the parameters of the model from real micofinance data.  Finally, the authors leverage the framework to provide answers to several policy questions.CRITIQUE:  By all metrics, this is a good paper.  It is clear, well-motivated, well-supported by analysis and data, introduces novel techniques, and has a potential for impact.  I point out a few specific problems below, and I do wonder whether this is a good fit for NIPS, but defer to more senior members for the latter.SUGGESTION:  The assumption that MFIs are non-profits is crucial to the entire analysis, yet in my mind is a source for some of the more interesting questions.  At the very least, it would be interesting to note what changes when MFIs are profit seeking, either analytically or empirically.  Specifically, could the social welfare actually increase if MFIs were even slightly profit seeking, and therefore requiring fewer subsidies over time?  It seems that if you expanded your model slightly you might be able to answer this question.SPECIFIC COMMENTS:page 2:. "[13] and joint-liability contracts would mitigate the risks of adverse selection [13]" -- odd to cite [13] twice in the same sentence. "study of causality" -- this term is not really defined, and to many people may seem extremely vague (i.e. all of science); even though you very nicely explain everything in the appendix, it might go a long way to describe the scope of "causality" here. "Put differently, what would be a game-theoretic analog of the do operation [21, p. 23] used for surgeries in probabilistic settings?" -- This sentence is somewhat meaningless unless you know the reference; please clarify the contextpage 3:. "subset of the village"[s]. "interest rate at which MFI i gives loan" -- the phrase "gives loan" seemed odd to me. "max_{r_i} 1" -- this is jarring without an explicit reassurance that you indeed intended max 1. "each MFI i is optimizing (P_M)" -- add "the left-hand side of" (one cannot optimize an inequality)page 4:. "we model the village side as non-corporate agents." -- Where does this come up in the model?. Property 3.2. -- Interpret this for us in words. "trivial allocations" -- Again, remind us what x is by saying why this would be trivialpage 5:. "that a more restricted case" -- this sentence has a bug. "Change ri as described later." -- Cite a specific equation number so the reader isn't left looking for where you fill this in (it's not obvious)page 6:. "Our model computes lower equilibrium interest rate"


Learning Multiple Tasks in Parallel with a Shared Annotator
We introduce a new multi-task framework, in which K online learners are sharing a single annotator with limited bandwidth. On each round, each of the K learners receives an input, and makes a prediction about the label of that input. Then, a shared (stochastic) mechanism decides which of the K inputs will be annotated. The learner that receives the feedback (label) may update its prediction rule, and then we proceed to the next round. We develop an online algorithm for multitask binary classification that learns in this setting, and bound its performance in the worst-case setting. Additionally, we show that our algorithm can be used to solve two bandits problems: contextual bandits, and dueling bandits with context, both allow to decouple exploration and exploitation. Empirical study with OCR data, vowel prediction (VJ project) and document classification, shows that our algorithm outperforms other algorithms, one of which uses uniform allocation, and essentially achieves more (accuracy) for the same labour of the annotator.

This paper studies the problem of online multi-task learning in which a single annotator with limited bandwidth is shared among tasks. An online algorithm is proposed and its performance is analyzed. Major Strengths:Online multitask learning is an important topic, and the field of multi-armed bandits has a rich history.   Seeking to leverage ideas and results from the latter to apply to the former domain will likely yield improved techniques, especially since for the multi-task learning problem some labels might not be observed.  Furthermore, this area is highly relevant to NIPS.Major Weaknesses:An important aspect to multi-task learning is that there is some dependency between the tasks so that there is utility between learning the tasks jointly.  In the work under review, no dependency is assumed.  This appears to be a significant departure from the main multitask learning literature with no apparent justification.  Without the ability to transfer knowledge between the tasks, it appears more as the multi-armed bandit analog in multi-task learning.  For instance, why can only one feature be annotated at a time though all features are allowed to be annotated?    In using Mechanical Turk, as suggested, for some applications the workers might not annotate labels for all features equally well.  Thus, only allowing a subset of features to be annotated while the rest cannot seems more relevant of a setting.  The article mentions it is a new multi-task framework, so it need not follow traditional assumptions, though more motivation and justification is needed for why the differences are important.One counter-example to the previous statement is Romera-Paredes et al. “Exploiting Unrelated Tasks in Multi-Task Learning” in AISTATS 2012.  There they use the knowledge that two tasks are unrelated to avoid overfitting by using the same features.  However, the work under review does not exploit the lack of dependence.This is less important, but to check the implication (lines 153-154) of Theorem 1, for the data analysis it would have been interesting to see how SHAMPO performs compared to the case when labels are available for all tasks.Minor notes:[25] “both allows”[209] “outputs multicalss"[347] remove comma


Bayesian Inference for Structured Spike and Slab Priors
Sparse signal recovery addresses the problem of solving underdetermined linear inverse problems subject to a sparsity constraint. We propose a novel prior formulation, the structured spike and slab prior, which allows to incorporate a priori knowledge of the sparsity pattern by imposing a spatial Gaussian process on the spike and slab probabilities. Thus, prior information on the structure of the sparsity pattern can be encoded using generic covariance functions. Furthermore, we provide a Bayesian inference scheme for the proposed model based on the expectation propagation framework. Using numerical experiments on synthetic data, we demonstrate the benefits of the model.

This paper introduces a new spike and slab sparse prior, where the sparse variables are allowed to be correlated and this structure is encoded by generic covariance functions. Therefore, this is a new spike and slab prior for modeling structured sparsity patterns. The authors also propose an algorithm for inference of spike and slab variables under the proposed prior, by applying the framework of expectation propagation. Approximations to a covariance matrix inverse are also proposed, in order to reduce the computational complexity of the algorithm. Numerical results demonstrate that the method significantly outperforms prior art for problems of sparse recovery that exhibit such form of structured sparsity.This paper is an original contribution to the field of sparse recovery and proposes a solution to a significant problem of inference for structured sparsity, one that has been a challenge for many researchers in the last decade. Although the paper does not solve the problem for any general joint distribution of sparse variables, it proposes a solution for a large set of distributions. Therefore, I think it will be of interest for a larger audience and probably useful to many researchers in the field.The paper is clearly written, though notation can be improved (see below for detailed suggestions). Acronyms are not always defined, for example MMV.One improvement I would like to see in the revised paper (besides detailed comments below) is the addition of real data examples where the structure in the sparse variables can be modeled by the set of distributions considered in the paper. The example given in the current version, with EEG measurement matrix, is not convincing since the sparse variables are still sampled from a distribution defined by the authors. I would not call it a real example, since the only real data is the measurement matrix and its properties are not really the target of the paper. Having such examples would make the paper much stronger and the proposed method would have a larger impact.Another point that would be good to discuss in the paper is the attenuation of correlation of sparse variables with respect to the correlation of parameters gamma (Fig. 1c). What does the attenuation depend on? Is there a way to control that dependence? How does the curvature of curves in Fig.1c change with other parameters? I believe this will be an important point to consider in real applications. Detailed comments:- Sec. 2 title: typo - should be "slab" instead of "slap"- line 093: "easy to generate" instead of "easy the generate"- Sec.3. When replacing f1 with f1 tilde, it would be good to explain the switch from the conditional distribution of y given x to a conditional distribution of x given m1 and V1 (where m1 depends on y)- Sec.3. It is not clear where the function d(x,y) is used in 18-21- Sec.3.1. Notation tilde(f)_a,i should be explained, previously was only used tilde(f)_a and now another subscript was added. Maybe it would be better to put i in the superscript? Or just define it in the text- Using bold A and regular A for two different variables is confusing, please consider modifying one.- Sec. 3.2: What is N? Rank of V1?- line 306: "slab" instead of "slap"


Asynchronous Anytime Sequential Monte Carlo
We introduce a new sequential Monte Carlo algorithm we call the particle cascade. The particle cascade is an asynchronous, anytime alternative to traditional sequential Monte Carlo algorithms that is amenable to parallel and distributed implementations. It uses no barrier synchronizations which leads to improved particle throughput and memory efficiency. It is an anytime algorithm in the sense that it can be run forever to emit an unbounded number of particles while keeping within a fixed memory budget. We prove that the particle cascade provides an unbiased marginal likelihood estimator which can be straightforwardly plugged into existing pseudo-marginal methods.

In particle filtering, the resampling step is a synchronous operation: one needs all the particles before computing the normalised weights (since the denominator is the sum of all the weights), and then resample. The reviewed paper propose an asynchronous resampling mechanism, where the number of children of particle k depends only the weights of particles 1 to k. The proposed idea is quite straightforward, but still interesting and potentially very useful. What is a bit lacking in the current version is some motivation for an asynchronous implementation of particle filtering.  Having a different thread for each particles seems quite involved for many practical cases (recall that not every user of particle filtering works in CS), so providing a more substantial discussion (beyond citing two 2014 papers) would be tremendously helpful. Perhaps one step in this direction is to clarify a bit the discussion of Figure 3. Is it the case that "SMC" means a version of SMC where the set of particles are distributed over the k cores, and then "re-united" before each resampling step? Then the better performance of the particle casade would be a convincing argument. Another motivation would be to look at more realistic examples, in particular cases where the running time of simulating a single particle is very variable (such as in e.g. models in stochastic chemometrics, off the top of my head)Two more points: 1. in most cases, people are interested in on-line algorithms: i.e. datapoints are collected sequentially, and as soon as y_n is available, computations may resume to deliver the filtering distribution at time n, etc. Do I understand correctly that the proposed algorithm is not on-line any more (see 305-306)? (If yes, then this point should be mentioned; also perhaps it's easy to make it on-line again? 2. Why did you compare particle casade to CSMC??? Sorry, I really did not understand this. CSMC samples from the smoothing distribution (if you take the selected trajectory), and is not really meant to be a filtering algorithm??? Minor remarks: 27: first sentence is a bit unclear; it's oddly constructed.  262: posterior marginals: do you mean filtering marginals (conditional on data up to time n), or smoothing marginals (conditional up to final time N)? I guess the former, but your terminology seems to mean the latter. Also does the MSE correspond to the filterting expectation of the state in Figures 1 and 2? See also 396 "true posterior distribution". 428-430: sorry, I did not understand this part.


Diverse Randomized Agents Vote to Win Tuomas Sandholm CMU
We investigate the power of voting among diverse, randomized software agents. With teams of computer Go agents in mind, we develop a novel theoretical model of two-stage noisy voting that builds on recent work in machine learning. This model allows us to reason about a collection of agents with different biases (determined by the first-stage noise models), which, furthermore, apply randomized algorithms to evaluate alternatives and produce votes (captured by the secondstage noise models). We analytically demonstrate that a uniform team, consisting of multiple instances of any single agent, must make a significant number of mistakes, whereas a diverse team converges to perfection as the number of agents grows. Our experiments, which pit teams of computer Go agents against strong agents, provide evidence for the effectiveness of voting when agents are diverse.

The paper provides a framework for analyzing the quality of a vote among a team of agents (compared to some underlying true ranking) where the teams are generated differently: by multiple instantiations of the same agent ("Uniform"), and by a single instantiation of several different types of agents ("Diverse"). Under some conditions, the authors show that as the number of agents grow the Diverse team converges in probability almost surely to choosing the correct choice, where the Uniform team always has a non-zero probability of making a mistake. The paper is quite clear and the technical quality is high. The results are significant and relevant. The main idea here is that voting among a diverse team leads to robust choices, whereas a population of fixed players does not have enough variance and will play "too predictably", but the authors provide a nice theoretical foundation to quantify these notions and show that even diversity obtained from a single (widely-used) Go program with different parameters can lead to significant performance boosts in 9x9 Go (against the same copies of the optimally-tuned parameter version of the Go program) when using an appropriate voting scheme. The authors also show how their assumptions and voting rules relate to other rules used in the literature. I have a some problems with the paper. The first is that there seems to be a disconnect between the experiments and the foundations that came before it. The experimental section is specific, focusing on the single application to Go, while the theoretical framework is quite general. I was expecting to see experiments comparing different noise models, something that validates the assumptions listed in section 3, or some stronger link between the first half of the paper and second. As far as I can tell there is no mention of whether the Go application even satisfies the assumptions 1-4, so does Theorem 2 even hold in this experiment? This is also true for the discussion section, which seems to again focus on the application to games while the first part is more general.Also, as this is follow-up work based heavily on past work ([18,19]) the authors should do a better job of highlighting what is new in this paper. The noise model approach certainly seems appropriate for the computer Go setting where there indeed is some underlying true ranking of alternatives, but this might not be true in other more general social choice settings such as elections.. can anything be said on whether this approach could be applied when the existence of a true ranking is unknown?


Cone-constrained Principal Component Analysis
Estimating a vector from noisy quadratic observations is a task that arises naturally in many contexts, from dimensionality reduction, to synchronization and phase retrieval problems. It is often the case that additional information is available about the unknown vector (for instance, sparsity, sign or magnitude of its entries). Many authors propose non-convex quadratic optimization problems that aim at exploiting optimally this information. However, solving these problems is typically NP-hard. We consider a simple model for noisy quadratic observation of an unknown vector v 0 . The unknown vector is constrained to belong to a cone C v 0 . While optimal estimation appears to be intractable for the general problems in this class, we provide evidence that it is tractable when C is a convex cone with an efficient projection. This is surprising, since the corresponding optimization problem is non-convex and -from a worst case perspective-often NP hard. We characterize the resulting minimax risk in terms of the statistical dimension of the cone δ(C). This quantity is already known to control the risk of estimation from gaussian observations and random linear measurements. It is rather surprising that the same quantity plays a role in the estimation risk from quadratic measurements.

The authors consider the model of noisy quadratic measurements (rank1 matrix plus noise), where we want to estimate the spike that generates the rank-1 matrix. They refer to this model as a quadratic observation model.Similar estimation problems that come with non-convex constraints (i.e., sparse PCA) have been analyzed where there is a gap between minimax risk and tractable estimators. Here the authors  consider convex cone constraints, (which don't render the problem any easier at first sight). Although the ML-optimization problem is computationally intractable, in the general case, they can show that a projected version of the power method is almost optimal in terms of minimax risk estimation, and approximates the ML performance within a constant gap. In the case of the nonnegative cone, they show that their analysis can be tightened to an almost zero gap between the ML and the projected power method.In summary, their contribution is three fold: - they present the information theoretic bounds for the estimation problem- they present the risk of the ML estimator using the gaussian dimension of the involved cone- they use a projected power method algorithm to essentially achieve ML performance.I generally really liked the paper; it’s an original and significant contribution to the relevant literature. I think that the use of the gaussian dimension of the cone for quadratic measurement problems seems new and very promising. The authors bound the minimax rate using that precisely. I wonder if there is any relevance to the recent phase recovery problem.Similarity with concurrent works of [1] and [2]:Some of the results look very similar to some recent works [1] and [2], where information theoretic bounds and a similar approximate message passing/ projected power method was used. It seems that section 5 establishes the same result as the work of [1]. However, the submission date of [1] is later than the present manuscript was submitted, and the works can be considered concurrent.The presented work here however is much more general and very significant to the relevant literature; the above comment is only made due to the relevance between the results.Some typos:typo in line 137, reference is missingequation (3), subjectto -> subject to[1] “Non-negative Principal Component Analysis: Message Passing Algorithms and Sharp Asymptotics”Andrea Montanari, Emile Richardhttp://arxiv.org/pdf/1406.4775v1.pdf[2] “Information-theoretically Optimal Sparse PCA”Yash Deshpande, Andrea Montanarihttp://arxiv.org/pdf/1402.2238v2.pdf


Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models
Gaussian processes (GPs) are a powerful tool for probabilistic inference over functions. They have been applied to both regression and non-linear dimensionality reduction, and offer desirable properties such as uncertainty estimates, robustness to over-fitting, and principled ways for tuning hyper-parameters. However the scalability of these models to big datasets remains an active topic of research. We introduce a novel re-parametrisation of variational inference for sparse GP regression and latent variable models that allows for an efficient distributed algorithm. This is done by exploiting the decoupling of the data given the inducing points to re-formulate the evidence lower bound in a Map-Reduce setting. We show that the inference scales well with data and computational resources, while preserving a balanced distribution of the load among the nodes. We further demonstrate the utility in scaling Gaussian processes to big data. We show that GP performance improves with increasing amounts of data in regression (on flight data with 2 million records) and latent variable modelling (on MNIST). The results show that GPs perform better than many common models often used for big data.

The authors have come up with a novel way to distribute the optimisation of variational Bayesian sparse GPs (for regression and LVMs). By reformulating Titsias' (2009) variational lower bound, such that the training data become independent given the inducing points, training can be parallelised across nodes on a cluster or network, via a Map-Reduce implementation.There is of course a bit of overhead, because of the optimisation of global parameters, which is negligible considering the overall speed-up and scaling, and the authors demonstrate that with a simple experiment.It is nice that the reformulation of the variational lower bound unifies both cases (regression becomes a special case of the LVM when the inputs are fixed).The paper convincingly demonstrates the benefits of the scheme in terms of balanced computational loads across nodes, and scaling up to millions of training datapoints (a number which they claim have never been attempted in the GP literature).I do not necessarily see a big improvement in terms of prediction accuracy, but this is because it is not clear what the units of the RMSE are. I suspect they are 'minutes of flight delay', in which case I can appreciate that a marginal improvement of a few minutes does matter to airlines.Lastly, the authors focus of scalable GPLVMs, by showing significant improvement in the classification accuracy of MNIST data (3%).Quality and clarity are good. I believe the work is significant to the GP and data mining community.Other comments:line 78: 'the optimal form'. Be more precise.eq.(3.1): Missing an integral symbol.Tables 1,2: How many inducing points are you using in Dist GP? Also, see my comment on units of the RMSE. (If it is 'min of delay' then Disp GP predicts more accurately by 2-3 minutes.)Figure 5: Add a more detailed caption. What are the left and right panels showing?line 395: By 'smaller model' I take it you also mean one using a smaller dataset.line 393: One possible cause for the unexpected result: SCG optimises the GP marginal likelihood (or var. bound) which is shaped by amount of data. The more the data, the sharper the modes are. Using less data, amounts to an annealing-like behaviour of the likelihood surface, thereby giving you better chances of approaching a good mode.


On Communication Cost of Distributed Statistical Estimation and Dimensionality
We explore the connection between dimensionality and communication cost in distributed learning problems. Specifically we study the problem of estimating the mean✓ of an unknown d dimensional gaussian distribution in the distributed setting. In this problem, the samples from the unknown distribution are distributed among m different machines. The goal is to estimate the mean✓ at the optimal minimax rate while communicating as few bits as possible. We show that in this setting, the communication cost scales linearly in the number of dimensions i.e. one needs to deal with different dimensions individually. Applying this result to previous lower bounds for one dimension in the interactive setting [1] and to our improved bounds for the simultaneous setting, we prove new lower bounds of ⌦(md/ log(m)) and ⌦(md) for the bits of communication needed to achieve the minimax squared loss, in the interactive and simultaneous settings respectively. To complement, we also demonstrate an interactive protocol achieving the minimax squared loss with O(md) bits of communication, which improves upon the simple simultaneous protocol by a logarithmic factor. Given the strong lower bounds in the general setting, we initiate the study of the distributed parameter estimation problems with structured parameters. Specifically, when the parameter is promised to be s-sparse, we show a simple thresholding based protocol that achieves the same squared loss while saving a d/s factor of communication. We conjecture that the tradeoff between communication and squared loss demonstrated by this protocol is essentially optimal up to logarithmic factor.

This paper explores the connection between dimensionality and communication cost in distributed learning problems. In this problem, the samples from the unknown distribution are distributed among m different machines. The goal is to estimate the mean  at the optimal minimax rate while communicating as few bits as possible. This paper shows that in this setting, the communication cost scales linearly in the number of dimensions. Applying this result to previous lower bounds for one dimension in the interactive setting [4] and to the improved bounds for the simultaneous setting, this paper proves new lower bounds of \Omega(md/ log(m)) and \Omega(md) for the bits of communication needed to achieve the minimax squared loss, in the interactive and simultaneous settings respectively. Besides, this paper demonstrates an interactive protocol achieving the minimax squared loss with O(md) bits of communication. Given the strong lower bounds in the general setting, this paper initiates the study of the distributed parameter estimation problems with structured parameters. Specifically, when the parameter is known to be s-sparse, it  shows a protocol achieving the minimax squared loss with high probability and with communication cost proportional to s rather than the dimension d of the ambient space.Quality: The authors need to assume that the readers have no knowledge on the problem in [4].Clarity:  In Section 2, definitions are missing for several notions:0. What is the definition of s-sparse?1. In the definition of R(\hat{\theta},\theta), why the expectaion is over \hat{\theta},X,\theta? How is  the mean \vec{\theta}(Y) of Y obtained? Why is Y in {\cal X}^n? Is there any example for transcript?2. I cannot follow below Private/Public Randomness. For example,a. What is private and public randomness?  What is a protocol? Give definitions to them. b. Why can the public randomness be shared among the machines before the start of the protocol?c. Why does the protocol work well on average over all public randomness?d. Why can the machine use private randomness to hide information from other machines in a protocol?e. The definition ``(\Pi,\vec{\theta}) solves T(d,m,n,\sigma^2,{\cal D}_\theta^d) with C and R.Oiginality: I believe that new lower and upper bounds are derived for the interactive and simultaneous settings, and that the negative result is obtained. It is fair that the authors explain how difficult the derivation fom [4] is.Significance: I do not understand why ''the communication cost scales linearly in the number of dimensions" is a news. In fact, ituitively, this seems to be obvious while they mathematically derived the property. The authors need to explain why the result is different from what they expected.


Augmentative Message Passing for Traveling Salesman Problem and Graph Partitioning
The cutting plane method is an augmentative constrained optimization procedure that is often used with continuous-domain optimization techniques such as linear and convex programs. We investigate the viability of a similar idea within message passing -for integral solutions in the context of two combinatorial problems: 1) For Traveling Salesman Problem (TSP), we propose a factor-graph based on HeldKarp formulation, with an exponential number of constraint factors, each of which has an exponential but sparse tabular form. 2) For graph-partitioning (a.k.a. community mining) using modularity optimization, we introduce a binary variable model with a large number of constraints that enforce formation of cliques. In both cases we are able to derive simple message updates that lead to competitive solutions on benchmark instances. In particular for TSP we are able to find nearoptimal solutions in the time that empirically grows with N 3 , demonstrating that augmentation is practical and efficient.

The authors studied how to solve NP-hard TSP and graph partition problem as an inference on a graphical model. They first showed how to design a graphical model for each problem. The corresponding graphs are cyclic and the inference problems are NP hard. Because of the success of the belief propagation (BP) method, which provides a good approximate inference with cheap computational cost in error correction codes and related problems, the authors applied min-sum BP for TSP and graph partition problem.Most part of the paper is clearly written. The approach is interesting, but this is an empirical study. My recommendation is positive but not strong.The paper has two main claims. One is how to design the graphical models and apply BP algorithm. Naïve BP may not work well, and the authors combined some tricks. The other point is to show the numerical results of min-sum. The authors claim the results are comparable to state-of-art methods but the computations cost is small.First point: The graphical model representation of TSP is straightforward. That for the graph partition is realized via a stochastic approach, which is not so difficult. The important part is how to apply BP successfully. The graphs have a lot of cycles and naïve min-sum BP may not work well. For each problem, they combined some approaches: the messages are normalized, and the subtour/clique constraints are treated carefully. They also combined and decimation for TSP and damping for both. These are heuristic and we would like to know if they really avoided local maxima and how often the algorithm converges to the optimal points.Second point: For graphs with many cycles, min-sum BP generally does not provide the optimal solution. We confirm this fact from numerical results of the paper. Although the authors claim the cheap computational cost and the good quality of the solutions, it is difficult to convince non-specialists to use this approach.Overall comment: The authors have shown numerical results for known data. What people want to know is what will happen for new unknown data. Some theoretical studies of the characteristics should be shown. For what kind of data, this method is better than other methods, and how the computational cost grows according to the size of the problem (the discussion in section 4 is not sufficient). Finally, I have one question. The graph partitioning method uses a randomized method. How the results are influenced with it?


Robust Kernel Density Estimation by Scaling and Projection in Hilbert Space
While robust parameter estimation has been well studied in parametric density estimation, there has been little investigation into robust density estimation in the nonparametric setting. We present a robust version of the popular kernel density estimator (KDE). As with other estimators, a robust version of the KDE is useful since sample contamination is a common issue with datasets. What "robustness" means for a nonparametric density estimate is not straightforward and is a topic we explore in this paper. To construct a robust KDE we scale the traditional KDE and project it to its nearest weighted KDE in the L 2 norm. This yields a scaled and projected KDE (SPKDE). Because the squared L 2 norm penalizes point-wise errors superlinearly this causes the weighted KDE to allocate more weight to high density regions. We demonstrate the robustness of the SPKDE with numerical experiments and a consistency result which shows that asymptotically the SPKDE recovers the uncontaminated density under sufficient conditions on the contamination.

Summary:This paper considers the problem of robust kernel density estimation, where the authors assume that most observations come from a target density f_tar, but some observations are drawn from a contaminating density f_con.  They consider the setting that the observation density f_obs is a known convex combination of f_tar and f_con, but we do not know f_tar or f_con. In practice, the convex combination is not known, but can be treated as a tuning parameter, to adjust between more and less robust estimates. They also have "Assumption A", which states that the contamination density f_con is uniform at r on the support of the target density f_tar, but is less than r otherwise.  Under this assumption, they present a decontamination operator that maps a contaminated density to the target density.  The decontamination operator rescales the observation density f_obs by a certain amount, then shifts it down, pointwise, to make the resulting function integrate to 1.  This addresses the infinite sample, or population version of the problem.  To apply this a finite sample, we could first approximate the observation density f_obs by a classic kernel density estimator, and then apply the decontamination operator.  However, this does not work in practice, because computing the amount of the shift is not computationally tractable.  Instead, they propose taking the L2 projection of the rescaled function onto the convex hull of kernel functions evaluated at each of the sample points.  Finding this projection amounts to a quadratic program involving the Gram matrix of L2 inner products of the kernel functions centered at each of the sample points, which can be computed efficiently for many kernels of interest.  They eventually prove that the L1 distance between the "SPKDE" estimator and the target density converges to 0 in probability. The authors say that the convergence of their estimator to f_tar, under sufficient conditions, is the first result of its type. In their experiments, they first compare the performance of SPKDE to traditional KDE on artificially generated data from distributions satisfying Assumption 1.  SPKDE clearly outperforms KDE. In the experiments on real data, the estimated density based on contaminated training data is compared to an estimated density based on uncontaminated [I assume uncontaminated -- this wasn't entirely explicit] test data.  The comparison was done using KL-divergence in both directions. The Wilcoxon signed rank test is used to see if the performance differences are significant across the N datasets.General Comments:The paper was quite interesting and readable.  I just have a few suggestions.  First, density estimation on R is so easy to visualize, I feel like you could do a better job of illustrating the differences between SPKDE, KDE, rejKDE, and RKDE.  You could pick synthetic datasets that play to the strengths of each.  I think it would be more impactful and interesting.On the real datasets, using KL divergence in both directions seems reasonable, but is a bit heavy, since you're measuring divergences with respect to another KDE estimate on the test data. [I've assumed that this test data is uncontaminated, but this could be explained more clearly.]  What about using test likelihood as another performance measure?  This seems very standard for measuring the quality of a distribution's fit to data, and there are no issues of symmetry or having to first "fit" the test data.With regard to RKDE performing better in one direction of the KL divergence, and the SPKDE performing better in the other direction: can this difference be visualized?  Under what assumptions does RKDE perform best, and how do RKDE and SPKDE compare in that situation, and vice versa?Specific Comments:- Seems like Proposition 2 is essentially proved by line 136 on page 3.  Lemma 1 seems unnecessary to prove Proposition 2.  Maybe state Proposition 2 before Lemma 1. And right after say that the transform used "is well-defined more generally" and then give Lemma 1?  Or even postpone Lemma 1 until you actually need it in Section 3?  As it is currently presented, it's seems like Proposition 2 depends on Lemma 1 (which has a long proof), when really it's quite simple (unless I've misunderstood).- The acronym SPKDE isn't formally introduced before it's used in the abstract or the introduction.- Justification for uniformity assumption on f_con: It's unclear what aspects of Assumption A are being supported by the second paragraph on page 3 (starting line 113).  Assumption A has two parts: 1) f_con is uniform on the support of f_tar, and 2) The values of f_con outside of supp(f_tar) is bounded above by its value on supp(f_tar).  It seems like both of these are important to the method.- Line 111 on page 3, I think you mean that the union is exactly all densities whose _support_ has finite Lebesgue measure?- The differences between what's happening in Figure 2 and Figure 3 are pretty subtle.  Might be there a more clear illustration of the differences?- It's rather difficult to find any of these contamination models particularly "natural" or "reasonable".  However, on line 182 of page 4, you characterize the assumption in terms of the type of target densities that can be recovered.  At that point, it might be worth reminding the reader of the comment on line 111, that any target density with finite support is recoverable under Assumption A your method.


Local Decorrelation for Improved Pedestrian Detection
Even with the advent of more sophisticated, data-hungry methods, boosted decision trees remain extraordinarily successful for fast rigid object detection, achieving top accuracy on numerous datasets. While effective, most boosted detectors use decision trees with orthogonal (single feature) splits, and the topology of the resulting decision boundary may not be well matched to the natural topology of the data. Given highly correlated data, decision trees with oblique (multiple feature) splits can be effective. Use of oblique splits, however, comes at considerable computational expense. Inspired by recent work on discriminative decorrelation of HOG features, we instead propose an efficient feature transform that removes correlations in local neighborhoods. The result is an overcomplete but locally decorrelated representation ideally suited for use with orthogonal decision trees. In fact, orthogonal trees with our locally decorrelated features outperform oblique trees trained over the original features at a fraction of the computational cost. The overall improvement in accuracy is dramatic: on the Caltech Pedestrian Dataset, we reduce false positives nearly tenfold over the previous state-of-the-art.

The paper introduces an efficient feature transform of local decorrelation, which when combined with boosted (orthogonal) decision trees, considerably improves over the state-of-the-art on pedestrian detection. Overall, it is a clearly (and nicely) written paper with good analysis, enough details and solid experiments. Pros:- Very well written and executed paper- Attention to detail- Solid results- Straight forward and intuitive methodCons:- Incremental from Hariharan et al. (not major, see later)- If it claims ``Improved Detection'', as opposed to ``Improved Pedestrian Detection'', then I would have liked to see some more results on object detection or likewise.- Some minor analysis and experiments I would live to see (see later)- Some parts can be better explained (see later)- Relevance to NIPS (see later)Some comments and concerns:Incremental nature -- Though it might seem that this paper is an incremental work, I would say that it has enough novelty to offer over [15]. Going from global to local decorrelation, and doing the right analysis for design decisions set it apart. Though the analysis are in line with [6], I don't see that as a negative; I rather welcome the analysis.Detection vs. Pedestrian Detection -- I would suggest the title to be changed to reflect that this work is currently targeted towards Pedestrian detection. Otherwise, I would like to see some experiments (may be similar to [15], or some work which uses Channel features) to show the effectiveness on standard detection tasks. Just to be clear: I am not asking for more state-of-the-art results.Experiments/analysis --1. L289-92: How does downsampling the final set of feature impact the overall performance? Does it improve quantitative performance, or just improves efficiency at the cost some quantitative performance?2. Table 2 (6. LDCF constant): This is a very interesting result for me. Using k DCT basis filters itself takes them from 17.3% to 14.2% MR. Can the authors perform more analysis on using these DCT basis? I'm thinking on the lines of Fig. 4 for DCT basis. Does varying the DCT basis or the size of the filters improves the performance further? What's the impact of using more/less tree depth and sampling interval? I think this analysis will be interesting, and will give a new perspective to using these DCT basis. However, I don't think higher performance from these constant filters undermines the impact/novelty of this paper.3. L375-77 (Fig. 4c): I see that the performance is still improving with decreasing tree depth and sampling interval. Did the authors try to push it more? Does the performance plateau? It would be nice to see that. If it wasn't feasible to push further, it can be mentioned.4. "Replacing oblique splits": I'm not in favor of stating that the current method can act as a replacement of oblique splits. To state that, I would like to see both are ultimately doing the same thing (may be by analyzing the hits/misses at test time etc.) But this is a personal preference.Clarity --1. Section 4 (Complexity): How are #patches=d? Use of \Sigma and \bar\Sigma etc. I think this part can be made clearer.2. Section 5 (observations): I think it might be a good idea to ref. each observation to a section/sub-section to make the obvious connection.3. (minor) L269: How many levels per octave in feature pyramid? Given that authors are releasing code, this might not be necessary.4. Section 5 (Estimating \Sigma): Which large collection of images did the authors use? #images and #patches?5. Caltech Results (L409-10): ACF-Caltech is the standard ACF paper trained on Caltech? ACF-Caltech+ is just using more training data in a more complex model? By the ``ability to effectively utilize ..", do the authors mean computational ability or something else? Some clarity in this part will help!6. Fig. 5: Having citations with each result in legends might be a good idea. If authors are not citing all paper due to space limitation, partial citation list will also be helpful.Relevance -- I think this paper might be slightly better suited for a vision venue as opposed to NIPS. But I definitely see merit and novelty in the paper, and its potential impact. I would argue for the paper!


Communication Efficient Distributed Machine Learning with the Parameter Server
This paper describes a third-generation parameter server framework for distributed machine learning. This framework offers two relaxations to balance system performance and algorithm efficiency. We propose a new algorithm that takes advantage of this framework to solve non-convex non-smooth problems with convergence guarantees. We present an in-depth analysis of two large scale machine learning problems ranging from 1 -regularized logistic regression on CPUs to reconstruction ICA on GPUs, using 636TB of real data with hundreds of billions of samples and dimensions. We demonstrate using these examples that the parameter server framework is an effective and straightforward way to scale machine learning to larger problems and systems than have been previously achieved.

This paper presents improvements on a system for large-scale learning known as "parameter server". The parameter server is designed to perform reliable distributed machine learning in large-scale industrial systems (1000's of nodes). The architecture is based on a bipartite graph composed by "servers" and "workers". Workers compute gradients based on subsets of the training instances, while servers aggregate the workers' gradients, update the shared parameter vector and redistribute it to the workers for the next iteration. The architecture is based on asynchronous communication and allows trading-off convergence speed and accuracy through a flexible consistency model. The optimization problem is solved with a modified proximal gradient method, in which only blocks of coordinates are updated at a time. Results are shown in an ad-click prediction dataset with O(10^11) instances as well as features. Results are presented both in terms of convergence time of the algorithm and average time spent per worker. Both are roughly half of the values for the previous version of the parameter server (version called "B" in the paper). Roughly 1h convergence time using 1000 machines each with 16 cores and 192Gb RAM, 10Gb Ethernet connection (800 workers and 200 servers). Other jobs were concurrently run in the cluster. The authors claim it was not possible to compare against other algorithms since at the scale they are operating there is no other open-source solution. In the supplementary material they do compare their system with shotgun [7] and obtain faster convergence (4x) and similar value of the objective function at convergence. Since this is really a report of a working system, I have basically no technical comments to make. The numbers are impressive, the system works and there is no other open source system to compare against at this scale, so this is clearly a significant achievement. A nice materialization of a number of simple but important insights from the design of robust distributed systems into a machine learning pipeline (eg ideas like bounded delay, asynchronous push-pull, caching keys, etc.)Novelty claimed with respect to "existing approaches":- Solving large-scale problems (1000's nodes)- Solving non-convex and non-smooth composite objective functionsDoesn't [10] addresses these aspects (apart from non-smoothness, not sure?) If you refer only to open source versions, then make this clear.The aspect of the paper that I didn't like was the presentation.The authors compare the results of their so-called "third-generation parameter server" to two other systems called "system A" and "system B" which are not cited and are presumably the previous versions of their system. One aspect of the paper that makes assessment harder is that the contributions of the paper are not very clear in a case in which this clarification is quite important. This is precisely because there have been previous versions of the system that have presumably not been published (otherwise systems A and B would have been cited, at least anonymously) and the current submission conflates novelties from the latest version with aspects that originates in the earlier systems. In my view it would have been better if either the paper was written as introducing the parameter server from scratch, and illustrating its evolution up to this point (if the goal were to sell the entire package) or instead that the *precise* differences between the third generation and the previous ones were made really clear in the paper (if the goal were to sell the 3rd generation, and not the entire package). From the presentation I believe the goal was the latter. The table in page 7 helps, but I was expecting a much more detailed discrimination.Also, it's unfortunate that all the comparisons against other approaches (not the parameter server) are relegated to the supplementary material. I think this is a major weakness of the presentation.My suggestion is that the authors improve the presentation of the paper. I suggest they tell the story as introducing the parameter server and describing 3 stages of its evolution. Also the paper would read better if the English was polished (although to me it's acceptable as it is since it doesn't compromise understanding).


Distance-Based Network Recovery under Feature Correlation
We present an inference method for Gaussian graphical models when only pairwise distances of n objects are observed. Formally, this is a problem of estimating an n × n covariance matrix from the Mahalanobis distances d MH (x i , x j ), where object x i lives in a latent feature space. We solve the problem in fully Bayesian fashion by integrating over the Matrix-Normal likelihood and a MatrixGamma prior; the resulting Matrix-T posterior enables network recovery even under strongly correlated features. Hereby, we generalize TiWnet [19], which assumes Euclidean distances with strict feature independence. In spite of the greatly increased flexibility, our model neither loses statistical power nor entails more computational cost. We argue that the extension is highly relevant as it yields significantly better results in both synthetic and real-world experiments, which is successfully demonstrated for a network of biological pathways in cancer patients.

Summary: This work proposes a new method for learning the structure ofa Gaussian graphical model.  It seeks to account for hidden featurecorrelations when determining the structure by modeling the featurecovariance matrix using a Matrix-Gamma prior.  In contrast, prior worksuch as TiWnet [19] assumes feature independence, which can lead to anoverly-dense graph.  Synthetic experiments, as well as experiments inwhich networks of genetic pathways are learned, clearly demonstratethe superiority of the proposed method over TiWnet [19].Clarity, quality, originality, significance: The exposition is fairlyclear.  There seem to be many similarities to the TiWnet work, so,while the background is a little sparse, most readers' questions canbe answered by referring to [19].  Figure 2 also provides a very nice,clean comparison to related work, and section 2.1.1's explanation ofwhy Sigma can't simply be removed by setting the derivative equal tozero is helpful.  While it's not a large theoretical delta over theTiWnet work, the experiments indicate the practical edge the use ofthe Matrix-Gamma prior gives, which testifies to the work'ssignificance.More detailed comments:It seems that there are many similarities between this work and theTiWnet work that aren't exactly explicit in the paper.  For instance,Algorithm 1 is nearly identical to the algorithm for TiWnet, exceptfor the beta sampling step.  It would be helpful to the reader ifthese sort of similarities and differences were more clearly marked.Please explain in more detail exactly what is done experimentally to"obtain the final W via annealing".  How is the temperature d adjustedto converge on a final W?In the experiments, how is lambda for the prior p_2 chosen?  Sincethis prior is meant to induce sparsity, why not just turn up lambdafor TiWnet?  This seems like it might at least partially solve themain problem observed in the experiments, where TiWnet learns far toodense of a graph.It would be interesting to see some discussion of future work and thedeficiencies of the proposed method.  Does TiMT have any obviousweaknesses like TiWnet has?  What sort of connections does it fail topredict and why?  Is the fact that the W matrix is only allowed threelevels (+1, -1, 0) something that could be easily improved upon?  Itseems like most of the analysis of Algorithm 1's efficiency dependsupon the fact that the update to W can be expressed in terms of asmall number of rank-1 matrices.  Perhaps this could be furtherleveraged to increase the number of levels in W without significantlyincreasing the computational cost.Thanks very much for including code with this submission.  One smallcomment --- while the demo_script is easy to parse, it would behelpful if the timt script had some comments in the code.Minor notes:0) "Gaussian Graphical Models" -> "Gaussian graphical models"1) "our approach is capable to infer" -> "our approach is capable ofinferring"2) "can be removed to receive" -> "can be removed to recover"3) "it is possibly to measure" -> "it is possible to measure"4) "levels of hundred thousands of genes" -> "levels of hundreds ofthousands of genes"5) "’a complete success’" -> "'a complete success'"6) "’a spectacular failure’" -> "'a spectacular failure'"7) Copy "Black/red edges refers to +/− edge weight." from the captionof Figure 5 up to the caption of Figure 4.8) "Bhattacharyya distance[11] are computed as" -> "Bhattacharyyadistances [11] are computed as"9) "and to which degree" -> "and to what degree"10) "cannot be verified, however, there" -> "cannot be verified;however, there"11) "information loss about underlying features" -> "information lostabout underlying features"12) Supplement: "we are interested to find" -> "we are interested infinding"


Mode Estimation for High Dimensional Discrete Tree Graphical Models
This paper studies the following problem: given samples from a high dimensional discrete distribution, we want to estimate the leading (δ, ρ)-modes of the underlying distributions. A point is defined to be a (δ, ρ)-mode if it is a local optimum of the density within a δ-neighborhood under metric ρ. As we increase the "scale" parameter δ, the neighborhood size increases and the total number of modes monotonically decreases. The sequence of the (δ, ρ)-modes reveal intrinsic topographical information of the underlying distributions. Though the mode finding problem is generally intractable in high dimensions, this paper unveils that, if the distribution can be approximated well by a tree graphical model, mode characterization is significantly easier. An efficient algorithm with provable theoretical guarantees is proposed and is applied to applications like data analysis and multiple predictions.

Description:The paper presents an efficient algorithm to compute modes of theprobability distribution over many discrete variables defined by atree-structured graphical model. More precisely, given a (eg Hamming)metric and a constant delta, a labeling is a delta-mode if it hasgreater probability than all other labelings within distancedelta. The proposed algorithm computes M best modes for each scaledelta. The complexity of the algorithm is exponential in theory butreasonable empirically.The algorithm uses a newly derived relation between local and globalmodes. A local delta-mode is a labeling on a subgraph whch has higherprobability than all other labelings on the subgraph within distancedelta and with the same labeling on the boundary. Given a rich enoughcollection of subgraphs of the original graph, a labeling is a globaldelta-mode iff its restrictions to subgraphs are localdelta-modes. This implies that local delta-modes that are consistent(agree on overlapping variables) define a global mode.For a tree-structured model, the collection of subgraphs has apolynomial size (they are delta-balls). The algorithm first finds alllocal delta-modes for each subgraph. This is done by enumerating allboundary labelings and then finding the maximum probability for eachsuch labeling (there is a non-trivial insight in this algorithm,though). Then, the algorithm constructs a junction tree graphicalmodel whose nodes are the subgraphs, labels are the local modes, andwhose probability is the probability of the global mode if the localmodes are consistent and zero otherwise. Now, M best labelings(computed by the existing algorithm [16]) in this junction tree modelcorresponds to M best delta-modes in the original tree. Thus, theproblem of finding M best modes has been reduced to the known problemof finding M best labelings.When increasing delta, a mode can only disappear but neverappear. Modes for various delta thus form somethng like a scale space.When we want to compute the modes for all scales delta, the modes forsome delta can be computed efficiently from already computed modes fordelta-1. Thus, the above costly enumeration must be done only fordelta=1.Besides the algorithm, the paper presents theoretical guarantees thatdelta-modes are stable under errors in estimating the tree fromsamples, using the method from [1].The algorithm is verified in several experiments. One uses syntheticdata, the other real biological data, and the third (in thesupplement) image segmentation from superpixels by CRF. Theexperiments suggest that the number of local as well as global modesis managable in practice. The runtime is tens of seconds for treeswith 2000 nodes. On image segmentation, the method almost always yields better results than the M-best diverse labeling method by [Batra et al, ECCV'12].Comments:This is very good paper with a lot of work behind it. It is a goodstep towards practical algorithms for finding multiple modes of agraphical model. I have no major objections. Minor objections are asfollows.In Figure 3a, the uncertainty bars are wrong because runtime cannot benegative.To my understanding, for a fixed delta the algorithm finds not alldelta-modes but only M best delta-modes. But you never state what M wasin your experiments.The text contains a number of very minor language mistakes, such as"it is a local optima" (l.17), "inversely logarithm proportional"(l.107).On l.247: Why don't you write y_c\in L istead of y_c=\ell, \ell\in L.


Large-Margin Convex Polytope Machine
We present the Convex Polytope Machine (CPM), a novel non-linear learning algorithm for large-scale binary classification tasks. The CPM finds a large margin convex polytope separator which encloses one class. We develop a stochastic gradient descent based algorithm that is amenable to massive datasets, and augment it with a heuristic procedure to avoid sub-optimal local minima. Our experimental evaluations of the CPM on large-scale datasets from distinct domains (MNIST handwritten digit recognition, text topic, and web security) demonstrate that the CPM trains models faster, sometimes several orders of magnitude, than state-ofthe-art similar approaches and kernel-SVM methods while achieving comparable or better classification performance. Our empirical results suggest that, unlike prior similar approaches, we do not need to control the number of sub-classifiers (sides of the polytope) to avoid overfitting.

This paper presents a large-margin learning algorithm for convex polytope machines for binary classification tasks. The optimization objective is obtained from relaxing the problem of maximizing the total margin. A preliminary generalization bound regarding the objective function is presented. The paper proposes to use SGD to solve the optimization problem Eq. (6).  The experiments, which are conducted on standard benchmarks, demonstrate convincing performance over baselines including RBF SVM. It seems that the proposed formulation of optimization objective could be further clarified or improved. The paper claims to optimize the total margin (line 170-176). Indeed, the first objectives (Eq. (1) and Eq. (2)) directly maximize the total margin. These objective functions are non-convex, as pointed out in the paper. Therefore, the authors propose Eq. (3-5) and Eq. (6) as the objective function to be optimized, suggesting that they are the relaxation of Eq. (1) and Eq. (2).  However, the quantitative connection between Eq. (3-6) and Eq. (1-2) is unclear to me and is unfortunately not clarified in the paper.  It would be better if the paper develops or explains in more quantitative way how minimizing Eq. (3-5) will maximize the total margin. Furthermore, even after relaxation, the objective Eq. (6) is still non-convex. Thus I doubt whether the proposed relaxation has served its purpose well. In sum, the proposed objective function Eq. (6) seem to be a choice that is not well-justified, since (1) it is unclear how well it maximizes the total margin; (2) the relaxed objective is still non-convex.The main theoretical result (Theorem 1) in the paper is proven using standard arguments of proving generalization bounds. The proof does not seem to contain much new techniques or insights; it would be better if the paper points out the novelties of its proof, if any. The proposed SGD algorithm is heuristic and lacks theoretical underpinnings. The paper suggests that pure SGD does not work (due to the non-convexity) and therefore the authors add an entropy term to encourage smaller clusters. However, besides some intuitions, there are no formal analyses on how this entropy term will influence the objective function. Further, the paper does not describe clearly how the entropy is computed; line 278 says that it is computed by assuming x~D^+, but I cannot find the definition of _distribution_ D^+ in the text. While these details are missing or unclear, the paper includes some implementation details (which are very standard and well-known techniques) that add little value to the paper, e.g., maintaining scaling factors, using dense/sparse matrices, the internal data structure and using C++11. I think that the space may be better utilized if it is used to present more algorithm/model related details. Nevertheless, I appreciate the authors’ promise of releasing the source code to the community.The experiment results shown in the paper are good. But many experimental results are omitted (including the influence of K on the rest of data sets) due to space constraints. These results (especially the influence of K) may be useful of understanding the advantages of large-margin convex polytope machines over ordinary SVMs. Perhaps the space of supplementary material can be utilized to present more experiments results.The writing of this paper could be improved in a few places. - line 248. “sparse high dimensional space”; - line 265-266. “a pure SGD approach could get stuck in a local optimum. We found that pure SGD gets stuck in low-quality local optima in practice”;