Fixed-Length Poisson MRF: Adding Dependencies to the Multinomial
We propose a novel distribution that generalizes the Multinomial distribution to enable dependencies between dimensions. Our novel distribution is based on the parametric form of the Poisson MRF model [1] but is fundamentally different because of the domain restriction to a fixed-length vector like in a Multinomial where the number of trials is fixed or known. Thus, we propose the Fixed-Length Poisson MRF (LPMRF) distribution. We develop AIS sampling methods to estimate the likelihood and log partition function (i.e. the log normalizing constant), which was not developed for the Poisson MRF model. In addition, we propose novel mixture and topic models that use LPMRF as a base distribution and discuss the similarities and differences with previous topic models such as the recently proposed Admixture of Poisson MRFs [2]. We show the effectiveness of our LPMRF distribution over Multinomial models by evaluating the test set perplexity on a dataset of abstracts and Wikipedia. Qualitatively, we show that the positive dependencies discovered by LPMRF are interesting and intuitive. Finally, we show that our algorithms are fast and have good scaling (code available online).

This paper proposes a tractable extension of the Poisson MRF model that can accommodate positive dependencies. This model, the Fixed-Length Poisson MRF (LPMRF), is then used as the emission distribution for an admixture model similar to LDA. In this model, each topic carries additional information about pairwise dependencies (correlations) between word types. Applying this model to text, the authors obtain interesting and competitive results using only 3 topics. My primary technical question for this work is: Where is the line between modeling topics and modeling word dependencies? If we had a perfect model of word dependencies, would we need a mixture model over topics? Are the topics simply accounting for higher-order (ternary, etc) dependencies that cannot be captured easily by this particular model of word dependencies? With enough topics, the top words are more than enough to look at as results from LDA. The intuitive nature of word type dependencies might be much more questionable if you used your model with more topics. This is especially true because each topic is capturing certain positive and negative dependencies which might interact across topics in ways that are very hard to analyze or present intuitively.A key concern is scalability of the approach. How well can this method scale with vocabulary size? Especially as the number of topics increases, it seems like the memory requirements might be prohibitive. How many word types were in your data set? Why didn't you test your model with more topics?This paper is very interesting in terms of its modeling contribution and the development of a fixed-length PMRF with different properties from the original PMRF distribution. The paper has a relatively small contribution in terms of inference, because existing techniques which optimize a pseudo-likelihood are applied. Why did you choose to optimize rather than sample your parameters?You mention the Admixture of Poisson MRFs model several times in your paper, but you do not discuss any details. Please address this in a related work discussion. In addition, please cite collocation literature from linguistics and contrast your approach; you look for long-range dependences rather than words following one another .The output of the model looks similar, except the order of words in the pairs don't matter.In section 3, you say that LDAdoes not "capture dependencies between words because there is only one word being drawn at a time". While this is true from a generative point of view, I would argue that from an inferential point of view, allowing enough topics helps us to capture many types of positive dependencies.The proposed model shows only small gains over LDA in terms of perplexity. Perplexity is known to be a bad indicator of topic quality, and there are many existing topic models with better perplexity scores than LDA.The negative dependencies in Table 1 don't make nearly as much sense as the positive dependences. Please comment on this.More interesting applications for your model might actually be outside of topic modeling, e.g. in event count modeling for political science where the relationships are less well understood and where the relationships themselves are more relevant as direct objects of study.Small commentsBring Figure 1 up to page 2. Figure 1 is very effective. What L did you use to produce these graphs?Standardize your model names and acronyms. What does CPMRF stand for in Eq 1?What size of dependency matrix did you use to produce Figure 3?For perplexity results, Figure 4, note in the caption that lower is better.Increase your font size in all figures!Do not use \theta and \Theta as different parameters! Likewise, don't use 'e' both as the well-known constant and as a vector of all 1's in your proof. These notations are hard to follow.Move your Monte Carlo sampling section before your Upper Bound section in 2.1.On Page 5, use bold font for "LPMRF" in the generative process to emphasize that this model is your paper's contribution.In section 4 you say you optimize the Dirichlet parameter \beta. Do you mean \alpha?


Sparse Linear Programming via Primal and Dual Augmented Coordinate Descent
Over the past decades, Linear Programming (LP) has been widely used in different areas and considered as one of the mature technologies in numerical optimization. However, the complexity offered by state-of-the-art algorithms (i.e. interior-point method and primal, dual simplex methods) is still unsatisfactory for problems in machine learning with huge number of variables and constraints. In this paper, we investigate a general LP algorithm based on the combination of Augmented Lagrangian and Coordinate Descent (AL-CD), giving an iteration complexity of O((log(1/ ))
2 ) with O(nnz(A)) cost per iteration, where nnz(A) is the number of non-zeros in the m × n constraint matrix A, and in practice, one can further reduce cost per iteration to the order of non-zeros in columns (rows) corresponding to the active primal (dual) variables through an active-set strategy. The algorithm thus yields a tractable alternative to standard LP methods for large-scale problems of sparse solutions and nnz(A) mn. We conduct experiments on large-scale LP instances from 1 -regularized multi-class SVM, Sparse Inverse Covariance Estimation, and Nonnegative Matrix Factorization, where the proposed approach finds solutions of 10 −3 precision orders of magnitude faster than state-of-the-art implementations of interior-point and simplex methods.

In this paper, the authors propose a method which combines augmented Lagrangian and coordinate descent techniques for solving large scale linear programs with a sparse constraint matrix.Pros and cons --------------+ Overall, this is a strong contribution. In the introduction, the authors did a good job motivating their paper and covering the related work. Coordinate descent is a good fit for solving the inner problem. + The authors give iteration complexity guarantees for the entire algorithm, even if the inner problem is solved inexactly.+ I particularly appreciated that the experiments cover three representative linear programs in machine learning: l1-regularized multiclass SVM, NMF and sparse covariance matrix estimation.- The projected Newton-CG (PNCG) method is not covered in the experiments.- The experiments are rather short and only indicate timing results. I would have preferred to see more experiments and move the PNCG part to the supplementary material.- There is no discussion whatsoever on how the parameter \eta_t was chosen. If the paper is accepted, it is very important to address this issue in the camera-ready version.- Same problem with regularization parameters in the experiments.- While much better than interior point and simplex methods, the timing results remain too large for practical use. In practice, one can add a little bit of l2 regularization to the problem (i.e., replace l1 regularization by elastic-net).Then the problem can be solved by modern solvers such as SDCA, which should be orders of magnitude faster than results shown in this paper.Detailed comments ------------------* The authors claim in several places in the paper that the conditioning of the problem doesn't worsen with the number of iterations. It would be interesting to briefly explain why.* line 101: non-negative constraint -> non-negativity constraint* line 176: g(y) is already defined in (2). Here you are overloading the definition so that g(y) becomes infinity if y is infeasible. I hadn't realized this at first glance and so (5) made no sense. Instead of overloading g(y), you could define a new function \tilde{g}(y) to make it more explicit.* problem (5): you didn't define \eta_t* line 230: you already defined \hat{F}(x) in (9)* line 236: add citation for generalized Hessian* line 259: clever -> efficient* Table 1-3: Barrier = IPM?


Robust Portfolio Optimization
We propose a robust portfolio optimization approach based on quantile statistics. The proposed method is robust to extreme events in asset returns, and accommodates large portfolios under limited historical data. Specifically, we show that the risk of the estimated portfolio converges to the oracle optimal risk with parametric rate under weakly dependent asset returns. The theory does not rely on higher order moment assumptions, thus allowing for heavy-tailed asset returns. Moreover, the rate of convergence quantifies that the size of the portfolio under management is allowed to scale exponentially with the sample size of the historical data. The empirical effectiveness of the proposed method is demonstrated under both synthetic and real stock data. Our work extends existing ones by achieving robustness in high dimensions, and by allowing serial dependence.

The paper presents a method to derive a portfolio based on minimizing the variance of the return estimator, where the covariance is estimated by a quantile-based scatter matrix. The authors discuss how to estimate this matrix R^Q from data. An important property of their estimator is that no assumptions on the tails of the distribution are made. In Section 4, the authors bound the error of the quantile-based scatter matrix and their empirical estimate. Moreover, they show that R^Q is proportional to the covariance matrix if the asset returns follow an elliptical distribution (including Gaussians).I'm inclined to accept the paper because especially the latter finding is of interest to the ML community. However, the authors should address the following issues.Minimizing the overall variance of the financial return estimator in Eq. 2.1 is a "proxy" to solve the actual problem of maximizing return. Even if the problem statement and terms such as long/short positions are well known in the finance community, they may be not in the ML community. Please provide a clear problem setting. In contrast, there is no need to introduce matrix or vector norms.Replacing the covariance by any matrix R^Q seems arbitrary and unmotivated in the first place. If I understand correctly, the actual problem is that the sample variance is a "bad" estimate of the real covariance (for several reasons including a lack of observations). The proposed estimator R-tilde^Q is supposed to be a more accurate estimate w.r.t. metric E[X^T,w], where w is defined by solving (3.2) given the covariance estimate and X is any stochastic process of asset returns. This should be made clear in the beginning of Section 3. Also I would like to see an empirical comparison of different covariance estimators for various matrix distances; not just for E[X^T,w] as in Figure 1.Is estimator Q-hat in (3.1) a consistent/unbiased estimator of Q? To me it seems that Q-hat overestimates the quantiles since pairs (z_s,z_t) are "drawn" from sample {z_i} without replacement.In terms of notation, there is no real distinction between random variables and their realization which is sometimes confusing.typos: line 70 and 71, M_jk should not be bold face line 72: ^q missing for the q-norm definition 4.2: P(A) - P(A|B) without |.| Eq. 8 contains equalities; the text thereafter refers to inequalities


Training Very Deep Networks
Theoretical and empirical evidence indicates that the depth of neural networks is crucial for their success. However, training becomes more difficult as depth increases, and training of very deep networks remains an open problem. Here we introduce a new architecture designed to overcome this. Our so-called highway networks allow unimpeded information flow across many layers on information highways. They are inspired by Long Short-Term Memory recurrent networks and use adaptive gating units to regulate the information flow. Even with hundreds of layers, highway networks can be trained directly through simple gradient descent. This enables the study of extremely deep and efficient architectures.

I like the paper, the idea is well described and the experiments are convincing to a certain degree. The best thing in my opinion is that the authors tried to analyze the learned networks with respect to the pattern of gate outputs.(1) The gate unit T is modeled with a standard sigmoid activation. Therefore, the response value is never exactly 0 or 1 (this was also stated by the authors) and the gradients in eq. 5 are not correct. The authors should explain in the paper how backpropagation is exactly performed in these networks.(2) The effect of the initialization is always tricky to analyze, however, the fact that the authors propose to use constant negative biases depending on the number of levels is also not satisfying. I would like to see a plot of the performance with respect to the initial value of the bias. The initial bias acts as a kind of regularizer in this case with a high negative bias supporting networks with "longer highways" (I like this comparison by the way). (3) "report by recent studies in similar experimental settings" This sentence is unsatisfying. Either the methods use the same experimental setting, which would allow for comparison, or not, which would forbid a direct comparison. The authors need to clarify the differences of the experimental setups in the paper. (4) Giving "max" accuracies in tables is rather uncommon and also not reasonable from a statistical point of view. The authors should therefore always give mean accuracy and standard deviation for every method (and when available). (5) It would be interesting to study the proposed approach from a learning theory perspective.


Interactive Control of Diverse Complex Characters with Neural Networks
We present a method for training recurrent neural networks to act as near-optimal feedback controllers. It is able to generate stable and realistic behaviors for a range of dynamical systems and tasks -swimming, flying, biped and quadruped walking with different body morphologies. It does not require motion capture or task-specific features or state machines. The controller is a neural network, having a large number of feed-forward units that learn elaborate state-action mappings, and a small number of recurrent units that implement memory states beyond the physical system state. The action generated by the network is defined as velocity. Thus the network is not learning a control policy, but rather the dynamics under an implicit policy. Essential features of the method include interleaving supervised learning with trajectory optimization, injecting noise during training, training for unexpected changes in the task specification, and using the trajectory optimizer to obtain optimal feedback gains in addition to optimal actions. Figure 1: Illustration of the dynamical systems and tasks we have been able to control using the same method and architecture. See the video accompanying the submission.

In this submission, authors propose an algorithm for real-time control of 3D models, where a neural network (NN) is used to generate animations, after having been trained offline to reproduce the output of Contact-Invariant Optimization (CIO). Importantly, CIO and NN training are not independent: they are performed in an alternating fashion, the output of one being used in the criterion optimized by the other. Another core element of the method is the injection of noise, through data augmentation by small perturbations as well as additive noise in the NN's hidden layers. Experiments show that such a network is able to produce realistic and stable control policies on a range of very different character models.This is a very interesting approach, considering that real-time control of arbitrary models is a challenging task. Although the proposed method may initially appear relatively straightforward ("learn a NN to predict the output of CIO"), the additional steps to make it work (joint training and noise injection) are significant new contributions, and shown in experiments to help get much better generalization.Those experimental results would be stronger, however, if comparisons were made with a larger dataset. Since the base training trajectories were generated with only "between 100 to 200 trials, each with 5 branched segments", it is not surprising for the "no noise" variant to overfit. The "no joint" variant might also benefit from more diverse training trajectories -- even if that is less obvious.Overall the comparative evaluation is the weak point of the paper (acknowledged in the introduction: "A systematic comparison of these more direct methods with the present trajectory-optimization-based methods remains to be done"). The only comparison to a competing method is a quick one with MPC at the end of the paper (and there is no reference for this method by the way).For the most part, the maths are clearly explained and motivated, although someone willing to replicate these results will definitely need to read up additional references, since some points are only addressed superficially (especially the CIO step -- see Eq. 6 whose notations are not defined). Providing code would certainly be much appreciated by the community. There is one part I was not able to properly understand: section 3.2 about generating optimal actions for noisy inputs. Notations got really confusing for me at this point, and it was not clear to me what was being done (in particular the time index "t" was dropped, but is it still implied for some of the quantities?).It would have also been useful to get some time measurements for computations during real-time control, since that could be a major bottleneck (and also knowing how these computations scale with the complexity of the 3D model). In an application like a video game, there is not much CPU available to animate a single character on screen.Additional minor remarks: - "biped location": locomotion? - "It is our aim to unity these disparate approaches": unite? unify? - Am I right that the target is only about x,y,z coordinates, and there is no constraint on the angle the character is facing when reaching it? If yes, would it be easy to add an extra target angle? (which seems important in practice) - I would not call "noise injection" the step where additional data is generated from noisy inputs, because the target is re-computed to match the modified inputs. To me, this is simply "data augmentation", and it has nothing to do with the idea behind denoising autoencoders. - Are you really using the initial (random) network weights theta in the very first iteration, to compute the first X's in Alg. 1? Or are you starting with X's computed without the regularization term? - What is the justification for using the same hyperparameter eta in the two steps of Alg. 1? - "it is non-trivial to adapt them to asynchronous and stochastic setting we have": missing "the"? - Acronym LQG is used one line before it is defined. - "While we can use Linear Quadratic Gaussian (LQG) pass": missing "a"? - l. 231 the star in the argmin is in the wrong place, and is \bar{s} missing in \tilde{C}? - "the minimizer of a quadratic around optimal trajectory": missing a word? - "sX and aX are subsets of \phiX": why is that guaranteed? - "a single pass over the data to reduce the objective in (4)": it would rather be the objective from Alg. 1 since only theta is optimized - The asynchronous joint optimization scheme described in Section 6 is very different from the alternating scheme from Alg. 1. It would be worth at least mentioning this earlier. - l. 315, f should be f^{des}? - "In the current work, the dynamics constraints are enforced softly and thus may include some root forces in simulation.": I did not understand this sentence - The "no noise" variant in experiments could be split between "no data augmentation" and "no hidden noise injection". - A question I would find interesting to investigate is whether alpha can be decreased all the way to zero during optimization (for CIO only). In other words, is it working mostly because of a "curriculum learning" effect (easier task initially), or is it always useful to keep the trajectories learned by CIO close to what the network can learn? - Caption of Table 1a says 4 layers but I believe this is 3


Submodular Hamming Metrics
We show that there is a largely unexplored class of functions (positive polymatroids) that can define proper discrete metrics over pairs of binary vectors and that are fairly tractable to optimize over. By exploiting submodularity, we are able to give hardness results and approximation algorithms for optimizing over such metrics. Additionally, we demonstrate empirically the effectiveness of these metrics and associated algorithms on both a metric minimization task (a form of clustering) and also a metric maximization task (generating diverse k-best lists).

First, I must say that the paper was very well written and was a joy to read.The authors analyse metrics of the form d(A, B) = F(A \triangle B), where F is a positive polymatroid function. As the authors state, it was known that even a weaker condition is sufficient for metricity, and the authors analyse the benefits of the additional submodular assumption. The resulting analysis is for both minimizing and maximizing the sum of distances. The algorithms mostly rely on approximating F(A \trinagle B) by F(A \ B) + F(B \ A), which is submodular in A and is a 2-approximation (as they prove). Moreover, this seems to hold even when f is only subadditive and positive, not necessarily submodular. Can the authors please clarify this?The theoretical results seem correct and they are clearly presented. I went only through some of the proofs, and they were also well written and easily understandable.The authors propose two new algorithms for minimization. It is a bit disappointing that in the homogeneous case, they could not improve on the existing algorithm B-Bound, which makes a much weaker assumption. I would have been much happier if there were experimental results for minimization as well, as it is not clear if this is due to the looseness of the bound of Union-Split. The proposed algorithm (Major-Min) for the constrained case is very natural as the same strategy (minimize the upper bound from a supergradient) has been already used.For maximization, the authors suggest the same approximation strategy and then their results follow from the 2-approximation. I believe that it would be more instructive if the authors present the fact that the maximization of any positive symmetric submodular function is a special case for m=2 in the main paper.Here are some questions I have regarding the experimental section: * You claim that the greedy algorithm has a 1 - 1/e approximation guarante for HM,  but adding the |A \triangle A_i| term destroys the monotonicity of the objective.  Isn't this a mistake? * Did you try experimenting with alpha * hamming? I would be interested in the  results under such a simple modification of the baseline model. * Do you present variance, or standard error in the results? Almost all of the  intervals seem to be significantly overlapping.


Community Detection via Measure Space Embedding
We present a new algorithm for community detection. The algorithm uses random walks to embed the graph in a space of measures, after which a modification of k-means in that space is applied. The algorithm is therefore fast and easily parallelizable. We evaluate the algorithm on standard random graph benchmarks, including some overlapping community benchmarks, and find its performance to be better or at least as good as previously known algorithms. We also prove a linear time (in number of edges) guarantee for the algorithm on a p, q-stochastic block model with where p ≥ c · N

This paper presents a graph partitioning algorithm for community detection based on random walk. It represents each node in the graph by a distribution, which is induced by performing a fixed-length random walk on the graph starting from that node. The nodes are the clustered by a kmeans-style algorithm. Empirical results are presented on some benchmark graphs for detecting non-overlapping and overlapping communities.The results show similar or better performance compared to two leading algorithms identified by a recent empirical published in 2009. A theoretical result is also presented that shows in some limited situations (k=2, identical partition size, 1-step random walk) the proposed algorithm with high probability converges to the true partition in one iteration. The algorithm is interesting and appears to be novel in how random walk is specifically used. I particularly liked the different interpretations that were presented by the paper in the objective (eg. equation 3).With that said, the use of random walk for graph partitioning has been considered in many studies, including walktrap (while different in details, this method is conceptually quite similar to the proposed method) and infomap as referenced by the paper. The paper discusses the distinction between the proposed approach and these two other methods, but I felt that the discussion stops a bit short in providing a clear argument as to why the proposed method should be preferred to the existing methods.The experimental results seem competitive (for the overlapping case). But I do have a few concerns. For the non-overlapping case, L is set to be 5 and for the overlapping case L is set to 2. Why use different L's? How are they decided? How sensitive is the method to the choice of L?A minor presentation issue. Figure 2 does not have a caption. For comparison purposes, results of the competing methods should be included in the paper other than pointing the readers to a different papers to make the comparison.For the theoretical result, I was not able to carefully read through the supplementary material to verify the full proof. Based on the stated result (theorem 5.1), I find it to be limited in the cases that it considers. That is, the only cases the result is applicable is whenk=2, and when the two clusters are of exact equal size. The equal-size assumption seems particularly restrictive to me. It is also stated that the result is only for L=1 (random walk of one step) and it is unclear how it would generalize to longer walks, which is used in the experiments.The presentation of the algorithm has some confusing parts to me. Algorithm 1 does not explicitly talk about random walk. I assume w_i is computed using random walk, but it is unclear to me if it is computed analytically using the stochastic matrix or by doing random walks starting from i. D is referred to as acting as a distance function. In the traditional Kmeans algorithm, distances are to be minimized. Here, however, obj 2 is to be maximized.Further, obj 2 is referred to as the cost. Generally, we try to minimize cost. The inconsistent use of the terminology is confusing.


Path-SGD: Path-Normalized Optimization in Deep Neural Networks
We revisit the choice of SGD for training deep neural networks by reconsidering the appropriate geometry in which to optimize the weights. We argue for a geometry invariant to rescaling of weights that does not affect the output of the network, and suggest Path-SGD, which is an approximate steepest descent method with respect to a path-wise regularizer related to max-norm regularization. Path-SGD is easy and efficient to implement and leads to empirical gains over SGD and AdaGrad.

Deep rectified neural networks are over-parameterized in the sense that scaling of the weights in one layer, can be compensated for exactly in the subsequent layer. This paper introduces Path-SGD, a simple modification to the SGD update rule, whose update is invariant to such rescaling. The method is derived from the proximal form of gradient descent, whereby a constraint term is added which preserves the norm of the "product weight" formed along each path in the network (from input to output node). Path-SGD is thus principled and shown to yield faster convergence for a standard 2 layer rectifier network, across a variety of dataset (MNIST, CIFAR-10, CIFAR-100, SVHN). As the method implicitly regularizes the neural weights, this also translates to better generalization performance on half of the datasets.As an algorithm, Path-SGD appears effective, simple to implement and addresses an obvious flaw in first-order updates to ReLU networks. As a paper, it could be improved however, especially with respect to notation (see details section below) which sometimes obfuscates simple concepts. Certain sections of the paper seem rushed and would require a careful rewrite. See details section below.There are also a number of glaring omissions to prior work. At its core, Path-SGD belongs to the family of learning algorithms which aim to be invariant to model reparametrizations. This is the central tenet of Amari's natural gradient (NG), whose importance has resurfaced in the area of deep learning (see e.g [R1-R4]). Path-SGD can thus be cast an approximation to NG, which focuses on a particular type of rescaling between neighboring layers. The paper would greatly benefit from such a discussion in my opinion. I also believe NG to be a much more direct way to motivate Path-SGD, than the heuristics of max-norm regularization.The experimental section could also benefit from a few extra experiments. Can the authors validate experimentally that $\pi(w)$ is more stable during optimization ? It is also regrettable that the authors chose a fixed model architecture for the experiments. One would expect the advantage of Path-SGD to be even more pronounced for deeper networks, which have longer paths from input to output. Such experiments would be much more informative that the dropout experiments of Figure 3, which do not add much to the narrative.Detailed Feedback: Theorems should be rather self contained and mathematically precise. Path-SGD is not invariant to arbitrary rescaling, contrary to the claims of Theorem 4.1, but only invariant to the rescaling function $\rho$ for c > 0. Please correct accordingly. Eq. 5 (and most others) are missing an outer summation over i and j. Eq. 5: notation $v_in[i] \rightarrow ... \rightarrow v_out[j]$ found throughout the paper is quite cumbersome. One could instead use $\mathcal{P}$ to denote the set of all possible paths in the network, and then for each path $p \in \mathcal{P}$, sum over edges $\e_k \in p$ ? Idem for line 198 and other equation involving paths. line 212: missing $\mathcal{1}{2}$ term line 212: missing absolute values before exponentiating by $p$. line 216: "hard to calculate". Please be more precise: is it intractable, ill-defined, expensive to compute ? line 216: "Instead, we will update each coordinate" -> "perform coordinate descent". The original form was confusing to me as SGD also updates each parameter "independently", in the sense that they do not exploit curvature / covariance information. Eq 7 vs line 224: inconsistent notation between $e'$ and $e_k$ line 227: similar to previous comments, it is clumsy for $v_in[i] -> ... e .. -> v_out[j]$ to be implicitly defined over all i and j.[R1] Enhanced Gradient for Training Restricted Boltzmann Machines, KyungHyun Cho, Tapani Raiko, Alexander Ilin. ICML 2011. [R2] Deep Learning Made Easier by Linear Transformations in Perceptrons. Tapani Raiko Harri Valpola Yann LeCun. AISTATS'12. [R3] Optimizing Neural Networks with Kronecker-factored Approximate Curvature, James Martens, Roger Grosse. ICML 2015. [R4] Scaling up Natural Gradient by Sparsely Factorizing the Inverse Fisher Matrix. Roger Grosse, Ruslan Salakhudinov. ICML 2015.


Fighting Bandits with a New Kind of Smoothness
We provide a new analysis framework for the adversarial multi-armed bandit problem. Using the notion of convex smoothing, we define a novel family of algorithms with minimax optimal regret guarantees. First, we show that regularization via the Tsallis entropy, which includes EXP3 as a special case, matches the O( p NT ) minimax regret with a smaller constant factor. Second, we show that a wide class of perturbation methods achieve a near-optimal regret as low as O( p NT log N ), as long as the perturbation distribution has a bounded hazard function. For example, the Gumbel, Weibull, Frechet, Pareto, and Gamma distributions all satisfy this key property and lead to near-optimal algorithms.

The paper considers the classic problem of multi-armed bandits and extends the algorithmic framework of "gradient-based prediction algorithms" (GBPA) to this setting. In this framework, the learner uses a convex potential function to measure its own performance and predicts according to the gradient of the potential function evaluated at the current state. The main result in the current paper is a sufficient condition on the potential function so that it guarantees near-optimal regret bounds in the multi-armed bandit problem, assuming that the learner estimates the losses by the standard importance-weighting technique. Based on this result, two further results are derived: an order-optimal learning algorithm based on running FTRL (Follow-the-Regularized-Leader) with the Tsallis entropy as a regularizer and a sufficient condition for the perturbation distribution in FPL (Follow-the-Perturbed-Leader) to give near-optimal performance guarantees.The main result and its consequences contribute greatly to the literature on multi-armed bandits: Such an easily verifyable sufficient condition on the regularization functions and the perturbation distributions were highly sought for by many researchers. The contributions of the current paper make the relatively scattered picture of existing results on bandits much clearer. The writing and the technical quality are both excellent, which, together with the strength of the results, make this a very good paper overall.There are, however, a few minor mistakes that need to be fixed before the paper is published. First, the authors argue that the PolyINF algorithm of Audibert and Bubeck (2009) is "completely different" from gradient-based methods. This, however, is not true: as shown by Audibert, Lugosi and Bubeck (2014, MOR), PolyINF is actually an instance of mirror descent with a polynomial regularizer. Actually, it turns out that the regularizer used by Audibert et al. is the same (up to an additive constant) as the Tsallis entropy as proposed in the current paper. Nevertheless, the analysis presented in the current paper is more elegant in many respects, in that it reveals a close connection between Exp3 and the algorithm giving optimal bounds. The authors also claim an improvement in the best known constant factors, this is actually not true: the leading constant of Audibert et al. is actually better (although this issue can be fixed, see below). All in all, the presentation in the current submission needs to be modified a bit to highlight these connections.A closer examination of the proofs reveals that the regret bounds presented in the paper can be improved by at least a factor of \sqrt{2}. This factor is lost in the inequality on line 207, where C/2 is upper-bounded by C (and \ell_i^2 is bounded by 1). Propagating this lost factor into Theorems 2.3 and 3.1 improves the bounds derived for Exp3 and PolyINF to match the best known regret bounds for the respective algorithms (see, e.g., the monograph by Bubeck and Cesa-Bianchi, 2012 and the paper of Audibert, Bubeck and Lugosi, 2014).A minor annoyance in the presentation that while the authors claim to study the loss game, they actually study the gain game with negative gains and aim to maximize the sum of these negative gains. To me, this was often the source of confusion, as a loss should always be intuitively minimized and not maximized as often referred to in the paper (e.g., in Eq. 3 and the following paragraphs). I suggest that the authors refer to gains throughout the paper, reminding the reader that the proofs only work for non-positive gains. (Alternatively, conduct the analysis with non-negative losses. However, this seems to be rather tedious as it gives rise to a bunch of confusing minus signs everywhere in the paper.)Detailed comments ----------------- 087: "and associated bound" -> "and the associated bound"? 088: "That is, noise must satisfy" -> "Precisely, we show that the noise must satisfy"? 107: "Nature's choices [...] are assumed deterministic throughout." -- Can you clarify this? Does this mean that nature fixes the loss vectors in the beginning of the game (i.e., obliviously) or can be a deterministic function of the past plays of the learner? Is there any particular reason to make this assumption? 119: What does "chosen (but fixed)" precisely mean? 122: What is i here? 126: This would be the right time to note that PolyINF is also closely related. (Actually, it is not FTRL but mirror descent. I think it should be easy to show that for this particular regularizer, MD and FTRL are identical on the simplex, as can be done for the negative Shannon entropy.) 132: I mostly buy this argument about the fluctuations of the loss estimates, but note that the estimates need not be unbiased for good performance. In fact, algorithms that give high-probability bounds all use biased estimates. Also, "O(1/p)" should be "\Theta(1/p)". 156: This proof is a bit too sketchy, and referring to "GBPA run without the estimation step" is rather confusing. Besides Abernethy et al. (2012), Jan Poland's work "FPL analysis for adaptive bandits" (2005, TCS tech. report) also does a great job in explaining this step (and is also a relevant reference for following the perturbed leader in bandit problems). 161: "i_1,...,i_t" -> "i_1,...,i_{t-1}"? 170: "hessian" -> "Hessian" 184: "Then divergence penalty" -> "Then, the divergence penalty"? 192: "i_t is conditionally independent given \hat{L}" -- I am not sure what you mean here. I think it should be more important to note here that i_t is distributed as \nabla\tilde\Phi(\hat{L}). Also, some hats and tilde's are missing in this line. 218: "and it was not until [...] did the authors obtain" -> "and it was not until [...] that the authors obtained"? 225 vs 244: sup vs max? 268: \eta is not part of the regularizer and should not show up here. On the other hand, \tilde{\Phi} should be defined as \eta S + I instead of S + I for everything to check out. 290: This section could use a short introduction to be on par with the previous one. 300: The passage about swapping the order of differentiation and integration comes before differentiation is mentioned at all. Also, \tilde\Phi should be replaced by \nabla\tilde\Phi here. 304 and 305: ">" -> "\ge" would be more consistent with the final expression. Also, technically this expression is a subgradient, as for some specific perturbation distributions, a tie between arms may arise with positive probability. 350: I suppose that "(sup h_D)" refers to the supremum of h_D over its domain, but this is not entirely clear (and Appendix B seems to suggest otherwise). 451 and 477: missing years. 473 and 475: "Boltzmann-Gibbs" and "Monte Carlo". 497 and 500: missing parens. 506 and 509: missing \nabla from the last terms.bonus remark: I don't like the title -- I first thought that "Fighting bandits" is yet another derivative bandit setting. "A new kind of smoothness" sounds good though.


Scalable Adaptation of State Complexity for Nonparametric Hidden Markov Models
Bayesian nonparametric hidden Markov models are typically learned via fixed truncations of the infinite state space or local Monte Carlo proposals that make small changes to the state space. We develop an inference algorithm for the sticky hierarchical Dirichlet process hidden Markov model that scales to big datasets by processing a few sequences at a time yet allows rapid adaptation of the state space cardinality. Unlike previous point-estimate methods, our novel variational bound penalizes redundant or irrelevant states and thus enables optimization of the state space. Our birth proposals use observed data statistics to create useful new states that escape local optima. Merge and delete proposals remove ineffective states to yield simpler models with more affordable future computations. Experiments on speaker diarization, motion capture, and epigenetic chromatin datasets discover models that are more compact, more interpretable, and better aligned to ground truth segmentations than competitors. We have released an open-source Python implementation which can parallelize local inference steps across sequences.

This paper applies to the sticky HDP-HMM some recently developed machinery for split/merge style adaptation of the truncation level for variational inference in Bayesian nonparametric models and evaluates the resulting algorithm on several tasks, emphasizing identifying the true number of states present in a dataset. The algorithm is an improvement over the algorithm presented in [7] particularly because of these extra steps, and possibly due to the improved treatment of the variational factor on the top-level HDP weights.The paper is written very clearly. However, given previous work on these subjects the novel contributions here are incremental. Furthermore, the experiments are unenlightening and the synthetic experiment is very confusing.## Experiments ## The synthetic data results presented in the first experiment don't look correct. The data, which I understand are shown in the top-left panel, are trivially separable and any algorithm, even a GMM without any temporal dynamics let alone sticky bias, would separate these clusters perfectly in just a few iterations (unless perhaps it had some adversarial initialization). Yet the reported experiment shows that a sampler (presumably operating on the entire dataset in each iteration) required 1000 iterations just to find these obvious clusters? Or that SVI (without split/merge moves) couldn't get away from modeling the data with 60 clusters after 1000 full passes through the data? These results contradict both reported performance in the literature and my own experience with related models, even vanilla GMMs.The other experiments are believable but they don't provide much insight into the potential tradeoffs here or failure modes of the proposed algorithm; each figure just serves to show that the proposed method always works great. Studying Hamming error and the number of recovered states is okay, but those metrics probably favor the proposed method's emphasis on searching explicitly over the model size; why not show predictive likelihood as well to measure whether the predictive distributions are converging for these other methods (as expected)? When does this method do badly, and when might its extra computational burden not be worthwhile? Instead of shedding light on these kinds of questions, these experiments are all salesmanship.## Originality / significance ## The originality of this work relative to [13] and [14] is minimal. The authors helpfully enumerate contributions over [14] in the conclusion section. Three of the four seem incremental while the last (an implementation with parallelism) isn't relevant to deciding the merits of the paper.The extension from the HDP-HMM to the Sticky HDP-HMM is pretty much the first thing to try: the sticky bias amounts to just adding an extra weight to a particular entry in each second-level weight vector in the HDP, and so the extent of the contribution here is just using the bound from [14] and splitting a logarithm over a sum (Eq. (36) in the supplement). (The algebra would be clearer if it were written as log(alpha beta_k + kappa) = log(beta (alpha + kappa) + (1-beta) kappa) >= beta log(alpha+kappa) + (1-beta) log(kappa).)The merge selection and delete acceptance rules are an improvement over [14], but they simply amount to a better heuristic. The birth moves also seem like a reasonable choice but the idea is pretty straightforward given the use of birth moves in [13].Parallelization would be a merit of the implementation, but not the method as described in the paper, especially since all the comparable methods admit the same (easy) parallel implementations. If we're meant to evaluate the quality of the implementation as a contribution of this paper, the implementation should be included.## Misc ## - The comments on model selection don't make sense (Lines 049 and 174). Are these comments meant to refer to using the variational lower bound on the model evidence to compare the HDP-HMM to other models? If so, it's not been made clear that these bounds for these models are useful proxies compared to direct estimates via AIS or SIS, and if that's a claimed merit of this approach, it should be investigated in the experiments. The comment on Line 174 that bounds which use a delta factor on the top-level weights "can favor models with redundant states" is not demonstrated; indeed, those do provide valid bounds on model evidence with just the top-level weights fixed instead in precisely the same way that the each majorization bound used in the EM algorithm is a bound on the likelihood of the data with the hidden variables marginalized out (but the parameters fixed). - Around line 236 it is claimed that memoized variational inference with one pass (one lap) is identical to SVB. Is that essentially a choice to use SVB to start the algorithm as opposed to initializing the cached statistics (M, S)? Since [13] was published at the same time as [20] and does not cite [20], it's surprising to learn that memoized variational inference actually contains streaming variational inference as a special case. - Is memoized variational inference just regular batch mean field with a different update schedule? Caching sufficient statistics, instead of keeping around individual factors' parameters only to throw them out when they are updated, seems to be the straightforward implementation strategy for such an update schedule. - In supplement C.1, a line reads "Here, all the infinite sums are convergent. This allows some regrouping" but infinite sums need to be *absolutely* convergent to allow regrouping. If all terms are nonnegative, regrouping is always okay. If the series is convergent but with indefinite terms, regrouping does not hold in general.


Black-box optimization of noisy functions with unknown smoothness
We study the problem of black-box optimization of a function f of any dimension, given function evaluations perturbed by noise. The function is assumed to be locally smooth around one of its global optima, but this smoothness is unknown. Our contribution is an adaptive optimization algorithm, POO or parallel optimistic optimization, that is able to deal with this setting. POO performs almost as well as the best known algorithms requiring the knowledge of the smoothness. Furthermore, POO works for a larger class of functions than what was previously considered, especially for functions that are difficult to optimize, in a very precise sense. We provide a finite-time analysis of POO's performance, which shows that its error after n evaluations is at most a factor of √ ln n away from the error of the best known optimization algorithms using the knowledge of the smoothness.

The problem of bandit optimization of a noisy function without an explicit knowledge of smoothness is considered.The paper introduces a new variant ofthe existing HOO algorithm to solve this problem. The main idea is that to run Ninstances HOO in parallel for different choice of smoothness and then to choose the instance with the highest average reward.Theoretical guaranteesin the form of simple expected regret bound has been provided. The paper is in most part well-written and easy to follow.Also the idea of using multiple instances of HOOwith different smoothness parameters seems a promising idea to deal with thecase of unknown smoothness. However I am not sure the currentversion of POO really addresses the problem of unknown smoothness in online optimization. The main reason is thatthe current version of proposed algorithm still needs to receivesome upper bounds of the smoothness as the input. Also regret bounds strongly depend on the value of these upper bounds(both the rate and constant). The paper suggests to deal with the problem of unknown \rho_\max and \nu_\max by gradually increasing thevalue of smoothness parameters until they becomelarger than \rho^* and \nu^*. In particular the paper suggests to increase \rho_\max and \nu_\max such that the factor D_\max \nu_\max D_\max increases with the slow (logarithmic) rate which keeps the rate of regretintact (up to a logarithmic factor). The problem with this approach is 1- it leads to an extra constant regret term with double exponential dependency on 1/log(1/\rho).The term 1/\log(1/\rho ) approximately equals to 1/(1-\rho) for\rho s close to 1 which makes this constant regret termhuge andthe whole regret bound vacuous. 2- the idea of increasing \rho and \nu, combined with doubling trick, could be already applied to theoriginal HOO algorithm without any need for parallelization. In fact I think by using doubling trick with increasing \rho and \nu one could achieve a better regret than what we can get with POO (the constant term would be smaller and the dependency on log(n) would be quadratic instead of cubicin the case of POO with increasing \rho_{\max}). Therefore I am not sure the new algorithm is the best way to deal with the problem of unknown smoothness since the standard methods (e.g., using doubling trick with increasing parameters) seems to provides us with a better solution.


Estimating Mixture Models via Mixtures of Polynomials
Mixture modeling is a general technique for making any simple model more expressive through weighted combination. This generality and simplicity in part explains the success of the Expectation Maximization (EM) algorithm, in which updates are easy to derive for a wide class of mixture models. However, the likelihood of a mixture model is non-convex, so EM has no known global convergence guarantees. Recently, method of moments approaches offer global guarantees for some mixture models, but they do not extend easily to the range of mixture models that exist. In this work, we present Polymom, an unifying framework based on method of moments in which estimation procedures are easily derivable, just as in EM. Polymom is applicable when the moments of a single mixture component are polynomials of the parameters. Our key observation is that the moments of the mixture model are a mixture of these polynomials, which allows us to cast estimation as a Generalized Moment Problem. We solve its relaxations using semidefinite optimization, and then extract parameters using ideas from computer algebra. This framework allows us to draw insights and apply tools from convex optimization, computer algebra and the theory of moments to study problems in statistical estimation. Simulations show good empirical performance on several models.

The paper presents a general framework for deriving methods of moments for learning mixture models. Its main contributions are: (1) showing how expressions for the moments of the base distribution can be bootstrapped to derive expressions for the moments of mixtures of base distributions; and, (2) providing recipes for solving the resulting moment equations combining SDP and generalised eigenvalue problems. The overall impression is that the paper does not contain a great deal of technical novelty, but it provides a fresh perspective on moment matching methods for mixture models. The main paper is well-written, but several crucial details are discussed only in the appendix.It is a bit disappointing that the paper pays very little attention to two aspects that initially attracted a lot of attention about spectral methods of moments: the possibility of proving finite-sample bounds, and their scalability as compared to vanilla EM. About the latter, my impression is that obtaining really efficient solutions for the SDP problems arising from different mixture models will require exploiting specific structural properties in each individual case. So the method might not be as "turnstyle" as the authors promise when scalability considerations are taken into account.The scope of the experiments is quite limited, but nearly enough due to the nature of the paper. It does not help dispel any doubts about the scalability of the proposed method, but it showcases the potential benefits of this approach when compared to EM. There are, however, two aspects of these results that could be improved: adding a comparison with the baseline from reference [9] for the linear regression case, and comparing the methods using more than one accuracy measure (eg. perplexity), since somehow one suspects that L2 distance to the true parameter is what poly is hiddenly optimizing, while EM is clearly optimizing another metric.The weakest point of the paper is perhaps its lack of awareness of some relevant previous results. In particular, although the paper does a good job at connecting with literature related to generalised moment problems and solutions of polynomial equations, it misses some quite direct connections to other methods of moments based on Hankel matrices. These follow by simply observing that the moment matrix M(y) is nothing else than a Hankel matrix with indices on the commutative free monoid \mathbb{N}^P. From this point of view, rank properties of M(y), the way in which the truncation M_r(y) is defined, and the shifting exploited in Section 4, are all direct consequences of the same properties for Hankel matrices on the non-commutative free monoid (eg. see [1]). Also, non-commutative versions of problems (6) and (7) were considered already in [2] and [3] with similar constraints as the ones used in the paper, and in [4] in a more general non-probabilistic setting. In view of this, the paper would probably do a better service to the community by making these connections explicit.After reading the authors' response I am more inclined to recognize the novelty in their contribution. However, the feeling of this reviewer (and apparently other reviewers as well) is that in general the paper glosses over a number of important points (eg. identifiability, sample bounds, scalability of SDP relaxations, possibility of special-purpose optimization algorithms). It is clear that a conference paper cannot be asked to tackle all these in detail, but the current paper could be improved with a more systematic and upfront discussion of what it tries and tries not to do. In summary, I think this work shows promise, but perhaps it could benefit from a more elaborate presentation.  *** Minor Comment ***--- In Line 193, the only immediate conclusion one can draw from \mu being K-atomic is that rank(M_r(y)) <= K (note the inequality, not equality like in the paper). To obtain the equality claimed in the paper one needs to: (1) suppose that r is large enough, and (2) somehow make an argument showing that when the theta^*_k are all different then the vectors v_r(theta^*_k) for k = 1...K are linearly independent. Since this point might have impact on the identifiability of the model, I think it is too important to be hand-waved like the authors do. [1] Borja Balle, Xavier Carreras, Franco M. Luque, Ariadna Quattoni: Spectral learning of weighted automata - A forward-backward perspective. Machine Learning 96(1-2): 33-63 (2014)[2] Raphael Bailly, Xavier Carreras, Ariadna Quattoni: Unsupervised Spectral Learning of Finite State Transducers. NIPS 2013[3] Raphael Bailly, Xavier Carreras, Franco M. Luque, Ariadna Quattoni: Unsupervised Spectral Learning of WCFG as Low-rank Matrix Completion. EMNLP 2013[4] Borja Balle and Mehryar Mohri: Spectral learning of general weighted automata via constrained matrix completion. NIPS 2012


Learning Causal Graphs with Small Interventions
We consider the problem of learning causal networks with interventions, when each intervention is limited in size under Pearl's Structural Equation Model with independent errors (SEM-IE). The objective is to minimize the number of experiments to discover the causal directions of all the edges in a causal graph. Previous work has focused on the use of separating systems for complete graphs for this task. We prove that any deterministic adaptive algorithm needs to be a separating system in order to learn complete graphs in the worst case. In addition, we present a novel separating system construction, whose size is close to optimal and is arguably simpler than previous work in combinatorics. We also develop a novel information theoretic lower bound on the number of interventions that applies in full generality, including for randomized adaptive learning algorithms. For general chordal graphs, we derive worst case lower bounds on the number of interventions. Building on observations about induced trees, we give a new deterministic adaptive algorithm to learn directions on any chordal skeleton completely. In the worst case, our achievable scheme is an ↵-approximation algorithm where ↵ is the independence number of the graph. We also show that there exist graph classes for which the sufficient number of experiments is close to the lower bound. In the other extreme, there are graph classes for which the required number of experiments is multiplicatively ↵ away from our lower bound. In simulations, our algorithm almost always performs very close to the lower bound, while the approach based on separating systems for complete graphs is significantly worse for random chordal graphs.

This paper addresses the problem of learning causal networks with interventions, when each intervention is limited to size k. The paper is generally well-written and addresses a relevant question, as it is generally not feasible to learn the causal structure from observational data alone. Moreover, in some cases, it may also not be possible to perform arbitrarily large interventions, but it is possible to perform interventions over smaller subsets of the variables. The authors prove a number of results around the number of interventions required to learn complete and chordal graphs, and, while I was not able to check all the proofs in detail, the results are as expected (and appear to be correct).The results on chordal graphs are applicable to general causal structures in the sense that application of conditional independence learning and Meek rules results in a chain graph with chordal chain components.In addition, the authors implement a novel adaptive deterministic algorithm and test it on randomly generated chordal graphs.The simulations show that the algorithm performs close to the lower bound for worst case chordal graphs using adaptive or non-adaptive algorithms.Additionally, it significantly outperforms the naive separating systems approach for complete graphs. Overall, this is a strong paper that should be accepted. I list below the specific contributions. - The paper gives a novel separating system construction, where the size is larger than that of Wegener (1979) but has a simpler description. This construction is later used for the results on adaptive and non-adaptive algorithms for a complete graph.- It had previously been shown that any non-adaptive algorithm needs a (n, k) separating system to learn a complete graph. The authors show that the same is true for any adaptive deterministic algorithm.- The authors show that to learn a complete causal DAG on n variables using k-size interventions, n/2k interventions are necessary. This bound holds for any algorithm.- The authors give a straightforward extension of a result by Hu et al. (2014), which showed that randomized adaptive algorithms need only log log n interventions with high probability for the unbounded case for complete graphs. The authors show that O(n/k log log k) interventions suffice with high probability when the interventions are bounded by k in size.- The authors extend the lower bound for adaptive algorithms for general chordal graphs and show that the number of experiments from a ((G), k) separating system is needed where (G) is the chromatic number of the skeleton graph.- The authors give two extreme examples; one for which the number of interventions is close to the lower bound is sufficient and the other for which the number of interventions needed is close to the upper bound.- The authors give a novel adaptive deterministic algorithm that uses the idea of separating systems together with adaptability to Meek rules. They test their algorithm on randomly generated chordal graphs and observe that it performs close to the (, k) separating system and significantly outperforms the (n, k) separating system (for complete graphs).** Additional Comments: - The "Meek rules" described in Section 2.2 were introduced by Verma and Pearl (1992) and Meek (1995) proved certain properties of them. You should cite Verma92.- Top of pg. 5, replace "also provided a tighter upper bound in [15]" with "also provided a tighter upper bound than the one in [15] -On pg. 8, the "Information Theoretic LB" is described as /2k.Where is this bound given? References: Verma, Thomas, and Pearl, Judea. "An algorithm for deciding if a set of observed independencies has a causal explanation." Proceedings of the Eighth international conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc., 1992.


Learning both Weights and Connections for Efficient Neural Networks
Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems. Also, conventional networks fix the architecture before training starts; as a result, training cannot improve the architecture. To address these limitations, we describe a method to reduce the storage and computation required by neural networks by an order of magnitude without affecting their accuracy by learning only the important connections. Our method prunes redundant connections using a three-step method. First, we train the network to learn which connections are important. Next, we prune the unimportant connections. Finally, we retrain the network to fine tune the weights of the remaining connections. On the ImageNet dataset, our method reduced the number of parameters of AlexNet by a factor of 9×, from 61 million to 6.7 million, without incurring accuracy loss. Similar experiments with VGG-16 found that the total number of parameters can be reduced by 13×, from 138 million to 10.3 million, again with no loss of accuracy.

It is good that the authors study reasonably modern and well-known (AlexNet and VGG) deep nets, so that the compression ratios seem directly relevant to recent CNN work. However, the authors do not seem to compare their work to ANY of the now-large literative on related recent methods for compressing CNN and other deep nets. They dismiss OBD in Section 2 as unsuitable for "today's large scale neural networks", but that seems unfair since OBD's saliencies only require the Hessian diagonals, and several modern methods to approximate the required Hessian diagonals could be used for baselines.But even without using ODB-related methods, there are many other baselines they could of and should of reported for comparison, including methods that reduce weights to fewer bits or approximate the CNN filter matrices using cheap SVD-based compression, or ones which approximate the fully connected layers using randomized projection (e.g. FastFood methods), and so on.The authors claim in Section 2 that such work is often "orthogonal to network pruning", but that seems to miss the point: once those methods are used, is there any real advantage to the proposed method?Importantly, some of those methods are in practice much easier to realize promised speedups (e.g. doing more multiplies in parallel using weights of fewer bits) in the dominate dense computations (i.e. BLAS ops) than the proposed sparse method (see below for more on this point).      The authors are not very clear on how (and when) their approach is necessarily efficient -- Section 4.2 mentions "900k" and "700k" iterations ... relative to AlexNet "450k".Presumably this means the authors are claiming their approach requires only 2x as long to train as AlexNet?Unfortunately they are not clear because they never report wall clock in seconds.The reason that is especially critical is that they later claim great "speedups" when their nets have 5-9x few weights.However, fewer FLOPS does not directly translate to similar drops in seconds, as is well-known and carefully mentioned in much of the other work on compressing nets.Specifically, the cache-aware and multicore efficiency of modern BLAS operations (both for CPU and GPU) makes dense matrix operations very competitive to sparse operations until sparsity is much lower than 10%.So it is not surprising that the authors never report runtime seconds -- they would almost surely NOT have noticed any actual runtime speed improvements with their approach.      This paper is also unclear on several other issues, including how (and why) they decided to reduce the learning rate (by 1/100 sometimes and by 1/10 for other experiments).If this requires a lot of experimentation, then that search time needs to be included in the cost of this compression process, making this approach even less competitive.           This paper basically tries the simplest method to sparsify a net: drop small weights and retrain.It reports impressive-sounding compression ratios (e.g. 5-9x) and thus claims victory.Finding the simplest ideas that work is in general good and should be encouraged, but the actual significance of the results and carefully describing necessary conditions required to achieve them (including tweaking the learning rates) becomes critical.Too much is left out of this paper, incluidng what stopping condition they used for the retraining process (e.g. how did they properly determine that "900k iterations" was enough (and fair), in Section 4.2?).Their reporting of only FLOPS, instead of seconds as well, makes it very clear that the authors are not very aware of the rich related literature on compressing networks, and the difficulty of actually achieving significant compressions in practice (due to the efficiency of modern dense-matrix operations over sparse ones).      In short, it is good to see a paper try to get away with the simplest approach, and show an existence proof that more of the weights in some modern CNNs could be zero and still get the same accuracy.This might have more modest appplications, such as storing networks on disk (sparsely) and then running them densely (filling in zeros explicitly for the net in RAM).But the authors seem to want and try to claim more significance than that, but without comparing to any other relevant compression methods. As stated this work is too prelimary, as the authors even basically admit in Section 4.3 about VGG ("we have not yet had a chance to prune the convolutional layers".Its limitatons and tradeoffs are not discussed, nor explored in relation to any other existing methods.


Learnability of Influence in Networks
We show PAC learnability of influence functions for three common influence models, namely, the Linear Threshold (LT), Independent Cascade (IC) and Voter models, and present concrete sample complexity results in each case. Our results for the LT model are based on interesting connections with neural networks; those for the IC model are based an interpretation of the influence function as an expectation over random draw of a subgraph and use covering number arguments; and those for the Voter model are based on a reduction to linear regression. We show these results for the case in which the cascades are only partially observed and we do not see the time steps in which a node has been influenced. We also provide efficient polynomial time learning algorithms for a setting with full observation, i.e. where the cascades also contain the time steps in which nodes are influenced.

The paper studies PAC learnability of three commonly used network influence models, namely linear threshold model, independent cascade model and voter model. The authors study both the fully observed model where the time steps of the influence events are included, and the partial observation model, where only the set of influenced nodes is given. The authors are able to show PAC learnability of the influence models in all the above cases, and polynomial time optimisation for the fully observed linear threshold and independent cascade models and both fully and partial observed voter model. Quality: I think the paper is of a good quality, in sofar I have been able to judge. I could not perform detailed check of the proofs in the supplementary material.Clarity:The paper is readable after some effort. There are some notation and presentation issues which I list below.Originality: The idea of providing the PAC analysis is original and welcomed. The proofrequire considerable insight to come up with as different setups are required (neural network, Vcdim for LT model, covering numbers, log likelihood for IC, regression for voter model), albeit the proof as such use standard tools.Significance: The results are significant, because they provide the learning theoretic justification for the widely studies influence function models. It is also foreseeable that the work can lead to new approaches in studying influence in complex networks. Is it for example possible to analyse the context-sensitive case (Su et al. ICML 2014, pp. 442-450) where the influence depends on the type of stimulus, not just the topology and the initial set?Details:- line 49-53:I don't get the purpose of the footnote. In the following you show that the models are PAC-learnable. Here, you wish to imply that they are hard to learn? You do not return to this kind of analysis later to this footnote is not connected to the rest of the paper. I would use the limited space allowed by NIPS by making your main arguments clear(er). - line 61-76: I am not sure this summary is needed, the contributions are clear, perhaps the space would be better used later- line 69: independent CASCADE model - line 167: You are overloading the subscript of set Y: here Y_t denotes time points, below Y_u denotes a node. This confused me a lot. I would be of great help to the reader if you used some other notation for one of the cases. - line 198: I don't think the subscripts _u should be there on the left-hand side of the equation (or, then I really don't understand) - line 245-246: you ... show ... provably ...=> you prove - line 251: subscripts _u on the left hand side again... - line 253: why does \bar{Y}_u^i need to defined through the union of time points.Isn't it so hat the last time point contains all actives nodes so the union should always equal Y_n^i? Or perhaps I miss something. - line 270: I am not completely satisfied with the description. There is not literature reference to the computational hardness and you do not hint how this is established. Why do sigmoidal activations and back-propagation help in resolving the time-points?- line 416: I would appreciate a literature reference to the standard results


Scalable Inference for Gaussian Process Models with Black-Box Likelihoods
We propose a sparse method for scalable automated variational inference (AVI) in a large class of models with Gaussian process (GP) priors, multiple latent functions, multiple outputs and non-linear likelihoods. Our approach maintains the statistical efficiency property of the original AVI method, requiring only expectations over univariate Gaussian distributions to approximate the posterior with a mixture of Gaussians. Experiments on small datasets for various problems including regression, classification, Log Gaussian Cox processes, and warped GPs show that our method can perform as well as the full method under high sparsity levels. On larger experiments using the MNIST and the SARCOS datasets we show that our method can provide superior performance to previously published scalable approaches that have been handcrafted to specific likelihood models.

This paper takes the pre-existing Gaussian Process inference method of "Automatic Variational Inference" and adapts it for scaling to larger datasets. Specifically, it draws on the method of inducing points to develop a new variational bound that factorises across the data, enabling the use of stochastic optimisation. It uses a mixture of Gaussians to generate approximate posteriors, without being tailored to a specific likelihood model.I think this article represents a very good contribution to the field: previous work on non-Gaussian likelihoods is generally either an expensive Monte Carlo method, likelihood-specific, or stuck with O(n^3) computational bound. This work represents a method to perform scalable and accurate inference for generic likelihood models: a previously unachieved target that this work makes good progress towards.The only strong objections I have to the presentation of the work is the representation of experimental results. There is no comparison to competing methods: it would be greatly of interest to compare against "Scalable variational Gaussian process classification." for binary classification, although for other likelihoods there are no clearly competing methods at this scale, so their absence is fair. There is also no timing data: without indication of how quickly the method takes to work, readers will not be able to make a judgement of the accuracy/time tradeoff involved. Ideally, there would be a figure plotting average test error against training time, so the model's learning behaviour can be elucidated. This should be achievable, however, by slight modification to existing code.Some comments working through the paper:The introduction is good: the preceding work on Automatic Variational Inference is presented well, as is the method of inducing variables.Introduction of uncoupled latent Gaussian processes is presented sensibly, along with iid likelihood properties.The scalability modifications to AVI, drawing on inducing point framework is presented well. Calculation of the of each term is presented well and very thoroughly, to a degree might make as much sense in a supplement.The introduction of stochastic optimisation is interesting, and could possibly be made more of: unbiased subsampling of data is a hugely useful property for large datasets, and competitors to existing SVI methods would be a good contribution to the literature.The experimental section is impressive, containing some results that are unprecedented in the literature. As mentioned above, it would greatly benefit from the presentation of timing results, and with comparison against existing methods for scalable Gaussian Process Binary Classification.A separate conclusion would be valuable to reiterate the main contributions of the work, and the related work section could possibly be moved to earlier in the article.The language is consistently clear, and the references are well-presented. The supplementary material is accurate and complete.----- after author rebuttal -----After the authors' rebuttal, I would be happy to see this paper accepted, if the points raised are fulfilled, i.e. comparison to SVIGPC, timing data and a full conclusion.


Accelerated Mirror Descent in Continuous and Discrete Time
We study accelerated mirror descent dynamics in continuous and discrete time. Combining the original continuous-time motivation of mirror descent with a recent ODE interpretation of Nesterov's accelerated method, we propose a family of continuous-time descent dynamics for convex functions with Lipschitz gradients, such that the solution trajectories converge to the optimum at a O(1/t 2 ) rate. We then show that a large family of first-order accelerated methods can be obtained as a discretization of the ODE, and these methods converge at a O(1/k 2 ) rate. This connection between accelerated mirror descent and the ODE provides an intuitive approach to the design and analysis of accelerated first-order algorithms.

The paper combines the ODE interpretation of mirror descent with the work of [25], which interprets AGD as a discretized ODE, to formulate an ODE system that corresponds to what should be accelerated mirror descent. Proofs of existence, uniqueness, and convergence follow standard ODE methods using compactness and contraction mappings. Discretizing the ODE yields the derivation of discrete-time accelerated mirror descent methods, one example of which is accelerated entropic descent. The authors provide experiments validating the improvement of accelerated entropic descent over its non-accelerated variant.     Quality: The paper is relatively well-written, outlining recent work in the field, motivating its contribution, and methodically arriving at concrete algorithms at the end.Clarity: The authors mention and leverage the Cauchy-Lipschitz theorem many times throughout the paper, but not once provide an exact statement or a reference. Since the theorem is sometimes presented in different forms, it seems important to cite it precisely. It is not clear why the 2nd inequality on line 251 is true. It is also not clear why the 2nd inequality in the proof of Lemma 2 on page 5 of the supplementary material is true. [15] is cited improperly without any journal or conference reference.EDIT: The authors addressed many of these complaints in their rebuttal.Originality: The formulation of accelerated mirror descent is a natural extension of recent work, but to the best of the reviewer's knowledge, new. Significance: The results are new, but the proof techniques and ideas are relatively standard extensions to what exists in the literature. The experiments seem to indicate that the algorithms present an improvement upon commonly used methods.


Combinatorial Cascading Bandits
We propose combinatorial cascading bandits, a class of partial monitoring problems where at each step a learning agent chooses a tuple of ground items subject to constraints and receives a reward if and only if the weights of all chosen items are one. The weights of the items are binary, stochastic, and drawn independently of each other. The agent observes the index of the first chosen item whose weight is zero. This observation model arises in network routing, for instance, where the learning agent may only observe the first link in the routing path which is down, and blocks the path. We propose a UCB-like algorithm for solving our problems, CombCascade; and prove gap-dependent and gap-free upper bounds on its n-step regret. Our proofs build on recent work in stochastic combinatorial semi-bandits but also address two novel challenges of our setting, a non-linear reward function and partial observability. We evaluate CombCascade on two real-world problems and show that it performs well even when our modeling assumptions are violated. We also demonstrate that our setting requires a new learning algorithm.

The authors investigate a class of combinatorial cascading bandit problems. The archetypal problem in this class seems to be that of routing packets in a network where edges are subject to random independent failures. I will restrict to this problem to simplify the description (though the authors also discuss the disjunctive version of this reward function). At each time step the agent chooses a path and receives unit reward only if none of the edges along the path failed. If some edge failed the agent gets a reward of zero and is also given the feedback of which was the first edge that failed. The algorithm they design for this problem called CombCascade, essentially builds UCB estimates for edge failure probability and picks the path with highest expected non-failure probability. They prove upper bounds for the regret of their algorithm one that is gap-dependent and one that is not. They argue that the upper bound matching the previously known tight upper bound for the stochastic semi-bandit problem (which is a more informative feedback setting) perhaps indicates that their algorithm is tight in settings that are "close" to the semi-bandit setting. The paper concludes with some experiments on synthetic and real datasets that show sub-linear growth of regret for their algorithm and they show that it performs better than CombUCB1 which is a stochastic combinatorial semi-bandit algorithm. There are a couple of limitations of the study. One, as the authors also point out, is the rather strong independent failure assumption of the graph edges. [Aside: I appreciate the authors' honest comments pointing out the strongest and weakest parts of their paper, I wish more papers did that, not so much to simplify the referee's task, but to stimulate any reader of the paper.] Secondly, their work points to tantalizing connections to the semi-bandit setting, but this is not clarified well. This is perhaps food for future work. For instance, the weight estimation stage of their algorithm hints at a possible reduction between the two settings. On the other hand, the presence of interesting practical problems in their setting argues for acceptance of the paper despite these issues.


Attention-Based Models for Speech Recognition
Recurrent sequence generators conditioned on input data through an attention mechanism have recently shown very good performance on a range of tasks including machine translation, handwriting synthesis [1, 2] and image caption generation [3]. We extend the attention-mechanism with features needed for speech recognition. We show that while an adaptation of the model used for machine translation in [2] reaches a competitive 18.7% phoneme error rate (PER) on the TIMIT phoneme recognition task, it can only be applied to utterances which are roughly as long as the ones it was trained on. We offer a qualitative explanation of this failure and propose a novel and generic method of adding location-awareness to the attention mechanism to alleviate this issue. The new method yields a model that is robust to long inputs and achieves 18% PER in single utterances and 20% in 10-times longer (repeated) utterances. Finally, we propose a change to the attention mechanism that prevents it from concentrating too much on single frames, which further reduces PER to 17.6% level.

The paper applies attention-based models to speech recognition. The model is extended by considering location awareness, sharpening and smoothing of the scores, and windowing. These extensions overcome the long sentence issues in attention-based models, and achieved the improvement from the straightforward applications of the attention-based models.Quality: This paper describes an original attention-based model, and mathematically explains the extension of a location-awareness score function in Eqs. (5) - (9). The other extensions (sharpening and smoothing) are also mathematically described by considering the long utterance issues. Therefore, the paper has sufficient quality in terms of mathematical/theoretical formulation.Clarity: The paper clearly describes an attention-based model, issues of long sentences, and analysis of experimental discussions. Originality: Although the novelty of this paper is incremental, each step of the incremental extensions is reasonably supported by the experimental results and analysis.Significance: Since the task is relatively small (TIMIT phone recognition), the experimental result is not so significant. Although the proposed method uses a window technique, introducing \alpha requires long-range computations in training and decoding, and the proposed method does not seem to be scalable to large-scale (practical) speech recognition tasks.I summarize the overall Pros. and Cons. as follows: Pros: Attention-based novel architecture provides end-to-end speech recognition. Cons: Scalability.Minor comment: 1. P.2, Section 2: L and T must be explicitly defined.


Human Memory Search as Initial-Visit Emitting Random Walk
Imagine a random walk that outputs a state only when visiting it for the first time. The observed output is therefore a repeat-censored version of the underlying walk, and consists of a permutation of the states or a prefix of it. We call this model initial-visit emitting random walk (INVITE). Prior work has shown that the random walks with such a repeat-censoring mechanism explain well human behavior in memory search tasks, which is of great interest in both the study of human cognition and various clinical applications. However, parameter estimation in IN-VITE is challenging, because naive likelihood computation by marginalizing over infinitely many hidden random walk trajectories is intractable. In this paper, we propose the first efficient maximum likelihood estimate (MLE) for INVITE by decomposing the censored output into a series of absorbing random walks. We also prove theoretical properties of the MLE including identifiability and consistency. We show that INVITE outperforms several existing methods on real-world human response data from memory search tasks.

An initial-visit emitting random walk is a random walk that emits an output only when visiting a state for the first time. Previous researchers have used these walks to model human behavior data on memory search tasks (e.g., name as many animals as you can). The contributions of this paper are that it proposes a new and efficient way of computing a likelihood function for the model, it proves that the resulting maximum likelihood estimates are consistent, it presents a way of estimating parameter values using a regularized likelihood function, and it shows, via computer simulation, that the proposed parameter estimation procedure performs better than alternatives on a toy task and on real-world data collected (by others) in experiments with people.I should start by stating that this paper relies on areas of mathematics that I know nothing about.That said, I have a mixed opinion of this paper. On the one hand, the proposed way of computing a likelihood for the model seems extremely clever and a clear technical advance over previous efforts. In addition, the paper is very well-written. On the other hand, the paper is likely to appeal to a very small segment of the NIPS community. Initial-visit emitting random walks would seem to have limited applicability. Indeed, to my knowledge, they have only been studied by psychologists with interests in mathematical models of semantic memory search (a surprisingly small sub-community). In addition, the point of using these random walks is that they reveal something interesting about human memory search. However, although the current paper provides an improvement in fitting parameter values for these walks, it does not demonstrate any payoff in terms of revealing new information about human memory search. That is, the paper demonstrates improved parameter estimation but no payoff from these improved parameter estimates.But, but, but...perhaps I'm being too harsh. Maybe someone else will be able to use the parameter estimation procedure proposed here to learn something new about human memory search. I hope so.


Non-convex Statistical Optimization for Sparse Tensor Graphical Model
We consider the estimation of sparse graphical models that characterize the dependency structure of high-dimensional tensor-valued data. To facilitate the estimation of the precision matrix corresponding to each way of the tensor, we assume the data follow a tensor normal distribution whose covariance has a Kronecker product structure. The penalized maximum likelihood estimation of this model involves minimizing a non-convex objective function. In spite of the non-convexity of this estimation problem, we prove that an alternating minimization algorithm, which iteratively estimates each sparse precision matrix while fixing the others, attains an estimator with the optimal statistical rate of convergence as well as consistent graph recovery. Notably, such an estimator achieves estimation consistency with only one tensor sample, which is unobserved in previous work. Our theoretical results are backed by thorough numerical studies.

after rebuttal: As the irrelevance of the initialization is one of the main contributions of this paper, it is strange to me that the authors adopted the same initialization as previous papers. This really raises a big concern on the work.============ In my opinion, this is a technically sound paper with nontrivial results. Following a recent thread of research on alternating minimization applied to different statistical problems, the authors established interesting results on recovering precision matrices from tensor normal distributions, and hence can be viewed as another manifestation of the power of alternating minimization to statistical problems. Theorem 3.1, though the simplest result in this paper, presents the core idea and hence is perhaps the most important observation in this paper for followup researches. The proofs of other theorems are more technical, but are clear enough for me to follow, and contain no unfixable problems. The proof of the last theorem is missing. The following are some errors or drawbacks I found in this paper:(1) The authors claim that one sample suffices to achieve strong statistical guarantees, but the current proof does not support such strong claim: In line 616 of Supplementary Material, the Frobenius norm of the error is only bounded for sufficiently large n.This problem may be avoided by considering fixed n and growing dimensions such that the the radius in the definition of $\mathbb{A}$ vanishes. For example, the scaling law considered in line 294 of the main text satisfies the above requirement, and therefore the conclusion is still valid. Only some parts of the proof need to be slightly modified.(2) In Supplementary Material there are some small errors in proofs: 2.1) Line 607, what is the "boundary" of $\mathbb{A}$? $\mathbb{A}$ is already a sphere-like set, so I see no point talking about the boundary of $\mathbb{A}$.2.2) Line 661, a typo in the second term of the first equality.2.3) Line 669, I don't see how the inequality here is applied. What I understood is to simply use sub-multiplicativity of the Frobenius norm and 667 follows.2.4) Line 699, "primal"-dual witness.2.5) In the proof of Lemma B.1, the constant  is not explicitly defined. I would suggest to include the definition of  in the statement of the lemma, so as to highlight the fact that the bound in Lemma B.1 depends on || - *||F .2.6) In the main text the authors state that their proof relies on Talagrand's concentration inequality. Where is the inequality used? Lemma C.2 is not Talagrand's concentration inequality; Gaussian concentration is a classical result that can be simply obtained through, for example, Gaussian log-Sobolev inequality.(3) Line 303 in the main text, the authors state that their bound is minimax optimal. Minimax with respect to what class?(4) Line 323, the Hessian := *-1 ? *-1?


Closed-form Estimators for High-dimensional Generalized Linear Models
We propose a class of closed-form estimators for GLMs under high-dimensional sampling regimes. Our class of estimators is based on deriving closed-form variants of the vanilla unregularized MLE but which are (a) well-defined even under high-dimensional settings, and (b) available in closed-form. We then perform thresholding operations on this MLE variant to obtain our class of estimators. We derive a unified statistical analysis of our class of estimators, and show that it enjoys strong statistical guarantees in both parameter error as well as variable selection, that surprisingly match those of the more complex regularized GLM MLEs, even while our closed-form estimators are computationally much simpler. We derive instantiations of our class of closed-form estimators, as well as corollaries of our general theorem, for the special cases of logistic, exponential and Poisson regression models. We corroborate the surprising statistical and computational performance of our class of estimators via extensive simulations.

- Summary of Paper   - The paper approaches the problem of estimating parameters in GLMs     when the dimension of the data is larger than the number of     samples. The authors claim to have a closed-form solution for     such a setting based on thresholding and projection operations.     The closed-form estimators enjoy some favorable frequentist     statistical properties.Finally, the authors compare their     estimates to l1 regularized MLE estimates for two simulated data     sets. - Quality   - I have questions (L120 comments below) about the work that trace     back to the way the authors formulated the exponential family     form of logistic regression and poisson regression. This issue     impacts how the authors define the subset of the interior of the     mean-response polytope and therefore the evidence provided in the     results section.   - L120: The authors' definition of the logistic regression is     confusing and I'm not sure if it is correct. I might write the     exponential family as $p(y_i | theta, x) = exp( y__i * \eta -     A(\eta)$, where $\eta = log(\mu/(1-\mu))$ is the log-odds, the     log-partition function is $A(\eta) = log(1+\exp(\eta)$, and $\mu     = (1+\exp(\theta*x))^{-1}$ is the logistic function. This     formulation is very different than the one presented: $\exp{y*z -     \log(\exp{z}+\exp{-z}}$ where z = < \theta, x > is their natural     parameter. In particular the stated log partition function is     A(a) = log(exp(-a) + exp(a)). We could write that as A(a) =     log(exp(-a^2)). How did the authors arrive at the stated form for     the logistic regression exponential family density function?   - L120: I am similarly confused by how the authors arrived at their     exponential family formulation for the poisson regression     density. How did the authors arrive at this formulation and can     they show that it is certainly correct?   - L296: The theorem seems to provide for not missing an important     predictor with high probability, but we would also like to make     sure to set coefficient to zero that should be. Is this theorem     one-sided in that it only provides for not missing a non-zero     coefficient?   - L237: The article develops reasons for truncating (X^T * X) and     for the projection on the subset of the polytope in the     estimator. But the reason for the soft-thresholdS_{\lambda_n}     is not clear. This should be clarified.   - L296: The theorem shows that the estimator includes all non-zero     supports with a probability bound. Is this bound non-trivial in     any specific instances? - Clarity   - The paper is fairly clear, though dense. A few edits outlined     below could make it more clear.   - L041: The introductory paragraph starts with talking about     estimation and ends with talking about prediction. While these     concepts are related, it would help to be consisent so early in     the manuscript.   - L046: References would help after "and others"   - L106: The notation for the natural parameter in the exponential     family model < \theta, x > is non-standard and confusing. The     bracket notation is used by some to denote dot-product and other     to denote expected value. A more familar notation or a     justification for this particular notation would help the article     clarity.   - L118: The use of "A(a)" is confusing and "a" could be changed for     the authors' notation for the natural parameter < \theta, x >.   - L190: The article seems to rely heavily on \hat{M} the subset of     the interior of the response moment polytope. But we don't get to     see what that is until we get to specific models. It would help     to remind the reader that \hat{M} will be defined for specific     models more frequently.   - L194: The estimator is stated without development. Either put the     estimator after the development or state that development will     follow. The "machinery" is not described, it is the notation that     is described earlier. The "machinery" seems to be described     later. The reason for "Elem" subscript on \theta is not described.   - L322: Remark 3 refers to "k". I could not find what "k" is     earlier in the paper. - Originality   - The paper builds on some previous work on linear regression, but     the extension to the GLM setting is novel. - Significance   - As the authors state, even the MLE in a low-dimensional setting p     < n does not have a closed-form solution. So, the development of     a GLM estimator that is both closed form and valid in high     dimensional settings is highly significant.