Improved Iteration Complexity Bounds of Cyclic Block Coordinate Descent for Convex Problems
The iteration complexity of the block-coordinate descent (BCD) type algorithm has been under extensive investigation. It was recently shown that for convex problems the classical cyclic BCGD (block coordinate gradient descent) achieves an O(1/r) complexity (r is the number of passes of all blocks). However, such bounds are at least linearly depend on K (the number of variable blocks), and are at least K times worse than those of the gradient descent (GD) and proximal gradient (PG) methods. In this paper, we close such theoretical performance gap between cyclic BCD and GD/PG. First we show that for a family of quadratic nonsmooth problems, the complexity bounds for cyclic Block Coordinate Proximal Gradient (BCPG), a popular variant of BCD, can match those of the GD/PG in terms of dependency on K (up to a log 2 (K) factor). Second, we establish an improved complexity bound for Coordinate Gradient Descent (CGD) for general convex problems which can match that of GD in certain scenarios. Our bounds are sharper than the known bounds as they are always at least K times worse than GD. Our analyses do not depend on the update order of block variables inside each cycle, thus our results also apply to BCD methods with random permutation (random sampling without replacement, another popular variant).

Report for "Improved Iteration Complexity Bounds of Cyclic Block Coordinate Descent for Convex Problems".## SummaryThe authors consider methods of block coordinate types with cyclic block selection rules. The focus is on a composite model involving a non-separable smooth term and a separable non-smooth term. The main methods of interest are block proximal gradient and cyclic minimization. The authors propose a refined analysis of these methods for problems for which the smooth term is quadratic. This leads to new convergence rate estimates for this class of problems. Finally an alternative analysis is presented for the general case with the restriction that blocks should have size one (coordinate proximal gradient). ## Main commentsThe analysis of sections 2 is original, relevant and quite impressive. In some settings, the complexity estimates match those of the usual proximal gradient method which was not known to be possible for such a general class of problems. The analysis removes a multiplicative dependance in the number of blocks compared to previous work. This constitutes a very strong theoretical contribution. Section 3 is also very interesting, but I have one concern with the proof of Theorem 3.1 (see next paragraph). The results of section 4 are a bit less spectacular but still of great interest. Overall, the proof arguments look reasonable and are quite original. As the questions adressed in this paper are strongly related to both machine learning and optimization, I warmly recommend this work for publication after taking into account the following remark:In proof of Theorem 3.1 at thebottom of page 15, I think that the last step (i) is wrong. It is mentioned that (i) can be derived from (49), but I am not sure it is true since Lmin is not the minimal eigenvalue of the matrix of interest in (49), but the minimum of the maximum eigenvalues of its diagonal blocks. I think it is possible to fix this by introducing a factor K, using | | sum b _ i | | ^ 2 leq K sum | | b _ i | | ^2It could also be possible to fix it by using the idea that exact minimization makes more progress than a proximal gradient step, and using arguments from Lemma 2.1, but I am not completely sure about it.In any case, if the authors agree with this remark, important changes have to be made in the proofs and/or in the main text. ## Additional comments and suggestions- An important variant of cyclic block decomposition methods is the random permutation approach where the order of blocks changes at each itteration. In some applications, this additional randomization gives a practical advantage. Most of results usually given for cyclic block decomposition methods still hold deterministically when the order of the blocks changes arbitrarily between iterations. I suspect that this is also the case for the proposed analysis which, if true, would be worth mentioning.- In section 4, the assumption that N=1 is not only for notational simplicity. The analysis falls appart when N > 1. Indeed, the analysis uses the mean valued-theorem in equation (78) which does not hold for vector valued functions. Therefore the analysis cannot be straightforwardly extended to blocks of size more than one without further arguments.- The quantity defined in (3) might very well be infinite, unless compactness of sublevel sets is assumed as in [3] for example. This could be precised.- Notation in (2) constitutes a slight abuse as the order of the arguments of g is not the same as in (1). Without needing to change notations, it could be a good idea to give precisions.- I do not think there is a result as (6) in [3] which is concerned mostly about BCGD and not BCD (only for the two block case). Maybe also give a precise pointer in reference [6].- In (8), all the blocks are assumed to have the same size. I guess the analysis can be extended to cases for which blocks have different sizes. Maybe emphasize that this is not an assumption made for restriction, but for simplicity.- In (7), the constant is independant of problem dimension for fixed ||x^0 - x^*|| and L. From a practical perspective, these values usually depend on the dimension of the problem.- (11) is true, although not very common. Maybe add a sentence explaining that both (9) and (11) are equivalent to the same subdifferential inclusion.- Kroneker - > Kronecker. Maybe precise the notation. I suspect that in the notations of the paper, the order of matrices involved in Kronecker products is reversed compared to other sources (e.g. wikipedia).- Line 661 and 688 in the appendix refer to optimality condition of w_{k+1}^{r+1}. I guess it should be optimality of x^(r)_{k}. Furthermore, this optimality condition could be written explicitely as (11) is for example.- Just before Thorem 4.1, it is mentioned that the results can be established "in the same way as the standard analysis of GD". It would be nicer to have the precise statement, and at least a reference (e.g. [Theorem 2.1.13 10]).- Quantification over r could be more precise in Theorem 4.1 and Corollary 4.1, in particylar, the estimates presented are not well defined for r = 0.- In the discussion of section 4, I think that another relevant feature of the newly proposed bound is that it does not increase with the artificial addition of new variables on which the problem does not depend (with Li = 0). This is not the case for the quantity KL which is problematic since such an artificial introduction does not modify the iterates an we therefore anticipate that it should not modify the complexity estimates.- In the abstract, it is worth precising that the iteration counter is the number of times all blocks have been updated. It would be OK to have a dependance in K if the counter consisted in the number of blocks updated as in Random block methods.- Page 5, line 228, I guess "same dependency on K" should be "same dependency on L".- Remark 3.1 "those derive here" - > "those derived here".- Toplitz - > Toeplitz.- Although transparent, the notations Pmax and Pmin are never defined.- In Lemma 4.1, the logical quantification over xi should be more precise. I think that the correct quantifier is "there exists"- In (36), I guess there are missing indices for A(:,i). This is matlab notation, maybe precise what it means.- In theproof of Lemma 2.1, there are missing indices k in $h_k(x^r)$ on line 504- In (45), there is a use of (11) on the fifth line which could be precised.- Line 637 "the fact" - > "and the fact"- Line 702 "Combining the above to inequalities". There are one inequality and one equality. "to" - > "two".- Section 13 should be "Proof of Corrolary 4.1"?


Active Learning from Weak and Strong Labelers
An active learner is given a hypothesis class, a large set of unlabeled examples and the ability to interactively query labels to an oracle of a subset of these examples; the goal of the learner is to learn a hypothesis in the class that fits the data well by making as few label queries as possible.
This work addresses active learning with labels obtained from strong and weak labelers, where in addition to the standard active learning setting, we have an extra weak labeler which may occasionally provide incorrect labels. An example is learning to classify medical images where either expensive labels may be obtained from a physician (oracle or strong labeler), or cheaper but occasionally incorrect labels may be obtained from a medical resident (weak labeler). Our goal is to learn a classifier with low error on data labeled by the oracle, while using the weak labeler to reduce the number of label queries made to this labeler. We provide an active learning algorithm for this setting, establish its statistical consistency, and analyze its label complexity to characterize when it can provide label savings over using the strong labeler alone.

1 SUMMARY OF THE PAPERIn this submission, the authors propose to tackle the problem of Active Learning from weak and strong labelers. That is, active learning when the labels can be queried from two different sources. One, the Strong Oracle (noted O in the paper), is costly but reliable while the other, the Weak Oracle (noted W) is marginally cheap but also locally unreliable. The idea behind this is to minimize the use of O by relying on the labels from W as much as possible.The authors present a new algorithm to learn in this setting. Their solution takes its root on the Disagreement-Based-Active-Learning (DBAL) literature which advocate to only query labels that are in an 'uncertainty region' on the input space. On top of that, the authors propose to learn a 'difference classifier' to determine whether or not O and W return different labels. Hence, for a given datapoint $x$ O is only queried if the two following conditions are met : 1) $x$ is in the disagreement region and 2) the 'difference classifier' predict that W may not return the same label than O.The main difficulty in this setting seems to come from the fact that the 'difference classifier' might produce false negative (FN) error --that is, when it wrongly predicts agreement between W and O. The proposed solution for this problem is to enforce a maximal FN rate with a cost-sensitive learning procedure for the difference classifier.The emphasis of the submission is put on keeping the setting as general as possible and this is clearly one of its strong point. The proposed algorithm is extremely general and can be used with any class of classifier. Nevertheless, the authors are able to obtain theoretical guaranteeson the number of query to O. Moreover, those results apply to the very general agnostic PAC setting.In addition to this, the authors propose a case study for their algorithm in the context of linear classification with uniformly distributed datapoints. They provide extended results for this special case based on their previous theoretical bound on the number of query to O.Finally, they also quickly discuss the case of learning with data labeled by a mixture of O and W and provide corollary results for this setting. 2 QUALITYFrom a scientific point of view, the submission is good and interesting. The generality of the results is remarkable and, to quote the authors, this represent "a step towards a theoretical understanding of active learning from multiple annotators". The proofs presented in the appendices are very thorough and with a great level of detail and every piece of result is fully and properly tackled. Some results are far from trivial and show a very good understanding of the setting and concepts involved in the proofs.However, I have major concerns about the exposition of this work.First, the paper seems extremely rushed: there is a plethora of typos, the notations seem also not completely though out, especially in the appendices and the whole dynamic of the paper feels clunky at times. While this clearly impairs the quality of this paper, it is mainly a matter of clarity and therefore I will defer more detailed arguments in the next section. Also, a comprehensive list of typos and style suggestion is given at the end of this review. My second concern is that a fair amount of the interesting stuff is actually in the appendices. A whole subsection of the proposed algorithm is deferredto them although it is clearly non-trivial, new, and pivotal to the whole learning procedure. Moreover, the submission only contains the major results with no proof. I understand that the proofs are very long and convoluted but a very rough sketch of them would help the reader understand the leading idea. Ironically, this is actually done at the beginning of the major proofs in the appendices. The same remark applies to some intermediate lemmas and theorems that are, in my opinion, core to the comprehension of the submission but only mentioned in the appendices. Overall the paper is interesting and seems mathematically solid but it feels very rushed and trimmed to fit in the 9 pages limit of the conference. Because of that, and despite its undeniable scientific quality, I do not think that this submission meet the quality standards of the conference. For me it is a typical case of a good and very interesting journal submission that is way too dense, in its present state, for a conference paper.  3 CLARITYThere are several things in this submission that, once added, greatly impair clarity.1) The various typos and sentences mismatch can be sometimes very confusing, although this can easily be fixed with careful proofreading (see the end of this review for a detailed list).2) The first subroutine of the proposed algorithm is actually in the appendices. This introduce unintuitive numbering across the different parts of the whole algorithm. In order to unfold it fully, one has to read Alg.1, then Alg.4 (which is in the appendices), followed by Alg.2 and 3. Moreover, it would help if each subroutine was presented as functions. This would vastly improve the readability of the subcalls in Alg.1, especially when it comes to parameters. Finally I think it is would help to have the full algorithm (that is, Alg. 1, 4, 2 and 3) somewhere in one block for easy reference; this may be put in the appendices provided that Alg. 4 is also in the main submission.3) The notations tend to obfuscate the submission to a point that it is, at least for me, really difficult to follow the reasoning of the authors. In my opinion, it is the whole notation system that should be re-though. In their present state, the notation choices seem somewhat random. This is especially true for classifiers and sampled subsets of examples (in the appendices) for which it is very hard to infer their purpose from their format only.4) The authors tend to paraphrase themselves at least two times, this is confusing because there is typically very little new information in those parts.This first happens at the beginning of the section 3 (Algorithm) which is basically is a rewriting of the second part of the introduction, with added notation. Given that the introduction is just 2 pages away, it seems unnecessary to go, again, through a complete explanation of the proposed approach, especially since space seems to be an issue here. The second case is in the beginning of the appendices. The authors reintroduce the notations of the paper. This is very confusing because some old notations are not re-introduced, yet they are used thereafter, while some new notations that could have been used in the main submission are defined, and at least one ($\sigma(n ,\delta)$) is defined for the first time here despite being used liberally in the main submission without any mention of its subsequent definition. 4 ORIGINALITYThe novelty of this paper is clearly the generality of the proposed setting. The authors do a good job on mentioning related works and adequately cite important papers related to their setting. It should be noted that the proposed approach (DBAL coupled with a difference classifier for weak and strong labelers) is extremely close to the one proposed in the work of Malago, Cesa-Bianchi and Renders (cf [14] in the submission's bibliography). The submission feels like an (highly non-trivial) extension of [14] to a far more general setting. While the theoretical results are clearly new and unprecedented, the proposed algorithm is more of an adaptation of the one of [14] to the setting of the submission. Nonetheless, it is my opinion that the theoretical results alone represent a step forward that by itself is worth of a publication. Especially given that the authors also tackle the special cases of linear classification and mixed oracle at the end of the submission. 5. SIGNIFICANCEThis work formally establishes that weak labeler can be used to reduce oracle queries in virtually any instance of active learning with any finite VC-dimension hypothesis class under very mild assumptions. While purely theoretical this is an interesting result that validate the intuitive idea that a strong labeler is not always needed. The assumption that the weak oracle is locally unreliable is very interesting because it matches real case problems where some part of the input space are difficult to classify while others are trivial.In summary, this submission represent a theoretical step in assessing the usefulness of adding a weak oracle to the regular active learning setting. In this respect, the authors establishes that, it can nearly always be beneficial to do so, provided one use their algorithm. They also quickly discuss the some limit cases where their algorithm provide only marginal benefits over regular DBAL approach. 6. TYPOS & STYLISTIC SUGGESTIONSPlease consider the following changes. While some are just typo correction, others should be taken as style suggestion despite being stated in the imperative for clarity's sake.l1. You use 'labeler' in your title with only one 'l' but later you use the form with two 'l' in 'labelled' (l.400). You should pick a form and stick with it for the remaining of the paper. Moreover, since 'labeler' is more related to American english and 'labeller' to British english, you should choose the form that fit best your writing style to stay consistent across your paper.l17-18: Remove "On the theory side" l121: Replace $\geq$ by "at least" l121: Clarify what "over the sample" means l130: Remove "large" l167: Having "df" as exponent is confusing because $d$ is also the VC-Dim of $H$ l175: Do not use the term 'disagreement' when speaking of the difference between O and W, this is confusing because of BDAL nomenclaturel190: I'm not sure of why $\epsilon_k$ has this value. Is it arbitrary ? l201: Better explain what are $\hat{C}_k$ and $A^U_k$. Right now it is very confusing because one depends on $k$ only but the other depends on $U$ and $k$. Also why the hat on $C$ and not on $A$. I understand this become more clear in the appendices but the reader does not know this at that point. Especially because Alg.4 is not in the paper.l209-210: Remove "A hard bound is" by "and" and merge with the previous sentence. l215: How $A^U_k$ is related to the disagreement region $DIS(V_t)$ and what is the link between $\phi_k$ and $\phi_t$ l215: the $\bigO$ notation is exactly the same that O the oracle l215: As $\phi_k$ diminishes, does the FN error in $O(\epsilon_k / \phi_k) increase too ? Alg1, l3: What is the use of $\delta_0 = \delta / 4$ ? You could use directly $\delta$ in l.4 because of the $\bigO$ Alg1, l5: $\sigma(.,.)$ is not defined until the appendices. Alg1, l10: You never draw $T^U_k$ prior to its use here. I assume this is a mistake since you explicitly mention drawing $\hat{S}_0$ in l5. Alg1, l10: You do not mention a threshold parameter $\epsilon_{k-1}$ when you introduce Alg.4 (l199-207) Alg1, l15: Place a "EndFor" line above rather than simply rely on indentation Alg1: Change the formatting of the comments or add empty lines to better separate the different subcalls. Right now, the whole algorithms seems messy Alg1: Call Alg 4, 2 and 3 through functions with parameters to improve the general clarity Alg2,l1: U is previously defined as the Unlabeled distribution. Use $A_k^U$ instead. Alg2,l3: It is not clear to me if you re-use the labels queried for $U'$ in the active learning procedure. If you don't, can you do it ? and why ? l263: Does this inclusion hold with probability 1 ? I am under the impression that it should hold with probability $1- \delta$ but it is not clear from the text. l300: Precise what similar means in this context. l315: I cannot find another definition of $\alpha(.,.)$, in that case it means that eq. (3) hold trivialy by definition l315: The probability's distribution is missing in $P(DIS(B_U(h^*, r)))$ l314-317: The whole phrasing here is unclear l336: Replace $\geq$ by "at least" l337: "[...] in epoch $k$ is at most [...]" Eq(5): Order the terms the same way than the DBAL's result, that is: [VC-Dim][\theta(...)][1 + ./.] l353: "firsts" l359: You just stated above that $\sup_{r \geq \epsilon}[...]$ is less than the disagreement coefficient. I get what you mean but you should rephrase this sentence. l375: If you want to use $\bar{}$ here, you should also use $\bar{h}$ instead of $h^*$ l377: The usual result is that, with linear classifier, the VC-dim is $n+1$ where $n$ is the dimensionally of your data. This is clearly not the case here, because you restrict your example to the unit ball. Nonetheless, this can be confusing because your VC-dimension is denoted $d$, which is usually the data dimension (which is also $2$ here). Add a few word to clarify that.l390: You never introduce the notations $h_w$ l396: See remark on Eq (5) l396: From your previous results, it is expected that $d'$ appears in the second term. You should mention in the main submission that $d' < 5d$ rather than only in the appendices. l407: Why not re-use $h^*$ rather than introduce another new notation ? l409: I don't get the whole $\alpha'$ thing. Is $\alpha$ just a short-hand to $\alpha'(r, \eta)$ ? And what is exactly $\alpha'$ ? l409: Write down the modified version of Alg.1 or at the very least explain in what it is different. l415: I'm not sure if it is supposed to be $O$ or $O'$ here. l421: The conclusion should be in its own section instead of just a paragraph. To some extends this also apply to the late subsections that have the same formatting than the theorems/corollaries/assumptions (inline, bold, with a '.' at the end) -- Appendices --l551: Missing Distribution in $B(h^*, r)$ l576: "We begin each epoch $k$ of Algorithm [...]" l579: Do not separate everything with colons in $(x_i, y_{i,O}, y_{i,W})$. This holds for the remaining of the appendices. l579: If you start with $T$, $A$, and $C$ explicitly conditioned on $U$ it should also be the case of $\mathcal{T}$, $\mathcal{A}$ and $\mathcal{C}$ l582: Remind what is $\hat{C}_k$ because you only mention it briefly (l201) l591: Say that Eq(14) is not yet given l593: Are the constants really important ? l597: "to denote" l605: $T^U_k$ vs $T_k$ vs $\mathcal{T}_k$ vs $\hat{T}_k$ is really confusing l608: What if $|T^U_k| < 2^t$ ? l610: I'm not sure of what $S^U_k$ actually is. l613: Does that mean that $S^t_k$ is actually independent of $\mathcal{S}^{t}_k$ ? l616: The way I understand the definition of $\hat{S}_k^t$, the third term is pointless l620: What is $m_{k,2}$ l621: "hatS_k" l622: Is it really necessary, given all the notation already defined, to add one simply to avoid writing $\hat{}$ ? At this point you already have at least defined $S_0$, $\hat{S_0}$, $S_k$, $S_k^t$, $\hat{S}_k^t$, $\mathcal{S}_k^t$, $S_k^{U,t}$. And this is only based on $S$ the same thing is true for $T$, $A$ and $C$.l626: Do not use them interchangeably, stick with one. l626: "interchangeably" l632&l635: Add a few words on where these constants come from and why it is relevant to explicitly write them. l641: "We will condition on later in the proof" This part seems odd. l684: From my understanding, it seems that it is the distance between the errors of any pair of classifier that will not increase much. I am not sure that it is equivalent to what you write though. l692: "that concentration of empirical [...] to their true errors." The whole phrasing here seems off. l700: I'm not sure that $\hat{h}_k$ is defined. If it is the case, you should recall what it is. l718: Again, this threshold is never mentioned before here and Alg1. l10. l790: Does that mean that $h$ does not make any mistake on $\hat{C}_k$, even in the agnostic case ? l886: What do you mean by "by feasibility" ? l945: "W or O" l948: From my understanding, this holds because $T_k = T_k^U = \mathcal{T}_k$. The right-hand part is only briefly mentioned in a proof above and you should remind it. As for the left-hand part I cannot find it anywhere but I probably missed it.l955: You have not used $\gamma$ in roughly $400$ lines, remind what it is. l1087: Directly write $F_0$ and "epoch $0$" l1187: "PR_U" I suppose a "\" is missing here l1283: Is the VC-dim always $d'$ or at most $d'$ ? l1288: See l608 l1291 (and below): Replace "for all" with $\forall$


Mixing Time Estimation in Reversible Markov Chains from a Single Sample Path
This article provides the first procedure for computing a fully data-dependent interval that traps the mixing time t mix of a finite reversible ergodic Markov chain at a prescribed confidence level. The interval is computed from a single finite-length sample path from the Markov chain, and does not require the knowledge of any parameters of the chain. This stands in contrast to previous approaches, which either only provide point estimates, or require a reset mechanism, or additional prior knowledge. The interval is constructed around the relaxation time t relax , which is strongly related to the mixing time, and the width of the interval converges to zero roughly at a √ n rate, where n is the length of the sample path. Upper and lower bounds are given on the number of samples required to achieve constant-factor multiplicative accuracy. The lower bounds indicate that, unless further restrictions are placed on the chain, no procedure can achieve this accuracy level before seeing each state at least Ω(t relax ) times on the average. Finally, future directions of research are identified.

1. The paper would be much more interesting to the NIPS audience if the authors provided an application of their work to a practical machine learning example, e.g., MCMC for reinforcement learning (as stated for motivation in the Introduction) or for marginalization of a graphical model. Without such an example, which would establish tightness of the proposed bounds and confidence intervals, it is difficult to appreciate the practical utility of the theoretical results presented in the paper. 2. After Theorem 1, in the statement that each state may have to be visited \log(d)/\gamma_* times on average, "at least" should be added considering the non-uniformity of the state frequencies. 3. In Theorems 1 and 2, since a lower bound on the number of observations is sought, it is more appropriate to present the results for n \geq c/\overline{\pi} and n \geq cd log(d)/\overline{\gamma}, instead of the reversed inequalities in these theorem statements. If the authors adopted this they would obtain upper bounds instead of lower bounds, on the error probabilities shown in each theorem.4. It is not clear how (4) and (5) in Theorem 3 imply n \tilde{O}\left(\frac{1}{\pi_* \gamma_*} \right) andn \tilde{O}\left(\frac{1}{\pi_* \gamma_*^3} \right), respectively. Indeed, one can easily see (suppressing the logarithmic terms) that (4) impliesn \tilde{O}\left(\max\left\{\frac{\pi_*}{\gamma_*}, \frac{1}{\gamma_*} \right\} \right) \tilde{O}\left(\frac{1}{\gamma_*} \right)and similarly, that (5) implies n \tilde{O}\left(\max\left\{\frac{\pi_*\gamma_*}{\gamma_*}, \frac{1}{\gamma_*} \right\} \right) \tilde{O}\left(\frac{1}{\pi_*\gamma_*} \right)However, these do not imply the desired result. More explanation is needed.  5. The lower bound on the number of samples stated in Main Results is not the same as the lower bound in Theorem 2. The former lower bound is \Omega((d\log d)/\gamma_*+1/\pi_*) while the latter lower bound is missing the term 1/\pi_*. Explanation of this discrepancy is required. 6. There seems to be a problem with the proof of Theorem 1 in Appendix B. The following counterexample violates the claims in the proof. Let \overline{\pi} 1/4 for which \epsilon 2/3 and \lambda_* -1/6 < 0, which violates the definition \lambda_* \max(\lambda_2,|\lambda_d|). Furthermore, for n < c(2+\epsilon)/\epsilon 4c, choosing c 1, for which n 3, we compute the probability of staying in state 1 as (1-\epsilon)^n 1/27 which is less than the presented result of 1/4.7. The last sentence of the proof of Thm 2 requires elaboration. It is not clear how it follows. Minor comments1. In the first paragraph of the Related Work the authors need clearly state the assumptions on the Markov chain, e.g. reversible, ergodic, aperiodic, irreducible, finite-state.2. The title of Section 3 is misleading since they are not performing "point estimation" but rather interval estimation. 3. The authors repeated claims of obtaining confidence intervals that are "empirical" and "based on a single sample path" are a overemphasized. Any useful confidence interval must be empirical and by definition, a confidence interval is an interval estimate, and always depends on a given sample.


High-dimensional neural spike train analysis with generalized count linear dynamical systems
Latent factor models have been widely used to analyze simultaneous recordings of spike trains from large, heterogeneous neural populations. These models assume the signal of interest in the population is a low-dimensional latent intensity that evolves over time, which is observed in high dimension via noisy point-process observations. These techniques have been well used to capture neural correlations across a population and to provide a smooth, denoised, and concise representation of high-dimensional spiking data. One limitation of many current models is that the observation model is assumed to be Poisson, which lacks the flexibility to capture under-and over-dispersion that is common in recorded neural data, thereby introducing bias into estimates of covariance. Here we develop the generalized count linear dynamical system, which relaxes the Poisson assumption by using a more general exponential family for count data. In addition to containing Poisson, Bernoulli, negative binomial, and other common count distributions as special cases, we show that this model can be tractably learned by extending recent advances in variational inference techniques. We apply our model to data from primate motor cortex and demonstrate performance improvements over state-of-the-art methods, both in capturing the variance structure of the data and in held-out prediction.

High-dimensional neural spike train analysis with generalized count linear dynamical systemsThis paper describes a general exponential-family model (called the "generalized count" (GC) distribution) for multi-neuron spike count data. The model accounts for both under-dispersed and over-dispersed spike count data, and has Poisson, Negative Binomial, Bernoulli, and several other classic models as special cases.The authors give a clear account of the relationship to other models, and demonstrate the need for a model to capture under-dispersed counts in primate motor cortex.They then describe an efficient method for maximum-likelihood fitting (and demonstrate concavity of the log-likelihood).It briefly discusses GC regression before turning to latent variable modeling of population data using a latent Gaussian linear dynamical system and GC observations.They derive an efficient variational Bayesian inference method and apply the model to data from primate motor cortex, showing that it accounts more accurately for variance and cross-covariance of spike count data, compared to a model with Poisson observations.The paper is clearly written, original, interesting, and thorough, and likely to have a major impact on the field of neural coding and latent variable modeling of neural data. While the quantitative improvement over existing models is somewhat modest, the model has very nice theoretical properties and is attractive for its ability to account for under- and over-dispersion within a single framework. An outstanding paper -- I expect it to attract considerable interest and attention.If I have one general criticism it's that it would be nicer to have a clearer idea for how to model single-neuron data, since the "full" model assigns zero probability to all unobserved counts (sending test log-likelihood negative infinity if the test data ever contains a count not observed in training), and the "simple" model requires a full population.The authors mention this shortcoming briefly in the discussion, but it might be nice to have a simple recommendation for how to handle this (e.g., add a small pseudocount for some reasonable set of possible spike counts). Technical comments:106: "this choice has been previously considered and found suboptimal [8]".No no no, that is not correct! I'm sorry, the Macke et al 2011 paper did NOT examine a fine timescale GLM at all, much less whether it could account for underdispersion (or cross-correlations) in population data. The smallest bin size used was 10ms (which is obviously too big to be effective for modeling short time-scale refractory effects).So please stop repeating this misleading claim from that earlier paper.Besides, there's no need for this attack here: this paper addresses an interesting and important problem its own right, whether or not there's some other approach for capturing underdispersion that might also suffice.(Someone should make a sincere attempt, but it's a topic for another paper.)Table 1:If you have space, it might be nice to include the mapping to \theta in the "GCGLM Parametrization", just to make it easier for people to mentally connect the GC model to the classic models.(e.g., \theta = \log \lambda for Poisson model). Not strictly necessary though if it takes too much space.eq 6: I found this a little puzzling when I first read it -- might be helpful to spell out what g_i is.294: "does not significantly overfit"?This is for how much data -- the same amount as the recording data?(Presumably there should be substantial overfitting for a small dataset.)Fig 2: is this all from training data (i.e., just confirming that the model is capable of exhibiting the desired mean-variance relationship), or is there some prediction on test data underlying this figure?


Communication Complexity of Distributed Convex Learning and Optimization
We study the fundamental limits to communication-efficient distributed methods for convex learning and optimization, under different assumptions on the information available to individual machines, and the types of functions considered. We identify cases where existing algorithms are already worst-case optimal, as well as cases where room for further improvement is still possible. Among other things, our results indicate that without similarity between the local objective functions (due to statistical data similarity or otherwise) many communication rounds may be required, even if the machines have unbounded computational power.

The paper gives lower bounds for the number of rounds needed for optimizing convex functions distributedly. In the main result of the paper (Theorem 1), it is assumed that we are optimizing a sum of delta-related and \lambda functions with a certain class of algorithms (which can basically take gradient, use local hessein to do preconditioning, and other related operations). It is shown that number of rounds for this task is at least \sqrt{\detla/\lambda}. These are matched by DiSCO under quadratic case. The class of functions concerned here contains the (probably the most) interesting situation, random partition of a set of functions, that people care about, though the former is a strict superset of the later. The paper also shows lower bounds for the case when the local functions can be arbitrary, and the case when only one round of communication is allowed. The main proof technique is similar in spirit to the query complexity of first order method. Essentially the constructed hard instances is such that starting from initial iterate 0, every round of communication can only provide information for one additional dimension and therefore without enough rounds, the iterate can only be in the span of the first several dimensions and dooms to have large error. In summary, the paper has nice results and is well written. I wouldn't say that results are very strong due to the restrictions to the algorithms and the large class of functions concerned. However, proving lower bounds for rounds of communications are typicallyhard (e.g. for the typical boolean function and two player case), and therefore restriction to a class of algorithms should be not be a problem. Finally, there are some minor issues that I hope the author could clarify/revise/strengthen: 1. the argument seems to depend on the initial point w to be 0 heavily. Is it possible to allow a random starting point (which doesn't depend on the function)? I don't see an easy fix for it. (Maybe previous proofs for first order methods (like Nemirovsky-Yudin) suffers from the same drawback? If so maybe it's good to clarify) 2. it seems to be possible to extend the lower bound to the query model in the flavor of increment first-order algorithm like in Nemirovsky-Yudin or Agarwal-Bottou? It would be nice if one can allow arbitrary update based on the iterate, gradient, preconditioned gradient. 3. It is mentioned in line 86 that "we conjecture that the optimal algorithm in this case is again .. ". I got confused about why this is a conjecture? Maybe it is good to spell out the gap more explicitly.


Minimax Time Series Prediction
We consider an adversarial formulation of the problem of predicting a time series with square loss. The aim is to predict an arbitrary sequence of vectors almost as well as the best smooth comparator sequence in retrospect. Our approach allows natural measures of smoothness such as the squared norm of increments. More generally, we consider a linear time series model and penalize the comparator sequence through the energy of the implied driving noise terms. We derive the minimax strategy for all problems of this type and show that it can be implemented efficiently. The optimal predictions are linear in the previous observations. We obtain an explicit expression for the regret in terms of the parameters defining the problem. For typical, simple definitions of smoothness, the computation of the optimal predictions involves only sparse matrices. In the case of norm-constrained data, where the smoothness is defined in terms of the squared norm of the comparator's increments, we show that the regret grows as T / √ λ T , where T is the length of the game and λ T is an increasing limit on comparator smoothness.

The paper is very well-written, both syntactically and logically: the ideas were easy to follow and the contribution is quite clear. The idea to analyze arbitrarily generated time series using the regret criteria is interesting in my opinion, yet it is not the first time such analysis is made (despite the fact that the authors do not cover the relevant literature in their related work section). I expect the authors to do a better literature review if the paper is accepted.One issue that bothers me the most is the non-standard definition of the regret: why should the comparator class be penalized on its smoothness complexity? In particular, many time series are not smooth and yet are simply predicted (for example, AR type time series). In my opinion, the logical thing to do is to restrict the model and not the predictions. For example, if we consider an AR model, the restriction should be on the smoothness of the AR coefficients and not on the smoothness of the predictions. The current regularization might hold as well, but needs to be better motivated by the authors.The authors mention that their setting can handle with ARMA models, which seems misleading to me. I am not sure what the authors mean here - if the time series is allowed to be arbitrarily generated (as the authors assume) then ARMA models are not well-defined (see the work of Anava et al (2013)). If the time series is stochastically generated (and is in fact an ARMA process), then a regret result is meaningful, but I expect the authors to clarify this point. In particular, such results should be better contrasted with existing results (both theoretically and empirically).I find the experimental section very poor (even though it appears only in the appendix). In particular, the authors do not compare their result to any baseline, error plots are not presented, the setting and the generation mechanism of the time series are not specified, and more. I suggest the authors to remove it completely or follow the standard requirements in this field.Despite the weaknesses I spot, I vote for acceptance and willing to upgrade my score according to the authors response (mainly motivating the regret definition).minor comments: (1) line 282 - sill -> still. (2) I would use similar notations in section 3 and 4 - \alpha should be c, \beta should be w, and c should be u. It will be easier to follow this way. (3) please use the word "intriguing" less than you do.UPDATE: The authors response did not allay my concerns regarding the regret formulation, and thus I chose not to upgrade my score. However, since the authors suggest a new approach for time series prediction, I think the paper should be accepted despite its weak theory and experiments. In fact, I am quite sure that this approach will be far behind existing approaches in practice, but might open the door for interesting future work. Missing literature: I would start with the standard textbooks (Hamilton, Box&Jenkins, Brockwell&Davis), and advance to newer papers that combine learning techniques for time series prediction (some even use the regret criterion). Some pointers: Generalization Bounds for Time Series Prediction with Non-stationary Processes, Nonparametric Time Series Prediction Through Adaptive Model Selection, Online Prediction of Time Series Data With Kernels, Online Learning for Time Series Prediction.


HONOR: Hybrid Optimization for NOn-convex Regularized problems
Recent years have witnessed the superiority of non-convex sparse learning formulations over their convex counterparts in both theory and practice. However, due to the non-convexity and non-smoothness of the regularizer, how to efficiently solve the non-convex optimization problem for large-scale data is still quite challenging. In this paper, we propose an efficient Hybrid Optimization algorithm for NOn-convex Regularized problems (HONOR). Specifically, we develop a hybrid scheme which effectively integrates a Quasi-Newton (QN) step and a Gradient Descent (GD) step. Our contributions are as follows: (1) HONOR incorporates the second-order information to greatly speed up the convergence, while it avoids solving a regularized quadratic programming and only involves matrixvector multiplications without explicitly forming the inverse Hessian matrix. (2) We establish a rigorous convergence analysis for HONOR, which shows that convergence is guaranteed even for non-convex problems, while it is typically challenging to analyze the convergence for non-convex problems. (3) We conduct empirical studies on large-scale data sets and results demonstrate that HONOR converges significantly faster than state-of-the-art algorithms.

This paper considers efficient implementations of non-convex sparse learning formulations. Recent studies have shown that many convex sparse learning formulations are inferior to their non-convex counterparts in both theory and practice. However, it is still quite challenging to efficiently solve non-convex sparse optimization problems for large-scale data. This paper presents a novel algorithm HONOR which is applicable for a wide range of non-convex sparse learning formulations.   One of the key ideas in HONOR is to incorporate the second-order information to greatly speed up the convergence, while unlike most existing second-order methods it avoids solving a regularized quadratic programming and only involves matrix-vector multiplications without explicitly forming the inverse Hessian matrix. Thus, HONOR has a low computational complexity at each iteration and it is scalable to large-size problems. The convergence for non-convex problems is typically challenging to analyze and establish. One major contribution of this paper is to establish a rigorous convergence analysis for HONOR, which shows that convergence is guaranteed even for non-convex problems. The key to the convergence analysis is the hybrid optimization scheme which chooses either a Quasi-Newton step or a Gradient Descent step per iteration. The presented analysis is nontrivial. It will be good to include a high-level description of the intuition behind the proposed hybrid scheme.       The empirical evaluation presented in this paper is convincing. The authors evaluate the proposed algorithm using large-scale datasets which include up to millions of samples and features. Two of the datasets include over 20 million features. Such scale of datasets is needed to evaluate the behavior of the algorithms. Results demonstrate that HONOR converges significantly faster than state-of-the-art algorithms. Some guidance on the selection of \epsilon will be helpful. It is mentioned that HONOR may have the potential of escaping from high error plateau which often exists in high dimensional non-convex problems. It will be interesting to explore theoretical properties of the HONOR solutions. The proposed algorithm empirically converges very fast. Is there any guarantee on the (local) convergence rate of HONOR?Can the proposed algorithm be extended to other sparsity-inducing penalties such as group Lasso and fused Lasso?


End-To-End Memory Networks
We introduce a neural network with a recurrent attention model over a possibly large external memory. The architecture is a form of Memory Network [23] but unlike the model in that work, it is trained end-to-end, and hence requires significantly less supervision during training, making it more generally applicable in realistic settings. It can also be seen as an extension of RNNsearch [2] to the case where multiple computational steps (hops) are performed per output symbol. The flexibility of the model allows us to apply it to tasks as diverse as (synthetic) question answering [22] and to language modeling. For the former our approach is competitive with Memory Networks, but with less supervision. For the latter, on the Penn TreeBank and Text8 datasets our approach demonstrates comparable performance to RNNs and LSTMs. In both cases we show that the key concept of multiple computational hops yields improved results.

Summary:This paper presents an end-to-end version of memory networks (Weston et al., 2015) such that the model doesn't train on the intermediate 'supporting facts' strong supervision of which input sentences are the best memory accesses, making it much more realistic. They also have multiple hops (computational steps) per output symbol. The tasks are Q&A and language modeling, and achieves strong results.Quality: The paper is a useful extension of memNN because it removes the strong, unrealistic supervision requirement and still performs pretty competitively. The architecture is defined pretty cleanly and simply. The related work section is quite well-written, detailing the various similarities and differences with multiple streams of related work. The discussion about the model's connection to RNNs is also useful. My main concern with the paper is that it presents various model decisions and choices without motivating them, without empirically ablating them, and also without apparently using a dev set for most of these decisions (and hence tuning and presenting too many results on the test set directly?). Some examples are: -- Eqn3 -- the various wight tying choices, and whether these generalize to other tasks (Q&A or otherwise) -- temporal encoding design choices -- the various training decisions in Sec 4.2 and Sec 4.4 -- Table 1, that does finally have ablations for some remaining, more important choices, is done directly on the test set, at the risk of reporting too many numbers on the test set and essentially choosing the best among those as their final number to compare to baselines. Ideally, all these decisions should be made on a dev/validation set and then the final setting choice should be run on test.Other issues or suggestions: -- Encoding with an RNN or LSTM-RNN would be good to compare to the current approach of BoW or temporal encoding. -- Also, for the MemNN-WSH baseline, why not have a better classifier to predict which sentences to use, instead of using >1 word match? -- the perplexity differences in Table2 from the LSTM baseline do not look large so authors should report statistical significance results. The gap might be even smaller or negative if the current model did not so many extra parameters to tune over, as compared to LSTMs. -- perplexity in language modeling is not very convincing as a metric, so maybe useful to also apply this model (and show its generalizability) to text categorization tasks like tagging, NER, sentiment, etc.Clarity: The paper is well-written and well-organizedOriginality and significance: The paper is original and has a useful impact in terms of moving away from the strong supervision assumption of memNNs, and using multiple hops per output; the related work is well-written to clarify the originality. However, there are issues with model tuning and decisions, comparison, and need of better tasks. ########## Post-rebuttal: Thanks for answering some of my questions. I have increased my score to a 7 but please include all the dev/test details, model choice ablations, and statistical significance tests for perplexity. Also try to add some non-perplexity metrics.


The Human Kernel
Bayesian nonparametric models, such as Gaussian processes, provide a compelling framework for automatic statistical modelling: these models have a high degree of flexibility, and automatically calibrated complexity. However, automating human expertise remains elusive; for example, Gaussian processes with standard kernels struggle on function extrapolation problems that are trivial for human learners. In this paper, we create function extrapolation problems and acquire human responses, and then design a kernel learning framework to reverse engineer the inductive biases of human learners across a set of behavioral experiments. We use the learned kernels to gain psychological insights and to extrapolate in humanlike ways that go beyond traditional stationary and polynomial kernels. Finally, we investigate Occam's razor in human and Gaussian process based function learning.

The paper is fantastic in many manners and very well-written. I will focus on (hopefully) constructive criticism:There is too much packed into the paper. As such, there are some details that should be in the paper, but are not. For example, a reader should be able to replicate the modeling and human experiments based on reading the article alone, if she desired. I do not think I could do either based on only the article. Although the supplement helps, it does not help with the experiment presented in section 5. My main other criticism is that the psychological implications of the studies presented by the authors is sloppy at times. Does it really get at THE human kernel? Their own results seem to contradict this as different kernels were learned depending on the experiment (compare Figure 2o to Figure 3d). I also suspect the kernel will be highly domain-dependent. Also, the authors write "they are more confident in their predictions...". As far as I can tell, the researchers did not get individual confidence ratings and without that, it seems inappropriate to say that they are more confident. I would say that they is less variability between participants. Finally, I have one last concern with respect to the stimuli. As the authors note at the end, the nature of the stimuli typically used in the human function learning is very different than that used in the experiments. I'm not a fan of the standard human function learning stimuli, but you should mention that it is going to be different right away, as it is confusing to cognitive science readers that are familiar with the human function learning literature. Further, the author's stimulus presentation introduces a confound: they could be studying human inductive biases over 1-d curves in images rather than functions themselves. It is probably worth mentioning this limitation of this method.More minor concerns: Paragraph 2 of 4.1: I don't understand what the prediction kernel is. Also, why use RBFs if later you focus on RQs? Also also, you never define RQ... I found Figure 2 difficult to parse. It might be worth considering other ways of organizing the subplots. If you are interested in human rule learning in number sequences, you might consider citing and/or taking a look at Austerweil & Griffiths (2011). Seeking Confirmation is Rational for Deterministic Hypotheses. Cognitive Science. line 351-352: "Participant were" -> "Participants were" The sections for experiments are mislabeled in the supplementary materials (and also the last 2/3 experiments are missing).


Policy Evaluation Using the Ω-Return
We propose the Ω-return as an alternative to the λ-return currently used by the TD(λ) family of algorithms. The benefit of the Ω-return is that it accounts for the correlation of different length returns. Because it is difficult to compute exactly, we suggest one way of approximating the Ω-return. We provide empirical studies that suggest that it is superior to the λ-return and γ-return for a variety of problems.

***After author response/discussion***I have moved my score up for two main reasons: 1) The authors helpful response did clarify the experimental set-up for me, which I agree does not need to include True TD(lambda). I have edited my review below to reflect this.2) I have come around to thinking of the approximate algorithm as a proof of concept rather than a primary contribution. Seeing this paper as describing a solid foothold in this direction, and some early evidence that there may be something to be gained by going further makes me less concerned about its practical impact, and more optimistic about its chances of provoking questions and possibly inspiring follow-on work.---Paper Summary---This paper proposes a novel replacement for the lambda return for mixing between bootstrapping and Monte Carlo estimates in reinforcement learning. Specifically, the Omega return in principle accounts for the covariance of returns at different depths (whereas the lambda return implicitly assumes the returns at various depths are independent). Computation of the Omega return is formulated as a generalized least squares problem, which impractically relies upon unavailable quantities. The paper describes one method for approximating these quantities using experience data. Empirical results in four domains suggest that the approximation produces qualitatively sensible weights, and that the use of the approximate Omega return yields more accurate off-policy estimates of policy quality.---Quality---I believe the derivation of the Omega return weights (the main technical result) is technically sound. The various steps of approximation are generally well-motivated. One argument that I did not totally follow starts on line 208. The argument is intended to motivate the assumption that the covariance between two different depth returns is approximately equal to the variance of the shallower return. Intuitively I can see why this might be reasonable; it seems sensible in many problems to expect the return at one depth to be very similar to a slightly deeper return (less so for much deeper returns, but therein lies the approximation). The thing I didn't get is the argument that the i-step return should be nearly independent of the depth-j TD-error. The claim seems to be that they should be nearly independent because they are *conditionally* independent given state. As I'm sure the authors are aware, there's no reason that two conditionally independent variables should also be independent. If you want to assume those two quantities are independent, it's important to develop a better founded sense of what that assumption really means.The claim that the approximation yields accurate weights is a little imprecise. It is mainly supported by eyeballing the approximated quantities to see that they are qualitatively similar to the targets. I'd have preferred a more quantitative study of the accuracy of the approximation, perhaps studying the relative impact of the various stages of approximation proposed in Section 4.The empirical claim that using the approximate Omega return yields more accurate value estimates than the usual lambda return seems well-supported. ---Clarity---I found the paper to be well-written for the most part. I particularly appreciated the introduction which I thought laid out the issues at hand clearly and effectively set the stage for the technical contributions. The main area for improvement in clarity I see is Section 4. The section takes the reader through multiple different stages of approximation and it starts to feel a bit like a run-on sentence ("and then...and then..."). I'd humbly suggest that the authors take another look at this section with eye toward providing the reader with more structure to hold on to. Some things that might help include additional subheadings, some text along the way to place the current step in the overall approximation plan, and/or perhaps an executive summary at the end of the various assumptions and their consequences.The other major impediment to clarity is the relegation of the description of the experiments to the supplementary material. This make it harder to determine the meaning of the results; the figures even contain terms that are not defined anywhere in the main text. The authors should be aware that they are permitted to place their references section on the 9th page, leaving nearly 3/4 page of available space. While I understand that the appendix as-is cannot fit in that space, I expect that with some judicious editing significantly more detail regarding the experiments can fit in the main text.---Originality---As far as I am aware, the Omega return is a novel concept, and the derivation and subsequent approximation presented here are also original, though they make use of well-known machine learning tools. ---Significance---The conceptual problem with the lambda return is very real, and effectively addressing the bias in the estimator could potentially have an important impact on RL practice. However, this is not a novel insight -- the issues with the lambda return have already been discussed in the literature. So the burden of significance falls to the Omega return itself.The principle underlying the Omega return is a good one, and I wouldn't say that its formulation was a priori obvious. So one source of potential significance (that the paper itself raises) is that the Omega return, as described here, may now serve as a target for approximation for future study.I do see potential value in having this idea available in the literature as a foundation for future work.The other possible source of significance is the particular approximation described here. Unfortunately, to my eyes it looks like this method is fairly complicated and limited compared to the lambda return. As George Box is famous for saying "All models are wrong, but some are useful." Both the approximate Omega return and lambda return are wrong, but from these results I'd be forced to conclude that the lambda return is currently more useful. I doubt the specific approximation method in this paper will significantly impact RL practice. At best it serves as a proof of concept that one can approximate the Omega return to some extent, giving hope that a better approximation will come down the road eventually.


Max-Margin Deep Generative Models
Deep generative models (DGMs) are effective on learning multilayered representations of complex data and performing inference of input data by exploring the generative ability. However, little work has been done on examining or empowering the discriminative ability of DGMs on making accurate predictions. This paper presents max-margin deep generative models (mmDGMs), which explore the strongly discriminative principle of max-margin learning to improve the discriminative power of DGMs, while retaining the generative capability. We develop an efficient doubly stochastic subgradient algorithm for the piecewise linear objective. Empirical results on MNIST and SVHN datasets demonstrate that (1) maxmargin learning can significantly improve the prediction performance of DGMs and meanwhile retain the generative ability; and (2) mmDGMs are competitive to the state-of-the-art fully discriminative networks by employing deep convolutional neural networks (CNNs) as both recognition and generative models.

This paper introduces a max-margin criterion to the learning of a deep generative model, essentially enhancing the discriminative abilities of the latent variables z.Unsurprisingly, this improves the classification abilities of the model, compared to the use of a simple linear classifier trained on the latent variables of the original generative model.On the MNIST and SVHN sets, the final model is reasonably competitive with purely discriminative models, though not state-of-the-art.However, it retains the advantage of being a generative model.The paper is well written.The technical sections are particularly clear, although I found the motivation section was weak and unclear in places.Although the idea of incorporating the max-margin criterion is not highly innovative, this paper adds a large amount of useful work regarding how the optimisation should be achieved.I am not an expert on the literature in this area, so am unable to make a strong judgement on the level of originality.I found that the work is not clearly motivated in the early sections.I was initially rather sceptical about whether there is any gain to using a discriminative criterion with a generative model - if you want generative behaviour, why not use a generative model; if you want discriminative behaviour, why not use a discriminative model?However, I am prepared to accept that there are advantages.In particular, section 4.4 is very interesting and in a sense, motivates the rest of the paper for me.However, there is a missed opportunity to compare the classification performance against other systems in the missing data scenarios - I would find good results in this area to be a compelling reason for using the proposed model.Simply comparing error rates against a series of discriminative models in Tables 1 and 3 aren't so interesting.The mathematics seems correct to me, but I have not checked every equation in detail.Guassian -> GaussianThere are some minor inconsistencies in the references section, eg. [4] vs. [33], and [13].


Biologically Inspired Dynamic Textures for Probing Motion Perception
Perception is often described as a predictive process based on an optimal inference with respect to a generative model. We study here the principled construction of a generative model specifically crafted to probe motion perception. In that context, we first provide an axiomatic, biologically-driven derivation of the model. This model synthesizes random dynamic textures which are defined by stationary Gaussian distributions obtained by the random aggregation of warped patterns. Importantly, we show that this model can equivalently be described as a stochastic partial differential equation. Using this characterization of motion in images, it allows us to recast motion-energy models into a principled Bayesian inference framework. Finally, we apply these textures in order to psychophysically probe speed perception in humans. In this framework, while the likelihood is derived from the generative model, the prior is estimated from the observed results and accounts for the perceptual bias in a principled fashion.

The paper describes a novel way to parameterize stimuli for generating space-time texture fields modeled after real-world deformations. The intent is to provide a more rigid framework by which scientists may psychophysically test motion perception. The model allows one to generate causal time-structured "Motion Clouds" in real-time. Motion Clouds are textured stimuli, which could be gabor-like or described by other functions. The model includes independent variables to describe motion, position, and warping. Warping describes texture zooming and rotations. They further suggest parameter distributions and stimulus properties that match well with existing psychophysical data. They use their model to generate stimuli for a speed discrimination psychophysical task and demonstrate their ability to assess how the stimulus spatial frequency affects speed discrimination. I interpret their contribution as a GPU ready method for generating real-time stimuli that is parameterized in a meaningful way for studying texture-motion related psychophysical phenomena. A further advantage of this formalism is that it allows for statements of the model within a Bayesian framework. This sets the stage for psychophysical experiments. The main result of these (although the dataset is small) is that spatial frequency has a positive effect on perceived speed: stimuli of lower frequency with respect to the reference are perceived as going slower. The Bayesian model was able to explain these systematic biases for spatial frequency as shifts in priors on speed during the speed judgements. Overall I find the data part of this paper to be less satisfying because there is insufficient data for detailed analysis within the Bayesian frame work and the authors are unable to interpret some of their results.Other comments:The paper has some formatting problems (figure Legends not sufficiently different from text)that make it difficult to read.Note: The discussion section is cut off in the second sentence and doesn't continue. It seems as though they lost some text somewhere...


Semi-supervised Sequence Learning
We present two approaches to use unlabeled data to improve Sequence Learning with recurrent networks. The first approach is to predict what comes next in a sequence, which is a language model in NLP. The second approach is to use a sequence autoencoder, which reads the input sequence into a vector and predicts the input sequence again. These two algorithms can be used as a "pretraining" algorithm for a later supervised sequence learning algorithm. In other words, the parameters obtained from the pretraining step can then be used as a starting point for other supervised training models. In our experiments, we find that long short term memory recurrent networks after pretrained with the two approaches become more stable to train and generalize better. With pretraining, we were able to achieve strong performance in many classification tasks, such as text classification with IMDB, DBpedia or image recognition in CIFAR-10.

This is a mostly empirical paper where the authors apply a new style of pretraining to LSTMs and show major improvements over randomly initialized LSTM on sentiment analysis, text classification, and character-level document classification. The main pretraining approach is to simply train the LSTM as auto-encoder to be able to echo a sequence in order.Generally this paper shows very striking results with gains over temporal convolutional networks and document vector methods like paragraph vectors.General comments:- "Unsupervised Sequence Learning" is a confusing title for this paper. While the pretraining method does use an unsupervised sub-task, there are no results reported on any unsupervised tasks or even analysis of this initial subtask.- The LSTM autoencoder itself (not for pretraining) does not seem novel. The seq2seq work for instance uses "unsupervised" reconstruction as an example (particularly in their talks).This should be made clear in the paper.- I was not able to follow the explanation in paragraph l.97- l. 182 "Random word dropout" seems important. Could you describe?- It was not clear exactly how pretraining was done. Unless I am mistaked, it seems like the autoencoder is trained on in-domain documents possibly from the same dataset? Is this important? What happens if you pretrain on wikipedia or external data. For section 4.3 how is SA pretrained?- l.243. It seems unfair that CNN-non-static is not given the Amazon reviews as well. Should be possible to run word2vec on amazon instead of google news for this comparison.  - One aspect that would improve this paper is some direct evidence of the claim that long-range dependencies are really being captured. It seems possible that the LSTM is just somehow better than CNN at picking up local phrases. Even anecdotal evidence of this process would be interesting.     ## Small - l103: typo "the weights" verb agreement - l114: typo verb agreement. - l188: same


Weighted Theta Functions and Embeddings with Applications to Max-Cut, Clustering and Summarization
We introduce a unifying generalization of the Lovász theta function, and the associated geometric embedding, for graphs with weights on both nodes and edges. We show how it can be computed exactly by semidefinite programming, and how to approximate it using SVM computations. We show how the theta function can be interpreted as a measure of diversity in graphs and use this idea, and the graph embedding in algorithms for Max-Cut, correlation clustering and document summarization, all of which are well represented as problems on weighted graphs.

The paper starts by situating the problem and motivating their approach - essentially, enabling embeddings for weighted graphs by extending the original implicit embedding performed in Lovasz "Shannon capacity of a graph" paper, which operated on binary graphs. The paper then presents the connection between kernel machines and the Lovasz number in the unweighted case, and extends previous work for vertex-weighted, edge-weighted, and LS-labelled graphs. Section 3 provides practical details for computation, and section 4 motivates the use of their approach for the clustering problem - setting the number of clusters by using their \vartheta^1 bound, and initialising the clusters by starting by vertices with large alpha_i values. Finally, they show results on max-cut, clustering, overlapping clustering, and summarization tasks. The paper ties together very different work to propose a coherent approach to graph embedding. The contributions are clearly laid out, and the references to previous work is well established and used.       Overall, the paper is well-written and the notation is clean.and the derivation of their technique is clearly explained.  This is significant work that can have impact in several domains of machine learning, as shown in the experiments section.  There are a few issues that deserve slightly better explanations:  a) To recover the embedding from the computed kernel matrix, the authors propose to use a truncated SVD. This is fine, but the efficiency of the embedding really hinges on the choice of the truncation order. How much better than a full-rank orthonormal representation (basically order n) can we do and still have good performance? Surely, this depends on the structure of the input adjacency or affinity matrix - something that could be generated from a stochastic block model, for example, would presumably call for a different choice of truncation order than a lattice graph? The authors choose \sqrt(n) for the truncation order, but it would be useful to have some intuition as to the impact on the quality of the embedding generated (the paper focuses a lot on how to obtain K and alpha).  b) In section 4, the authors suggest using vertices with the largest alphas as initial centroids. In the SVM analogy, these would be support vectors (i.e. non-zero alphas), which would put them on the decision boundary. However, in the SVM setting these points are typically outliers or extreme values and don't represent the bulk of the data. I understand that the authors want 'diversity' here, but the explanation in lines 263-265 seems to be at odds with the stated goal of finding good centroids in section 4.1 - do we want representative points (center of mass) or influential points (extremes, on the boundary in the discriminative setting) ?             c) In the experiments on maxcut (table 2), the authors have selected 6 graphs from the Helmberg and Rendl set of 24 weighted graphs. These are toroidal graphs, which have a known upper bound on the cut size. Because the comparison to the best known algorithm (Scatter search or VNSPR) if empirical anyways, is there a particular reason to select these? How would the method perform on random graphs (e.g. G6...G10) or planar graphs (e.g. G18...G21) ?


Semi-Supervised Factored Logistic Regression for High-Dimensional Neuroimaging Data
Imaging neuroscience links human behavior to aspects of brain biology in everincreasing datasets. Existing neuroimaging methods typically perform either discovery of unknown neural structure or testing of neural structure associated with mental tasks. However, testing hypotheses on the neural correlates underlying larger sets of mental tasks necessitates adequate representations for the observations. We therefore propose to blend representation modelling and task classification into a unified statistical learning problem. A multinomial logistic regression is introduced that is constrained by factored coefficients and coupled with an autoencoder. We show that this approach yields more accurate and interpretable neural models of psychological tasks in a reference dataset, as well as better generalization to other datasets.

The paper combines unsupervised learning on rest-state fMRI data (using a linear autoencoder) with supervised learning on task-based fMRI data (factored linear regression on task-based fMRI data). The combination is done by sharing the first matrix and using an objective which is a linear combination between reconstruction error (autoencoder) and classification error (regression).Comments:The paper reports a surprising result: The combined method obtains an out-of-sample accuracy of greater or equal than 95% whereas the best chain of independent steps for dimensionality reduction and regression achieves 87% or less. This is potentially a very interesting finding but missing information in the manuscript make it hard to understand the reason for this result.Specific comments:1) Eq. 4 is missing a closing parenthesis and, more importantly, the rationale behind this objective for regression needs to be explained. A potential problem with this objective is that it diverges to minus infinity if one class membership probability is exactly zero.2) It should be explained why \lambda = 1 is chosen for table 2 and in parts of table 3. For this setting the objective does not at all reflect reconstruction error - basically only regression performance is optimized.3) It would be good to also report the reconstruction performances achieved by the different models.


Collaboratively Learning Preferences from Ordinal Data
In personalized recommendation systems, it is important to predict preferences of a user on items that have not been seen by that user yet. Similarly, in revenue management, it is important to predict outcomes of comparisons among those items that have never been compared so far. The MultiNomial Logit model, a popular discrete choice model, captures the structure of the hidden preferences with a low-rank matrix. In order to predict the preferences, we want to learn the underlying model from noisy observations of the low-rank matrix, collected as revealed preferences in various forms of ordinal data. A natural approach to learn such a model is to solve a convex relaxation of nuclear norm minimization. We present the convex relaxation approach in two contexts of interest: collaborative ranking and bundled choice modeling. In both cases, we show that the convex relaxation is minimax optimal. We prove an upper bound on the resulting error with finite samples, and provide a matching information-theoretic lower bound.

summary of paper: This paper proposes a new approach to inference for a low-rank matrix factorization model with ordinal data and another application.It is not clear if MultiNomialLogit (MNL) models have already been applied to these particular problems and the contribution is just the inference approach (nuclear norm minimization), or if the models are contributions as well.Regardless, the authors give performance guarantees for their approach, but do not demonstrate it on real or even synthetic data.quality: good - The theory appears to be thorough.The contributions obviously required substantial work. clarity: poor - Needs significant editing to address both minor issues and also major flow of ideas within sections.Should have more discussion of implications of contributions. originality: mediocre - Low rank matrix factorization has already been addressed from many, many angles; there, the contribution appears incremental.I am less familiar with the other application. significance: poor - The authors do not compare to any other methods (there are many well-established ones for matrix factorization), either empirically or theoretically.Even a small improvement can have a large impact, but the reader is left to assume that this paper would have no practical and only minor theoretical impact on the problems described.some detailed points - line 20 (abstract), Awkward/unclear/editing mistake?: "A popular discrete choice model of multinomial logit model captures..."- line 18, we're obviously talking about personalized user preferences, but that could be more clear-the user isn't mentioned until after the problem of preference prediction, which seems backward; then, we're talking about comparisons without talking about what we're comparing.This sequence of ideas is very convoluted. - abstract starts off with motivating applications, but then the contribution bears to only be theoretical (presenting an approach and proving a bound), with no justification as to why the theory is important, aside from it being a "natural approach" to for inference on "a popular model."No hint of how the theory impacts application, nor mention of experiments on real or even synthetic data. - line 33: should be comma after applications: "applications, such as..." - 1st line of intro and abstract are identical; while not strictly prohibited, it disengages the reader - line 44, 53: elements in a list (i.e., (a) something, (b) something) should have some kind of parallel structure.Here, (b) is the goal and (a) is the means.Again, the sequences of ideas seem convoluted. - line 57: MultiNomialLogit model not capitalized consistently with abstract; needs citation - line 61: "The" should be capitalized at start of sentence - line 62: it might have been better to only use one example in the intro (the bundling is more novel, so I'd pick that); you can always add the other in the experiments/results - line 70: it's great to have your contribution highlighted - line 70: why is it natural to take this approach? - contribution: I think you're saying that MNL is already applied to ordinal low-rank matrix factorization, but that you're doing a new kind of inference for the model and applying that to two cases.Is that right?Or has the MNL only been applied to non-ordinal MF problems and you are applying the MNL to ordinal data for the first time? - line 76: RUM should also have a citation here - line 94: spacing issues - Sections 3 and 4, generally: the practical implications of the theorems and their corollaries are unclear.Perhaps I need to study them in greater detail, but the authors could highlight in words, or empirically, the take home message.The single figure does not immediately contribute to greater understanding. - line 289: RMSE comes out of nowhere.Further, while RMSE is commonly used, it's not actually a good metric for rank-based item recommendation. - Discussion shouldn't just be a to-do list for the future.What should the reader take home?


Interpolating Convex and Non-Convex Tensor Decompositions via the Subspace Norm
We consider the problem of recovering a low-rank tensor from its noisy observation. Previous work has shown a recovery guarantee with signal to noise ratio O(n K/2 /2 ) for recovering a Kth order rank one tensor of size n × · · · × n by recursive unfolding. In this paper, we first improve this bound to O(n K/4 ) by a much simpler approach, but with a more careful analysis. Then we propose a new norm called the subspace norm, which is based on the Kronecker products of factors obtained by the proposed simple estimator. The imposed Kronecker structure allows us to show a nearly ideal O( √ n + √ H K−1 ) bound, in which the parameter H controls the blend from the non-convex estimator to mode-wise nuclear norm minimization. Furthermore, we empirically demonstrate that the subspace norm achieves the nearly ideal denoising performance even with H = O(1).

This paper introduces a new theoretical bound for recovering a low-rank tensor from a noisy observation as well as a new norm on tensors.Among the neatest results in this paper is a demonstration and a proof that there is a two-phased behavior of the recovery.There are a number of problems with the paper, but quality, clarity and significance are my main concerns.The papers quality suffers mainly due to poor grammar, poor terminology, notation and terms that aren't properly and clearly defined.It is unclear how significant any of the results are as the authors have not demonstrated a meaningful improvement in a practical setting.The sub-space norm for example improves things when the relative error is close to 1, but why do I care what happens in a situation where the model is anyways useless?I would have expected a discussion on the practical value of the result - especially given that the signal to noise ration required is quite large (even if reduced dramatically from earlier papers).I can only presume that a lack of such a discussion means the value is marginal and that this should be seen as mainly a theoretical contribution?Here are some comments"Tensor is a natural way" --> "The tensor is a natural way" "for variety of data" --> "for a variety of data" "ranging from ..., neuroimaging" --> "ranging FROM ... TO neuroimaging" what is model 2 in the comment of table 1? "(Vershynin, 2010)" --> this should be a proper reference.This is one of many demonstrations that the paper was not really completed. "mn" vs "nm" : Just be consistent with the order of the product so as not to distract the reader with irrelevant details.Definition (4) is completely bogus.You can't redefine "k-1, ..., k+1" to mean "1,...,k-1,k+1,...,K".Also, you only sometimes use this definition.Even the right hand of (4) is written in a terrible way.In stead of writingA^{(k-1)}\otimes \cdots \otimes A^{(1)} \otimes A^{(K)} \otimes \cdots \otimes A^{(k+1)} I would prefer A^{(1)}\otimes \cdots \otimes A^{(k-1)} \otimes A^{(k+1)} \otimes \cdots \otimes A^{(K)} which clearly shows that the term A^{(k)} is the one left out.Even clearer and more succinct would be \otimes_{i\neq k} A^{(i)}"n_1\times \cdots n_K" should be "n_1\times \cdots \times n_K".This sort of mistake is made repeatedly in the paper.M is not properly defined.I would in the least expect to see the sizes of M defined. Is there a missing sum in the definition of Z_{(k)}^{(k)}?If not then you may as well say that Z has a rank one factorization as opposed to a low-rank factorization?In any case things it's not exactly clear to me what exactly Z is."r=1,\ldots R" --> "r=1,\ldots,R"All the graphs are too busy (and extremely small) to make sense of.  The experiments use a cheating type of method to find lambda, and the rank is assumed known.I assume that in reality both lambda and the rank will be unknown.The only real data experiment was the amino acids data example.It's unclear that the paper added any practical value to analyzing these sort of data sets.


Galileo: Perceiving Physical Object Properties by Integrating a Physics Engine with Deep Learning
Humans demonstrate remarkable abilities to predict physical events in dynamic scenes, and to infer the physical properties of objects from static images. We propose a generative model for solving these problems of physical scene understanding from real-world videos and images. At the core of our generative model is a 3D physics engine, operating on an object-based representation of physical properties, including mass, position, 3D shape, and friction. We can infer these latent properties using relatively brief runs of MCMC, which drive simulations in the physics engine to fit key features of visual observations. We further explore directly mapping visual inputs to physical properties, inverting a part of the generative process using deep learning. We name our model Galileo, and evaluate it on a video dataset with simple yet physically rich scenarios. Results show that Galileo is able to infer the physical properties of objects and predict the outcome of a variety of physical events, with an accuracy comparable to human subjects. Our study points towards an account of human vision with generative physical knowledge at its core, and various recognition models as helpers leading to efficient inference.

This is a high quality paper, clearly written, original, and has the potential for considerable significance.  As I see it, the major contribution and innovation here is that the authors connect a symbolic physical model to an image-level recognition and tracking algorithm.As such, I think the authors need to dedicate a bit more space to discussing the image-level recognition/tracking model, its capabilities and limitations.The scenarios they assess are rather impoverished images; how well would this model fair with new scene configurations? with surfaces that are not smooth planes?With objects that are not blocks?Insofar as this is indeed as general an algorithm as it might seem at first blush, this is worth advertising clearly.If it is limited to fairly simple scenes with known geometric configurations (which would be an easy way to make something that operates over image data, but only because the domain of objects is very impoverished), this needs to be clearly stated.Either way, this reflects useful progress, but the former scenario is far more impressive.A few other comments: - In the introduction (lines ~96-98) the authors make the point that a "computational solution asserts that the infant starts with a relatively reliable mental physics engine, or acquires it soon after birth". This statement sounds like a nativist claim that I don't believe the authors would hold fast to - if infants developed a mature physics engine throughout the first year or two of life, the findings from this paper would still hold (and I would not count this as 'soon after birth'). In fact, much of Renee Baillaregon's work demonstrates that physical reasoning performance develops throughout the first year of life. If the authors wish to claim that mental physics engines are developed (and well formed) very early, they should give a nod to this work; however, I believe this statement distracts from the overall message of the paper and could be dropped.- On line 198 the authors fix sigma (the velocity noise) to 0.05. However, it is not clear how this value was set, or how sensitive to changes in this parameter the model is. It would be helpful to provide units for this value (is it in m/s? normalized to average velocity?) and briefly demonstrate that the model predictions do not change significantly under a couple other reasonable values for this parameter.- In the Outcome Prediction experiment (6.1), it is mentioned that the target object is either cardboard or foam, but the x-axis in Fig 5 represents only (I believe) the material of the initially moving object. Are the errors in Fig 5 averaged across both target objects, or was each initial material associated with only one of the two target objects?- It seems useful to estimate error correlations between people, the uniform model, and POM on the data in figure 5, given the wide variation in errors across all scenes.- For the other experiments (Mass Prediction & "Will It Move") it's not clear how the data is being analyzed. In both cases, I believe the human decision is dichotomous (obj1 heavier vs obj2 heavier for Mass Prediction and moving vs not for Will it Move), and assume that this is aggregated for each trial to get the proportion of people making each decision. Then in both cases, the model samples from the posterior a number of times to get a proportion of either choice for each model? Further clarity on this analysis would help readers understand these results- In the Mass Prediction experiment, the correlations between human and model data are calculated using Spearman's rho because the relationship is claimed to be very non-linear. However, if the POM is capturing human judgments accurately then there should be a linear, 1:1 relationship. Therefore, Pearson's correlation would be more appropriate for this relationship (unless the correlation would be skewed by outliers). Showing scatterplots of these relationships by trial would also help readers understand this relationship.- In the first sentence of the paper (line 37) it is a *Rube* Goldberg machine


Distributionally Robust Logistic Regression
This paper proposes a distributionally robust approach to logistic regression. We use the Wasserstein distance to construct a ball in the space of probability distributions centered at the uniform distribution on the training samples. If the radius of this ball is chosen judiciously, we can guarantee that it contains the unknown datagenerating distribution with high confidence. We then formulate a distributionally robust logistic regression model that minimizes a worst-case expected logloss function, where the worst case is taken over all distributions in the Wasserstein ball. We prove that this optimization problem admits a tractable reformulation and encapsulates the classical as well as the popular regularized logistic regression problems as special cases. We further propose a distributionally robust approach based on Wasserstein balls to compute upper and lower confidence bounds on the misclassification probability of the resulting classifier. These bounds are given by the optimal values of two highly tractable linear programs. We validate our theoretical out-of-sample guarantees through simulated and empirical experiments.

This paper presents distributionally robust logistic regression (DRLR); the uncertainty set under consideration here is the Wasserstein ball of distributions, centered at the empirical (feature, label) distribution.The authors reformulate the problem setup so that it leads to a tractable convex optimization problem.The authors go on to provide a generalization error guarantee (which depends, counter-intuitively, on the assumption that the data-generating distribution is not heavy-tailed), and also present (tractable) convex optimization problems for computing upper and lower bounds on this quantity.The authors present simulation results confirming the intuitive behavior of DRLR, and also show (on real data) that the method (essentially) doesn't hurt.Overall, I thought this was a nice contribution to the growing line of work on distributional robustness, and is likely to spur follow-on work.The paper was very well written (although there were a few typos scattered throughout, e.g., "week" --> "weak" on L62).I did, however, have a few comments re: the empirical evaluation:- Firstly, it is not really that surprising (to me, anyway) that your accuracy is essentially the same as that of your competitors on the real data (since typically, you have to sacrifice something when you gain in robustness) -- did you consider reporting CVaR on the test set for all the methods (you might see a better gain here, and this might more precisely characterize the performance of your method)?- Secondly, could you provide some discussion on how tractable the optimization problems (7, 10a, 10b) actually are?What are the specs of the computer you are using to solve these problems?Do you have any timing results?- How sensitive is your method to the choice of norm (you seem to use the Linfinity, L1, and L2 norm in your experiments, which indicates to me that no one norm is suitable for all problems) and kappa?Is the guidance for selecting these hyper-parameters simply (some kind of) cross-validation (which, btw, adds significantly to the computational cost, but does hinge on my previous question re: timing results)? - Part of your (stated) motivation for pursuing robustness is sparsity in the design matrices -- I wonder if you could conduct simulations where you play with fat vs. skinny design matrices with different levels of sparsity.- I wonder if you could conduct some simulations to check the accuracy of Thm. 2, as the light-tailed assumption does seem restrictive to me (it is good that you included Fig. 3c in the paper).