Efficient Globally Convergent Stochastic Optimization for Canonical Correlation Analysis
We study the stochastic optimization of canonical correlation analysis (CCA), whose objective is nonconvex and does not decouple over training samples. Although several stochastic gradient based optimization algorithms have been recently proposed to solve this problem, no global convergence guarantee was provided by any of them. Inspired by the alternating least squares/power iterations formulation of CCA, and the shift-and-invert preconditioning method for PCA, we propose two globally convergent meta-algorithms for CCA, both of which transform the original problem into sequences of least squares problems that need only be solved approximately. We instantiate the meta-algorithms with state-of-the-art SGD methods and obtain time complexities that significantly improve upon that of previous work. Experimental results demonstrate their superior performance.

The paper considers a stochastic version in CCA.  CCA is a useful tool that, given two snapshots of different set of variables (that come, however, from the same set of subjects), we are interested in revealing the most important correlations between the variables. State of the art algorithms work mostly in an offline fashion: given all datapoints from each set of variables, the task is to solve the maximization of a quadratic bilinear form, over quadratic constraints.  While the problem has a "closed-form" solution (after matrix transformations such as matrix inversions), only until recently some online/stochastic methods have been proposed. This paper considers such scenario for CCA and proposes 2 algorithms that solve the online CCA problem with global approximation guarantees.  The authors claim that this is the first time such guarantees are provided for such setting.  Experimental results show the superiority of the algorithms compared to state of the art.
Novelty: The novelty of the algorithm lies in providing global convergence guarantees for the online version of CCA. The algorithm has potential - however, this would have been better highlighted if real-applications were used (e.g. neuroscience experiments)  Impact/Significance: CCA is an important algorithm that finds applications in Cognitive Neuroscience, NLP, speech recognition, molecular biology, etc. To this end, having an efficient online algorithm for such settings is valuable.  Clarity: The authors do a great job presenting exactly the necessary information needed: from introducing CCA to the power method version of CCA, and to its ALS and stochastic formulation. At each step of the description, the authors provide remarks about what can go wrong: what happens when there is a gap, what about the normalization step (is it expensive?), what about input sparsity, etc. The flow of the paper is divided into small paragraphs/subsections, each of which provide an additional piece of information about the proposed algorithms. Really helpful for easy reading.  Other comments: 1. The authors should consider comparing also with the recent paper: Raman Arora, Poorya Mianjy and Teodor Marinov. Stochastic optimization for multiview learning using partial least squares. In Proceedings of the 32nd International Conference on Machine Learning (ICML), 2016. In their approach, the authors also consider a (partial) least-squares procedure for solution.  2. It would be helpful if the authors could describe in 1-2 sentences the findings, presented in Table 1. I.e., how much does log^2(1/eta) affect the performance over log(1/eta)? Is ASVRG always better than SVRG? If such overall comments are present, these will be really appreciated by readers. (Some of these comments are scattered over the text - maybe you could consider gathering all the comments together). 3. What is not clear from the text is whether the algorithm 1, which is taken from [7], wasn't proved to converge until now. If this is the first time this algorithm is proved, this should be stressed out in the text. If not, then the proof in the appendix is redundant and Theorem 1 should have a reference. This point is to discussed also with the reviewers, after the rebuttal. 4. How much significant is to set \epsilon(T) as proposed in Theorem 2? It seems that we need to know some information about the problem at hand (singular values, etc)?  5. It is not clear where the upper bound on the sub-optimality of initialization for SVRG is crucial for the proof of Lemma 3, or the overall analytical complexity of Algorithm 2. Can you please elaborate? 6. Maybe a small description of the technique shift-and-invert-preconditioning would be helpful to understand the idea behind it. Currently, the authors directly apply the steps, following similar notions from analogous PCA approaches.   Overall, this is a good paper and for the moment I have no reason to push for rejection. However, some re-writing of parts of the paper are needed for publication.


Spatio-Temporal Hilbert Maps for Continuous Occupancy Representation in Dynamic Environments
We consider the problem of building continuous occupancy representations in dynamic environments for robotics applications. The problem has hardly been discussed previously due to the complexity of patterns in urban environments, which have both spatial and temporal dependencies. We address the problem as learning a kernel classifier on an efficient feature space. The key novelty of our approach is the incorporation of variations in the time domain into the spatial domain. We propose a method to propagate motion uncertainty into the kernel using a hierarchical model. The main benefit of this approach is that it can directly predict the occupancy state of the map in the future from past observations, being a valuable tool for robot trajectory planning under uncertainty. Our approach preserves the main computational benefits of static Hilbert maps -using stochastic gradient descent for fast optimization of model parameters and incremental updates as new data are captured. Experiments conducted in road intersections of an urban environment demonstrated that spatio-temporal Hilbert maps can accurately model changes in the map while outperforming other techniques on various aspects.

The paper presents an approach to maintain and update a spatio-temporal occupancy map. Given a spatial position and time, the map gives a probability that location is occupied. Their first approach uses pre-processing of laser scans to get observations of motion for objects and fit a GP per object to predict its future motion. These GPs are then used to define features for a logistic regression classifier that predicts the probability a location is occupied. They also propose an approach that uses random Fourier features. They provide experiments to verify the utility of both of these approaches with respect to prior art.
Overall, the paper was well-written and proposes an improvement on an important problem.   As a general point, I'm not sure about the utility of including the Fourier features approach. Right now, that part of the paper seems under motivated and insufficiently evaluated. The best motivation is at the end of section 4, I think it would be better to move this further forward in the paper. If you decide to include this part of the paper, the experiments should be expanded. 5.2.1 currently shows that RFF features perform better in the temporal domain, but does not give a case for why RFF features are good for learning general motion dynamics or priors.  I think the explanation of the approach is section 3 is fairly approachable but it isn't until the end of the section that the reader gets a sense of what is being conveyed. I would include a simple example or step through of the process at the beginning of the section to give a broad strokes overview of the approach. It could also be useful to move Figure 1 higher up in the paper and refer to it earlier. I think it is also important to give some intuition for why the approach uses a feature embedding (rather than directly putting a GP on the occupancy map). It is done for efficiency reasons, but this is not made explicit for the reader.   Finally, the experiments section could use some work to make them more accessible. There are 6 experiments to evaluate the HF-STHM, please make it easier for readers to determine which figures and tables correspond to which experiments. Figure 2 shows the results of experiment 5.1.4, Figure 3 shows an illustration of the environment, Figure 4 shows the results of experiment 5.1.1, Figure 5 is not mentioned in the text (but I guess that it is the results of experiment 5.1.5). Please try to place figures in the order the appear in the text and include informative captions that explain the significance of the image. Please also make sure that each experiment clearly states: 1) the methods compared; 2) the dependent variables measured; and 3) the results of the experiment.    Figure 1 (b) also has some errors in it and should be revised.


Adaptive Maximization of Pointwise Submodular Functions With Budget Constraint
We study the worst-case adaptive optimization problem with budget constraint that is useful for modeling various practical applications in artificial intelligence and machine learning. We investigate the near-optimality of greedy algorithms for this problem with both modular and non-modular cost functions. In both cases, we prove that two simple greedy algorithms are not near-optimal but the best between them is near-optimal if the utility function satisfies pointwise submodularity and pointwise cost-sensitive submodularity respectively. This implies a combined algorithm that is near-optimal with respect to the optimal algorithm that uses half of the budget. We discuss applications of our theoretical results and also report experiments comparing the greedy algorithms on the active learning problem.

The paper addresses the problems of adaptive optimization with budget constraint. While previous studies address this problem for the average case, this study focuses on the worst case. In contrast to previous works on adaptive optimization with budget constraint, the authors of this paper consider not only modular cost functions but also general, possibly non-modular, cost functions. The authors investigate the near-optimality of two greedy policies for the worst-caseadaptive optimization problem with budget constraint (a policy is near-optimal if its worst-case utility is within a constant factor of the optimal worst-case utility). The authors prove that in the general case, these two policies cannot achieve near-optimality with non-uniform modular costs, but the best between these two greedy policies always achieves near-optimality. Following that, the main result of the paper is derived: The two greedy policies into one greedy policy that is near-optimal with respect to the optimal worst-case policy that uses half of the budget.  In addition, the authors propose a novel class of utility functions satisfying a property that is a generalization of cost-sensitive submodularity to the adaptive setting. For this new class of utilities, the authors prove similar near-optimality results for the greedy policies as in the case of modular costs.  The proposed algorithm is compared experimentally to other algorithms, and results show that even though it does not yield the best result in almost any case (which is not surprising and supported by the theoretical results in the paper), it is a robust choice for an algorithm that achieves the second best results in all cases.
Definitions and mathematical proofs are concise and easy to understand. Related work is covered in a sufficient manner. The paper exhibits a novel mathematical analysis relevant to adaptive optimization with budget constraint, a problem with many practical implications. The setting in which the analysis is given is wider compared to previous papers on the subject. Two greedy algorithms are analyzed, and despite negative results regarding their near-optimality, results (proofs) show that a combination of the two has interesting properties and can be sound and useful in some cases. In addition, a subset of utility functions under which strong positive results can be proved is correctly identified. The proposed algorithm is derived naturally from the theoretical results, and experiment results support the theoretical results. On the down side, the proposed algorithm does not perform better than other algorithms (but also never performs catastrophically bad), and was not examined in many different experiment settings.


Matching Networks for One Shot Learning
Learning from a few examples remains a key challenge in machine learning. Despite recent advances in important domains such as vision and language, the standard supervised deep learning paradigm does not offer a satisfactory solution for learning new concepts rapidly from little data. In this work, we employ ideas from metric learning based on deep neural features and from recent advances that augment neural networks with external memories. Our framework learns a network that maps a small labelled support set and an unlabelled example to its label, obviating the need for fine-tuning to adapt to new class types. We then define one-shot learning problems on vision (using Omniglot, ImageNet) and language tasks. Our algorithm improves one-shot accuracy on ImageNet from 87.6% to 93.2% and from 88.0% to 93.8% on Omniglot compared to competing approaches. We also demonstrate the usefulness of the same model on language modeling by introducing a one-shot task on the Penn Treebank.

The article describes a one-shot learning model called matching network. The basic idea of the model is to compute the output category of one given test example by computing a kind of ‘similarity’ with all the training examples that are stored in a memory. This similarity (denoted a(x,x_i)) is based on two representation functions f and g which goals are to encode both the new example, but also the examples stored in memory. The proposed model is thus very close to siamese networks, but using an asymmetric architectures since g and f can be different functions. The main originality is contained in two particular aspects: first, instead of computing the g function on each training example, the model proposes a sequential model (bidirectional LSTM) to learn to encode the training examples based on the previously seen examples. Second, the set of examples is also used to learn to encode any incoming datapoint. These two components will be learned over a set of one-shot learning tasks, resulting in a ‘learning to learn’ problem. Experiments are made on different images datasets and one language dataset. The proposed approach is compared to baseline models, MANN and siamese nets (not on all the datasets…) and shows interesting results
Comments: the idea contained in the paper is not a very big contribution to the field but still remains interesting, and the experiments on different datasets provide important information concerning the behavior of the model. But the paper has many different problems that make it difficult to understand. The first problem is in the structure of the paper since the problem formulation is in fact only given in Section 2.2 while I think it is important to well define the one-shot learning problem at the beginning of the paper. It would make the article easier to read and to follow. Sections  2.1 and 2.1.1 are well written but could include more precise definitions. For example, in Equation 1, y_i is not defined : is it a real number, a one hot vector (I suppose it is a one hot vector). The core of the contributions in Section 2.1.2 is very difficult to well understand. It is not clear how the bidirectional LSTM is used, but the more difficult part concerns the way f(\hat(x)) is defined. The authors could clearly give more details since this section is in fact the main contribution of the article. Written as it is, it makes the model difficult to understand, and almost impossible to reproduce. At last, if these sections can be understood by researchers very familiar to deep learning, it will not be the case for ML researchers used to read more ‘classical’ papers while the principle of the paper is more general than deep learning and can be for example understood from a metric learning point of view. I clearly advise the authors to improve the way the model is described. Concerning the experiments, it is really interesting to see that this type of approach improves the quality of the system w.r.t Siamese networks. But the Siamese net results are only provided for omniglot and not for imagenet which is a little bit strange. The language model part also does not contain any table/figure showing a summary of the results which make the discourse difficult to follow in this section.   To conclude, if the article provides a simple but interesting idea for one-shot learning, the way it is written must be drastically improved to allow a publication in a top conference.


Multistage Campaigning in Social Networks
We consider the problem of how to optimize multi-stage campaigning over social networks. The dynamic programming framework is employed to balance the high present reward and large penalty on low future outcome in the presence of extensive uncertainties. In particular, we establish theoretical foundations of optimal campaigning over social networks where the user activities are modeled as a multivariate Hawkes process, and we derive a time dependent linear relation between the intensity of exogenous events and several commonly used objective functions of campaigning. We further develop a convex dynamic programming framework for determining the optimal intervention policy that prescribes the required level of external drive at each stage for the desired campaigning result. Experiments on both synthetic data and the real-world MemeTracker dataset show that our algorithm can steer the user activities for optimal campaigning much more accurately than baselines.

The paper builds on existing work in shaping social activity on social networks modeled as multivariate Hawkes processes (developed in citation [8] for constant-in-time control schemes). The paper characterizes a relationship between time-dependent exogenous activity and average activity rate in the network for exponential kernels, and uses this to derive an approximate dynamic programming control algorithm for driving activity using piecewise-constant control signals. The paper compares the performance of their control algorithm to baselines on synthetic networks with simulations, as well as real-data (using a custom performance metric) and shows promising although somewhat difficult to interpret results.
The theoretical contributions (relationship between time-dependent exogenous intensity to average activity) appear significant, and the use of this result to derive a closed-form control algorithm appears to be a contribution to the field of shaping activities in social (and other) networks modeled as multivariate Hawkes processes.  Regarding the clarity of this paper:  The paper does not frame itself relative to prior work clearly. Clearly indicating where and how this work is a generalization of prior work [8] would improve clarity of the paper, and highlight the core contribution of handling *time-dependent* exogenous events. Stating that the paper "establishes theoretical foundations of optimal campaigning over social networks where user activities are modeled as multivariate Hawkes processes" does not properly localize the work relative to [8]. This could be improved in part by mentioning [8] in the introduction, and specifically stating that prior work has done optimal control with constant exogenous control, and that this paper addresses multi-stage exogenous intensity. Similarly, in section 3 “From Intensity to Average Activity”, it should be made clear that this is an extension of a previous analogous result for constant-time activity. Finally, the three objective functions of section 4 appear to mirror the three objective functions from [8], but the text does not reveal this when it states “In what follows, we introduce several instances of the objective function...”. These changes may help the reader focus on the core and cohesive technical contribution.  The description of the closed-loop approximate dynamic programming control algorithm requires more precise language. For example, Lines 241-252 are unclear: The excerpt “By observing the counting process in previous stages summarized in a sequence of x_m) and taking future uncertainty into account, the control problem is…” needs to be more explicit (using mathematical variables) about what specific data is given to the control algorithm, and what are the specific random variables causing the "future uncertainty". For example, in Line 257: “However, when control policy \pi_m is to be implemented, only x_m is observed and there are still uncertainties in future..” Why is this a problem? Assuming “implemented” means that the policy was already determined, then isn’t implementing the policy a matter of evaluating the function \pi_m on input x_m?. Being more clear about the *computation* represented by Equation 19 and the *computation* representing Equation 18 would make it more clear how Algorithm 1 is derived in a stage-wise process (e.g. on Line 268: “Solving for .. is analytically intractable [due to ...]").  How significant is the choice of exponential kernel? It seems it may be critical to the control algorithm. If this is indeed a critical requirement, it deserves some more discussion besides just calling it “standard”.  Based on the text, in Figure 1 the error bars seem to represent standard deviation and not a form of confidence intervals, but this could be made more explicit in the figure caption.  Only the acronyms of the baselines used in Figure 1 and Figure 2 were given in the main text. The identity of these baselines could be briefly stated in the caption of Figure 1 (with citations).  Regarding the potential impact of the work:  The experimental results on synthetic data did not seem as significant of an improvement over open-loop one-time optimization over all stages as might be expected given that the closed-loop algorithm has access to significantly greater information. A demonstration for network parameter settings that better highlight the novel algorithm's strengths relative to baselines, paired with a demonstration of network parameter settings for which the distinction is less striking, would make the potential impact of the technique more clear. A discussion of in which contexts the proposed algorithm is likely to significantly improve performance would also strengthen the clarity as well as make it easier to assess impact.  The experimental results on real-world data are based on a performance metric of “predicting which cascade will reach the objective function better. They should be able to answer this by measuring how similar their prescription is to the real exogenous intensity...” This metric needs a clearer and more mathematically precise definition---the results of Figure 2 seem promising, but are hard to interpret based on the lack of clarity in the prose description of this key metric.


Boosting with Abstention
We present a new boosting algorithm for the key scenario of binary classification with abstention where the algorithm can abstain from predicting the label of a point, at the price of a fixed cost. At each round, our algorithm selects a pair of functions, a base predictor and a base abstention function. We define convex upper bounds for the natural loss function associated to this problem, which we prove to be calibrated with respect to the Bayes solution. Our algorithm benefits from general margin-based learning guarantees which we derive for ensembles of pairs of base predictor and abstention functions, in terms of the Rademacher complexities of the corresponding function classes. We give convergence guarantees for our algorithm along with a linear-time weak-learning algorithm for abstention stumps. We also report the results of several experiments suggesting that our algorithm provides a significant improvement in practice over two confidence-based algorithms.

The papers considers a new boosting algorithm for classification with abstention. In each round, the algorithm selects a a base learner which is a pair of functions (h(x),r(x)), where h concerns classification into positive/negative labels, while r indicates the abstention region. The paper contains extensive theoretical analysis of the proposed approach, including: convex surrogates for the proposed loss function, calibration of exponential-loss convex surrogate, generalization bounds by means of Rademacher complexity, derivation of the boosting algorithm and its convergence guarantees. A short experimental study concludes the paper.
The paper presents an interesting, and (to the best of my knowledge) novel approach to binary classification with abstention. The standard approach to this problem is via confidence-based abstention: "estimate" the class conditional probability function \eta(x) (or some monotonic transformation thereof) and reject (abstain from classification) when \eta(x) is close to 1/2. The authors points out that while this is all that is needed to find the optimal solution in the class of all classification functions (because the Bayes solution is of that form), in practical cases, when the class of functions is constrained, it can be beneficial to use a more general approach, where there are two functions, h and r, h being responsible for classification, and r -- for deciding upon rejection.  The paper contains various theoretical results concerning the proposed approach: - Convex surrogates for the original (task) loss function, with the proof of calibration of exponential convex surrogate, - Guarantees on the generalization error of the ensemble by means of the convex surrogate and the Rademacher complexity of the function class which depends on the Lipschitz constant of the convex surrogate (truncated at 1), - Derivation of new boosting algorithms based on the proposed convex surrogates using the method of projected coordinate descent, - Proof of the convergence of the projected coordinate descent, - A new type of based learner suitable for this problem -- abstention stumps, together with a method for fast selection of the optimal stump at each iteration. - A short experimental study, which is actually the weakest part of the paper due to small size of the data sets used for the comparison.  The paper is generally clearly presented, but due to numerous theoretical results, it is quite dense. I have some minor suggestions on the presentation (below) that may help the final version.  Overall, I think the contribution of the paper is strong. I am in favor of accepting it.  Minor suggestions:  l. 68: "part of our analysis is applicable to the general case" -- please explain which part. l. 82-84 and Fig. 1: the Bayes solution is not unique here. l. 98-99: repeated sentence ("ham based on the presence of some other word") l. 60, 69, 105: unify notation for open and closed intervals eq. between lines 113-114: h(x)) -> h(x) l. 121: I think it better to write \Phi_1(x) = \Phi_2(x) = \exp(x)  Theorem 1: it is not clear what are h^* and r^* -- is (h^*,r^*) a Bayes classifier for the task loss L? Also the proof of Theorem 1 is unclear. In l. 467 (Appendix): in fact when \eta(x) \in {0,1}, there is no minimizer (for example, when \eta(x)=1, the infimum is attained by r(x) -> infinity and h(x) - r(x) -> infinity). Furthermore, in l. 476-477 (Appendix) you write that the Bayes classifier (of the task loss L?) satisfies h^*(x) = \eta(x) - 1/2 and r^*(x) = |h^*|-1/2+c, but this is not true. In fact there exists Bayes classifier which satisfies that, but this is not the only one: any (h^*, r^*) is a Bayes classifier as long as sgn(h^*) = sgn(eta(x)) and sgn(r^*) = sgn(|\eta(x)-1/2|-1/2-c). This is satisfied for the solution of the surrogate if and only if b/a = 2sqrt((1-c)/c), so Theorem 1 is correct, but I think the proof should be improved.  Theorem 2: explain the notation -- R is the Rademacher complexity (define it in the Appendix to make it more self-contained).  Theorem 2: Perhaps you could add a comment below Thm 2 that it also applies to L_SB since it is an upper bound on L_MB (similar to the comment below Corollary 3)  Figure 3: (algorithm) It is unclear, what are the parameters of the algorithm. From what I understand, they are c and \beta? I would suggest to say anything about the role of \beta in the algorithm (from what I understand, it acts as a regularization strength).  Experiment: it is not explicitly stated what kind of convex surrogate is used in the BA algorithm. I guess this is exactly the algorithm presented on Figure 3 (with exponential loss, L_SB-type).  ==========  I have read the rebuttal which clarified most of the issues raised in my review.


Online and Differentially-Private Tensor Decomposition
Tensor decomposition is an important tool for big data analysis. In this paper, we resolve many of the key algorithmic questions regarding robustness, memory efficiency, and differential privacy of tensor decomposition. We propose simple variants of the tensor power method which enjoy these strong properties. We present the first guarantees for online tensor power method which has a linear memory requirement. Moreover, we present a noise calibrated tensor power method with efficient privacy guarantees. At the heart of all these guarantees lies a careful perturbation analysis derived in this paper which improves up on the existing results significantly.

NOTE TO AUTHORS AND TPC: I was asked to provide an additional review for this paper after the rebuttal period, but did not read the other reviews or rebuttal prior to writing this review. I received some assistance in this review from a graduate student who also read the paper.  This is a "light" review. I did not have an opportunity to verify the proofs or verify correctness.  SUMMARY: This paper proposes a new method for online tensor decomposition based on the power iteration and in particular on its robustness to noise. They then (perhaps inspired by Hardt and Price) suggest that this method could give effective differentially private algorithms for the power iteration, provided the noise level is sufficiently low (or \epsilon is sufficiently large).
 STRENGTHS: - The problem addressed by the authors is important - The proposed algorithm is relatively novel and seems effective - While I didn't verify the proofs, the arguments appear correct.  WEAKNESSES: - I found the application to differential privacy unconvincing (see comments below) - Experimental validation was a bit light and felt preliminary  RECOMMENDATION: I think this paper should be accepted into the NIPS program on the basis of the online algorithm and analysis. However, I think the application to differential privacy, without experimental validation, should be omitted from the main paper in favor of the preliminary experimental evidence of the tensor method. The results on privacy appear too preliminary to appear in a "conference of record" like NIPS.  TECHNICAL COMMENTS:  1) Section 1.2: the dimensions of the projection matrices are written as $A_i \in \mathbb{R}^{m_i \times d_i}$. I think this should be $A_i \in \mathbb{R}^{d_i \times m_i}$, otherwise you cannot project a tensor $T \in \mathbb{R}^{d_1 \times d_2 \times \ldots d_p}$ on those matrices. But maybe I am wrong about this...  2) The neighborhood condition in Definition 3.2 for differential privacy seems a bit odd in the context of topic modeling. In that setting, two tensors/databases would be neighbors if one document is different, which could induce a change of something like $\sqrt{2}$ (if there is no normalization, so I found this a bit confusing. This makes me think the application of the method to differential privacy feels a bit preliminary (at best) or naive (at worst): even if a method is robust to noise, a semantically meaningful privacy model may not be immediate. This $\sqrt{2}$ is less than the $\sqrt{6}$ suggested by the authors, which may make things better?  3) A major concern I have about the differential privacy claims in this paper is with regards to the noise level in the algorithm. For moderate values of $L$, $R$, and $K$, and small $\epsilon = 1$, the noise level will be quite high. The utility theorem provided by the author requires a lower bound on $\epsilon$ to make the noise level sufficiently low, but since everything is in "big-O" notation, it is quite possible that the algorithm may not work at all for reasonable parameter values. A similar problem exists with the Hardt-Price method for differential privacy (see a recent ICASSP paper by Imtiaz and Sarwate or an ArXiV preprint by Sheffet). For example, setting L=R=100 and K=10, \epsilon = 1, \delta = 0.01 then the noise variance is of the order of 4 x 10^4.  Of course, to get differentially private machine learning methods to work in practice, one either needs large sample size or to choose larger $\epsilon$, even $\epsilon \gg 1$. Having any sense of reasonable values of $\epsilon$ for a reasonable problem size (e.g. in topic modeling) would do a lot towards justifying the privacy application.  4) Privacy-preserving eigenvector computation is pretty related to private PCA, so one would expect that the authors would have considered some of the approaches in that literature. What about (\epsilon,0) methods such as the exponential mechanism (Chaudhuri et al., Kapralov and Talwar), Laplace noise (the (\epsilon,0) version in Hardt-Price), or Wishart noise (Sheffet 2015, Jiang et al. 2016, Imtiaz and Sarwate 2016)?  5) It's not clear how to use the private algorithm given the utility bound as stated. Running the algorithm is easy: providing $\epsilon$ and $\delta$ gives a private version -- but since the $\lambda$'s are unknown, verifying if the lower bound on $\epsilon$ holds may not be possible: so while I get a differentially private output, I will not know if it is useful or not. I'm not quite sure how to fix this, but perhaps a direct connection/reduction to Assumption 2.2 as a function of $\epsilon$ could give a weaker but more interpretable result.  6) Overall, given 2)-5) I think the differential privacy application is a bit too "half-baked" at the present time and I would encourage the authors to think through it more clearly. The online algorithm and robustness is significantly interesting and novel on its own. The experimental results in the appendix would be better in the main paper.  7) Given the motivation by topic modeling and so on, I would have expected at least an experiment on one real data set, but all results are on synthetic data sets. One problem with synthetic problems versus real data (which one sees in PCA as well) is that synthetic examples often have a "jump" or eigenvalue gap in the spectrum that may not be observed in real data. While verifying the conditions for exact recovery is interesting within the narrow confines of theory, experiments are an opportunity to show that the method actually works in settings where the restrictive theoretical assumptions do not hold. I would encourage the authors to include at least one such example in future extended versions of this work.


Learning Deep Embeddings with Histogram Loss
We suggest a loss for learning deep embeddings. The new loss does not introduce parameters that need to be tuned and results in very good embeddings across a range of datasets and problems. The loss is computed by estimating two distribution of similarities for positive (matching) and negative (non-matching) sample pairs, and then computing the probability of a positive pair to have a lower similarity score than a negative pair based on the estimated similarity distributions. We show that such operations can be performed in a simple and piecewise-differentiable manner using 1D histograms with soft assignment operations. This makes the proposed loss suitable for learning deep embeddings using stochastic optimization. In the experiments, the new loss performs favourably compared to recently proposed alternatives.

The authors provide a new loss function for learning embeddings in deep networks, called histogram loss. This loss is based on a pairwise classification: whether two labels belong to the same class or not. In particular, the authors suggest to look at the similarity distribution of the embeddings on the L2 unit sphere (all embeddings are L2 normalized). The idea is to look at the distribution of the similar embedding (positive pairs) and the distribution of the non-similar ones (negative pairs) and make the probability that positive pairs  has smaller score then negative pairs, smaller.  After reviewing previous work in the area (Section 2), in Section 3 they develop a method how to estimate the Histogram loss. They begin with the definitions of Histogram (eq. 1, and 2) and based on them give the definition of $p_reverse$ which later we they want to minimize (eq.3). Afterward, they present an approximation to $p_reverse$ and show that it is differentiable. They conclude  Section 3 with showing a connection to the quadruplet loss.  In Section 4 the authors compare their method to other state of the art embedding loss, specifically, they consider the binomial deviance loss (eq. 7 and 8) and Lifted Structured Similarity Softmax (LSSS; eq. 9). They try these loses on 4 datasets: CUB200-2011 [25], CUHK03 [11], Market-1501 [29] and Online Products.   In addition, different architectures for the neural network are used: For training on the CUB-200-2011 and the Online Products they use GoogleNet while for CUHK03 and Market-1501 they used the Deep Metric Learning architecture. The optimizatin algorithm used in this work is Adam (for stochastic optimization).  They show the results using the Recall@K performance on all the datasets with different methods in Figures 3 and 4. The authors show how their method is sometime competitive and sometimes outperforms previous state of the art algorithms in the embeddings area.
I like and see the benefit of the authors' method for embeddings. The (almost) parameter free method is very appealing and have its benefits. The authors simply show the results of their algorithm and demonstrate that it works. Also, after many works where I had to read the supplementary material in order to understand the work, I appreciate the authors non-supplementary paper! Also, their throughout comparison to other methods in terms of loss functions, datasets, and different parameters, is very good. Also, I like the choice of the datasets, where there are many classes, which seems natural to this kind of loss.   Two things bother me in this work:  1) The authors do not explain what is recall@K, where I needed to go back through the papers "Deep Metric Learning via Lifted Structured Feature Embedding" and then "Product quantization for nearest neighbor search". I could postulate what is the definition in this case is, but still it affected the self-contained nature of this work.  2) After doing so much profound work, it was not clear to me why each dataset has different set of graphs. I miss a good reason why such comparison is adequate in this context, especially in the light of that the authors did most of the work for throughout comparison.   Typos: * In figure 5 caption, should it read |Red is for *positive* pair, green is for *negative* pairs?


One-vs-Each Approximation to Softmax for Scalable Estimation of Probabilities
The softmax representation of probabilities for categorical variables plays a prominent role in modern machine learning with numerous applications in areas such as large scale classification, neural language modeling and recommendation systems. However, softmax estimation is very expensive for large scale inference because of the high cost associated with computing the normalizing constant. Here, we introduce an efficient approximation to softmax probabilities which takes the form of a rigorous lower bound on the exact probability. This bound is expressed as a product over pairwise probabilities and it leads to scalable estimation based on stochastic optimization. It allows us to perform doubly stochastic estimation by subsampling both training instances and class labels. We show that the new bound has interesting theoretical properties and we demonstrate its use in classification problems.

The authors introduce a lower bound on the softmax probability frequently used in multi-class learning, based on combining one-vs-one probabilities. They demonstrate that this bound can be used to train large-scale multi-class models.
In my view, the main reason the proposed lower bound is interesting is that it offers a potential way to speed up training for multi-class models with a very large number of classes. While it is useful to understand other properties of the lower bound, the paper could be improved by emphasizing this primary use case in machine learning.  Figure 1c and Figure 3 need a more clear explanation of what is being displayed, and why it is important. In particular, what value is being plotted on the y-axis, and at what setting of the parameters w. Here is how I understand it, for Figure 1c say:  Blue Line - value of Eq. (13) at the setting of parameters w that maximize 13  Red Line - value of Eq. (13) at the setting of parameters w that maximize 14  Green Line - value of Eq. (13)? at the setting of parameters w that maximize the Bouchard lower bound (?) Red dashed line - value of Eq. (13)? at parameters w based on the given iterations of training? But if it is always the actual log-likelihood on the y axis (Eq 13), then I'm not sure why the y-axis is labelled "lower bound" and you use the phrase "maximized (approximate) log likelihoods". The same comments apply to Figure 3.  In order to effectively demonstrate that the proposed bound can significantly speed up training, additional experiments and comparisons are needed. Most importantly, another natural proxy loss that can be used in training is to only compute the softmax over a random sample of the full output space (vocabulary.) I believe this is a fairly standard trick (e.g., implemented in tensorflow as tf.nn.sampled_softmax_loss, https://www.tensorflow.org/versions/r0.9/api_docs/python/nn.html#sampled_softmax_loss). See also Section 3 of http://arxiv.org/pdf/1412.2007v2.pdf and references therein.  A critical parameter in both approaches is the number of classes used in the approximation of the softmax. I'd like to see that parameter explored, with some guidance on how to choose it in practice. For example, in Figure 2(c) I'd like to see lines for the number of classes in the stochastic approximation ranging from 1 to 10^4, as well as a line for SGD with computation of the full softmax. But ideally this parameter would be explored on the real datasets as well, with the key metric being wall-clock-time to reach a given level of accuracy.  For the small-scale experiments, in Table 2, the Softmax column has its fields in a different order, and is (critically) missing the error column. It's not clear from the text if the metrics are on the training set or the test set --- I assume the test set. But in that case it's hard to dismiss the Bouchard method as overfitting if it is actually getting the lowest test-set error. This method is more computationally expensive, but if that is the only downside, you should include experiments that show this (in particular to justify not including it in the comparison on the large-scale problem).  For the amazon-13k results, no comparisons are given, so it is hard to draw conclusions. To be meaningful, there should be a comparison to the sampled softmax mentioned above. You could also compare to full softmax but with a random projection or hashing trick on the input features to a lower-dim space (few thousands) where this would be more practical --- this would be a natural baseline. Again, since computational speedups are the main potential advantage here, those should be clearly illustrated.  Minor comments: Figure 1(a): I assume these are probabilities estimated by optimizing using the given loss, but then with probabilities all computed using Eq (2) (rather than say computing estimated probabilities for the Bouchard method using Eq. (11)). Please clarify this. Several typos and missing words, e.g. line 10 (allows *us*), 37 (*wheneven*), 114 (Not only *is F(f)* globally …), 122 (in order *for* the bound). Line 244 should refer to Table 2 not Table 1. Lines 235-237: These methods are all naturally singly stochastic, and can (and probably should) be solved with SGD (sampling examples) not GD. It should be clarified that while the global optimum is the same for Eqs. (6) and (8), it is not the case that (13) and (14) share the same minimum.


Without-Replacement Sampling for Stochastic Gradient Methods
Stochastic gradient methods for machine learning and optimization problems are usually analyzed assuming data points are sampled with replacement. In contrast, sampling without replacement is far less understood, yet in practice it is very common, often easier to implement, and usually performs better. In this paper, we provide competitive convergence guarantees for without-replacement sampling under several scenarios, focusing on the natural regime of few passes over the data. Moreover, we describe a useful application of these results in the context of distributed optimization with randomly-partitioned data, yielding a nearly-optimal algorithm for regularized least squares (in terms of both communication complexity and runtime complexity) under broad parameter regimes. Our proof techniques combine ideas from stochastic optimization, adversarial online learning and transductive learning theory, and can potentially be applied to other stochastic optimization and learning problems.

The paper studies the problem of minimizing the average of a finite sum of convex functions over a convex domain using stochastic algorithms that, opposed to most popular methods, apply WITHOUT-replacement sampling to the data. More specifically, the paper considers methods that first randomly permute the functions, and then process the functions via incremental updates one at a time, making at most a single pass over the data (hence the data is only shuffled once). There are three main motivations for considering stochastic methods with without-replacement sampling: 1. it is observed many times empirically (and to the best of my knowledge this is what ML practitioners really do) that applying random shuffles to the data and then apply incremental updates works better than with-replacement SGD. 2. After the data is randomly permuted, these algorithms require only sequential access to the memory, which is much more efficient than standard with-replacement SGD methods that require random access to perform the updates. Since the main setting under consideration here is when making at most a single pass, it sounds plausible to assume that the data is already stored in some random permutation, and hence the algorithm is fully sequential, and there is no need to even artificially permute the data. 3. In a distributed setting in which data is partitioned to several machine (not assuming the data is sampled i.i.d from a distribution), it is more natural and efficient to analyze without-replacement algorithms.    While this setting is well motivated there has not been a large body of work with strong positive results on it which is understandable, since analyzing without-replacement sampling algorithms is much more difficult than the with-resampling variants. Once such previous recent result focused on the regime in which the algorithm makes a large number of passes over the data, re-permuting the data before each new pass. This current paper focuses on a regime which I find more interesting and relevant to ML applications, in which at most a single pass is made over the data. Indeed the celebrated with-replacement SGD method is most interesting exactly in this regime (though suffering from the short-coming of with-replacement sampling discussed above).  The main results are as follows: 1. the paper shows that any algorithm for which one can give a regret bound in the online learning adversarial model (which holds for SGD for instance) can be applied in the setting above with convergence rate resembling the average regret bounds. In particular, when all functions are convex Lipschitz losses for a linear predictor, wo-replacement SGD exhibits roughly the same convergence rate as standard SGD.   2. when the average over all functions is also strongly convex, wo-replacement SGD gives an improved rate, again similar to standard SGD up to log factors. 3. The authors gives a linearly converging wo-replacment variant for the celebrated SVRG algorithm in the specific yet important case in which the loss functions are the squared-loss. 4. The authors use the previous result to improve the application of SVRG for distributed optimization in the wo-replacement setting, improving the regime of parameters (i.e. the condition number) for which the method is highly efficient (in terms of communication).
As I detailed above, I believe the setting under consideration (without-replacment sampling for finite sum optimization using single pass) is very well motivated and widely applicable in ML problems. The results, to the best of my knowledge, are very novel and will be potentially interesting to many in the community, both theorists and practitioners. The paper is also very well written and results are carefully explained.  Some minor comments: -when defining F_{1:t-1}, I think it will also be beneficial for the reader to add a subscript of \sigma, reminding the reader this function is random. - in Lemma 1, please define s_{1:t-1} and s_{t:m}. - in Theorem 2, in the centered equation, it should be \mathcal{R}_{t-1,m-t+1}, i.e., comma instead of colon. - first eq. in proof of Theorem 3: on the RHS it should be the squared norm of w_{t+1}.


Conditional Image Generation with PixelCNN Decoders
This work explores conditional image generation with a new image density model based on the PixelCNN architecture. The model can be conditioned on any vector, including descriptive labels or tags, or latent embeddings created by other networks. When conditioned on class labels from the ImageNet database, the model is able to generate diverse, realistic scenes representing distinct animals, objects, landscapes and structures. When conditioned on an embedding produced by a convolutional network given a single image of an unseen face, it generates a variety of new portraits of the same person with different facial expressions, poses and lighting conditions. We also show that conditional PixelCNN can serve as a powerful decoder in an image autoencoder. Additionally, the gated convolutional layers in the proposed model improve the log-likelihood of PixelCNN to match the state-ofthe-art performance of PixelRNN on ImageNet, with greatly reduced computational cost.

This paper investigates the problem of conditional image generation based on the pixel (R/C)NN framework. Building upon the previous pixel (R/C)NN framework, this paper proposes a gated extension of pixel CNN using multiplicative interactions which can be trained efficiently. The main focus of this paper lies in the conditioning modeling of image distribution p(x|h), where h is assumed to be a vector (e.g., one-hot vector or feature vector from pre-trained encoder networks). In addition, the proposed gated pixel CNN can be jointly trained with an encoder network, which is referred as pixel CNN auto-encoders. The model is validated in experiments on CIFAR-10, ImageNet, and Portraits dataset. For unconditional modeling, the performance of gated extension is on par with (slightly better on ImageNet) original pixel CNN. For conditional modeling, the model is able to generate reasonably-looking and diverse samples that agrees with the conditioning vector (either class label or face embedding).
The paper solves a significant problem in generative modeling and the paper is quite interesting. However, reviewer feels the current version is not polished well due to several issues in the experimental section. For rebuttal, please focus on the (*), (**), (***) and (***) mentioned in the following paragraphs. Reviewer is willing to change the score if all the concerns are addressed in the rebuttal.  Novelty: The proposed model is technically novel in the sense that it explores the conditional modeling with the recent pixel (R/C)NN framework. The extension of pixel-CNN to auto-encoders is interesting.  In a high-level, however, generating images with conditioning variables has been explored to some extent: -- Learning to Generate Chairs with Convolutional Neural Networks by Dosovitskiy et al. -- Deep Generative Image Modeling using a Laplacian Pyramid of Adversarial Networks by Denton et al. -- Deep Convolutional Inverse Graphics Network by Kulkarni et al. -- Attribute2Image: Conditional Image Generation from Visual Attributes by Yan et al. -- Generating Images from Captions with Attention by Mansimov et al.  Clarity: The paper is easy to follow and clearly written in general. However, it is hard to reproduce the results solely based on the description. Reviewer suggests the authors to submit supplementary document or consider code release in the future. References are incomplete.  Technical quality: The current experimental result fails to convince reviewer for the following reasons: -- In unconditional modeling, whether the proposed gated pixel CNN is indeed significant enough compared to original pixel CNN [21]? In the paper, the only comparisons between original pixel CNN and gated pixel CNN are detailed in Table 1 and Table 2. However, it is possible that performance gap comes from more parameters and deeper architecture of gated pixel CNN. *For rebuttal, please describe the quality of samples (comment the difference compared to original pixel CNN) and report the numbers under fair comparison. -- In conditional modeling, whether the improvements in visual quality is a sign of overfitting? It would be more convincing if the nearest training examples are added along with generated samples (as has been done in LAP-GAN paper). **For rebuttal, please comment on the overfitting risk and visual similarity between generated samples and nearest training examples. -- In conditional modeling, whether the proposed model is superior to original pixel CNN and other existing previous work is unknown. It seems that Eq.(4) and Eq.(5) can also be applied to original pixel CNN [21] as well. For other existing work on conditional image generation, please refer to class-conditional generation on CIFAR-10 dataset (LAP-GAN) and text-conditional generation on MS-COCO dataset (alignDRAW [10]). It would be more convincing if the proposed model is evaluated against the original pixel CNN [21] or other existing previous work. ***For rebuttal, please comment on the performance of conditional generation using original pixel CNN.   Usefulness: One issue with the original pixel (R/C)NN framework is that the generation cannot be made in parallel (one needs to generate one pixel at a time). This would be an issue in practice since the competitor models (condVAE and condGAN) can generate images in a much more efficient way. ****For rebuttal, please make it clear if this problem has been resolved in the proposed method.  Detailed Comments: -- Eq. (2): “W_{g,f}” → “W_{k,g}” -- Eq. (4) vs. Eq. (5): are you applying conditioning variable to every pixel CNN layer or not? Which one has been used in the experiments (imagenet and portraits)? Any suggestions when choosing Eq. (4) or Eq. (5) in practice? -- The description of the “portraits” dataset that being used is missing. How many images are there in the dataset? Whether the identities are overlapping between training set and testing set?


Learning Multiagent Communication with Backpropagation
Many tasks in AI require the collaboration of multiple agents. Typically, the communication protocol between agents is manually specified and not altered during training. In this paper we explore a simple neural model, called CommNet, that uses continuous communication for fully cooperative tasks. The model consists of multiple agents and the communication between them is learned alongside their policy. We apply this model to a diverse set of tasks, demonstrating the ability of the agents to learn to communicate amongst themselves, yielding improved performance over non-communicative agents and baselines. In some cases, it is possible to interpret the language devised by the agents, revealing simple but effective strategies for solving the task at hand.
First, we simplify and extend the graph neural network is architecture can be used to control groups of cooperating f multilayer neural networks f i that take as input vectors odel takes as input a set of vectors {h
} (we will call K the number of hops in the network). output them directly, so that the model outputs a vector an feed them into another network to get a single vector or by a nonlinearity :
d, and the matrix can be dynamically sized because the oordinate.
(we will call K the number of hops in the network). tput them directly, so that the model outputs a vector feed them into another network to get a single vector or y a nonlinearity :
, and the matrix can be dynamically sized because the ordinate.
tion Processing Systems (NIPS 2016). Do not distribute.
uthor(s) n t implify and extend the graph neural network re can be used to control groups of cooperating r neural networks f i that take as input vectors as input a set of vectors {h l call K the number of hops in the network).
directly, so that the model outputs a vector into another network to get a single vector or nearity :
matrix can be dynamically sized because the ssing Systems (NIPS 2016). Do not distribute.
ically sized, and the matrix can be dynamically sized because the r than by coordinate.
eural Information Processing Systems (NIPS 2016). Do not distribute.

The paper deals with centralized learning in the case of multi-agent collaboration problems. More specifically, it proposes a deep-learning approach for learning to collaborate when every agent needs to communicate with others (each agent only gets a partial view of the environment), but no communication protocol has been stated beforehand.
The model is a deep network which consists of a stack of layers, with parameter sharing between modules of a same layer. This parameter sharing allows the number of agents to vary during the task. Also, it allows to drastically reduce the number of parameters to be learned. The key idea of the paper is to use the output of every module of a given layer to build the communication input for the next layer. While this appears to obtain interesting results in the reported experiments, I find this proposal very straightforward and poorly innovative, as it corresponds to a quiet classical neural network structure. In fact, that is a classical network with shared parameters, except that the output of any agent $i$ is concatenated  with (rather than added to) the summed outputs of every others to form the input of this agent $i$ for the next layer. I am not sure about the intuition of what this could bring in the context of the considered tasks.   Experiments should have at least compared the proposal with a classical network with parameter sharing to highlight the benefits of the proposed mechanism. Also, a centralized controller that has access to all observations of all agents should perform better than the studied setting. So, if the dense baseline corresponds to such a centralized controller scheme (as suggested by the authors answer), there is no reason for me why this baseline does not get greatly better results... Since getting the whole information, this baseline should perform better than approaches with communication between partial views. Maybe that is because it has too many parameters and this is due to an optimization problem ? That's also why I asked a baseline as dense but with shared parameters between modules of a same layer to be able to compare with the proposal and to lower the learning complexity. Anyway, I am not sure on how the dense baseline works in the various settings...    Other comments:    - As the main argument is to maintain a network where the structure can be dynamically sized, it would have been useful to give in introduction some examples of applications where this dynamicity is particularly important    - I am not sure about the relevance of presenting the reinforcement policy gradient in section 2, as : 1) the proposed model can be learned via supervised learning in some applications and 2) this is not at the heart of the proposal at all. I would give it (with more details to be more self-contained) after the presentation of the model (maybe in a section Learning or something like that). Like this it confuses the discourse a lot from my point of view    - Not enough details of the experimental settings are given in the main paper. Very important points such as inputs given to the agents are given in the supplementary material. This complicates the analysis of the results    - Not enough explanations / intuitions are given w.r.t. what agents communicate for the different tasks. Wouldn't it be possible to give a simple example of what is learned for instance in a tiny lever pulling instance ? The section 5.2.2 does not help a lot to understand what is exchanged between agents


Fast Active Set Methods for Online Spike Inference from Calcium Imaging
Fluorescent calcium indicators are a popular means for observing the spiking activity of large neuronal populations. Unfortunately, extracting the spike train of each neuron from raw fluorescence calcium imaging data is a nontrivial problem. We present a fast online active set method to solve this sparse nonnegative deconvolution problem. Importantly, the algorithm progresses through each time series sequentially from beginning to end, thus enabling real-time online spike inference during the imaging session. Our algorithm is a generalization of the pool adjacent violators algorithm (PAVA) for isotonic regression and inherits its linear-time computational complexity. We gain remarkable increases in processing speed: more than one order of magnitude compared to currently employed state of the art convex solvers relying on interior point methods. Our method can exploit warm starts; therefore optimizing model hyperparameters only requires a handful of passes through the data. The algorithm enables real-time simultaneous deconvolution of O(10 5 ) traces of whole-brain zebrafish imaging data on a laptop.

This paper introduces a new algorithm for inferring the spiking pattern from a univariate calcium-imaging time-series. Calcium imaging is a popular technique for recording activity from thousands of individual neurons simultaneously.  An neuronal spike causes a sharp increase but a slow decrease in the observed calcium image signal.  Most neuroscience applications require deconvolving the observed signal to recover the underlying spikes.  This paper follows quite closely the exponential-decay and noise model [5], introducing an active set algorithm that solves this model much faster than previous methods (they report a 20x difference from previous competitor). They show the speed gain (with apparent successful deconvolution) on both simulated and real data, though it is hard to tell if the quality of spike inference has been improved or degraded.
 I enjoyed reading this paper, as it provides a clear and well argued algorithmic alternative for the spike inference problem.  The authors take a different approach than previous work solving the model for this problem, by modifying the isotonic regression into a locally constrained estimation problem. The authors make a strong case that algorithmic acceleration could potentially broaden the experimental scenarios in which large-scale calcium imaging can be used.    Main Comments: 1. The paper really only focuses on fitting the non-negative LASSO problem described in Eq (3)  [l 105] or on providing the best reconstruction of the path [see how accuracy is measured in Figure 3].  The reconstruction is never directly measured in terms  of the spikes location or strength. It is telling that the "goal of calcium deconvolution" is defined in terms of this  penalised optimisation, rather than in terms of the underlying spikes.   These are related goals, but are not exactly the same. Note that reconstruction equivalency to other algorithms is taken for granted.   In particular, its unclear that the hard noise constraint in Part 4 induces better recovery of the spikes  compared to other values. A different metric than signal correlation should be used to estimate this.   2. How sensitive is the hard-constraint of part-4 to the exactness of the model. The mean shape in the model (an exponential decay) is quite restrictive; what happens if the model mildly deviates from it? Is the noise still well estimated?    Minor comment:  The model in Eq 2, if based on previous literature should be referenced [is it from Ref 5?].


Orthogonal Random Features
We present an intriguing discovery related to Random Fourier Features: in Gaussian kernel approximation, replacing the random Gaussian matrix by a properly scaled random orthogonal matrix significantly decreases kernel approximation error. We call this technique Orthogonal Random Features (ORF), and provide theoretical and empirical justification for this behavior. Motivated by this discovery, we further propose Structured Orthogonal Random Features (SORF), which uses a class of structured discrete orthogonal matrices to speed up the computation. The method reduces the time cost from
where d is the data dimensionality, with almost no compromise in kernel approximation quality compared to ORF. Experiments on several datasets verify the effectiveness of ORF and SORF over the existing methods. We also provide discussions on using the same type of discrete orthogonal structure for a broader range of applications.

This paper presents a very interesting finding, that using orthogonalized random projections significantly reduces the variance (error) of Gaussian kernel approximation. The author(s) further propose a structured variant to reduce the computational cost from O(d^2) to O(d log(d)) using the Walsh-Hadamard transformation. Some theoretical insights are presented to justify the procedure proposed. The gain over existing solutions is demonstrated with numerical experiments.
This is a well-written paper, the motivations and main results are clearly stated in the main text, and technical details are put in the supplements. I do however have a few questions needs to to clarified.   1). Line 98, what does it mean by 'uniformly chosen random orthonormal basis' ? I do not think the set of orthonormal basis is compact, how can there be an uniform measure defined on it?  2). Line 100, again, 'uniformly from N(0,I_d)' does not make sense. And why does w_i follow a Gaussian distribution? It does not seem to be trivial. Please clarify.   3). For very high dimensional data (large d), the random projections will be effectively orthogonal. Does the proposed scheme offer any advantage other than the computational efficiency in such a setting.   4). Line 308 (Supp), the extra term f(z) seems very large, but it is omitted in Eq. (3) (line 108). And in Figure 3, there is this 'theoretical curve' for the variance. I wonder how D is set in this Figure and how is this theoretical curve computed?  And following might be some typos or negligences:   Line 86 : I presume the 'orthogonal' actually means 'orthonormal'?  Line 108 : Ratio of variance, shouldn't the 'K(x,y)' encapsulated by 'Var[]'?  Line 202-206 : I guess poly(x) means polynomial, but I could not find them in the main text or supplements.   ==================================================================   I have read the author(s)' rebuttal which is also well-written, and my major concerns have been clarified. My score remains unchanged and I recommend the area chair to accept this paper.


The Forget-me-not Process
We introduce the Forget-me-not Process, an efficient, non-parametric metaalgorithm for online probabilistic sequence prediction for piecewise stationary, repeating sources. Our method works by taking a Bayesian approach to partitioning a stream of data into postulated task-specific segments, while simultaneously building a model for each task. We provide regret guarantees with respect to piecewise stationary data sources under the logarithmic loss, and validate the method empirically across a range of sequence prediction and task identification problems.

The authors propose a scheme suitable for the predictions of piecewise "stationary" sequences, where here "stationarity" merely means that the sub-sequence corresponding to each "stationary" component is efficiently learnable by a given prediction scheme. The main challenge in this setting is therefore efficient identification of the boundaries between these components. The paper builds upon Bayesian mixing schemes and particularly PTW, and its novelty is in being able to exploit repetition within the components. The authors present an algorithm, along with several practically crucial complexity reduction heuristics, as well as several experiments studying the efficacy of their approach.
Overall, I believe this is a good paper - the algorithm appears non-trivial and innovative, and the experiments are convincing. However, the paper also has, in my opinion, several shortcomings:  1) There were several technical aspects of the algorithm and its implementation that I could not understand: A) In general, have found the pseudocode  for the algorithm hard to follow, despite reading the entire paper; I would recommend more verbal description (perhaps as comments next to the pseudocode) and if possible, less notation. B) The combination of the method with MADE is not clear to me. In particular, when and for which models gradient steps are taken? Are mini-batches used? Does the AdaGrad optimizer keep a separate state for each model? While I understand that the details of the training of the prediction models can be abstracted away while considering the mixing scheme, I think presenting these details when describing the algorithm is important and can make the presentation far more accessible. C) Two aspects of the complexity reducing heuristics were not clear to me: I) If rho* is selected as the best performing model for the segment, it seems unlikely for the condition of option 2 of the pruning heuristic to ever happen. In particular, if no sub-sampling is used, shouldn't rho*(s) be greater than xi(s) by construction? II) In option 3 of the pruning heuristic, how exactly is the "age" of a model determined?   D) How were the hyper-parameters determined in each experiment?  2) The efficacy of the approach in exploiting repetition is not established theoretically. Instead, the guarantees of PTW are shown to be inherited, and efficacy for repetition is deferred to experimental studies. Moreover, the theory doesn't seem to consider the complexity reducing heuristics, without which the approach has very limited practical value. I believe that the "Theoretical Properties" paragraph of the paper is sufficiently weak to be relegated to supplementary materials, and the freed up space can be used for more explanations of other aspects of the paper.  3) Motivation for the work is lacking in the sense that no convincing applications are described. The only "real-world" application for the proposed algorithm mentioned in the paper is multiple-task reinforcement learning, with a stream of different Atari games given as an example. However, it is not clear to me why should the identity of the game played be hidden from the agent learning multiple Atari games. Note that while I feel motivation is lacking in the presentation, I do believe the suitable applications exist, and answered question 7 of the review accordingly. I believe the authors should provide additional motivating examples in the introduction to the paper.  Here are some additional comments and suggestions: a) The abstract refers to the innovative aspect of this work with only one word ("repeating"). I believe this should be expanded to at least one sentence (I only understood that this is the innovative aspect after reading the introduction). b) In the denominator just below line 86, I believe it should be rho instead of nu c) In line 129, I believe it should be rho[x_{a_{i-1}:b_{i-1}}] rather than rho[x_{a_{i}:b_{i}}] d) In line 166, is "marginal probability" the right term here? Shouldn't it be "joint probability"? e) In line 177, the assumption that prediction takes O(1) time is perhaps not detailed enough when considering neural networks as statistical models; instead, it should be interesting to count exactly the number of forward and backward passes, as well as parameter updates, that the algorithm uses. f) The use of a generative model such as MADE in an online fashion is, to my knowledge, quite non-standard. It should be interesting to see a graph of the commutative regret as a function of time in order to get more insight into the learning dynamics and convergence behavior of the model. g) In line 259, the oracle is reported for the FOD domain, not the MBOC.


Learning under uncertainty: a comparison between R-W and Bayesian approach
Accurately differentiating between what are truly unpredictably random and systematic changes that occur at random can have profound effect on affect and cognition. To examine the underlying computational principles that guide different learning behavior in an uncertain environment, we compared an R-W model and a Bayesian approach in a visual search task with different volatility levels. Both R-W model and the Bayesian approach reflected an individual's estimation of the environmental volatility, and there is a strong correlation between the learning rate in R-W model and the belief of stationarity in the Bayesian approach in different volatility conditions. In a low volatility condition, R-W model indicates that learning rate positively correlates with lose-shift rate, but not choice optimality (inverted U shape). The Bayesian approach indicates that the belief of environmental stationarity positively correlates with choice optimality, but not lose-shift rate (inverted U shape). In addition, we showed that comparing to Expert learners, individuals with high lose-shift rate (sub-optimal learners) had significantly higher learning rate estimated from R-W model and lower belief of stationarity from the Bayesian model.

This paper compares an R-W model (with various constant learning rates) and a Bayesian model (with various constant beliefs of environmental stationarity). Section 2 verifies assumptions about the relation between the learning rate parameter and the performance and behavior of in-silico simulation runs of the R-W model, in three volatility conditions (section 2.1), and for three behavior types (section 2.2). Section 3 verifies assumptions about the relation between the belief of stationarity parameter and the performance and behavior of in-silico simulation runs of the Bayesian model, in three volatility conditions (section 3.1), and for three behavior types (section 3.2). Section 4 compares both models (R-W and Bayesian) "fitted" to simulate experiments performed by (Yu & Huang, 2014) with human subjects in a visual search task.
You cite (Gershman 2015), but you do not provide an inductive analysis of your experimental results in such an unifying framework. Most of your analysis is deductive or consists in verifying that your simulation results match intuitive reasoning such as "learning rate is expected to increase as the volatility increases" (at line 137).  My main comments on the form are related to clarifications of your approach and methods: - A presentation of your simulation method could improve readability. Notably I guess that you have manipulated the R-W learning rate η (i.e. you ran the simulation with various learning rates) and the belief of stationarity α, in which case η and α are simulation inputs; this is not obvious at first reading of "Learning rate is expected to increase as the volatility increases", which suggests that the learning rate is an output of the simulation. - In section §4 "Experiment", whether η and α are inputs or outputs is less clear. Maybe this is due to the "fitting" process mentioned at line 268: in my understanding, fitting consists in running several simulations, setting (inputting) different parameter (η or α) values for each run, then selecting the parameter value which produces the best match with experimental data; hence a given parameter is both an input of each run and an output of the whole fitting process. Maybe a qualitative and a quantitative descriptions of such a fitting process (and how you select the best match) would clarify both your approach and your method. - You should define how you measure "model accuracy" (figure 3 and line 241), or maybe refer to the "Data Analysis: Model Predictive Accuracy" section of (Yu et al. 2014). - At line 268 you mention "subjects", I understand from (Yu et al. 2014) that they are human subjects, but you could recall this fact in your paper.  Below are detailed comments on the form: - The text is not aligned with line numbers. In my comments I refer to the line number just above the commented text line. - Line 40: you cite "(Mathys et al. 2011)", but the referenced paper is "(Mathys et al. 2014)". - The last paragraph of section 1 (lines 91-98) refers to two questions. Apparently, the first question starts at line 66, and the second question starts at line 86 ("In addition ..."). Maybe you could highlight the two questions in the next to last paragraph of section 1. - Line 132 "..., with a large β approximates ’maximization’ strategy.": "..., where a large β..."? - Line 160 (or 161) "... agents strictly follow WSLS have ...": "... agents which strictly follow WSLS have ..."? - Line 161 "... indicating their choices are heavily relied on recent experience": is "relied" the correct term here? Do you mean "linked"? Or "... recent experience heavily impacts their choices"? - Line 230-231 "with α = 0 leads to": same comment as line 132. - Line 238-239 "This is consistent with their estimation of higher volatility reﬂected by higher learning rate than Expert estimated from R-W model (Fig 2c).": I do not understand this sentence. - Line 268 "with change points occur": same comment as line 132. - Line 287 "inverse decision parameters": you define the R-W parameter β at line 132, but you do not define the "inverse decision parameter" used with your DBM simulations. Maybe you should refer to the corresponding parameter β of (Yu et al. 2014 p283-284). - Line 311 "In a visual search task with relative reward frequency 1:3:9 and change of the reward contingency occur based on N(30, 1)": quoting the settings, notably "change of the reward contingency occur based on N(30, 1)", may help readability. - Line 321: "Fig. 6c" should be replaced by "Fig. 5c". - Fig. 5c and lines 321-323: what is/are the subject categorie/s related to fig. 5c? What is/are the model results or human data related to fig. 5c? - Line 356 / Fig. 6a caption: what is "η0"? - Lines 424-425: You should not include acknowledgments in the submitted paper. - Line 370 "with a fixed learning rate and a Bayesian model with a fixed belief of stationarity": these formulation suggest a constant value for each parameter (see my comment above about fitting). You could formulate this way: "with various constant learning rates and a Bayesian model with various constant beliefs of stationarity"   Response to author rebuttals: Thanks for addressing many of my concerns. Considering your response to methods clarification requests, I have increased the "clarity" note. Considering other reviewers' qualitative assessment, I've increased the "technical" note. I am less confident about an improvement of novelty and impact criteria. As stated your response, your aim is rather experimental than theoretical.


Cooperative Graphical Models
We study a rich family of distributions that capture variable interactions significantly more expressive than those representable with low-treewidth or pairwise graphical models, or log-supermodular models. We call these cooperative graphical models. Yet, this family retains structure, which we carefully exploit for efficient inference techniques. Our algorithms combine the polyhedral structure of submodular functions in new ways with variational inference methods to obtain both lower and upper bounds on the partition function. While our fully convex upper bound is minimized as an SDP or via tree-reweighted belief propagation, our lower bound is tightened via belief propagation or mean-field algorithms. The resulting algorithms are easy to implement and, as our experiments show, effectively obtain good bounds and marginals for synthetic and real-world examples.
Probabilistic inference in high-order discrete graphical models has been an ongoing computational challenge, and all existing methods rely on exploiting specific structure: either low-treewidth or pairwise graphical models, or functional properties of the distribution such as log-submodularity. Here, we aim to compute approximate marginal probabilities in complex models with long-range variable interactions that do not possess any of these properties. Instead, we exploit a combination of structural and functional properties in new ways.
The classical example of image segmentation may serve to motivate our family of models: we would like to estimate a posterior marginal distribution over k labels for each pixel in an image. A common approach uses Conditional Random Fields on a pixel neighborhood graph with pairwise potentials that encourage neighboring pixels to take on the same label. From the perspective of the graph, this model prefers configurations with few edges cut, where an edge is said to be cut if its endpoints have different labels. Such cut-based models, however, short-cut elongated structures (e.g. tree branches), a problem known as shrinking bias. Jegelka and Bilmes [1] hence replace the bias towards short cuts (boundaries) by a bias towards configurations with certain higher-order structure: the cut edges occur at similar-looking pixel pairs. They group the graph edges into clusters (based on, say, color gradients across the endpoints), observing that the true object boundary is captured by few of these clusters. To encourage cutting edges from few clusters, the cost of cutting an edge decreases as more edges in its cluster are cut. In short, the edges "cooperate". In Figure 1, each pixel takes on one of two labels (colors), and cut edges are indicated by dotted lines. The current configuration cuts three red edges and one blue edge, and has lower probability than the configuration that swaps X 3,1 to gray, cutting only red edges. Such a model can be implemented by an energy (cost) h(#red edges cut) + h(#blue edges cut), where e.g. h(u) = √ u. Similar cooperative models can express a preference for shapes [2].
While being expressive, such models are computationally very challenging: the nonlinear function on pairs of variables (edges) is equivalent to a graphical model of extremely high order (up to the number of variables). Previous work hence addressed only MAP inference [3,4]; the computation of marginals and partition functions was left as an open problem. In this paper, we close this gap, even for a larger family of models.
We address models, which we call cooperative graphical models, that are specified by an undirected graph G = (V, E): each node i ∈ V is associated with a random variable X i that takes values in X = {1, 2, . . . , k}. To each vertex i ∈ V and edge {i, j}, we attach a potential function θ i : X → R and θ i,j : X 2 → R, respectively. Our distribution is then
where we call y : X n → {0, 1} E the disagreement variable 1 , defined as y i,j = x i = x j . The term ν : X n → R ≥0 is the base-measure and allows to encode constraints, e.g., conditioning on some variables. With f ≡ 0 we obtain a Markov random field.
Probabilistic inference in our model class (1) is very challenging, since we make no factorization assumption about f . One solution would be to encode P (x) as a log-linear model via a new variable z ∈ {0, 1} E and constraints ν(x, z) = y(x) = z , but this in general requires computing exponential-sized sufficient statistics from z. In contrast, we make one additional key assumption that will enable the development of efficiently computable variational lower and upper bounds: we henceforth assume that f : {0, 1} E → R is submodular, i.e., it satisfies
where the min and max operations are taken element-wise. For example, the pairwise potentials θ i,j are submodular if θ i,j (0, 0) + θ i,j (1, 1) ≤ θ i,j (0, 1) + θ i,j (1, 0). In our introductory example, f is submodular if h is concave. As opposed to [3], we do not assume that f is monotone increasing. Importantly, even if f is submodular, P (x) neither has low treewdith, nor is its logarithm sub-or supermodular in x, properties that have commonly been exploited for inference.
Contributions. We make the following contributions: (1) We introduce a new family of probabilistic models that can capture rich non-submodular interactions, while still admitting efficient inference. This family includes pairwise and certain higher-order graphical models, cooperative cuts [1], and other, new models. We develop new inference methods for these models; in particular, (2) upper bounds that are amenable to convex optimization, and (3) lower bounds that we optimize with traditional variational methods. Finally, we demonstrate the efficacy of our methods empirically.

This paper provides variational lower/upper bound of partition of a special class of graphical models called `cooperative graphical model’. Cooperative graphical model is a probability distribution which is similar with pairwise graphical models but its energy function contains some extra submodular function of disagreement variable. The main technique for formulating variational lower/upper bound of cooperative graphical model is finding pairwise graphical model and its associate polytope which upper/lower bound the original cooperative graphical model. The author also provides algorithms for suggested bounds and experimental results for image segmentation.
I think the proposed graphical model is interesting and novel. The authors provide a lower bound and a fully convex upper bound for partition function. In particular, variational upper/lower bound are formulated by using pairwise graphical models, e.g. mean field approximation and SDP, and therefore, one can apply existing variational methods for pairwise graphical models for cooperative graphical models. The author also provides algorithms, Frank-Wolfe, PGD and BP, for finding upper/lower bound. The experimental results shows that cooperative graphical models and its variational methods performs better than pairwise graphical models for image segmentation.  Variational bounds provided in the paper are formulated by bounding the cooperative graphical model by pairwise models and use the existing variational bounds for pairwise models to bound cooperative graphical model. However, as the author applies two bounds consecutively, the suggested bound might be loose.  Lower bound is not tractable. Since the optimization over a polytope U(f) is not tractable, the variational lower bound is also intractable. The author suggests an algorithm based on the block coordinate descent which finds the lower bound for the suggested variational lower bound, however, the algorithm only explore the finite points of U(f) and it may not provide a tight lower bound.  The technique of constructing B(f) and U(f) seems generic. Can this technique applied to a wider class of f or other probability distribution? Furthermore, the optimization objectives (3), (4) for the variational upper/lower bound does not seems to be dependent with the submodularity of f (while the algorithms do).  Typos  - In page 2, line 59~60, for all x,y -> for all y,y^\prime - In page 5, procedure Linear-Oracle, `)’ is missing for procedure 4,5 - In page 6, line 211, U(F) -> U(f) - In page 5, line 179, smoth -> smooth


Globally Optimal Training of Generalized Polynomial Neural Networks with Nonlinear Spectral Methods
The optimization problem behind neural networks is highly non-convex. Training with stochastic gradient descent and variants requires careful parameter tuning and provides no guarantee to achieve the global optimum. In contrast we show under quite weak assumptions on the data that a particular class of feedforward neural networks can be trained globally optimal with a linear convergence rate with our nonlinear spectral method. Up to our knowledge this is the first practically feasible method which achieves such a guarantee. While the method can in principle be applied to deep networks, we restrict ourselves for simplicity in this paper to one and two hidden layer networks. Our experiments confirm that these models are rich enough to achieve good performance on a series of real-world datasets.

This paper presents a nice theory that demonstrates that a certain class of neural network has a global optimal which can be achieved in a linear convergence rate. The main constrain is that all the input data and the weights of the linear layer are positive and activation functions are generalized polynomial. The proposed theory is first proved on the neural network with a single hidden layer and later extend to the networks with multiple hidden layers. The authors also proposed an optimization algorithm based on nonlinear spectral method. Experiments show that the proposed algorithm works quite well on a small 2D synthetic dataset, and it also achieves a reasonable performance on some small real-world datasets (UCI datasets).
Pros: As claimed by the authors, this is the first work that proposed a potentially practical neural work with optimal guarantee and convergence guarantee. Previous work that have convergence guarantees requires a complicated method to check its preconditions, some of them even impossible to check in practice, as they depend on the data-generating measure. The theory proposed in this work has comparatively simple preconditions that only depends on the spectral radius of a nonnegative matrix consists of the parameters of the network. The authors also demonstrate that the proposed theory can be applied to network of arbitrary depth and show a reasonable result on small real dataset.  Cons: The proposed network still has limited applications. The theory requires both the input data and network parameters be positive, activation functions be generalized polynomials, and spectral radius (Theorem 1) be smaller than 1. I think the non-negative constrains on the input data should not block too much potential usage, as most of train data can be easily shift to be all positive. However, the fact that network parameters are positive might limit the expressiveness of the model. Also, the fact that all activation functions of all hidden layers are different is unnatural and it might also limit its application. At last, the authors said that in order to satisfy the spectral radius constrains, they need to increase p1 and p2 and decrease pw and pu, which effectively decreases the weights of linear units. And if I understand correctly, in order to satisfy the spectral radius constrains, the higher dimension of the input and network parameters, the lower the weights of linear units. Since when the linear weight is very small, the activation functions will almost act the same as a linear function. In that case, adding more layers is just adding another linear function, which won't increase the expressiveness of the network. Thus, I am little worried that this method might not easy to extend it to deep architectures. Despite all these concerns, this is still an encouraging result as an initial attempt and I think this paper should be accepted.  Minor comments:  I am curious the how p1, p2, pw, pu, the dimension of the input and the size of the network parameters affects the spectral radius rho(A). For example, when the dimension of input increases, does rho(A) increases linearly or exponentially, and in order to counter effect that, should we linearly or exponentially decreases pw and pu. I know it might not be easy to prove that, but it might be possible to empirically plot that relationship on a synthetic data.


Beyond Exchangeability: The Chinese Voting Process
Many online communities present user-contributed responses such as reviews of products and answers to questions. User-provided helpfulness votes can highlight the most useful responses, but voting is a social process that can gain momentum based on the popularity of responses and the polarity of existing votes. We propose the Chinese Voting Process (CVP) which models the evolution of helpfulness votes as a self-reinforcing process dependent on position and presentation biases. We evaluate this model on Amazon product reviews and more than 80 StackExchange forums, measuring the intrinsic quality of individual responses and behavioral coefficients of different communities.

The paper suggests Chinese Voting Process (CVP) to model user response patterns in online community where users can write a response or vote helpfulness of existing responses. The response patterns are divided into two phase: 1) selection phase and 2) voting phase. In the selection phase, a user decides to write a new response or vote an existing response. And once the user decided to vote, the user decides to give a positive vote or negative vote in the voting phase. With Amazon review and StachExchange datasets, the model shows that each community reveals different response patterns characterised by trendiness and conformity.
The exchangeability under a probabilistic model gives a great flexibility in building and inferencing, while preventing the representation power of a model. This paper well captures two different levels of non-exchangeability occurring in online voting.  Although the problem is interesting, the presentation of this paper makes hard to understand the proposed approach. Especially, the structure and notations hinder readers from understanding some key ideas.   Below I listed some suggestions and comments to improve the paper:  * a short description of the reinforcing mechanism between the CRP and ddCRP: ddCRP is an important concept of the proposed approach, which breaks the general exchangeability assumption. But It is quite hard to get an intuition about how this model works to compare with the CRP. A brief comparison between two processes would be good.  * intrinsic quality: In the generative process described in table2, each item has an unique intrinsic quality over time. However, in equation (1), it seems the model jointly learns different intrinsic qualities at a different time. I assume that only one intrinsic quality is estimated in the experiment. Please clarify the notation across the manuscript.  * Table 2 [generative process]: When table 2 is introduced in the main text, there is no explanation about the notations in this table, and it's impossible to understand the process. Maybe move the table after section 2.2? It would be better to combine table 2 and 4 since both tables are talking about the same thing.  * Table 5: There is no description of table 5 in the main text. The paragraph between line 208 and 218 seems a relevant part, but the description of each table should be included in the main text. Also the voting is a binary classification problem, so it might be more intuitive to interpret the result if it's written in terms of accuracy(?).  * Prediction: since the suggested model is fully generative, once the model learns parameters, it would be also possible to have a full prediction task which predict two phases together. Without this, the propose model seems just a combination of two independent models.