Linear Contextual Bandits with Knapsacks
We consider the linear contextual bandit problem with resource consumption, in addition to reward generation. In each round, the outcome of pulling an arm is a reward as well as a vector of resource consumptions. The expected values of these outcomes depend linearly on the context of that arm. The budget/capacity constraints require that the total consumption doesn't exceed the budget for each resource. The objective is once again to maximize the total reward. This problem turns out to be a common generalization of classic linear contextual bandits (linContextual) [8,11,1], bandits with knapsacks (BwK) [3,9], and the online stochastic packing problem (OSPP) [4,14]. We present algorithms with near-optimal regret bounds for this problem. Our bounds compare favorably to results on the unstructured version of the problem [5,10] where the relation between the contexts and the outcomes could be arbitrary, but the algorithm only competes against a fixed set of policies accessible through an optimization oracle. We combine techniques from the work on linContextual, BwK and OSPP in a nontrivial manner while also tackling new difficulties that are not present in any of these special cases.

********************************************************* POST REBUTTAL:   Thanks for the corrections. I now vote for acceptance.   I hope the authors will spend the effort to improve the readability.  *********************************************************  A new linear contextual bandit setting is proposed that includes the addition of a d-dimensional budget. This generalises existing budgeted bandits to the linear case. Everything is assumed to be i.i.d., and the budget usage of an action is assumed to be a linear function of the context for that action and some unknown parameter.  Besides the setting (which is quite natural) the main contribution is a new algorithm and analysis showing that  ignoring logarithmic factors enjoys a regret of O((1+OPT/B) m Sqrt(T)) where m is the dimension of the action space, B is the budget and OPT is the payoff of the optimal static strategy. Although there are no lower bounds, by combining existing lower bounds the authors argue that this is not much improvable, which is quite convincing.
The algorithm operates mostly in a quite standard way. Constructing confidence ellipsoids around the unknown parameters and acting optimistically subject to a budget penalty. The latter part is the most interesting because one has to try and learn the direction in the budget space for which the algorithm is most constrained, for which the authors use online mirror descent. As far as I know this is a completely novel idea.  I found a lot to like in this paper. The setting and algorithm are interesting and the analysis is mostly quite good.    There were also some weaknesses, however. For of all, there is a lot of sloppy writing, typos and undefined notation. See the long list of minor comments below. A larger concern is that some parts of the proof I could not understand, despite trying quite hard. The authors should focus their  response to this review on these technical concerns, which I mark with ** in the minor comments below. Hopefully I am missing something silly.  One also has to wonder about the practicality of such algorithms. The main algorithm relies on an estimate of the payoff for the optimal policy, which can be learnt with sufficient precision in a "short" initialisation period. Some synthetic experiments might shed some light on how long the horizon needs to be before any real learning occurs.  A final note. The paper is over length. Up to the two pages of references it is 10 pages, but only 9 are allowed. The appendix should have been  submitted as supplementary material and the reference list cut down.   Despite the weaknesses I am quite positive about this paper, although it could certainly use quite a lot of polishing. I will raise my score once the **  points are addressed in the rebuttal.  Minor comments:  * L75. Maybe say that pi is a function from R^m \to \Delta^{K+1}  * In (2) you have X pi(X), but the dimensions do not match because you dropped the no-op action. Why not just assume the 1st column of X_t is always 0?   * L177: "(OCO )" -> "(OCO)" and similar things elsewhere  * L176: You might want to mention that the learner observes the whole concave function (full information setting)  * L223: I would prefer to see a constant here. What does the O(.) really mean here?  * L240 and L428: "is sufficient" for what? I guess you want to write that the sum of the "optimistic" hoped for rewards is close to the expected actual rewards.   * L384: Could mention that you mean |Y_t - Y_{t-1}| \leq c_t almost surely.  ** L431: \mu_t should be \tilde \mu_t, yes?  * The algorithm only stops /after/ it has exhausted its budget. Don't you need to stop just before? (the regret is only trivially affected, so this isn't too important).  * L213: \tilde \mu is undefined. I guess you mean \tilde \mu_t, but that is also not defined except in Corollary 1, where it just given as some point in the confidence ellipsoid in round t. The result holds for all points in the ellipsoid uniformly with time, so maybe just write that, or at least clarify somehow.  ** L435: I do not see how this follows from Corollary 2 (I guess you meant part 1, please say so). So first of all mu_t(a_t) is not defined. Did you mean tilde mu_t(a_t)? But still I don't understand. pi^*(X_t) is (possibly random) optimal static strategy while \tilde \mu_t(a_t) is the optimistic mu for action a_t, which may not be optimistic for pi^*(X_t)? I have similar concerns about the claim on the use of budget as well.  * L434: The \hat v^*_t seems like strange notation. Elsewhere the \hat is used for empirical estimates (as is standard), but here it refers to something else.   * L178: Why not say what Omega is here. Also, OMD is a whole family of algorithms. It might be nice to be more explicit. What link function? Which theorem in [32] are you referring to for this regret guarantee?  * L200: "for every arm a" implies there is a single optimistic parameter, but of course it depends on a  ** L303: Why not choose T_0 = m Sqrt(T)? Then the condition becomes B > Sqrt(m) T^(3/4), which improves slightly on what you give.  * It would be nice to have more interpretation of theta (I hope I got it right), since this is the most novel component of the proof/algorithm.


A Multi-step Inertial Forward-Backward Splitting Method for Non-convex Optimization
We propose a multi-step inertial Forward-Backward splitting algorithm for minimizing the sum of two non-necessarily convex functions, one of which is proper lower semi-continuous while the other is differentiable with a Lipschitz continuous gradient. We first prove global convergence of the algorithm with the help of the Kurdyka-Łojasiewicz property. Then, when the non-smooth part is also partly smooth relative to a smooth submanifold, we establish finite identification of the latter and provide sharp local linear convergence analysis. The proposed method is illustrated on several problems arising from statistics and machine learning.

In this paper, a multi-step inertial forward-backward method is proposed   to minimise an objective function that can be written as the sum of a smooth (possibly non convex) function and a proper l.s.c. function. With respect to most common inertial schemes, the authors enlarge the possible choices for inertial parameters, allowing for combinations (also negative) of more than two previous iterates. Convergence to a critical point for arbitrary initialisation assuming a priori boundedness of the iterate is proved under KL property. Local linear convergence is established under partial smoothness. The results extend significantly what is known in the convex setting. The experimental results compare multi-step IFB with classical FB.
The paper provides new insight into the theory of inertial methods. The authors prove convergence to a critical point for non convex and non smooth objective functions.  Theoretical analysis: The theoretical results extend known results in two directions: to more general objective functions and to more general inertial schemes.  The more interesting aspect is the generalisation to non convex problems.  However, I am not sure of the correctness of the algorithm under the assumptions made in the paper.  Indeed, without additional conditions (e.g. boundedness from below) on the function R, the proximity operator can be empty, and in this case the iteration is not well-defined.  One of the main results, Theorem 2.2 establishes convergence of the iterates, assuming boundedness of the iterates. I think that a more deep analysis of the class of allowed inertial parameters would be useful. For instance, are classical choices included in the considered class (e.g. the FISTA ones)?   The local linear convergence result extend what was known in the convex setting to the non convex one.  Experiments: The experimental section is interesting, but to me the gains of using s\geq2 are limited.  Moreover, in the comparison with FB, why is the step-size so small?  FB should work with gamma\le 1/L. Does a choice of s\geq 1 require smaller step-sizes?  In view of the previous remarks, I think that the paper presents an interesting theoretical analysis, but the practical impact of the generalisation studied in the paper is not clear to me. In particular, the choice of several additional parameters with respect to standard inertial schemes is required, and the advantages shown in the experiments look  limited.


Backprop KF: Learning Discriminative Deterministic State Estimators
Generative state estimators based on probabilistic filters and smoothers are one of the most popular classes of state estimators for robots and autonomous vehicles. However, generative models have limited capacity to handle rich sensory observations, such as camera images, since they must model the entire distribution over sensor readings. Discriminative models do not suffer from this limitation, but are typically more complex to train as latent variable models for state estimation. We present an alternative approach where the parameters of the latent state distribution are directly optimized as a deterministic computation graph, resulting in a simple and effective gradient descent algorithm for training discriminative state estimators. We show that this procedure can be used to train state estimators that use complex input, such as raw camera images, which must be processed using expressive nonlinear function approximators such as convolutional neural networks. Our model can be viewed as a type of recurrent neural network, and the connection to probabilistic filtering allows us to design a network architecture that is particularly well suited for state estimation. We evaluate our approach on synthetic tracking task with raw image inputs and on the visual odometry task in the KITTI dataset. The results show significant improvement over both standard generative approaches and regular recurrent neural networks.

The paper describes that the standard Kalman Filter (KF) equations can be expressed in a computation graph, thereby integrating probabilistic state estimation with a per-frame convnet. This enables optimizing the convnet and the resulting low-dimensional features with respect to the filtered state directly on a batch of images with known target location using standard back-propagation. The hypothesis is that a KF provides useful domain knowledge for state estimation (i.e. through the linear motion equations and the parametric state probability distribution) over a generic LSTM.   The trained network yields thereby a better representation for state estimation of the target in novel tracking sequences, as compared to training the convnet on individual frames independently without considering the KF. Also, the convnet uses a novel normalization layer which resembles batch normalization. The method is evaluated on an artificial tracking problem consisting of a red disc moving through and occluded by various other colored discs, and compared to independent training of the convnet, and a version where the KF is replaced by LSTM. The results show that the proposed method outperforms these baselines on the artificial dataset.
I like the idea to investigate integration of the well-known Kalman Filtering in back-propogation, to obtain best of both-worlds: data-driven feature learning and informed motion models. The paper is generally well written and easy to follow. Since one could expect learning motion models would also benefit from non-linear data-driven models (e.g. LSTM) with large amounts of data, instead of using highly informed models with few free parameters, the question when/where this trade-off occurs would be very interesting. The idea that this could improve state estimation in tracking specifically is also appealing,  especially if empirical evidence on real-world tracking tasks can be given.  However, this paper does not really explore the first question. It is unclear how much data is used, and if the exhibited movements are realistic or already very similar towards the linear equations used in the Kalman filter. Also, the presented tracking use-case seems very ad-hoc and not very relevant to any real-world tracking use-case: it presuposes that we posses extensive training data of the instance that we wish to track (a red circle) with a distinct and static appearance, sufficient to train a convnet for the specific appearance. Standard model-based tracking (e.g. multi-person tracking) or model-free tracking (e.g. track unknown object from initial bounding box) do not fit this task, as the instance appearances are not known a-priori.  So while the artificial scenario is good for illustration and explaining the method, it does not really "reflect typical challenges in visual state estimation" (line 202). Due to the limited experimental evaluation and unclear use-case, I rated both Technical Quality and the Potential Impact as substandard for NIPS. More details comments follow below.  ## Detailed comments: - line 104: In this example, what would the hidden system state x_t be? If the state represents the 'true' position and velocity, do you consider y_t a noisy version of x_t (if yes, then why consider y_t the output instead of x_t, if no why call it an observation in a Kalman Filter)?   - line 181. Should L not be defined as the negative log likelihood? I.e. higher likelihood = lower loss.  - line 136: Aren't the two terms in the brackets ("dphi/ds . dL/dphi" and "dL / ds") the same? I would expect the recursion to take the form of "dL / ds_{t-1} = dL / dphi_{t-1} . dphi_{t-1} / ds_{t-1} + ds/ds_{t-1} . dL/ds"  - line 183. introduces a new contribution, namely a novel response normalization layer. Such a novelty warrants extra experiments (e.g. comparing to batch normalization [9]), or at least some insight into why it is "substantially more effective" (line 188). Can you expand on/cite why standard approach "is know(n) to require additional care"?   - line 195.5: Are the KF parameters also trained using back propagation, e.g. Q, A, B, C ? If not, how are they determined?  - The experimental section provides no details on how much training, testing data is used, what kind of back-propogation technique, or any other details to reproduce results. Are the results averaged over multiple folds / new batches of artificial data? For instance, are the up-down patterns in Fig. 6b systematic or due to random effects in a single artificial sequence?  - In the presented artificial dataset, the target seems to have a clearly distinct and static appearance (its color) compared to almost all distractors. So, while technically the model is processing "raw pixel data", this seems to overstate the complexity of dealing with real-world raw pixel data a lot: Creating a good per-frame target response map appears trivial (a simple color threshold), which undermines the whole point of using a convnet to learn a robust representation.  - The main question therefore remains: does this approach help to improve state estimation in real tracking scenarios where the target is not known a-prior, where the appearance is dynamic, has lighting and shape changes, and fore and background as more prone to false positives? E.g. comparing tracking on VOT 2014/2015 sequences, or PETS pedestrian sequences would be more convincing. Also, in normal tracking scenarios you do not have labeled prior examples of the specific target *instance*. At best, you have labeled examples of the target class (e.g. pedestrians). A tracker still needs to online learn a way to discriminate other instances of the same class. For instance, could the method be used to improve and initial tracking results from a baseline tracking method providing inaccurate object labels?   ## Minor comments - Equations are not numbered, annoying for anyone referring to parts of the paper.  - line 138.5: At this point I had several questions (though these were somewhat clarified later in Section 5.2): How are these gradient terms obtained in practice? Why not provide the closed form solutions, or appropriate citations? Or do the authors rely on automatic derivation by a software package? Is there any risk of a gradient descent step leading to non positive-definite covariance?  - line 169: Extended Kalman Filter should have a citation, it was not introduced anywhere before.  - line 261.: "although the accuracy of the BKF can be matched by a recurrent LSTM network". Where is this shown in the experiments? How could one conclude this from Table 1, or Figure 6?


Graph Clustering: Block-models and model free results
Clustering graphs under the Stochastic Block Model (SBM) and extensions are well studied. Guarantees of correctness exist under the assumption that the data is sampled from a model. In this paper, we propose a framework, in which we obtain "correctness" guarantees without assuming the data comes from a model. The guarantees we obtain depend instead on the statistics of the data that can be checked. We also show that this framework ties in with the existing model-based framework, and that we can exploit results in model-based recovery, as well as strengthen the results existing in that area of research.
extension, we define the degree matrix D and the Laplacian L associated to any matrix A ∈ R n×n , with A ij = A ji ≥ 0, in a similar way.
Let C be a partitioning (clustering) of the nodes of G into K clusters. We use the shorthand notation i ∈ k for "node i belongs to cluster k". We will represent C by its n × K indicator matrix Z, defined by Z ik = 1 if i ∈ k, 0 otherwise, for i = 1, . . . n, k = 1, . . . K.
Note that Z T Z = diag{n k } with n k counting the number of nodes in cluster k, and Z TÂ Z = [n kl ] K k,l=1 with n kl counting the edges in G between clusters k and l. Moreover, for two indicator matrices Z, Z � for clusterings C, C � , (Z T Z � ) kk � counts the number of points in the intersection of cluster k of C with cluster k � of C � , and (Z TD Z � ) kk � computes � i∈k∩k �di the volume of the same intersection.
"Block models" for random graphs (SBM, DC-SBM, PFM) This family of models contains Stochastic Block Models (SBM) [1,18], Degree-Corrected SBM (DC-SBM) [17] and Preference Frame Models (PFM) [20]. Under each of these model families, a graph G with adjacency matrixÂ over n nodes is generated by sampling its edges independently following the laŵ A ij ∼ Bernoulli(A ij ), for all i > j. The symmetric matrix A = [A ij ] describing the graph is the edge probability matrix. The three model families differ in the constraints they put on an acceptable A. Let C * be a clustering. The entries of A are defined w.r.t C * as follows (and we say that A is compatible with C * ).
SBM : A ij = B kl whenever i ∈ k, j ∈ l, with B = [B kl ] ∈ R K×K symmetric and nonnegative. DC-SBM : A ij = w i w j B kl whenever i ∈ k, j ∈ l, with B as above and w 1 , . . . w n non-negative weights associated with the graph nodes. PFM : A satisfies D = diag(A1), D −1 AZ = ZR where 1 denotes the vector of all ones, Z is the indicator matrix of C * , and R is a stochastic matrix (R1 = 1, R kl ≥ 0), the details are in [20] While perhaps not immediately obvious, the SBM is a subclass of the DC-SBM, and the latter a subclass of the PFM. Another common feature of block-models, that will be significant throughout this work is that for all three, Spectral Clustering algorithms [15] have been proved to work well estimating C * .

The paper presents a generic template for theoretical guarantee for the stability of a clustering C that fits the data well. The stability is measured in terms of the distance between C and any other C' that fit the same data well. The paper also provides specific example of using the template for two block models, PFM and SBM, and all the quantities involved in the guarantees can be computed explicitly from the data. Furthermore, it also uses the stability results thus obtained to show exact recovery guarantees for the models. However, the simulations suggest that either the goodness fit used or the conditions of the theorems provided render the results not too useful.
1) The paper considers the problem of providing generic template for theoretical guarantees for clustering without assuming that the data is from a specific model. The goal is to obtain such guarantees with quantities that can be computed from the data and the output of the clustering algorithms being compared. Providing such model free theoretical guarantees for clustering is of importance for both theoretical and practical purposes.  2) Given that PFM includes DC-SBM as a special class which in turn includes SBM, it is not clear why the two cases (PFM and SBM) are treated differently. Given that Spectral Clutering works well for all the models specified, why not use the same model estimator? In particular, it is not clear why the Laplacian is used for PFM while the adjacency matrix is used for the SBM. Also, the results for PFM is for weighted ME whereas for SBM it is in terms of ME. How do they compare with each other? Further, why are the results for DC-SBM not given?  3) In Proposition 10, what is the definition of < M', M >_F ?  4) In Assumption 5: Is it ||L^{hat}^2 - L^2||_F or ||L^{hat} - L||_F^2 ? If former, what is the intuition behind squaring the Laplacians?  5) The paper has focused mainly on disjoint clusters. Do the techniques/recipe used in this paper extend beyond block models to other models for which Spectral Clustering works? For example, Spectral clustering is known to work well for hierarchical clustering and there are theoretical guarantees available ( "Noise Thresholds for Spectral Clustering", S. Balakrishan et. al. 2011). How would the template work for this case?  6) The evaluation metric considered as well as the estimators considered both in the cases of PFM and SBM are limited to C and C' having the same number of clusters. Different clustering algorithms can potentially give different number of clusters. For example, let the eigenvalues of A might suggest K clusters for C and that of the Laplacian of A might suggest K' =  K + 1 clusters for C' such that all the clusters in C and C' are the same except for one cluster that splits into two.  C' is still "close" to C, but it cannot be captured by the given framework.  7) Related work section does not reflect a lot of model free results that exist: for example:  "Clustering with Spectral Norm and the k-means Algorithm", Kumar and Kannan, 2010. "Clustering under Perturbation Resilience", Balcan et. al. 2012. "Clustering under Approximation Stability", Balcan et. al. 2013. How does the notion of stability considered in this work related to those in these works? In particular, how is the stability in this paper related to the stability measured for perturbation of centers for two clusterings?  8) The perturbations tolerated by simulations for PFM are too small. Line 252-253 casually states "large eigengap makes PFM(G, C) differ from PFM(G, C_0) even for very small perturbations" without really qualifying the statement. It is unclear why it is so. On the other had, the political blog data is smoothed out by raising the power of A which would further exaggerate the gaps in eigenvalues, but in this case it seems to help stability?  Also, if such small perturbations can affect the model fit, then it renders the guarantee not of much use for practical purposes (which is one of the main selling points since the quantities can be actually calculated from the data). Lines 248-250 state that SBM guarantee conditions seem to be very hard to satisfy which is unfortunate.  9) Figure 1 caption mentions "For SBM, \delta is always greater than \delta_0". Which of the subplots is this referring to? If SBM is not plotted, then it should be made clear in the caption.  10) The guarantees in the theorems for PFM are for weighted ME, whereas in the experiment the perturbation is done for ME.     Minor Points: 1) Labels for axes in Figure 1 are missing.  2) (Line 44) For block models, A_{ij} for i > j are independent _given the cluster assignments_ for i and j.  3) B, C and Y have been used with _different_ definitions for PFM and later in 3.2 for SBM. This can be confusing in general. Given that the paper has a lot of notations, it is better to maintain consistency throughout. For example, may be use Y_{\orthogonal} and C_D respectively instead of B and C for PFM. Also, for the estimator of SBM, Y_Z is not required. It is better to use the expression of Y_Z directly in Proposition 8.  Similarly for \lambda in PFM and SBM, using \lamda^L an \lambda^B would clearly distinguish which eigenvalues are being talked about.  4) 4.4 is better suited to be a subsection in related work.


Scaled Least Squares Estimator for GLMs in Large-Scale Problems
We study the problem of efficiently estimating the coefficients of generalized linear models (GLMs) in the large-scale setting where the number of observations n is much larger than the number of predictors p, i.e. n p 1. We show that in GLMs with random (not necessarily Gaussian) design, the GLM coefficients are approximately proportional to the corresponding ordinary least squares (OLS) coefficients. Using this relation, we design an algorithm that achieves the same accuracy as the maximum likelihood estimator (MLE) through iterations that attain up to a cubic convergence rate, and that are cheaper than any batch optimization algorithm by at least a factor of O(p). We provide theoretical guarantees for our algorithm, and analyze the convergence behavior in terms of data dimensions. Finally, we demonstrate the performance of our algorithm through extensive numerical studies on large-scale real and synthetic datasets, and show that it achieves the highest performance compared to several other widely used optimization algorithms.

The paper is based on the fact that, when the predictors are normally distributed, the regression coefficients of a GLM (with canonical link) are proportional to the corresponding GLM coefficients.  The authors propose to exploit this fact to reduce the computational burden of fitting GLMs. In particular, they estimate the coefficients by OLS and then they find the proportionality constant using an univariate root-finding method.  They also show that the results approximately apply outside the Gaussian context, and provide bounds on the discrepancy between GLM and (scaled) OLS coefficients.
The main idea of the paper is an interesting and, at least to me, novel one. Sections 1 to 3 are extremely well written, while the second half of the paper seems a bit more hurried.  It would be worthwhile pointing out early in the paper that the proposed approach applies only when the canonical link is used (currently stated in line 92). However, the authors say: "Motivated by the results in the previous section, we design a computationally efficient algorithm for any GLM task that is as simple as solving the least squares problem; it is described in Algorithm 1." The "any" suggests that the canonical link is not required. Please clarify.  In the right plot of Figure 1 it might be more useful plotting the relative accuracy of the proposed method, that is MSE(SLS)/MSE(MLE). Also, are you using sub-sampling or the full sample for SLS here?  In Figure 1 STS does strictly worse than MLE in terms of accuracy. Given that here the design is Gaussian and given Proposition 1, it is not entirely clear to me why this should happen. It is because estimating the constant c introduces extra variance?  I am a bit puzzled by the decision of using sub-sampling here. The STS method should be applicable whether or not subsampling is used. Hence using subsampling makes the results in Section 5 harder to interpret. That is, in terms of speed-up and accuracy what is the contributions of STS and what is that of subsampling?  L204 The authors say: "On the other hand, each of the MLE algorithms requires some initial value for , but no such initialization is needed to find the OLS estimator in Algorithm 1. This raises the question of how the MLE algorithms should be initialized, in order to compare them fairly with the proposed method." It would have been useful to include also Iteratively Reweighted Least Squares (IRLS) in this comparison, given its popularity. Also, Wood (2006) page 66 explains that a default initialization is available for this method.    L215 The authors say: " For each dataset, the minimum achievable test error is set to be the maximum of the final test errors, where the maximum is taken over all of the estimation methods." Using this approach, if one of the methods fails, the test error will be set to a fairly high number. Have all methods converged on all datasets?   In Section 5 the authors use two simulated datasets. Are the results in Figure 2 and Table 1 the output of a single run and an average of several runs? At least a handful of run are needed in order to provide some confidence in the results.  Figure 2 is quite crammed. Maybe some space could be saved by eliminating redundant legends and axes labels. The lines in the legend are extremely small. Maybe it is also worth pointing out in the caption that STS does not start from zero on the x axis because of the fixed cost paid by OLS.  Lines 253-257 are quite confusing. What is the take home message of Corollary 1? I don't immediately see what the lasso results implies for the unpenalized GLM regression. Corollary 1 makes the assumption beta > eta / s, what is the meaning of this assumption and is it realistic? Similarly the authors set lambda = eta / (c*s), why this choice?  Lambda_min seems to be undefined in Proposition 2. It this defined in the Appendix, but something about it must be said also in the main text.   MINOR POINTS:  L46 I am not sure about this sentence: "For logistic regression with Gaussian design (which is equivalent to Fisher’s discriminant analysis)". Friedman et al. (2008) on page 127 say that they are quite different, even with Gaussian design. In particular, discriminant analysis is more efficient in this case, because it exploits the normality of the predictors.  L161 "that we can attain up to a cubic rate of convergence (by, for instance, using Hayley’s method)" this is a repetition.  L296 "Another interesting line of research is to find similar proportionality relations between the parameters in other large-scale optimization problems such as support vector machines. Then the problem complexity can be significantly reduced as in the case of GLMs." The latter sentences should be conditional on the former, but at the moment it is an assertion.  REFERENCES:  - Wood, Simon. Generalized additive models: an introduction with R. CRC press, 2006.  - Friedman, Jerome, Trevor Hastie, and Robert Tibshirani. The elements of statistical learning. 2nd Edition. Springer, Berlin: Springer series in statistics, 2008.


Infinite Hidden Semi-Markov Modulated Interaction Point Process
The correlation between events is ubiquitous and important for temporal events modelling. In many cases, the correlation exists between not only events' emitted observations, but also their arrival times. State space models (e.g., hidden Markov model) and stochastic interaction point process models (e.g., Hawkes process) have been studied extensively yet separately for the two types of correlations in the past. In this paper, we propose a Bayesian nonparametric approach that considers both types of correlations via unifying and generalizing the hidden semiMarkov model and interaction point process model. The proposed approach can simultaneously model both the observations and arrival times of temporal events, and automatically determine the number of latent states from data. A Metropoliswithin-particle-Gibbs sampler with ancestor resampling is developed for efficient posterior inference. The approach is tested on both synthetic and real-world data with promising outcomes.
with a variable duration time for each state. In addition to the HMM components, the HSMM models the duration of a state as a random variable and a state can emit a sequence of observations. The HSMM allows the flexibility of variable inter-arrival times, but it does not consider events' correlation on arrival times. In many real-world applications, one event can trigger the occurrences of others in the near future. For instance, earthquakes and epidemics are diffusible events, i.e., one can cause the occurrences of others. Trading events in financial markets arrive in clusters. Information propagation in social network shows contagious and clustering characteristics. All these events exhibit interaction characteristics in terms of arrival times. The likelihood of an event's arrival time is affected by the previous events' arrival times. Stochastic interaction point process (IPP), e.g., Hawkes process [6], is a widely adopted framework for capturing such arrival time correlation. It models the correlation via a conditional intensity function that depicts the event intensity depending on all the previous events' arrival times. However, unlike the SSMs, it lacks the capability of modelling events' latent states and their interactions.
It is clearly desirable in real-world applications to have both arrival time correlation and observation correlation considered in a unified manner so that we can estimate both when and how events will appear. Inspired by the merits of SSMs and IPPs, we propose a novel Bayesian nonparametric approach that unifies and generalizes SSMs and IPPs via a latent semi-Markov state chain with infinitely countable number of states. The latent states governs both the observation emission and new event triggering mechanism. An efficient sampling method is developed within the framework of particle Markov chain Monte Carlo (PMCMC) [1] for the posterior inference of the proposed model.
We first review closely related techniques in Section 2, and give the description of the proposed model in Section 3. Then Section 4 presents the inference algorithm. In Section 5, we show the results of the empirical studies on both synthetic and real-word data. Conclusions are drawn in Section 6.

This paper develops a joint model for the occurrance times and observed outcomes of discrete events: for example the onset time and electricity usage for variance appliances or the posting time and content of messages on a social network. The model is a sensible extension of a hidden Markov model (HMM) where each state has the standard transition and emission probabilities as well as a state-specific Hawkes process which generates the time until the next event. Using a hierarchical DP prior, the number of active states can be inferred from data.  To fit the proposed model to data, a particle Gibbs sampler algorithm is developed. The observed data in a sequence includes a time value t_n and an observed vector y_n at each discrete index n. The sampler must infer the state sequence s_1 ... s_N as well as the emission parameters, transition parameters, and Hawkes process parameters. Particle methods are used to sample the state sequence given all other variables, and then the remaining variables have standard Gibbs updates, except for the triggering kernels \psi and background rate \mu of the Hawkes process, which use random-walk Metropolis proposals.  Experiments compare the proposed method, called iHSMM-IPP, agains several baselines from the literature including the sticky HDP-HMM (Fox et al) and the semi-Markov HDP-HSMM (Johnson & Willsky '14) and two methods (variational [11] and Metropolis sampling [17]) for the marked Hawkes process. On toy data generated from the proposed model, the proposed algorithm recovers noticeably better Hamming distance scores. On an energy disaggregation task (REDD) and a pipe burst event task, the proposed method does better as well, likely due to its joint modeling of both event times and event emissions.
Overall I recommend accepting this paper as a poster presentation. I can see NIPS attendees benefiting from learning more about the model, inference algorithm, and the practical applications to event failure modeling. My chief concerns are questions about scalability and lack of within-method experiments to validate different features of the algorithm. The technical content could also use revision to improve readability, though the paper is organized well.  Novelty and related work ========================  This is the first paper I know of that combines Hawkes processes, HMMs with semi-Markov durations, and nonparametric priors like the HDP. However, it does look like there is some missing citations to previous efforts to synthesize Hawkes processes and state space models. See for example this somewhat paper on a switching state model where each state has a Hawkes process, which might be worth mentioning in related work (I haven't read carefully):  Markov-modulated Hawkes process with stepwise decay Wang, Bebbington, and Hart Ann Inst Stat Math (2012) 64:521–544 http://www.ism.ac.jp/editsec/aism/pdf/10463_2010_Article_320.pdf  On the inference side, this looks like a nice application of particle Gibbs methods. I hope the authors can share code so others can take advantage of this contribution. The description in the paper itself is likely too dense to easily allow reproducing this algorithm.  Algorithm feedback ================== Sec. 4 provides lots of gory detail about the various substeps of the algorithm, but is somewhat lacking in high-level description. In particular, I'd like to see discussion of the overall runtime complexity of the sampler: What is the big-oh cost of the particle sampling of the state sequence? How does it scale with sequence length, number of states K, number of particles, etc? How would it compare with standard forward-backward type inference for simpler HMM models?  How does the implementation do in a practical sense? That is, can you provide details like: it takes ___ seconds to handle a sequence of length ___ with ___ states.  What kind of hyperparameters/settings is the algorithm sensitive to? I'd expect that the number of particles needs to be set somewhat carefully to balance speed and accuracy. How is this chosen? (I didnt see it anywhere, apologies if I missed it).  How exactly is the algorithm initialized? I'd expect performance to be quite sensitive to initialization, especially if the number of active states is much lower than it needs to be.  The details of the truncated ancestor resampling (Sec 4.2) were confusing to me. What is being truncated? Why is this necessary? Big picture details would be useful here.  Experiment feedback =================== Overall I was pleased with the experiments: they include comparison to a large set of reasonable baselines and consider two challenging datasets (Redd and Pipe). I thought the biggest weakness was a lack of comparison to different possible settings of the proposed method: for example, using different numbers of particles, or with or without some hyperparameter resampling, etc. There are many speed/accuracy tradeoffs that could be made in practice, and the present paper doesn't provide much guidance about how to manage these concerns, or even indicate that such concerns should exist.  For the Pipe data experiments that predict the last 10% of each sequence given the first 90%, I am confused how the number of failure events is estimated for methods like the HDP-HMM that don't have any built-in model for durations. Can you clarify?  For REDD data, it would be interesting to visualize and explore qualitative relationships discovered by the learned triggering kernels. For example, is there a clear relationship between washing machine and dryer events, as expected?  Question: for REDD data, are the Hamming and LogLik score reported on the training set, or a heldout set? Please clarify. Seems that these numbers are for the training set of the Pipe data.  I'm curious how much the poor performance of other methods might be due to local optima, instead of modeling issues. To test this, I'd suggest initializing the HDP-HMM sampler or some other baseline method from the final state of the proposed iHSMM-IPP, or even from the "ground truth" labels, and report the Hamming distance after the sampler has run for a while.   Presentation feedback ==================== I like Fig 1 (illustrating model features alongside the graphical model) a lot. Thanks!  Line 196 typo: "infinite countable with the HDP" should be countably infinite  The graphical model in Fig 1 uses symbol T to indicate the total length of the sequence, but later the symbol N is used. Would be good to just pick one.  Fig 2: Plot on far left is missing x-axis ticks (# iterations). Please fix.


Adversarial Multiclass Classification: A Risk Minimization Perspective
Recently proposed adversarial classification methods have shown promising results for cost sensitive and multivariate losses. In contrast with empirical risk minimization (ERM) methods, which use convex surrogate losses to approximate the desired non-convex target loss function, adversarial methods minimize non-convex losses by treating the properties of the training data as being uncertain and worst case within a minimax game. Despite this difference in formulation, we recast adversarial classification under zero-one loss as an ERM method with a novel prescribed loss function. We demonstrate a number of theoretical and practical advantages over the very closely related hinge loss ERM methods. This establishes adversarial classification under the zero-one loss as a method that fills the long standing gap in multiclass hinge loss classification, simultaneously guaranteeing Fisher consistency and universal consistency, while also providing dual parameter sparsity and high accuracy predictions in practice.

This is a very interesting and well-written paper that closes the gap between theory and practice by providing a Fisher- and universally consistent loss function for multiclass classification that performs empirically well in practice. The authors show how adversarial loss minimization in the multiclass setting can be recast into an empirical risk minimization setup under the adversarial 0-1 loss function. They further prove that the adversarial 0-1 loss is both Fisher and universally consistent (so that it has advantageous theoretical properties) and demonstrate a novel algorithm that for efficient optimization that uses a constraint generation QP method (allowing the method to be computationally efficient).  The experiments are detailed and thorough, testing on 12 UCI datasets and comparing the adversarial 0-1 loss to three other multiclass SVM methods.
The main theorems of the paper are technically rich, and support the conclusion of the paper. The proofs are extremely well documented in the supplementary material, and easy to follow proof sketches are given in the main body of the work.   The experimental work compares the proposed method to three other competing state of the art multiclass SVM solvers across 12 datasets. The statistical significance of the experimental results are considered; 20 dataset splits are created and a paired t-test is performed. Additionally, the results are convincing in that they agree with previous work, as explained in lines 233-234.  However, I am curious as to why the authors did not compare against OVA multiclass classification since it has been extremely popular for its computational efficiency and shown to perform as well statistically as other methods (see Rifkin “In defense of One vs. All.”)  One main weakness of the experimental results is that they claim that the new algorithm is computationally efficient and provide a proof of convergence, but do not include any performance vs. time results that back up these claims.   Novelty/Originality  The work is highly novel in that it proposes a new loss function and a new, efficient algorithm to solve the associated optimization problem.   Potential impact or usefulness  The impact of the work is high since it combines work done from two somewhat distinct areas (adversarial classification and consistency of empirical risk minimization methods). In addition, the SVM algorithm for multiclass is highly popular, so I expect this result to affect a large number of people.   Clarity and presentation  Overall, the paper is well written. The intro very clearly lays out the gaps in existing approaches, and explains how the AL-01 approach bridges these gaps. The background gives a clear introduction to the main ideas needed to understand the theoretical results of the paper. The explanations in the experimental section make the findings easy to understand.  Some specific comments…  Figure 1 and Figure 2. These are difficult to understand, but I feel could be potentially very elucidating. Spend a little more time in text going over how to interpret these figures?  Table 2 #b - not clear what this means immediately. Should explain in Figure caption.


On Robustness of Kernel Clustering
Clustering is an important unsupervised learning problem in machine learning and statistics. Among many existing algorithms, kernel k-means has drawn much research attention due to its ability to find non-linear cluster boundaries and its inherent simplicity. There are two main approaches for kernel k-means: SVD of the kernel matrix and convex relaxations. Despite the attention kernel clustering has received both from theoretical and applied quarters, not much is known about robustness of the methods. In this paper we first introduce a semidefinite programming relaxation for the kernel clustering problem, then prove that under a suitable model specification, both K-SVD and SDP approaches are consistent in the limit, albeit SDP is strongly consistent, i.e. achieves exact recovery, whereas K-SVD is weakly consistent, i.e. the fraction of misclassified nodes vanish. Also the error bounds suggest that SDP is more resilient towards outliers, which we also demonstrate with experiments.

This paper introduces a semidefinite programming relaxation for the kernel clustering problem, then proves that under a suitable model specification, both the Kernel-SVD and Kernel-SDP approaches are consistent in the limit, albeit SDP is strongly consistent, i.e. achieves exact recovery, whereas K-SVD is weakly consistent. Experimental results on toy datasets are presented.
Here are few comments.   1. The analysis assumes that all inlier clusters are of equal size and uses this fact for analysis. However, this seems to be a very restrictive assumption. In real life, it is very unlikely that all clusters will be of equal size. It will be nice to have a discussion describing how much additional work is necessary to remove this unrealistic assumption.  2.  Using the current notation both the algorithms takes only inliers as input (Y-1,... Y_n are inliers). It will be nice to change the notation so that input to both algorithms can be both inliers as well as outliers.  3. Spectral clustering is based on normalized/unnormalized "Laplacian matrix" and not kernel matrix. It will be nice to have a discussion how they are related. Note that in case of spectral clustering using Laplacian matrix, eigenvectors corresponding to the lowest eigenvalues of Graph laplacian is used for embedding and k-means clustering, where as in case of kernel-PCA eigenvectors corresponding to the largest eigenvalues of the Gram matrix are used for k-means clustering.  4. Theorem 1 is based on Gaussian kernel (line 135) so is Lemma 1. Therefore, notation wise it will be easier to use K, \tilde{K} and exponential in place of function f etc.  5. The theoretical results of this paper are " high probability" results based on theorem 1. Theorem 1 uses an union bound and the failure probability is n^2p^{-c^2\rho}, where c and \rho are constants. Note that c is controllable depending on how much deviation one wants but \rho is just an existential constant. Therefore, for fixed data dimension p, as n tends to infinity the failure probability tends to one. For any fixed p, this limits the maximum size of inliers that the algorithm can handle. In particular if we want a failure probability of \delta, that implies n can be at most \sqrt{\delta p^{c^2\rho}}$.   6. Statement of Lemma 1 is a little unclear. In particular, the inequality in the proof is not clear. Note that only a lower bound of the eigengap is proved. No upper bound Big-Oh is proved and therefore using \Theta notation may not be correct.  7. In the supplementary material (proof of Lemma 1), no justification is not provided for the inequality between line 67 and 68. It is not clear how this inequality is obtained.  8. In proof of theorem 3, first only inliers are considered when no outliers are present and Davis-Kahan theorem is applied. However, in this case the corresponding expression, equation 4, involves $m$ which is the number of ouliers. This sounds strange because this analysis does not consider any outlier and since presence of ouliers are handled right after equation 4 and is given in equation 5.  9. Proof of theorem 3 utilizes results of [22]  which requires that centroids are well separated. It guaranteed in proof of theorem 3 that the centroids are well separated  so that results of [22] can be applied.   10. The accuracy of the clustering process is measured based on the criteria that inliers are assigned to the correct respective clusters. However, in presence of outliers, if an outlier is assigned to a cluster (consists of inliers) that is also an error.  How is this error taken into consideration in the analysis?   11. All experimenal results are on toy datasets involving same \sigma for all clusters. Experimental results can be improved by considering, (a) different sigma values  for different clusters in case of toy (synthetic) datasets, (b) varying dimensionality p for synthetic datasets (choce of specific p=160 seems arbitrary),  (c) applying the proposed method on realworld datasets.


A Posteriori Error Bounds for Joint Matrix Decomposition Problems
Joint matrix triangularization is often used for estimating the joint eigenstructure of a set M of matrices, with applications in signal processing and machine learning. We consider the problem of approximate joint matrix triangularization when the matrices in M are jointly diagonalizable and real, but we only observe a set M' of noise perturbed versions of the matrices in M. Our main result is a first-order upper bound on the distance between any approximate joint triangularizer of the matrices in M' and any exact joint triangularizer of the matrices in M. The bound depends only on the observable matrices in M' and the noise level. In particular, it does not depend on optimization specific properties of the triangularizer, such as its proximity to critical points, that are typical of existing bounds in the literature. To our knowledge, this is the first a posteriori bound for joint matrix decomposition. We demonstrate the bound on synthetic data for which the ground truth is known.

The paper addresses the problem of approximate non-orthogonal joint diagonalization (NOJD) of several matrices, where each matrix is of the form $V_o \Lambda_n V_o^{-1}$ plus some noise. Such problem appears, e.g., when performing estimation in some latent variable models via the method of moments. The authors approach this NOJD problem via related simultaneous Schur decomposition (SSD), which is formulated as a non-convex optimization problem (OP).  The main result of this paper is an a posteriori bound (which depends solely on a posteriori quantities). The only a priori quantity appearing in the bound is the noise parameter $\sigma$, which the authors assume to be known (see lines 57-59). For the obtained bound, an approximate joint triangularizer (the output of an algorithm for OP) is not required to be neither a global solution nor a critical point and the bound depends on the respective objective value. This bound is the first a posteriori bound for the SSD problem and, more generally, for joint matrix decomposition problems.
Update of the review: I read the rebuttal and other reviews and argue for acceptance of the paper. I would ask the authors to pay attention to the application of the Weyl's theorem which holds true only for Hermitian matrices.  The main result of the paper -- an a posteriori bound for the SSD problem -- and the techniques used to obtain the result are novel both in the context of the SSD problem and in the context of joint matrix decomposition problems. Since joint matrix decomposition methods are known to work well in practice for computing tensor factorizations, this result can be a step forward in understanding this empirical observation. Therefore, the contributions of this paper are important and can be of interest to a certain community.   The paper, however, is not written in a reader friendly way. I especially disagree with the way the authors motivate the problem since it looks like confusing several different matrix joint decomposition problems (see the detailed explanation in the next paragraph), which (a) can easily be interpreted in a wrong way and (b) can look like hiding some important assumptions. Therefore, I would insist on rewriting the introductory section in a clearer way. Moreover, some part of the proof of Theorem 1 (the main result; see the detailed discussion below) are subtle and have to be clarified. Given the authors successfully fix the mentioned problems, I would support the acceptance of the paper.  The citations are not in accordance with the NIPS template (Authors, year format vs [number]).  ---  *Introduction.* One of my points of criticism is how the problem is motivated, e.g., in lines 24-26. Basically, there are two different types of simultaneous diagonalization problems: problem (1) in lines 19-20 and joint diagonalization by congruence (see comment #1). The latter is one of several techniques to obtain *symmetric* tensor CP decomposition, which, e.g., naturally arises in estimation for some latent variable models (as in the cited papers by Anandkumar et al, Balle et al, Kuleshov et al where the considered matrices are tensor contractions/projections or slices). The former naturally arises, e.g., in (De Lethauwer et al, 2004; section 5) as a technique to obtain *non-symmetric* tensor CP decomposition. One can transform the matrices in joint diagonalization by congruence to problem (1) (see comment #2). This, however, requires some additional assumptions (invertibility of $V_o$?) and poses some additional questions, e.g., whether this transformation simplifies the original problem or instead makes it more complicated. These differences should be explicitly addressed in the paper given the authors want to motivate the problem with applications discussed in Anandkumar, et al, Balle et al, Kuleshov et al. As an alternative or in addition, the authors could motivate the problem with applications which boil down to a non-symmetric CP decomposition (as, e.g., discussed in De Lethauwer et al, 2004; [4] would be a more recent machine learning reference). In any case, the differences, assumptions, and connections between the problems have to be explicitly addressed.  Comments: #1: There are two types of simultaneous diagonalization problems: the problem in (1) and joint diagonalization by congruence, where the matrices have the form $V_o \Lambda_n V_o^{\top}$, i.e. the last matrix goes with the *transpose* instead of the *inverse*. The two problems are equivalent only if $V_o$ is orthogonal and then it is the problem of orthogonal joint diagonalization considered, e.g., by Cardoso and Souloumiac (1996). The setting where $V_o$ is orthogonal is, however, not the main setting in the paper being reviewed.  #2: In joint diagonalization by congruence the matrices are in the form $V_o \Lambda_n V_o^{\top}$. Assuming $V_o$ and say j-th matrix are invertible, multiply i-th matrix by the inverse of the j-th matrix to obtain a matrix $V_o \Lambda_i \Lambda_j^{-1} V_o^{-1}$ in the form suitable for problem (1) (see, e.g., De Lethauwer et al, 2004, section 5).  This being said, we recall that the perturbation analysis is only obtained for the SSD problem, but not for related tensor factorization problems. Do the results of the analysis easily extend to this case?  ---  *The proof of Theorem 1.* I have several questions regarding the proof: - The definition (line 212) of the quantity $T_{\beta}$ is not clear and, possibly wrong (see comment #4). This definition does not influence the overall result significantly, but this definition has to be clarified. - Is it obvious that min and max can be exchanged in (29)? (see comment #5) - Lemma 2 is provided without proof and I can not check whether the result is correct, esp., given the uncertainty in the definition of $T_{\beta}$. - The expansion in Eq. (32) does not obviously follow from the Weyl's theorem applied to the singular values of the matrices in line 224. The expression is in therms of the condition numbers ($kappa$?) of $V$ and $\hat{V}$ and not the singular values of $M$ and $\hat{M}$.   Comments: #4: In definition of the quantity $T_{\beta}$ in line 212, it not clear what Low means. Assuming the first Low refers to the lower triangular part of a matrix, the LHS of (25) is not equal to the LHS of (24). E.g., $low( M_{\beta} \otimes 1 ) vec(X)$ is actually the zero matrix and $low( 1 \otimes M_{\beta}^{\top} ) vec(X)$ is not equal to neither $vec( low( low(X) M_{\beta} ) )$ nor $vec( low( M_{\beta} low(X) ) )$. Obvious modifications of this result do not seem to help. Clearly, there is a matrix $T_{\beat}$ can be expressed via $M_{\beta}$, but the question is how simple / difficult this expression is? #5: The problem is that the sets in min and max are not convex and compact. Is it obvious to show that these constraints can be relaxed to some convex and compact sets such that the solution is tight? I can see that the min problem can be reformulated as minimization over the simplex, but the objective function is not linear in $\beta$ anymore (an upper envelope of piecewise linear functions) and, hence, it does not seem to be straightforward to relax the unit sphere to the unit ball for $\beta$. Is there a different explanation? If not, does it help that by weak duality the first equality in (29) can always be replaced by the less or equal sign.  ---  Other less important questions and comments:  I cannot check the correctness of Lemma 1 since the proof was not provided. However, the results of the lemma are not used in other proofs, so I see it as a complimentary result. Lemma 1 provides a kind of identifiability results for the SSD problem. Is there some intuition behind these results (e.g., analogy to permutation and scaling unidentifiability in the joint diagonalization by congruence case)?  After introducing problem (1), the authors may consider mentioning that this problem was already addressed in the literature, e.g., in [1,2,3] (see below) or by De Lethauwer et al (2004). However, papers [1,2,3] do not provide theoretical analysis and the first order perturbation analysis is from section 6 of De Lethauwer et al (2004) is weaker than the results of this paper. One could also add an explanation of these differences.  In lines 38-39, reference to Cardoso and Souloumiac (1996) is not appropriate since their optimization is also over the manifold of orthogonal matrices. The related optimization problems which do not involve the manifold of orthogonal matrices are, e.g., the ones by Afsari (2006) or Kuleshov et al (2015) or [9]. Moreover, the statement in lines 38-39 is a bit vague since the problems have different objective functions.  In lines 40-41, one could add references, e.g., to De Lethauwer et al (2004) and [6].  The known in the literature bounds (Cardoso, 1994; Afsari, 2008; Kuleshov et al, 2015) are obtained for joint diagonalization by congruence and, therefore, are formally different from the SSD problem considered in this paper. On the contrary, an a priori bound for the SSD problem was recently obtained [6] and the authors may consider to cite this.   List of typos/comments: 1) One should define the set GL(d) in equation (5). 2) It could be useful to specify which matrix norm is used, esp. in the proof of Theorem 1. 3) In the transition between equations (22) and (23), it worth mentioning that the expansion $e^{\alpha X} = 1 + \alpha X + O(\alpha^2)$ is used. 4) In equations (24), (25) and further the term $O( (\alpha + \sigma)^2 )$ is lost (either explain why it is omitted or put it back in). 5) In equation (28), the rightmost $\sigma$ has to be squared. 6) ``R'' is missing in the sum sign in line 223. 7) It would be useful to define $\kappa(\cdot)$ and $\sigma_{max/min}(\cdot)$ appearing in the proof - the condition number and the singular values? 8) In line 230, "min" and "max" are probably mixed up. 9) Is there some intuition behind the identifiability results in Lemma 1 (like permutation and scaling)? 10) Comment to line 132: there is only one way to obtain a slice of a tensor, but the matrices which are to be diagonalized can be constructed in different algebraic form. 11) In lines 158=159, the whitening procedure does not necessary lead to poor performance (and this is what the cited paper says). Moreover, it can be (and often is) used for dimensionality reduction.   ---  References: [1] ``Joint eigenvalue decomposition using polar matrix factorization'' by Luciani and Albera from 2010;  [2] ``A new Jacobi-like method for joint diagonalization of arbitrary non-defective matrices'' by Iferroudjene, Abed Meraim, and Belouchrani from 2009;  [3] ``Simultaneous diagonalization with similarity transformation for non-defective matrices'' by Fu and Gao from 2006; [4] ``Beyond CCA: moment matching for multi-view models'' by Podosinnikova, Bach, and Jacoste-Julien from 2016; [5] ``Nonorthogonal joint diagonalization by combining Givens and hyperbolic rotations'' by Souloumiac from 2009; [6] ``Tensor decomposition via joint matrix Schur decomposition'' by Colombo and Vlassis from 2016;


Inference by Reparameterization in Neural Population Codes
Behavioral experiments on humans and animals suggest that the brain performs probabilistic inference to interpret its environment. Here we present a new generalpurpose, biologically-plausible neural implementation of approximate inference. The neural network represents uncertainty using Probabilistic Population Codes (PPCs), which are distributed neural representations that naturally encode probability distributions, and support marginalization and evidence integration in a biologically-plausible manner. By connecting multiple PPCs together as a probabilistic graphical model, we represent multivariate probability distributions. Approximate inference in graphical models can be accomplished by message-passing algorithms that disseminate local information throughout the graph. An attractive and often accurate example of such an algorithm is Loopy Belief Propagation (LBP), which uses local marginalization and evidence integration operations to perform approximate inference efficiently even for complex models. Unfortunately, a subtle feature of LBP renders it neurally implausible. However, LBP can be elegantly reformulated as a sequence of Tree-based Reparameterizations (TRP) of the graphical model. We re-express the TRP updates as a nonlinear dynamical system with both fast and slow timescales, and show that this produces a neurally plausible solution. By combining all of these ideas, we show that a network of PPCs can represent multivariate probability distributions and implement the TRP updates to perform probabilistic inference. Simulations with Gaussian graphical models demonstrate that the neural network inference quality is comparable to the direct evaluation of LBP and robust to noise, and thus provides a promising mechanism for general probabilistic inference in the population codes of the brain.

A fundamental challenge for probabilistic interpretations of brain computation is to figure out how can neural networks represent complex probability distributions and use them to guide actions. Two main classes of solutions have been proposed - PPCs which work well for univariate distributions and very restricted parametric families of multivariate distributions and sampling based representations which have more flexibility but at the cost of potentially long sampling times. This paper considers a tempting alternative within the class of variational algs, using a continuous time approximation of TRP. They show that the resulting network dynamics are biologically plausible (and similar to traditional PPCs) and does numerically not much worse than the original. Overall, a neat idea which is worth exploring further in particular in terms of neural implications.  A few comments: - the main advantage of such a representations seem to me to be twofold a) easy access to posterior marginals throughout the computation (which is what probably drives individual decisions) and b) convergence speed.  It would be nice to have a better illustration of this in the results, In particular how long does it take to reach convergence in neural terms (some way to tie dynamics to membrane time constant? how slow are the slow dynamics relative to this)? How much faster is this compared to a standard sampling based alternative implementation? - the idea of state space expansion when mapping random variable is something which has been put forward in the context of sampling as well (Savin& Deneve, Nips 2013). It would be interesting to make these parallels more explicit ...
Would be nice to have a clearer understanding of what the different scales of the dynamics mean in neural terms, beyond the fact that the network activity looks complicated. Fig.4,6: what are the units for the time axis (fast time scale = membrane time const? slow timescale = synaptic facilitation/depression or NMDA/GABAb currents or sth even slower ?)? Still not 100% sure how the graphical model structure translates into the network connectivity structure   Minor typo: last page "biologically plauble interpretable"


Iterative Refinement of the Approximate Posterior for Directed Belief Networks
Variational methods that rely on a recognition network to approximate the posterior of directed graphical models offer better inference and learning than previous methods. Recent advances that exploit the capacity and flexibility in this approach have expanded what kinds of models can be trained. However, as a proposal for the posterior, the capacity of the recognition network is limited, which can constrain the representational power of the generative model and increase the variance of Monte Carlo estimates. To address these issues, we introduce an iterative refinement procedure for improving the approximate posterior of the recognition network and show that training with the refined posterior is competitive with state-of-the-art methods. The advantages of refinement are further evident in an increased effective sample size, which implies a lower variance of gradient estimates.

This paper proposes a sequential Monte Carlo algorithm to improve posterior estimation in a class of generative models.  The main motivation is to reduce variance during training of autoencoder models that use discrete random variables in their latent space, which are unsuitable for standard variational autoencoder (VAE) training as they prevent the use of the so-called "reparametrization trick". Backpropagation through discrete stochastic unit is possible by using Monte Carlo gradient estimation, but these techniques tend to suffer from high estimation variance and hence poor convergence speed.  The proposed approach, iterative refinement for variational inference (IRVI), applies at each training step a sequential Monte Carlo algorithm to refine the posterior in order to reduce the estimation variance. The approach depends on the choice of a refinement procedure, which in the experiments reported in the paper is adaptive importance refinement (AIR).  The paper compares the proposed methods with other method described in the literature, including SOTA methods.
The paper is very clearly written and describes technical concepts in a very comprehensible way.  The approach is sound and well motivated and the experimental comparisons with other approaches are fair, though they could have been more extensive in terms of datasets.  My greatest concern is about the execution time of the proposed approach, since this is a sequential Monte Carlo method that performs multiple refinement passes for each step of the training process. The authors report convergence curves vs epochs but not vs wall clock time, which should be provided as the main motivation of the paper is to speed up training for this class of generative methods.  The experimental section is good in terms of which methods it compares against, but a bit lacking in terms of datasets. Comparisons on larger images and color images should be used. The paper reports SOTA or near SOTA likelihood, which is good, but it has been argued in the literature that this measure may not well correlate with subjective quality of generative models, hence larger and richer color image examples would be a valuable addition to the experimental section.  In conclusion I think that this paper provides a valuable contribution, it is well written well placed in comparison with existing approaches. It does however need some discussion and empirical evaluation of wall clock time complexity and it might use some more extensive choice of datasets in the experimental section.


Learning Tree Structured Potential Games
Many real phenomena, including behaviors, involve strategic interactions that can be learned from data. We focus on learning tree structured potential games where equilibria are represented by local maxima of an underlying potential function. We cast the learning problem within a max margin setting and show that the problem is NP-hard even when the strategic interactions form a tree. We develop a variant of dual decomposition to estimate the underlying game and demonstrate with synthetic and real decision/voting data that the game theoretic perspective (carving out local maxima) enables meaningful recovery.

This paper introduces a max margin framework for learning potential games where the underlying interactions have a tree structure. A hardness result is presented first, followed by a dual decomposition algorithm for learning the tree structure and parameters of the game. Lastly, experimental results on synthetic and real datasets show that the algorithm recovers reasonable results.
I like the underlying idea of this paper, that of applying max margin learning to potential games. This works well with viewing observed behavior as local equilibria of the potential function. However, I have a couple serious concerns about the execution and technical content.  My major concern is that the paper very quickly glosses over the issue of learning N_i, the set of neighbors of each node. This appears to be the key part of the inference: as the authors point out, finding the optimal parameters with the tree structure fixed is just a quadratic program. It is claimed that special properties like submodularity could be used, but no such cases are explored in the paper. This leaves brute force, which is clearly not sufficient to tackle realistically sized problems. Also, it is never mentioned which method is used in the experiments.   Next, it would be helpful to see more detailed experiments. The synthetic and real datasets are good for illustrating that the algorithm is on some level reasonable, but two dimensions are somewhat lacking. First, all of the datasets considered are very small (and no runtime results are shown). So, it's hard to know how the algorithm scales, or whether it gives good results outside of small, clearly structured examples. Second, there is no comparison to other techniques for learning games. I'm not able to give a comprehensive list of recommendations, but a couple of papers which tackle similar problems are  Quang Duong, Yevgeniy Vorobeychik, Satinder Singh, Michael P. Wellman. "Learning Graphical Game Models". IJCAI 2009.  Jean Honorio, Luis Ortiz. "Learning the Structure and Parameters of Large-Population Graphical Games from Behavioral Data". UAI 2012.   Some comparison to alternative ways of learning graphical games would be serve to validate the overall max margin approach.  A few smaller issues:  1) It's not clear where the assumed tree structure makes the inference easier. I assume that it simplifies learning the parameters, but this should be explicitly discussed somewhere.  2) The meaning of the "context" is not clear. What was the context in the two real world datasets?  3) In the equation just before line 59, "y" is undefined.  4) In the Lagrangian (just after line 118), is delta_ij the Kronecker delta? If so, isn't the summation identically zero?


Observational-Interventional Priors for Dose-Response Learning
Controlled interventions provide the most direct source of information for learning causal effects. In particular, a dose-response curve can be learned by varying the treatment level and observing the corresponding outcomes. However, interventions can be expensive and time-consuming. Observational data, where the treatment is not controlled by a known mechanism, is sometimes available. Under some strong assumptions, observational data allows for the estimation of dose-response curves. Estimating such curves nonparametrically is hard: sample sizes for controlled interventions may be small, while in the observational case a large number of measured confounders may need to be marginalized. In this paper, we introduce a hierarchical Gaussian process prior that constructs a distribution over the doseresponse curve by learning from observational data, and reshapes the distribution with a nonparametric affine transform learned from controlled interventions. This function composition from different sources is shown to speed-up learning, which we demonstrate with a thorough sensitivity analysis and an application to modeling the effect of therapy on cognitive skills of premature infants.

This paper proposes a two-layered prior model for dose-response learning. A Gaussian process is first learned from observational data and then is reshaped with a nonparametric affine transformation from controlled interventions. The key modelling idea is expressed in Eq. (5), where a nonstationary covariance function is constructed from the observational data. The advantages of the newly proposed model is empirically demonstrated by two examples involving simulation.
Generally speaking the paper is written pretty clearly (except some questions and minor issues that I will list at the end). The underlying reason of the proposed model can be easily understood and the two case studies (which both involve simulations) have shown some advantages of the proposed model.   My main concern is that the idea of the paper does not seem significantly novel. The “elementwise affine” transform a . f_{obs} + b is simple and borrowing the information from K_{obs} to k_a also seems a natural way to address the problem. Overall this is more like developing a model suitable for one particular kind of problems. In addition, since both of the two examples involve simulation, the performance of the model has not been evaluated in any real data. (After all, the competitors to the proposed model in the paper seem naturally worse just from their semantics.)  Some questions and minor issues that I have found in the paper are listed as follows. (1). In Line 110: Why do you choose a Matern 3/2 covariance function for g(.,.)? Have you tried other (common) covariance options or Matern with different parameters for the comparison before choosing a Matern 3/2 covariance?  (2). In Line 120: What is the reason that “the cost of computing the covariance matrix K_{obs} of f_{obs}(X) is O(T^2 N^2)”? Do you assume that the TN * TN covariance matrix of the joint predictive distribution of {g(x, z^{(i)}} (mentioned at the end of Line 114) has already been computed? What is the running time of obtaining the posterior over functions g(.,.) (in your two studied cases)?  (3). For Eq. (6) (which is right after Line 135) 10^{-5} seems a magic number. Should it depend on the learned value of \lambda_a? How do you choose such a number?   (4). In Line 136: In “(\lambda_a, \sigma_h) are hyperparameters”, \sigma_h should be \sigma_a.  (5). Right after Line 146: Again, 0.5 in N(0, 0.5) and 0.1 in N(0, 0.1) are magic numbers. Should they be fixed or adjusted according to different data cases?    (6). In the third line of the caption for Figure 2: What is the meaning of “adjusted” in “adjusted \mu_{obs}”?  What is the meaning of “unadjusted” in “unadjusted Gaussian process regression”?


A scaled Bregman theorem with applications
Bregman divergences play a central role in the design and analysis of a range of machine learning algorithms through a handful of popular theorems. We present a new theorem which shows that "Bregman distortions" (employing a potentially non-convex generator) may be exactly re-written as a scaled Bregman divergence computed over transformed data. This property can be viewed from the standpoints of geometry (a scaled isometry with adaptive metrics) or convex optimization (relating generalized perspective transforms). Admissible distortions include geodesic distances on curved manifolds and projections or gauge-normalisation. Our theorem allows one to leverage to the wealth and convenience of Bregman divergences when analysing algorithms relying on the aforementioned Bregman distortions. We illustrate this with three novel applications of our theorem: a reduction from multi-class density ratio to class-probability estimation, a new adaptive projection free yet norm-enforcing dual norm mirror descent algorithm, and a reduction from clustering on flat manifolds to clustering on curved manifolds. Experiments on each of these domains validate the analyses and suggest that the scaled Bregman theorem might be a worthy addition to the popular handful of Bregman divergence properties that have been pervasive in machine learning.
Problem A Problem B that Theorem 1 reduces A to Reference Multiclass density-ratio estimation Multiclass class-probability estimation §3, Lemma 2 Online optimisation on L q ball Convex unconstrained online learning §4, Lemma 4 Clustering on curved manifolds Clustering on flat manifolds §5, Lemma 5 Table 1: Applications of our scaled Bregman Theorem (Theorem 1) -"Reduction" encompasses shortcuts on algorithms and on analyses (algorithm/proof A uses algorithm/proof B as subroutine).
differentiable scalar convex ϕ, for g(x) = 1 + x andφ(x)
= g(x) · ϕ(x/g(x)), g(x) · D ϕ (x/g(x) y/g(y)) = Dφ(x y) , ∀x, y.
Since the binary class-probability function η(x) = Pr(Y = 1|X = x) is related to the classconditional density ratio r(x) = Pr(X = x|Y = 1)/ Pr(X = x|Y = −1) via Bayes' rule as η(x) = r(x)/g(r(x)) ( [24] assume Pr(Y = 1) = 1/2), anyη with small D ϕ (η η) implicitly produces anr with low Dφ(r r) i.e. a good estimate of the density ratio. The Bregman property of eq.
(1) thus establishes a reduction from DRE to CPE. Two questions arise from this analysis: can we generalise eq.
(1) to other g(·), and if so, can we similarly relate other problems to each other?
This paper presents a new Bregman identity (Theorem 1), the scaled Bregman theorem, a significant generalisation of Menon and Ong [24, Lemma 2]. It shows that general distortions Dφ -which are not necessarily convex, positive, bounded or symmetric -may be re-expressed as a Bregman divergence D ϕ computed over transformed data, and thus inherit their good properties despite appearing prima facie to be a very different object. This transformation can be as simple as a projection or normalisation by a gauge, or more involved like the exponential map on lifted coordinates for a curved manifold. Our theorem can be summarized in two ways. The first is geometric as it specializes to a scaled isometry involving adaptive metrics. The second calls to a fundamental object of convex analysis, generalized perspective transforms [11,22,23]. Indeed, our theorem states when "the perspective of a Bregman divergence equals the distortion of a perspective", for a perspective (φ in eq. 1) which is analytically a generalized perspective transform but does not rely on the same convexity and sign requirements as in Maréchal [22,23]. We note that the perspective of a Bregman divergence (the left-hand side of eq. 1) is a special case of conformal divergence [27], yet to our knowledge it has never been formally defined. As with the aforementioned key properties of Bregman divergences, Theorem 1 has potentially wide implications for ML. We give three such novel applications to vastly different problems (see Table 1):
• a reduction of multiple density ratio estimation to multiclass-probability estimation ( §3), generalising the results of [24] for the binary label case, • a projection-free yet norm-enforcing mirror gradient algorithm (enforced norms are those of mirrored vectors and of the offset) with guarantees for adaptive filtering ( §4), and • a seeding approach for clustering on positively or negatively (constant) curved manifolds based on a popular seeding for flat manifolds and with the same approximation guarantees ( §5).
Experiments on each of these domains ( §6) validate our analysis. The Supplementary Material (SM) details the proofs of all results, provides the experimental results in extenso and some additional (nascent) applications to exponential families and computational information geometry. = {1, 2, ..., k} for k ∈ N. For any differentiable (but not necessarily convex) ϕ : X → R, we define the Bregman distortion D ϕ as
If ϕ is convex, D ϕ is the familiar Bregman divergence with generator ϕ. Without further ado, we present our main result.
Theorem 1 Let, ϕ : X → R be convex differentiable, and g : X → R * be differentiable. Then,   Remark. If X g is a vector space, ϕ satisfies eq. (5) if and only if it is positive homogeneous of degree 1 on X g (i.e. ϕ(αz) = α · ϕ(z) for any α > 0) from Euler's homogenous function theorem. When X g is not a vector space, this only holds for α such that αz ∈ X g as well. We thus call the gradient condition of eq. (5) "restricted positive homogeneity" for simplicity.
Remark. Appendix IV gives a "deep composition" extension of Theorem 1.
For the special case where X = R, and g(x) = 1 + x, Theorem 1 is exactly [24, Lemma 2] (c.f. eq. 1). We wish to highlight a few points with regard to our more general result. First, the "distortion" generatorφ may be 1 non-convex, as the following illustrates.
Example. Suppose ϕ(x) = (1/2) x 2 2 , the generator for squared Euclidean distance. Then, for g(x) = 1 + 1 x, we haveφ(
Whenφ is non-convex, the right hand side in eq. (3) is an object that ostensibly bears only a superficial similarity to a Bregman divergence; it is somewhat remarkable that Theorem 1 shows this general "distortion" between a pair (x, y) to be entirely equivalent to a (scaling of a) Bregman divergence between some transformation of the points. Second, when g is linear, eq. (3) holds for any convex ϕ (This was the case considered in [24]). When g is non-linear, however, ϕ must be chosen carefully so that (ϕ, g) satisfies the restricted homogeneity conditon 2 of eq. (5). In general, given a convex ϕ, one can "reverse engineer" a suitable g, as illustrated by the following example.
2 )/2. Then, eq. (5) requires that x 2 2 = 1 for every x ∈ X g , i.e. X g is (a subset of) the unit sphere. This is afforded by the choice g(x) = x 2 .
Third, Theorem 1 is not merely a mathematical curiosity: we now show that it facilitates novel results in three very different domains, namely estimating multiclass density ratios, constrained online optimisation, and clustering data on a manifold with non-zero curvature. We discuss nascent applications to exponential families and computational geometry in Appendices V and VI.

This paper aims to provide further understanding of how the *Bregman divergence* behaves under a certain type of transformation, in particular where the inputs are modified by a scaling function. The Bregman divergence is a generalization of a "distance measure" that arises in many areas of Machine Learning and is a key tool in understanding various aspects of convex analysis and optimization. The key result of the paper is to give full conditions under which the Bregman divergence is "well behaved" under scaling; that is under the mapping x --> x / g(x) where g() is some given function. The paper gives a central result along these lines, and then proceeds to describe three applications of the theorem: (a) a new way to compute multiclass density ratios, (b) an interesting approach to dual-norm mirror descent, and (c) some analysis tools for clustering on manifolds.
 My overall assessment is mixed. Here are my high level points:  1. As someone who has worked a lot with Bregman divergences, I do find the main result to be quite appealing and intriguing. For general transformations, whenever you do map your inputs from one space to the other, it's typically never that simple to see what happens to the Bregman divergence on the resulting space. So this result seems to provide some clarity here in addition to being a generic tool. Also, the three applications provided are quite diverse and surprising, and these lend credibility to the claim that the result is broadly useful.  2. On the other hand, when you think about what the result is saying, it is fundamentally just a calculus identity. The authors are just showing that one has two different ways in which to compute "divergence from the linear approximation", under certain constraints on the function g(). When stated this way, it's more difficult to see this as an ML-ish result.  3. Finally, upon looking at the paper again, it does feel like the authors never really explain what the main theorem really means. The thing that gets me is that the transformation, x --> x / g(x), is just a weird one. The result seems to want to let g() be generic, but outside of normalization (i.e. dividing by a p-norm), how often does this type of transformation arise? And what do you do when g(x) = 0? The paper seems to want us to rely on the applications to answer these questions, but I would like further intuition as well.   The authors may want to provide additional thoughts in their rebuttal.


Cooperative Inverse Reinforcement Learning
For an autonomous system to be helpful to humans and to pose no unwarranted risks, it needs to align its values with those of the humans in its environment in such a way that its actions contribute to the maximization of value for the humans. We propose a formal definition of the value alignment problem as cooperative inverse reinforcement learning (CIRL). A CIRL problem is a cooperative, partialinformation game with two agents, human and robot; both are rewarded according to the human's reward function, but the robot does not initially know what this is. In contrast to classical IRL, where the human is assumed to act optimally in isolation, optimal CIRL solutions produce behaviors such as active teaching, active learning, and communicative actions that are more effective in achieving value alignment. We show that computing optimal joint policies in CIRL games can be reduced to solving a POMDP, prove that optimality in isolation is suboptimal in CIRL, and derive an approximate CIRL algorithm.

The paper presents a general framework for cooperative interaction between a human and a robot. This framework encompasses well-known settings such as inverse reinforcement learning (IRL) or optimal teaching. The authors argue that it can be cast as a POMDP problem and then present in more details how apprentice learning can be formalized in their framework. This leads to the following insight: if the human knows that the robot uses IRL, she may be better off not demonstrating with an optimal policy, but with a “best response” to the robot’s strategy. The authors also provide an approximation scheme in the case where rewards are expressed as linear combinations of state features and when the human demonstrate only one trajectory. Finally, experimental results in the navigation problem support the previous insight.
           My opinion on this paper is quite mixed. On the one hand, the paper is generally clear. I like this general unifying framework and the insight it provides. On the other hand, the exposition could be made more precise and more rigorous. Some technical explanations and the proofs are a bit hand wavy. Besides, the theoretical results seem quite straightforward.                      Here are some more detailed comments:                      - Nash equilibrium is a concept developed in non-cooperative games. As the proposed framework is cooperative, the terminology from non-cooperative games may not be the best.                      - I find it strange to assume that both agents know the probability distribution over \theta. H doesn’t simply choose \theta? The model is a representation from the viewpoint of the robot and so maybe P_0 could simply be interpreted as its prior belief.                      - I think the authors should give the formal definition of the coordination POMDP to make things clearer. For instance, the explanations given in the paper and the supplementary material do not match: for instance, for the POMDP states (l.212, l.288 in paper and Def.2 in supplementary material).                      - In Sec.4.2, does the IRL approach also demonstrate only one trajectory? Assuming this is the case, could the experimental results be explained by the fact that only one trajectory is demonstrated?                      Some typos:           l.88: Should the histories be defined by S x (A^H x A^R x S)^*?           l.250: it gets           l.270: gives use the ability -> gives the ability?           l.272: although obvious, br is not defined           l.279: Remark 3 seems not to be really related to what precedes it.           l.287: a POMDP           l.321: Why is Figure placed on p.3?           l.326: The robot reward function -> The reward function           l.407: Kand -> K, and