Generalization of ERM in Stochastic Convex Optimization: The Dimension Strikes Back *
In stochastic convex optimization the goal is to minimize a convex function
where D is some unknown distribution and each f (·) in the support of D is convex over K. The optimization is commonly based on i.i.d. samples f 1 , f 2 , . . . , f n from D. A standard approach to such problems is empirical risk minimization (ERM) that optimizes
Here we consider the question of how many samples are necessary for ERM to succeed and the closely related question of uniform convergence of F S to F over K. We demonstrate that in the standard p / q setting of Lipschitz-bounded functions over a K of bounded radius, ERM requires sample size that scales linearly with the dimension d. This nearly matches standard upper bounds and improves on Ω(log d) dependence proved for 2 / 2 setting in [18]. In stark contrast, these problems can be solved using dimension-independent number of samples for 2 / 2 setting and log d dependence for 1 / ∞ setting using other approaches. We further show that our lower bound applies even if the functions in the support of D are smooth and efficiently computable and even if an 1 regularization term is added. Finally, we demonstrate that for a more general class of bounded-range (but not Lipschitz-bounded) stochastic convex programs an infinite gap appears already in dimension 2.

The paper considers the the question of when empirical risk minimization suffices to to actually find a true risk minimizer, a question which is fundamental to many statistical tasks. Rigorously, the setup is as follows: we wish to minimize E[f] where f is drawn from some distribution D, assuming the f's are "nice" in some sense (here they consider various settings, including convex, Lipschitz, etc). We get samples from f_1, ..., f_n from this distribution, and can form the empirical risk, which is the average of the sample f's. The question is whether or not minimizing this empirical risk will always get us a minimizer for the ground truth, or something close.  In this work the authors provide strong lower bounds for this problem, in a variety of settings. Their most basic result is that for convex, Lipschitz functions over the unit l2 ball, to ensure that the ERM is O(1) close to the true minimizer requires Omega(d) samples, where di is the dimension. This is a tight lower bound in terms of the dimension. To the best of my knowledge, the previous best lower bounds were surprisingly weak--only Omega (log d). They can also modify their construction to show that a similar lower bound also holds other lp norms, for smooth, convex, Lipschitz functions, and for l1 regularization. Finally, a further modification shows that ERM never works for range constrained stochastic convex optimization (without a Lipschitz constraint), although other techniques work for this setting.
I like this result quite a bit. In my opinion, the question they consider is quite important. The lower bounds seem to give the right tradeoffs for different lp norms (at least, in terms of dependence on d). The reductions are well presented and clear, and I don't see any issue with them.   The main drawback of their approach I see is that it does not easily generalize to also obtain tight bounds in terms of eps, the desired error. Moreover, often times (as they point out) algorithms can bypass the ERM entirely and get better sample complexities, so perhaps ERM is simply not the right quantity to study. However, since it is so pervasive in practice, I feel like ultimately understanding generalization of ERM is an important question and the authors make an important contribution to it, and for that reason I think it is worthy of inclusion in NIPS.


Tree-Structured Reinforcement Learning for Sequential Object Localization
Existing object proposal algorithms usually search for possible object regions over multiple locations and scales separately, which ignore the interdependency among different objects and deviate from the human perception procedure. To incorporate global interdependency between objects into object localization, we propose an effective Tree-structured Reinforcement Learning (Tree-RL) approach to sequentially search for objects by fully exploiting both the current observation and historical search paths. The Tree-RL approach learns multiple searching policies through maximizing the long-term reward that reflects localization accuracies over all the objects. Starting with taking the entire image as a proposal, the Tree-RL approach allows the agent to sequentially discover multiple objects via a tree-structured traversing scheme. Allowing multiple near-optimal policies, Tree-RL offers more diversity in search paths and is able to find multiple objects with a single feedforward pass. Therefore, Tree-RL can better cover different objects with various scales which is quite appealing in the context of object proposal. Experiments on PASCAL VOC 2007 and 2012 validate the effectiveness of the Tree-RL, which can achieve comparable recalls with current object proposal algorithms via much fewer candidate windows.

This paper proposes a method for generating object proposals by using a sequential search strategy that is trained with reinforcement learning (Q-learning specifically). The paper has two major contributions: one, at test time, instead of taking just one action at each step, it takes two actions (one best action each from two classes of actions) and bifurcates. This allows the model to actually predict multiple objects and deal with the associated multimodality better. Two, the reward signal is carefully designed and while not imposing any order, provides rewards both when the localization of an object improves and when the model latches on to a new object. The recall achieved by the method seems impressive, and when plugged into the fast r-cnn improves over RPN by about 2 points.
I liked the ideas present in the paper. This is the first sequential search strategy I have seen that tries to output all objects in an image, and does not impose any arbitrary and hard-to-justify order on the boxes during training. I have a minor clarification, and then some ways of strengthening the experimental section, which to me would be the difference between a poster and an oral.  A point of clarification: It seems that the ranking of the proposals output is simply the depth of the tree at which they are discovered. Why is this a good ranking? Is there anything in the training that encourages the model to discover objects quickly? I can think of some intuitive explanation but it would be good for the authors to provide an explanation in the paper.  Proposals are a fast moving subfield of computer vision and this paper would do well to add a few more experiments: 1) Comparison to more proposal methods. In particular, I would want comparisons to  MCG[1] and DeepMask[2], both of which are significantly better than, eg. edge boxes. MCG proposals are also available online 2) Comparison on COCO. COCO captures a much larger range of scale variations especially when it comes to small objects and methods like RPN typically do not fare very well. I would like to see how this fares. In addition, if the authors envisage this as a proposal method, then it should generalize to object categories that the network has not seen during training. It would be good to see if this is indeed true and if so, to what extent; COCO has a few categories that are not present either in PASCAL or Imagenet, so I would be curious how this method fares when trained on PASCAL and tested on COCO  It would also be nice to see what the detection AP looks like for higher overlap thresholds: this method seems to improve recall for higher IU so it might provide larger gains under more stringent evaluation metrics.


Error Analysis of Generalized Nyström Kernel Regression
Nyström method has been successfully used to improve the computational efficiency of kernel ridge regression (KRR). Recently, theoretical analysis of Nyström KRR, including generalization bound and convergence rate, has been established based on reproducing kernel Hilbert space (RKHS) associated with the symmetric positive semi-definite kernel. However, in real world applications, RKHS is not always optimal and kernel function is not necessary to be symmetric or positive semi-definite. In this paper, we consider the generalized Nyström kernel regression (GNKR) with 2 coefficient regularization, where the kernel just requires the continuity and boundedness. Error analysis is provided to characterize its generalization performance and the column norm sampling strategy is introduced to construct the refined hypothesis space. In particular, the fast learning rate with polynomial decay is reached for the GNKR. Experimental analysis demonstrates the satisfactory performance of GNKR with the column norm sampling.

The authors focus on the kernel ridge regression task with l_2-coefficient regularization [Eq.~(3)], on a compact domain of R^d. The primary goal is to analyze the performance of the Nystrom technique in this task, with 'Mercer kernels' (continuous bounded functions), which are not necessarily positive symmetric or positive definite. The main contribution is a generalization error bound on Nystrom based estimator (Theorem 1) which can give rise to fast rates (Theorem 2) if the true regression function satisfies an exponential approximation requirement (Definition 2) with a C^\infty kernel. The authors numerically illustrate that in the studied task the Nystrom method with column-norm sampling typically performs better compared to Nystrom technique relying on uniform sampling.  The paper is 1)novel: it bridges the gap between two popular directions, l_2-coefficient regularized objective functions and approximation with the Nystrom scheme. 2)well-organized, clearly written.   There are a few statements in the paper which should be proved and some bugs to be corrected. I detail these comments, and my minor points in the 'Qualitative assessment' section.
I use the 'lX' = 'line X' shorthand below. 'X->Y' will mean "Please change 'X' to 'Y'".  Major comments: ==============  A)Main part: ----------- 1)I found the 'Introduction' section redundant: the issues and the contributions are quite similar, this part can be significantly compressed. Throughout the paper the notion of 'Mercer kernel' is left a bit undefined, even in the 'exact' definition ('line 96'); currently, it has to be decoded from fragments of the abstract and the introduction. I also suggest adding  i)more emphasis on the motivation of general kernels (not necessarily symmetric and positive definite): What are the properties (a few illustrative examples can always help) which are not captured by the  classical reproducing kernels?, further potential impact, ... ii)Nystrom alternatives in the introduction such as the random Fourier feature technique [1-5] or the sketching method [6].  References: [1] A. Rahimi and B. Recht. Random features for large-scale kernel machines. In NIPS-2007, p. 1177–1184. [2] D.J. Sutherland and J. Schneider. On the Error of Random Fourier Features. In UAI-2015, p. 862-871. [3] B. Sriperumbudur, Z. Szabo. Optimal Rates for Random Fourier Features. In NIPS-2015, p. 1144-1152. [4] F. Bach. On the equivalence between quadrature rules and random features. Technical Report, 2015. (http://arxiv.org/abs/1502.06800) [5] A. Rudi, R. Camoriano, L. Rosasco. Generalization Properties of Learning with Random Features. Technical Report, 2016. (https://arxiv.org/abs/1602.04474) [6] Y. Yang, M. Pilanci and M. J. Wainwright. Randomized Sketches for Kernels: Fast and Optimal Non-Parametric Regression. Annals of Statistics (to appear; http://arxiv.org/abs/1501.06195).   2)There are some statements/conjectures in the paper without proof: i)l103-106: Solution of (1). Note: the cited [18] paper does not include 'inf_{\alpha}' in the regularization term. In fact, without this 'inf_{\alpha}' the regularization is not even a function of \alpha (unless the k(\cdot,x_i) points are assumed to be linearly independent). ii)l110-112: Solution of (2). iii)l130-132: (F,||\cdot ||_F) is a Banach space. iv)l203: Equivalent form of (9). Please, add proofs for these statements.  3)The computational complexity of the p=(p_i) vector in the column-norm sampling scheme seems to be O(n^2). This should also increase the complexity of (4) from O(mn+m^3), shouldn't it?   4)Section 3-4: In these sections there are several function definitions (f_\rho, f_z, \pi(\f_z), f_\lambda, g_\lambda, \hat{g}_\lambda). Based on the main text it is not obvious how these quantities will be connected, i.e. what will be the bridges. If there is place left [there are many displayed equations which can be made inline, or collected (see for example H, ||g||_H) to save space], it would be great to give a few-lines-long proof idea in the main part of the paper, 'the intuition of the bridges'.  5)Could the authors provide some concrete/illustrative examples, easy-to-check sufficient conditions for f_\rho in Def. 2? It is nice show that actually there are problems where this assumption holds.  6)The interpretation of Theorem 2 could be made more accessible by a quick discussion on the value of p and its impact, such as "p\in (0,2), m = n^{1/(2+p)} means that m can be chosen between n^{1/4} and n^{1/2}". Similarly noting that 'The smoothness of K is encoded in Def. 2.' can help understanding.  7)l195-206: This part should be improved. The displayed equation is hard to interpret; one could write instead "S(p_1,\ldots,p_n) = ..., task: \min_{p_1,\ldots,p_n}S(p_1,\ldots,p_n).". In line 203 "L" is undefined. "h_ii" is undefined; based on the supplement it should be "L_ii". "the minimizer of (9) can be approximated by": from the main text it is not clear in which sense this holds. A short sentence/discussion on the intuition of the "L_ii->0" assumption is also useful to add.  B)Supplement: ------------- 1)Unproved statements: l25: \phi'(0) = 0.  l26-27: "according to the arbitrariness of h, we get \lambda f_\lambda(t)=...". Note: I suppose that the authors wanted to construct some (h_n) \subset \L^2_{\rho_X} sequence and apply a limiting argument. l42-43: last inequality in (2). l102-103: order of "\sqrt{\frac{\log(2/\delta)}{m \lambda}}", see 2) below. Lemma5: 'R can be large enough', see 3) below. Please prove these gaps.  2)l102-103: Check the constants of the 3rd term on the r.h.s. [=D(\lambda) x (...)]. An "\sqrt{\frac{\log(2/\delta)}{m \lambda}}" type term is missing (it is coming from the E_2 bound = l37-38); if it is dominated by some other term, justify its reason. The same '\sqrt{...}' comment holds for Theorem 2.  3)l94-95: It looks like an 'for R is large enough' assumption has been implicitly used here.   Note: By assumption D(\lambda) \le c_\beta \lambda^{\beta}, where \beta (0,1]. R = \sqrt{D(\lambda)}/lambda \le \sqrt{c_\beta \lambda^{\beta}} / \lambda is just an exploding upper bound as \lambda ->0. Show that 'R is large enough' for sufficiently small \lambda, or assume it in Theorems 1-2.  Minor points:  ============  A)Main part: ----------- 1)l96-97: The boundedness of y is assumed; please make this assumption explicit -- otherwise w.l.o.g. one can not have y \in [-1,1]. In fact, it would be nice to leave the general kernel bound (\kappa) and the bound on y to make it explicit how these constants appear in the final performance guarantee (in a linear/quadratic/... form). 2)l123: This part of the sentence reads a bit strange. 3)l127-128: 'Let F be a square integrable space on X...' -> 'Let F be the square integrable space on X...'. Note: For a subset of L^2_{\rho_X}, the range of L_K has to be studied separately. 4)l130-132: in the definition of H: 'g(\cdot)' -> 'g', 'f(\cdot)' -> 'f'. 5)l130-132: in the definition of ||\cdot||_H: 'g(x)' -> 'g', 'f(x)' -> 'f', 'x \in X' -> ''. 6)l143, l165: 'the generalization error bound' -> 'our generalization error bound', 'we provide the main results' -> 'we provide our main results' [reading the text it is not really explicit that these are the authors' results] 7)After Definition 2: a quick note could be useful on the meaning of '\beta', such as 'larger \beta means easier task'. 8)line 215: in the K_E definition: '(...)' -> '\left(...\right)'. 9)Section 5.1: The authors could refer back when listing f_i-s, that these are the true f_\rho-s. 10)l241: 'Figure 1 shows the performance of the three sampling strategies' -> 'Figure 1 shows the performance of the two sampling strategies' [3->2]. Note: the subfigures in Fig. 1 could be arranged to a 3X3 format to save space. 11)References: [1,2,8,14]: missing page numbers. [3]: 'monte carlo' -> 'Monte Carlo'. [6]: 'reguularization' -> 'regularization', 'bouunds' -> 'bounds'. [20,25]: missing url-s.  B)Supplement: ------------ l3: 'provides the detail proof' -> 'provides the detailed proof' l12-13: the inequality seems to hold with equality as well l22: 'E_z(0) = 1' -> 'E_z(0) \le 1' since |y_i| \le 1 [i.e., this equality should be inequality]. l23: 'a auxiliary function' -> 'an auxiliary function' l24: 'h is an any fixed function' : at least h\in L^2_{\rho_X} seems to be needed to make the integral well-defined l57: "N_{2,u}(F,\epsilon) = N_{2,u}(...)": on the r.h.s. the '2,u' subscript should be deleted (to be compatible with Def. 1), l62: "Ef \le c (Ef)^\alpha" -> "Ef^2 \le c (Ef)^\alpha", i.e. f should be squared on the l.h.s. l77: r.h.s. of |g_1(z)-g_2(z)| should be '| |2y-\pi(f_1)-f_\rho(x)| |\pi(f_1)(x)-f_\rho(x)| - |2y-\pi(f_2)-f_\rho(x)| |\pi(f_2)(x)-f_\rho(x)| |'. l80: "true for and g" -> "true for any g" l81: please add the "\alpha = 1" choice. l87: 'It is easy to check that \hat{g}_{\lambda} \in G' -> 'By the definition of \hat{g}_{\lambda}, using v_i = \bar{x_i}, it follows that \hat{g}_{\lambda} \in G.' l98-99: 'E_{11}' -> 'E_1' l102-103: '\pi(f_{\rho})' -> 'f_{\rho}' [2x]; l106-109: similarly [3x]. In other words, the \pi clipping is superfluous on f_\rho. l112-113: I guess a \sum_i is missing on the r.h.s of the inequality. l113: 'inequality is follows from Hoder inequality' -> 'inequality follows from the Cauchy-Schwarz inequality' ['is': superfluous, 'Hoder' -> 'Holder' -> 'Cauchy-Schwarz' (it is sufficient)] l114: the inequality in Cauchy-Schwarz holds under equality of the arguments up to multipliers: \sqrt{1-L_{ii}} / \sqrt{p_i} ||K_i||_2 k^{-1} = \sqrt{p_i}; the conclusion is OK.


Sequential Neural Models with Stochastic Layers
How can we efficiently propagate uncertainty in a latent state representation with recurrent neural networks? This paper introduces stochastic recurrent neural networks which glue a deterministic recurrent neural network and a state space model together to form a stochastic and sequential neural generative model. The clear separation of deterministic and stochastic layers allows a structured variational inference network to track the factorization of the model's posterior distribution. By retaining both the nonlinear recursive structure of a recurrent neural network and averaging over the uncertainty in a latent path, like a state space model, we improve the state of the art results on the Blizzard and TIMIT speech modeling data sets by a large margin, while achieving comparable performances to competing methods on polyphonic music modeling.

The authors combine state space models with RNNs. The RNNs form a stochastic layer while the state space model creates a stochastic layer. Their model is trained with the tractable variational evidence lower bound (ELBO). The model is evaluated on several speech and music datasets and also compare it to similar models like STORN, VRNN and Deep Kalman Filters.
Updates after Rebuttal:  In the rebuttal the authors said (to clarify a concern of mine): "In the caption to Figure 1, we mentioned setting u_t=x_{t-1}, e.g. for language modelling. We used u_t=x_{t-1} for STORN and VRNN so that they could be compared more easily with the SRNN architecture in Figure 2. We will reiterate this in Figure’s 4 caption and elsewhere to keep it clear." I overlooked this information and it should indeed be mentioned in other captions and in the text. It is correct that when setting the action equal to the time-shifted observations all four models will have a more similar structure. However similar structure does not mean that the models are comparable (e.g. DKF still uses actions instead of observation as u_{1:T}). But this also means that there now is almost no difference between VRNN and the proposed method. The only difference is the missing connection from z to the deterministic hidden state. This does indeed "separate the noisy stochastic units from the deterministic ones", as the authors said, but this is not favorable when someone wants good generalization capabilities. After this clarification from the authors and the resulting changes in the model structure, I have to reduce the rating in the technical quality section and the novelty section.     Initial review: The generative model for STORN in Figure 4(a) is wrong. STORN has no input from any action u_t and the deterministic hidden state d_t needs input from the last generated observation x_{t-1}. VRNN in Figure 4(b) is also missing a connection from the last generative sample x_{t-1} to the hidden state d_t.  Technical quality: Experiments only evaluate negative log likelihoods and reconstruction quality. The generative model is never evaluated in itself, e.g., visualisations that show that the state transition is creating valid sequences even for multiple time-steps.  Novelty/originality: The algorithm is different from STORN and VRNN but does not provide any novel improvements upon them. The experiments show that other algorithms perform even better on most datasets.  Potential impact or usefulness: Impact is similar to STORN or VRNN since no clear improvements.  Clarity and presentation: Quite clear.


Neural Universal Discrete Denoiser
We present a new framework of applying deep neural networks (DNN) to devise a universal discrete denoiser. Unlike other approaches that utilize supervised learning for denoising, we do not require any additional training data. In such setting, while the ground-truth label, i.e., the clean data, is not available, we devise "pseudolabels" and a novel objective function such that DNN can be trained in a same way as supervised learning to become a discrete denoiser. We experimentally show that our resulting algorithm, dubbed as Neural DUDE, significantly outperforms the previous state-of-the-art in several applications with a systematic rule of choosing the hyperparameter, which is an attractive feature in practice.

This paper proposed a neural universal discrete denoiser. It is an extension of the universal discrete denoiser (DUDE) based on a deep neural network. The DNN can be trained using the so-called "pseudo-labels" under a novel objective function. Experimental results show some significant improvements over the conventional DUDE.
The core of the work is to use a DNN as an estimator of the empirical distribution of symbols given the context. The authors also designed a novel objective function that does not require labeled data. Not surprisingly, the DNN used in the presented Neural DUDE turns out to be a pretty good estimator with shared parameters. Significant improvements were observed.  This is an elegant piece of work, an interesting extension of the convention DUDE. I don't have problems with the Neural DUDE per se. But I have following concerns or comments.  1. Both DUDE and Neural DUDE will see a serious data sparsity problem when the alphabet becomes large. So far, a lot of application examples of DUDE are limited to very small alphabets, e.g. black-white images (|X|=2) and DNA sequencing (|X|=4). Neural DUDE uses a DNN with parameter sharing can mitigate it to some degree but I guess it will quickly run into problems when the context is long with a big alphabet. For example, the authors mentioned literature [4] and [22] in the introduction as related work using supervised training. But I wonder how the neural DUDE will perform when dealing with grayscale images as used in [4] and [22] rather than the black-white binary images used in this work? The alphabets are much bigger in [4] and [22]. So, although it is elegant in theory, it is hard to see it makes an impact to the real image denoising applications.  2. Another concern about the experiments is that the authors assume the channel is known. In other words, the channel transition matrix is known before hand. This is not a very realistic assumption in real world scenarios. Most cases, you don't know the channel and you have to estimate it. This is another crucial issue for the neural DUDE to make a real impact. How will the estimated channel transition matrix affect the performance? When the alphabet is big, the estimation of the matrix and its inversion itself may pose a problem.


CMA-ES with Optimal Covariance Update and Storage Complexity
The covariance matrix adaptation evolution strategy (CMA-ES) is arguably one of the most powerful real-valued derivative-free optimization algorithms, finding many applications in machine learning. The CMA-ES is a Monte Carlo method, sampling from a sequence of multi-variate Gaussian distributions. Given the function values at the sampled points, updating and storing the covariance matrix dominates the time and space complexity in each iteration of the algorithm. We propose a numerically stable quadratic-time covariance matrix update scheme with minimal memory requirements based on maintaining triangular Cholesky factors. This requires a modification of the cumulative step-size adaption (CSA) mechanism in the CMA-ES, in which we replace the inverse of the square root of the covariance matrix by the inverse of the triangular Cholesky factor. Because the triangular Cholesky factor changes smoothly with the matrix square root, this modification does not change the behavior of the CMA-ES in terms of required objective function evaluations as verified empirically. Thus, the described algorithm can and should replace the standard CMA-ES if updating and storing the covariance matrix matters.

The manuscript describes a novel CMA-ES update scheme, the Cholesky-CMA-ES, that approximates the original covariance matrix adaptation (with complexity O(d^3)) by a novel scheme that runs in O(\mu d^2) where d is the optimization problem dimension and \mu the number of samples per iterations (typically O(log(d)). The principle idea of the update is to maintain and update a lower-triangular Cholesky matrix rather than a full covariance matrix (and corresponding sqrt of that). The difference between the Cholesky and the original CMA-ES is due to the lack of estimating a (random) orthogonal matrix E_t in every step. Under quite restrictive assumptions (i.e., assumptions about convergence of a sequence of covariance matrices to an unknown target covariance matrix) the authors give an asymptotic argument that the sequence of E_t will converge to an identity matrix, and thus the two schemes become equivalent.  The validity and the general speed-up of the Cholesky-CMA-ES is illustrated by using standard runtime and convergence plots on selected objective functions and by comparing it to two state-of-the-art CMA-ES alternatives that delay the full covariance matrix update by different means.
This paper makes a solid contribution to speeding up the celebrated CMA-ES algorithm which is arguably the best algorithm for non-convex black-box optimization when little is known about the topology of the objective function and first or higher-order information is difficult to access or not helpful.   The proposed method speeds up the algorithm considerably and shows no deterioration of optimization performance on the synthetic test cases. Although this is of general importance, one weak point of ANY speed-up proposal for CMA-ES is that, in practice, the overall optimization process is dominated by the runtime of the black-box oracle (the objective function), and, often, the actual runtime of the CMA-ES is negligible compared to that. In addition, in typical scenarios where CMA-ES is used, the dimensionality of the problems does not go beyond a few tens of variables. Only in the case where a single function evaluation is on the order of milliseconds or smaller and the dimensionality is higher, the runtime of CMA-ES plays an important role, and this is targeted scenario here (which may be less realistic in practice).  As theoretical guarantees are notoriously difficult to get for an involved scheme such as CMA-ES, I would like to propose a couple of further empirical experiments and plots that may strengthen the presentation of the results. First, it may be instructive to see the evolution of E_t on selected optimization trajectories, e.g., in different norms. I envision an experiment where the full CMA-ES is used and the approximation is calculated simultaneously and compared. This would give further empirical evidence about the actual fluctuations of E_t over time. Second, it may be instructive to see actual runtimes and performance of CMA-ES without delay strategies in this contribution for completeness. Third, although the used test functions are standard in the evolutionary optimization community I would prefer, on quadratic functions, a simulation study where the spectrum of quadratic forms is varied continuously (see, e.g., Stich and Mueller, PPSN 2012), and compare the solution quality and error in E_t  in dependence of the spectral distribution.  Forth, it would be instructive to see a real-world experiment that accompanies the present simulation results. For instance, policy gradients in real-time robotics applications might be a good choice where speed indeed matters and the parameter space is not too large.


A Consistent Regularization Approach for Structured Prediction
We propose and analyze a regularization approach for structured prediction problems. We characterize a large class of loss functions that allows to naturally embed structured outputs in a linear space. We exploit this fact to design learning algorithms using a surrogate loss approach and regularization techniques. We prove universal consistency and finite sample bounds characterizing the generalization properties of the proposed method. Experimental results are provided to demonstrate the practical usefulness of the proposed approach.

The paper presents a framework for stuctured prediction that allows for a wide range of loss functions through a "loss trick" that allows the losses to be represented through embeddings in RKHS for the outputs. They present an elegant framework for optimising the models, essentially boiling down to solving a kernel ridge regression type problem and then computing a pre-image (decoding) to arrive at the prediction. The authors provide analysis of consistency and present a generalization bound for the framework. The method is experimentally compared to alternative approaches in different structured prediction tasks with good results
 In my view, this is a beautiful paper that will advance the field of structured prediction significantly and provides a platform for further development.   Nevertheless, the paper should be better related to existing work on vector-valued regression for structured output. A recent related work is but there are others: C ́eline Brouard, Florence D’Alch ́e-Buc, Marie Szafranski. Input Output Kernel Regression: Supervised and Semi-Supervised Structured Output Prediction with Operator-Valued Kernels. 2015.  The paper is generally well written, I have only few remarks: - line 70-72: you might note already here that this amounts to a ridge regression problem  in the output Hilbert space. - line 74: (7) is generally known as the pre-image (or decoding) problem. Good to mention it already here. - line 129: closing paranthesis missing: (\psi(y, - line 171: extra full stop  . [3]. - line 186: somehow convoluted grammar here: "of the same order of the generalization bounds...". Please rephrase. - line 211:  T => \ell in the dimension of the label space, I think. - line 212: i does not appear inside the argument of argmax or argmin - line 223-230: I found this subsection somehow cryptically written. Please write more to the point. What is the relationship of your results to SVMstruct. Can you cover the max-margin models such as SVMstruct and others (e.g. those in Bakir et al. 2007) as well or are those models outside your theory? - Figure 1: Please use only one of the terms, KRR or KRLS.


R-FCN: Object Detection via Region-based Fully Convolutional Networks
We present region-based, fully convolutional networks for accurate and efficient object detection. In contrast to previous region-based detectors such as Fast/Faster R-CNN [7,19] that apply a costly per-region subnetwork hundreds of times, our region-based detector is fully convolutional with almost all computation shared on the entire image. To achieve this goal, we propose position-sensitive score maps to address a dilemma between translation-invariance in image classification and translation-variance in object detection. Our method can thus naturally adopt fully convolutional image classifier backbones, such as the latest Residual Networks (ResNets) [10], for object detection. We show competitive results on the PASCAL VOC datasets (e.g., 83.6% mAP on the 2007 set) with the 101-layer ResNet. Meanwhile, our result is achieved at a test-time speed of 170ms per image, 2.5-20× faster than the Faster R-CNN counterpart. Code is made publicly available at: https://github.com/daijifeng001/r-fcn.

This paper is an important extension of recent seminal object detection work Faster R-CNN proposed by Ren et al [18]. The main contribution of the paper is to redesign Faster R-CNN into a fully convolutional variant R-FCN through introducing so called position-sensitive score maps. Benefiting from the property of fully convolutional architecture, the proposed method can not only bring about 2.7 & 2.5X speeding up in training & testing, but also can obtain on par performance with standard Faster R-CNN method. Extensive experimental results are also provided to show its effectiveness and to analyze various choices regarding the network design, training and testing.
Faster R-CNN [18] is one of the latest top-performing object detection methods on several popular benchmark datasets such as PASCAL VOC, Microsoft COCO and ImageNet. The main novelty of this paper is to redesign Faster R-CNN method into a fully convolutional variant R-FCN through introducing so called position-sensitive score maps. Note that the basic idea of position-sensitive score maps is extended from instance-sensitive score maps proposed by Dai et al. [3] recently. However by properly combining it with Faster R-CNN [18] and powerful ResNet [9], the proposed R-FCN method can not only bring about 2.7X & 2.5X speeding up in training & testing, but also can obtain on par performance with standard Faster R-CNN method. Extensive experiments are conducted for evaluation and analysis.  Overall, this paper is a good submission. But it can be further improved if the following concerns can be addressed.  - Obviously, the performance improvement of R-FCN is not just from position-sensitive score maps. The modification of ResNet with “hole algorithm” [2, 15] and the improved training with OHEM [22] also play important role. In Table 4, 5, 6 and two additional Tables (without captions) on page 8, it is important to clearly clarify that OHEM is used or not used in R-FCN. According to the results shown in Table 3, I think OHEM should be used for all results of R-FCN in above mentioned Tables. This is also important for fair comparisons with standard Faster R-CNN and Faster R-CNN +++ since they do not use OHEM. - As the authors describe that Faster R-CNN +++ further incorporates iterative BBR, global context and multi-scale testing, yet R-FCN incorporates “hole algorithm” and OHEM (and also multi-scale testing on Microsoft COCO dataset), how about object detection accuracy of R-FCN when using the completely same settings regarding its best results or all tricks, e.g., adding iterative BBR and global context. This is important to show the best potential of R-FCN. Will this provide best results on PASCAL VOC and Microsoft COCO datasets? - In line 250 and 251, the authors claim that R-FCN performs better on the objects of small sizes (in Microsoft COCO dataset) than standard Faster R-CNN. But according to Table 6, two methods show very similar performance, i.e., ~0.5% accuracy difference with respect to mAPs, does this mean that R-FCN performs less better/similar on the objects of middle sizes or large sizes? If yes, this is somewhat confusing because larger object instances usually activate more in feature maps compared with smaller object instances.  - Note that R-FCN still adopts 4-step (even 5-steps alternating training strategy on Microsoft COCO dataset) proposed in standard Faster R-CNN, it would be interesting if the authors can provide a single-run training scheme and make it more straightforward to train the detector, as like YOLO [17] and SSD [14]. - A minor typo should be corrected. According to Table 6, in line 252, “51.5%/29.2%”->”51.9%/29.9%”. - One related work (Gidaris and Komodakis, Object detection via a multi-region & semantic segmentation-aware cnn model, in ICCV 2015), in which the use of multi-region contextual information was first explored, should be cited. - The authors provide detailed info on training and testing, it will be better if the entire training time can be given for each benchmark dataset.


Exponential expressivity in deep neural networks through transient chaos
We combine Riemannian geometry with the mean field theory of high dimensional chaos to study the nature of signal propagation in generic, deep neural networks with random weights. Our results reveal an order-to-chaos expressivity phase transition, with networks in the chaotic phase computing nonlinear functions whose global curvature grows exponentially with depth but not width. We prove this generic class of deep random functions cannot be efficiently computed by any shallow network, going beyond prior work restricted to the analysis of single functions. Moreover, we formalize and quantitatively demonstrate the long conjectured idea that deep networks can disentangle highly curved manifolds in input space into flat manifolds in hidden space. Our theoretical analysis of the expressive power of deep networks broadly applies to arbitrary nonlinearities, and provides a quantitative underpinning for previously abstract notions about the geometry of deep functions.

The main results in this paper: 1) Deep neural networks are expressive. Specifically, an L-deep network will typically (with the right scaling of the weights) map any 1D curve in its input to a curve which has non-vanishing curvature and euclidian length exponentially increasing in L.  2) Shallow neural networks are not expressive. Specifically, for any given 1D curve in the input, a bound is derived on the elucidation length of the curve at the hidden layer.  Thus, deep networks are more expressive then shallow networks.
This is a very interesting work. However, I have a few major concerns:   1) I believe Theorem 1 is wrong, as can be seen from the counterexample at the bottom of this review. As can be observed from this counterexample, the main problem in the proof is the inaccurate sentence on lines 110-112 in the supplementary material. I'll wait to author's feedback before deciding if this a fatal flaw.  2) I believe the statement that h_i^l are uncorrelated random variables (line 69 in paper + line 22 in supplementary material) is wrong for l > 1 (i.e., second hidden layer, or deeper). In this case, h_i^l are all composed of different linear sums of the same random vector x^{l-1}, and are therefore dependent. I agree this dependency should vanish in the limit of large N and random W, but this should be clearly stated and quantified (to prevent potential issues when applying similar calculations for trained networks).  3) The authors should explain the novelty of their results in comparison to those in the recent ICML paper “Convolutional Rectifier Networks as Generalized Tensor Decompositions” by Cohen et al.  Minor comments:  4) Section 1 is very similar to section G in the supplementary material of “Exact solutions to the nonlinear dynamics of learning in deep linear neural networks” by Saxe et al. I feel this similarity should be stated and discussed. For example, Saxe et al. suggested that it is good to initialize near the transition to Chaos. It would be nice if the authors can clarify if this is related to the results in the current paper.  5) It would be nice if the authors could clarify how one should to use eq. (9) for non-differentiable activation functions (e.g., the popular ReLU).   6) Most of the paper is written very clearly, except Section 5, which feels a bit rushed. I think it should be improved or removed.  7) There are a few typos. For example, in the caption of fig. 3:  “5 and 10” -> “5,10 and 15” “ 6 “ -> “ (6)”  (in two places).  *** Counterexample *** x^0(t) = ( -2t , 2t+sin(t) ) which is a curve over some range 0 < t < 2*pi*M, where M is an positive integer (and we replaced \theta from the paper with t). Taking the derivative: d x^0(t) / dt = ( -2 , 2+cos(t) ) we see that s=0. Choosing N1=1,W=(1,1), b=0 and activation function  \phi(h) = min(1,max(-1,h)) (so R=2) we get  d x^1(t)  / dt  = d h^1(t)  / dt = cos(t) Therefore, the total Euclidian length is L^E=int _ 0 ^ {2*pi*M} |cos(t)| dt = 4*M , which is larger than the bound in Theorem 1:  N1*(1+s)*R = 1*1*2 = 2.   %% After Rebuttal %% 1) Correction of Theorem 1 looks good. I hoped this would be possible. 2) Yes, I was wrong here. Sorry for the confusion. 3) Sure, I understand your method is more true for more general activation functions. Still it would be very helpful to the reader if the authors place their results within the right context. For example, if you restrict yourself to the setting (ReLU+MaxPooling) in Cohen&Shashua 2016, can you get different/better results?  Thanks for this nice paper!


Generative Shape Models: Joint Text Recognition and Segmentation with Very Little Training Data
Abstract: We demonstrate that a generative model for object shapes can achieve state of the art results on challenging scene text recognition tasks, and with orders of magnitude fewer training images than required for competing discriminative methods. In addition to transcribing text from challenging images, our method performs fine-grained instance segmentation of characters. We show that our model is more robust to both affine transformations and non-affine deformations compared to previous approaches.

This aim of this paper is to recognize non-handwritten text in images. The model is generative, and inference proceeds using belief propagation on a factor graph for both finding characters in images and parsing words. Word parsing is based on a combination of bigram and trigram language models. Learning proceeds by learning structural models of characters for fonts, and then by greedily selecting fonts to be modelled. Experiments are performed on two font datasets, where the experimental results are incrementally better than OCR, though with a greatly reduced amount of training examples.
First, the experimental results seem fine, albeit a incremental. They slightly outperform competing methods. Although it's true the authors achieve good results with less training data than competing methods, their method is also expensive and it doesn't seem easy to make use of more training data anyway.  Next, this does not seem to be an actual generative model, but a probabilistic model. Because of how the character recognition is setup (a loopy factor graph), and how parsing is performed, it doesn't seem like one could actually generate text with this model. In other words, sampling would seems prohibitively expensive. It would be more fair to say this is a probabilistic model, but not a generative one for which sampled texts could be drawn. The authors state that the framework can generate segmentation boundaries (line 436); can it really? How do you sample from the factor graphs? There are these big normalizing constants in there, and the model is not a DAG, so sampling doesn't seem easy. How are boundary segmentations generated? It would be helpful to see samples from the model.  The inference step seems a bit odd. The authors build a minimum-spanning-tree over the edges of the factor graph. How is this done? Factor graphs don't have weights, so how is this spanning tree being created? I found this unclear. How necessary is the MST? How many edges does the loopy graph have, typically? Are the potentials constructed in such a way that message passing is linear in clique size, or is it more expensive? These details are missing in the paper.  Also, the authors state that they have "long constraints" in their factor graph for inferring characters, to help model the structure of characters. These constraints of a parameter (\gamma), and it's unclear how this value was set, and whether its value is important. Is the model/inference sensitive to the choice of parameter value?  Another modelling problem is the need to model fonts individually. Why is this necessary? Could there not be a richer model of what characters in general look like? What happens when one tries to perform inference on a font that the system hasn't seen (and hasn't seen anything similar)? Would the system be able to state that it's highly uncertain about the parse it produces for this unseen font?  Lastly, there is a factor graph for detecting characters which produces marginals, and another factor graph for word parsing. Why not combine the two into one factor graph to do joint character detection and parsing? Why separate the two stages?  Overall, the inference and learning seem ad-hoc, and a lot of details are missing. Inference in a large factor graph is challenging, but from the setup of the paper, there is some machinery in the framework that is complicated and seems unnecessary (eg, dividing the factor graph into two stages). Also, the modelling of characters seems brittle and that in-plane word rotations would cause the system to fail; although rotations could be built into the system in theory, it seems impractical here. The experimental results are incremental. Lastly, the margins are way too small, so there is a lot more content here than would normally be allowed.


Hierarchical Clustering via Spreading Metrics
We study the cost function for hierarchical clusterings introduced by [16] where hierarchies are treated as first-class objects rather than deriving their cost from projections into flat clusters. It was also shown in [16] that a top-down algorithm returns a hierarchical clustering of cost at most O (α n log n) times the cost of the optimal hierarchical clustering, where α n is the approximation ratio of the Sparsest Cut subroutine used. Thus using the best known approximation algorithm for Sparsest Cut due to Arora-Rao-Vazirani, the top-down algorithm returns a hierarchical clustering of cost at most O log 3/2 n times the cost of the optimal solution. We improve this by giving an O(log n)-approximation algorithm for this problem. Our main technical ingredients are a combinatorial characterization of ultrametrics induced by this cost function, deriving an Integer Linear Programming (ILP) formulation for this family of ultrametrics, and showing how to iteratively round an LP relaxation of this formulation by using the idea of sphere growing which has been extensively used in the context of graph partitioning. We also prove that our algorithm returns an O(log n)-approximate hierarchical clustering for a generalization of this cost function also studied in [16]. We also give constant factor inapproximability results for this problem.

I enjoyed reading this paper which shows that the hierarchical clustering problem [6] can be approximated within a multiplicative O(log n) factor. This improves the O(\alpha log n) approximation of [6] where \alpha is an approximation factor of a subroutine used to solve the “Sparsest Cut” problem. In order to avoid losing the approximation factor of the “Sparsest Cut” problem the authors develop a new approach that is based on a careful LP rounding algorithm. In a nutshell, an ILP formulation is used to give a characterization of ultrametrics associated with the problem. The key observation here is that the ultrametrics associated with the hierarchical clustering problem have to satisfy a non-triviality condition (Def 1 + Lem 5). This condition is then captured by the ILP together with the ultrametric condition (for the latter the techniques of [8] are used). Then an LP-relaxation based algorithm with iterative rounding is given that achieves the desired approximation using Leighton-Rao “sphere growing” technique. It is quite remarkable that a fairly complex LP-formulation used in this work does in fact admit such a rounding approach.   I also very much enjoyed the overall comprehensiveness of the approach taken in this paper as it includes experimental evaluation on datasets from the UCI database + negative results that show superpolynomial lower bounds on the LP/SDP relaxations of the problem.
I enjoyed reading this paper which shows that the hierarchical clustering problem [6] can be approximated within a multiplicative O(log n) factor. This improves the O(\alpha log n) approximation of [6] where \alpha is an approximation factor of a subroutine used to solve the “Sparsest Cut” problem. In order to avoid losing the approximation factor of the “Sparsest Cut” problem the authors develop a new approach that is based on a careful LP rounding algorithm. In a nutshell, an ILP formulation is used to give a characterization of ultrametrics associated with the problem. The key observation here is that the ultrametrics associated with the hierarchical clustering problem have to satisfy a non-triviality condition (Def 1 + Lem 5). This condition is then captured by the ILP together with the ultrametric condition (for the latter the techniques of [8] are used). Then an LP-relaxation based algorithm with iterative rounding is given that achieves the desired approximation using Leighton-Rao “sphere growing” technique. It is quite remarkable that a fairly complex LP-formulation used in this work does in fact admit such a rounding approach.   I also very much enjoyed the overall comprehensiveness of the approach taken in this paper as it includes experimental evaluation on datasets from the UCI database + negative results that show superpolynomial lower bounds on the LP/SDP relaxations of the problem.


Poisson-Gamma Dynamical Systems
We introduce a new dynamical system for sequentially observed multivariate count data. This model is based on the gamma-Poisson construction-a natural choice for count data-and relies on a novel Bayesian nonparametric prior that ties and shrinks the model parameters, thus avoiding overfitting. We present an efficient MCMC inference algorithm that advances recent work on augmentation schemes for inference in negative binomial models. Finally, we demonstrate the model's inductive bias using a variety of real-world data sets, showing that it exhibits superior predictive performance over other models and infers highly interpretable latent structure.

This paper considers modelling sequential count data, which arises in many real world applications but results in challenges in its analysis due to its high dimensionality, overdispersion and potentially non-linear dynamics governing its structure over time. The authors develop a novel Gamme-Poisson dynamical system (PGDS), assuming a latent sequence of Gamma variables and Poisson observations. They impose shrinkage priors that allow for automatic tuning of the latent state dimensionality and extend the model to a non-parametric approach, defining two novel stochastic processes. For inference in this model, the authors develop an MCMC algorithm with efficient sampling updates, making use of two equivalent generative processes of the model and auxiliary variables. The authors perform a series of experiments on different datasets, comparing the PGDS to the classic Gaussian linear dynamical system and the Gamma process dynamic Poisson Factor Analysis, showing that the  PGDS outperforms the other two models in a number of applications, in particular in cases with overdispersed data. The authors further illustrate the interpretability of the learned model parameters in regards to real world applications.
Technical quality:  This is a rigorous and well-written paper with a lot of technical detail. The details included in the main paper are sufficient to develop a general understanding of the methods used, with the supplementary material offering proofs and more details on the sampling scheme. The experiments show the model's applicability to a variety of real world datasets, which ties in nicely with the way the method was motivated.  Minor details: the introduction mentions an Extended Kalman filter as a non-Gaussian extension to the LDS. I think this generally refers to non-linear extensions of the LDS, rather than non-Gaussian ones. There is also a larger body of work in the neuroscience literature that has been concerned with extensions of the LDS to Poisson observations (PLDS), combining the expressive power of Gaussian latent dynamics with Poisson observations. This could be included in the literature review and might be a nice contrast to the approach presented in this paper.   Novelty/originality:  The paper makes notable novel contributions by introducing the PGLD together with novel stochastic processes.  Potential impact or usefulness:   Novel approaches to modelling sequential count data should be applicable to a variety of real world processes, such as social networks or Neuroscience.   Clarity and presentation:  The paper is well-written and coherent. At points there are small errors (e.g. line 76) or typos which would require corrections, but these are very minor.


Doubly Convolutional Neural Networks
Building large models with parameter sharing accounts for most of the success of deep convolutional neural networks (CNNs). In this paper, we propose doubly convolutional neural networks (DCNNs), which significantly improve the performance of CNNs by further exploring this idea. In stead of allocating a set of convolutional filters that are independently learned, a DCNN maintains groups of filters where filters within each group are translated versions of each other. Practically, a DCNN can be easily implemented by a two-step convolution procedure, which is supported by most modern deep learning libraries. We perform extensive experiments on three image classification benchmarks: CIFAR-10, CIFAR-100 and ImageNet, and show that DCNNs consistently outperform other competing architectures. We have also verified that replacing a convolutional layer with a doubly convolutional layer at any depth of a CNN can improve its performance. Moreover, various design choices of DCNNs are demonstrated, which shows that DCNN can serve the dual purpose of building more accurate models and/or reducing the memory footprint without sacrificing the accuracy.

The paper proposes to increase parameter sharing in convolutional layers by making all filters part of larger "metafilters". Filters are obtained by convolutionally extracting overlapping parts from the metafilters, hence the name "doubly convolutional neural networks". Models with this form of parameter sharing are compared with regular CNNs and some other variants on the CIFAR-10, CIFAR-100 and ImageNet datasets.
Added after rebuttal / discussion:  Although I still contend that the CIFAR-10 baseline network used in the paper is unnecessarily weak, I appreciate that results on a large-scale dataset such as ImageNet (on which the method is shown to be quite competitive) are much more relevant, and I no longer consider it a major issue.  I still disagree with the presentation of the idea (asserting that the filters are "translated versions of each other", when the reason the method works is precisely because they are _not_ exact translated versions of each other, only approximately so), but I guess that can be put down to a matter of taste.  That leaves the problem with the CyclicCNN baseline, which I maintain is unnecessarily crippled by removing its ability to use relative orientation information. My issue was not with the fact that this is not explained in enough detail (as the rebuttal seems to imply), but rather that the model is wrong. This form of parameter sharing is pointless, except in very rare cases where relative orientation information is not relevant to the task at hand (I can't think of any situations where this is true, but there might be some). In natural image classification problems like the ones tackled here, I think it is clear that relative orientations between low-level features encode crucial information that should not be discarded.  I really feel that this should be addressed, either by fixing the model architecture of the CyclicCNN so it actually benefits from the parameter sharing, rather than being crippled by it (i.e. not pooling after every layer), or by simply removing this model from the paper altogether. As it stands it doesn't really add anything because there are other, more interesting baselines to compare to, and it unnecessarily weakens the paper.  The paper probably should not be held back by an issue with an additional baseline. In light of all this I have decided to increase the technical quality score to 3, in the hope that the CyclicCNN issue will be addressed.   ---------   I'm not sure if the averaged maximum 1-translated correlation shown across different layers in Figure 2 is all that meaningful. A stroke detecting filter such as the one in the top-left corner of Figure 1, and its horizontal flip, will presumably also have a relatively high correlation compared to random filters. It would be useful to compare with something like this as well, because it would be more meaningful than the comparison with random filters (arguably a filter and its horizontal flip are not translated versions of each other, so if they also have high correlation it means the measure is not particularly meaningful).  The paper states that "it is interesting to see for most filters, there exist several filters that are roughly translated versions of each other". I agree that this is interesting, surprising even, considering that convolution involves exactly filter translation. I would have expected a bit more in-depth discussion of why convnets tend to learn approximately translated filters despite this. My intuition after reading the paper is that the value of the proposed approach seems to be in masking larger filters in various ways. Clearly these are _not_ translated versions of each other (only approximately so), else the resulting feature maps would be redundant (the would also be shifted versions of each other). So the fact that they are only _approximately_ translated versions of each other seems to be key, and this is not really stressed in the paper.  Although the experimental section is quite extensive, which is commendable, there are a few issues with the evaluation:  - The baseline models for CIFAR-10 and CIFAR-100 have the same number of units in all convolutional layers. This is non-standard and it constitutes an inefficient use of the parameter budget. It means that the total size of the representation (number of feature maps * height * width) shrinks considerably throughout the network. Usually, later layers are given more feature maps to compensate for smaller feature map sizes. This is reflected in the fairly poor performance of the baseline CNN (10.02% without / 9.59% with data augmentation), which reduces the impact of the results.  - The CyclicCNN model differs significantly from the models described in "Exploiting Cyclic Symmetry in CNNs" by Dieleman et al., so its name is a bit misleading. The CyclicCNN has rotation invariant layers (because after each layer, the activations are pooled across different orientations), whereas in the original paper, the entire network is invariant but the individual layers are instead equivariant (local vs. global invariance). This is an important difference because it means that the CyclicCNN is unable to learn high-level features that depend on relative orientations of low-level features. This cripples the network compared to the baseline, so it is not surprising that it performs worse. The CyclicCNN is actually more similar to the models described in "Flip-rotate-pooling Convolution and Split Dropout on CNNs for Image Classification" by Wu et al. than to any of the models described in the cyclic symmetry paper (although their framework could arguably be used to construct this type of model).  - At the end of Section 4.2.1 it is stated that, to ensure a fair comparison, the number of layers and the number of units per layer is kept constant across models. However, this does not ensure a fair comparison at all: instead, the number of parameters should be kept (approximately) constant. In Table 2, the DCNN is shown to outperform all other models, but it has almost twice as many parameters so this comparison is not fair. Instead, the number of feature maps in each layer of the DCNN should be reduced so the number of parameters becomes comparable.    Minor comments:  - A nonstandard symbol (circle with middle dot) is used for the convolution operation. This symbol is usually used for elementwise multiplication instead, so it's a bit confusing.  - Regarding MaxoutDCNN, it would be interesting to also evaluate other pooling function (mean pooling in particular, possibly after the application of a nonlinearity).  - The related work section should also discuss "Learning Invariant Features through Topographic Filter Maps" by Kavukcuoglu et al., which details another approach for grouping similar filters together (through a group sparsity penalty rather than explicit weight tying).  - Regarding [11] it is said that "this explores the scale invariant property of images", but I don't see how this is the case unless the weights across different layers are tied.  - [15] is said to "investigate the combination of more than one transformation of filters", but in fact STN transform the input, not the filters.  - In section 4.2.1, for MaxoutCNN it is said that the stride is k. Is this a stride or a window size? Or are both equal?


Designing smoothing functions for improved worst-case competitive ratio in online optimization
Online optimization covers problems such as online resource allocation, online bipartite matching, adwords (a central problem in e-commerce and advertising), and adwords with separable concave returns. We analyze the worst case competitive ratio of two primal-dual algorithms for a class of online convex (conic) optimization problems that contains the previous examples as special cases defined on the positive orthant. We derive a sufficient condition on the objective function that guarantees a constant worst case competitive ratio (greater than or equal to 1 2 ) for monotone objective functions. We provide new examples of online problems on the positive orthant that satisfy the sufficient condition. We show how smoothing can improve the competitive ratio of these algorithms, and in particular for separable functions, we show that the optimal smoothing can be derived by solving a convex optimization problem. This result allows us to directly optimize the competitive ratio bound over a class of smoothing functions, and hence design effective smoothing customized for a given cost function.

In this paper, the authors study a family of methods for online optimization in the (for lack of a better general descriptive term--if the authors know one, that would be great) generalized concave adwords problem. I am not as familiar with the area as I ought to be, so my review may be somewhat that of a neophyte.  The general problem is to understand the difference between the online solution of the problem  max. \psi(\sum_t A_t x_t)   s.t. x_t \in F_t  and its offline solution, where we measure quality by competitive ratio. The authors give a fairly clean analysis of the competitive ratio, based on convex analysis and the FTRL family of algorithms, and provide an extension to show how smoothing the function \psi can yield improved behavior.
I am pro-acceptance for this paper, though I have (of course) criticisms.  Below are a number of comments I have about the paper, not particularly organized, but hopefully cogent enough.  The approach via convex analysis/online regret/duality is cleaner than the literature I have seen on this problem, which focuses on the "continuous update" idea of Devanur and Jain ([DJ] from here)--frankly, a somewhat obtuse and confusing idea--so that using duality more carefully is nice.  The main assumption on the superdifferentials relative to the cone inclusion, Assumption 1, certainly is important for the analysis, but I had a hard time understanding how much more generality it allowed over the standard separable case in [DJ]. The PSD cone is a nice example that makes clear that the generality is helpful, though I wonder if other natural examples exist. That would be quite helpful, as the notation in the paper is somewhat heavy, and I found constructing examples on the fly for intuition more challenging than I usually do.  Because the algorithm in the simultaneous case is the same (I believe?) as the "continuous update" of [DJ], I thought that when specialized to linear problems one should attain a competitive ratio of 1 - 1/e rather than 1/2. Am I missing something? Is this a weakness in the analysis? Is there actually a gap? The smoothing appears to achieve 1 - 1/e, so some similar algorithm achieves the correct competitive ratio... I found this a bit unclear.  A somewhat problematic issue with the paper is that the interaction of the smoothing operation with Assumption 1---the cone ordering condition on the superdifferential of \psi---is quite unclear. It appears that separable functions maintain good behavior on the positive orthant, which falls back into the [DJ] analysis. But the authors do not mention non-separable functions in Section 3, which I understood to be one of the major points of the current paper. Why the lack of discussion? To be clear, also, the "optimal" smoothing is chosen by optimization of a *bound* on the competitive ratio, not the ratio itself.  A second issue with the paper is the lack of compelling examples. The plots of smoothed functions are fine, but I would really like to see an example of behavior of these operations on a real problem. Just because one has improved a bound does not always mean one has improved the algorithm.  Frankly, I wish there were a "resubmission" period for these reviews, because this paper could use (1) a bit more space for clarity and (2) a few better experiments so that we could see the real effects of smoothing. I think there are nice ideas, but with a bit of touching up, they could become very nice.   Minor comments:  The authors never defined competitive ratio; it might be useful to have early on.  I found the notation a tiny bit confusing, as I had to swap all standard notation from convex analysis to the concave case (e.g. convex vs. concave conjugate, etc.). I am not sure if this is avoidable, though the NIPS audience is more generally familiar with the convex case and it might be clearer to form the primal problem as minimization. Not a deal-breaker or anything.  Pg. 4, line 112: calling D_seq the dual value was a little strange to me, as the sequence $\hat{y}_t$ does not obviously yield a dual feasible point, though as Theorem 1 shows D_sim and D_seq are actually tighter than D^*, the optimal dual value. It might be worth noting this earlier.  Pg. 8, lines 284-end. A relevant reference for understanding regret vs. competitive ratio, at least in a restricted set of problems, might be Andrew et al.'s "Tale of Two Metrics" (http://users.cms.caltech.edu/~katrina/papers/twometrics.pdf).


Coordinate-wise Power Method
In this paper, we propose a coordinate-wise version of the power method from an optimization viewpoint. The vanilla power method simultaneously updates all the coordinates of the iterate, which is essential for its convergence analysis. However, different coordinates converge to the optimal value at different speeds. Our proposed algorithm, which we call coordinate-wise power method, is able to select and update the most important k coordinates in O(kn) time at each iteration, where n is the dimension of the matrix and k  n is the size of the active set. Inspired by the "greedy" nature of our method, we further propose a greedy coordinate descent algorithm applied on a non-convex objective function specialized for symmetric matrices. We provide convergence analyses for both methods. Experimental results on both synthetic and real data show that our methods achieve up to 23 times speedup over the basic power method. Meanwhile, due to their coordinate-wise nature, our methods are very suitable for the important case when data cannot fit into memory. Finally, we introduce how the coordinatewise mechanism could be applied to other iterative methods that are used in machine learning.

A coordinate descent method of the power method is proposed with convergence analysis. An even faster algorithm based on greedy coordinate descent is proposed. Empirical results suggest that the proposed method is fast in practice compared to the baseline or vanilla power method. The authors show that the algorithms proposed have a local linear convergence.
Overall I enjoyed reading the paper, it is well written and flows well with the intuition for the algorithms clearly written in the main paper which is helpful. However I am not sure how the comparison is fair: the convergence analysis only applies to the problem of computing eigenvectors. If this is the case, it is important to compare with recent algorithms that compute the same, comparing just with the vanilla power method does not say much about usefulness of the proposed algorithms, many state-of-the-art algorithms are often randomized/parallelized see for example [1], [2], [3] (I am not familiar with recent papers in SIAM, Matrix analysis, Numerical Linear Algebra but these seem like good baselines to compare with). A discussion of how the proposed algorithms can or cannot be parallelized WITH convergence would greatly enhance the utility.   The authors claim that their method is useful when the entire matrix cannot fit in the memory but is not made concrete: usually in this scenario, communication costs become significant see [4].  Finally, local convergence is already known, see [5] although showing that the function is strongly convex is still a valid contribution.  1. A Parallel Eigensolver for Dense Symmetric Matrices based on Multiple Relatively Robust Representations (2005) by P. Bientinesi, I. S. Dhillon, and R. A. van de Geijn;   2. Finding Structure With Randomness: Probabilistic Algorithms For Constructing Approximate Matrix Decompositions (2010) by N. Halko, P. G. Martinsson AND J. A. Tropp.    3. An Algorithm for the Principal Component Analysis of large Data Sets (2011) by N. Halko, P.G. Martinsson, Y. Shkolnisky, M. Tygert.  4. Communication-Avoiding Krylov Subspace Methods in Theory and Practice (2015) PhD Thesis by  Erin Carson, URL = {http://www.eecs.berkeley.edu/Pubs/TechRpts/2015/EECS-2015-179.html}  5. Coordinate Descent Converges Faster with the Gauss-Southwell Rule Than Random Selection (ICML 2015) by Julie Nutini, Mark Schmidt, Issam Laradji, Michael Friedlander, Hoyt Koepke


Tractable Operations for Arithmetic Circuits of Probabilistic Models
We consider tractable representations of probability distributions and the polytime operations they support. In particular, we consider a recently proposed arithmetic circuit representation, the Probabilistic Sentential Decision Diagram (PSDD). We show that PSDDs support a polytime multiplication operator, while they do not support a polytime operator for summing-out variables. A polytime multiplication operator makes PSDDs suitable for a broader class of applications compared to classes of arithmetic circuits that do not support multiplication. As one example, we show that PSDD multiplication leads to a very simple but effective compilation algorithm for probabilistic graphical models: represent each model factor as a PSDD, and then multiply them.

The authors describe algorithms for operations on arithmetic circuits suitable for representing probability distributions. Such circuits can be used as a sparse and efficient representation for probability distributions, therefore the manuscript has high potential impact for practical applications in probabilistic reasoning and fits NIPS.  Concretely the authors work on PSDDs (probabilistic sentential decision diagrams) and describe theoretical results on multiplying such representations, compiling PSDDs into arithmetic circuits, multiplying two PSDDs, summing out probabilities, several representation results and an empirical comparison with other approaches.
The manuscript advances the theoretical framework of PSDDs and contains contributions that can make applications on probabilistic representations more efficient in the future.  The only drawback of the manuscript is, that it is not self- contained and requires background reading to intuitively understand, because the example (figures) are not explained in detailed and because some preliminaries are not given.  Nevertheless I recommend acceptance for publication.  Detailed drawbacks of examples: * Figure 1 shows a circuit for a network A -> B   and it contains numbers attached to nodes,   however it is not clear what these numbers are,   and how they are related to A -> B. Also the text   does not explain this.   From my interpretation, this is just an arbitrary   example situation, where \theta_a = 0.3, ...   Curiously, \lambda_b = 1 and \lambda_\not{b} = 1   although these should be inverse if they are indicators.    It does not help that Figure 1 and Figure 2 seem to   describe two complete distinct examples (Fig 2 has 3 variables,   Fig 1 has only two). * Figure 2 shows several numbers (1, 1, 3, 4) that are   not intuitive. In (b) and (c) they are attached to the    topologically same or-gates, however in (d) in addition   there is "0" and "2", and only one "1". I recommend to revise the presentation of these examples, putting the figures later (to the text that describes it) and perhaps putting the figures on separate pages because they seem to be unrelated.  (I wonder if the and/or gate way of drawing PSDDs is a good choice, as drawing them just as graphs with edge labels would be more close to the original ACs, however this seems to be a choice of the authors of previous papers.)  Minor comments: * 18: compilating -> compiling * Figure 4: top-aligning the three trees could be nicer * Table 1: winning (shortest) times could be marked in bold * Table 1: what is the unit of size? not MB as it seems. nodes?


A Unified Framework for Extensive-Form Game Abstraction with Bounds
Abstraction has long been a key component in the practical solving of large-scale extensive-form games. Despite this, abstraction remains poorly understood. There have been some recent theoretical results but they have been confined to specific assumptions on abstraction structure and are specific to various disjoint types of abstraction, and specific solution concepts, for example, exact Nash equilibria or strategies with bounded immediate regret. In this paper we present a unified framework for analyzing abstractions that can express all types of abstractions and solution concepts used in prior papers with performance guarantees-while maintaining comparable bounds on abstraction quality. Moreover, our framework gives an exact decomposition of abstraction error in a much broader class of games, albeit only in an ex-post sense, as our results depend on the specific strategy chosen. Nonetheless, we use this ex-post decomposition along with slightly weaker assumptions than prior work to derive generalizations of prior bounds on abstraction quality. We also show, via counterexample, that such assumptions are necessary for some games. Finally, we prove the first bounds for how ✏-Nash equilibria computed in abstractions perform in the original game. This is important because often one cannot afford to compute an exact Nash equilibrium in the abstraction. All our results apply to general-sum n-player games.

SUMMARY The paper presents a bound on the quality of the strategy for extensive-form games computed using abstraction. This bound is general enough to generalize previous works and extend beyond it. The paper is purely theoretical and does not include any experiments.  QUALITY The proofs presented in the supplementary materials seem to be correct. However, they take a lot of effort to verify, because of     i) the technical nature of the problem,     ii) the choice of notation (see Clarity),     iii) the fact that a lot of work is left to the reader (proofs of Lemma 1 and Proposition 1),                                    iv) using mathematical objects which were not properly introduced (see Q1 and Q2).      CLARITY   * The paper is clearly organized and well written. The only unclear part is in Section 3, and the problem there is minor (see C1).   * The paper is very technical, which is unavoidable. However, the corresponding notation is sometimes unnecessarily complicated, and the logic behind it is inconsistent (see the C) comments). If the paper aims to present a unified framework for EFG abstraction bounds, I consider this a major drawback.  ORIGINALITY   * The main result of the paper is an exact formula describing the error caused by working with an abstracted version of an extensive form game. This yields a unified description of the error from which two different prior results can be derived. However, the presented formula can also be applied to problems which none of the prior results was able to address (Section 7).   * The authors claim that their results produce tighter bounds than the prior results. This is definitely true formally, but I do not know whether it makes a useful difference in practice. While this wouldn't be the main selling point of the paper, it would be nice to illustrate the bounds on a specific example.   SIGNIFICANCE  The paper addresses an important problem: abstractions are a technique which is very useful (and very often used) in practice. The paper makes a major contribution to this problem by presenting a widely applicable bound on the error caused by abstraction.   DETAILED COMMENTS OF SERIOUS IMPORTANCE:   Q1) The explanation of the notation around pure strategies at the beginning of Section 6 is unclear and confusing. I do not understand what is meant by \pi^{\cdot}(I|\vec a) in the RHS of the bound in the proposition. As a result, I am unable check the correctness of Proposition 1. Q2) A symbol \delta_{I'_I} suddenly appears on line 208 with no explanation or definition (and is used more times after that). It should be explicitly defined when introducing abstractions. Q3) It is not obvious how the conditional reach probabilities on line 101 are defined. Since this is central to the paper, this should be noted explicitly. How are they defined when the reach probability is zero? Q4) The definition of the counterfactual value on line 158 seems to be wrong --  shouldn't there be one term which only depends on \sigma_{-i} and another which depends on the whole \sigma? Also cf. (5) from Zinkevich et al. 2007, cited at line 155. (This seems to be a typo rather than an error.)    C1) On line 121, an abstraction is explained as a two-stage process, and a high-level description of the first step is given. This step is then described in more detail -- however this description spans over 20 lines. It would be better to first give a high-level description of both steps, and only then describe these steps in more detail (or improve the clarity some other way).  C2) The paper deals with an 'original' game \Gamma = (H,Z,A,P,\pi_o,{\mathcal I_i},{u_i}), it's perfect-recall abstraction (H',Z',A',P',\pi'_o,\mathcal P',{u'_i}), and a general abstraction \Gamma' = (H',Z',A',P',\pi'_o,{\mathcal I'_i},{u'_i}). I have several issues related to this choice:     i) For an information set I in the original game, the authors for example the object Z_I (as derived from Z and I). If I' is an information set in the abstraction, they denote the analogous object as Z_{I'}. This seems wrong, as this object is derived from Z' and I', and should therefore be denoted as Z'_{I'}. This issue concerns most of the symbols used throughout the paper and I believe the suggested change would make the technical statement much easier to process.     ii) The symbols P' (for the abstract player function) and \mathcal P' (for perf.recall inf. sets) look too similar. Consider changing one of them.     iii) Most of the objects underlying the perfect-recall abstraction and the general abstraction are identical. However, the reader needs to be aware at all times whether he is currently working with an object from the p.r. abstraction or the general abstraction, as different methods apply in these two contexts. I propose using different symbols in each context but mentioning that some of them in fact correspond to the same objects. For example:  "\Gamma = (H,Z,A,P,\pi_o,{\mathcal I_i},{u_i}) for the original game,   \hat \Gamma = (\hat H,\hat Z,\hat A,\hat P,\hat \pi_o, {\hat \mathcal I_i},\hat {u_i}) for the perfect recall abstraction,  \Gamma' = (H',Z',A',P',\pi'_o,{\mathcal I'_i},{u'_i}) for the general abstraction,  where with the exception of \hat \mathcal I_i and \mathcal I', all of the corresponding \hat \cdot and (\cdot)' objects are identical."     Other solutions are possible, but I believe that some change is necessary.     iv) The authors consistently use u_i instead of u'_i for the abstract utility function. Either fix this or mention explicitly.  C3) When using two objects of the same type (e.g. two information sets), authors consistently denote the second object with a hat (e.g. I and \hat I). This is consistent with some of the prior work, but it gets extremely impractical in the presence of indices, primes etc. (e.g. \hat I'_{\hat I} on line 195). Strongly consider using adjacent alphabet letters instead (e.g. I and J instead of I and \hat I). Also applies to terminal nodes.  C4) In mapping f : I \mapsto f(I)=I'_I  (line 121-123), I' is understood as constant and the subscript I as a variable. This dual usage of I is unfortunate. Also, it clashes with using I' as a generic (non p.r.) abstract information set. Throughout the paper, f(I) and I'_I is used interchangeably. Moreover, a generic p.r. abstract inf. sets are also denoted as I'_I, even in context where no original inf. set I exists. I strongly suggest only using f(I), and never I'_I, and denoting a generic p.r. abstraction inf. set as \hat I (or \tilde I, I^R, I^P, I^{PR}, or anything else which is simple and doesn't clash with I'). The second part also applies to terminal nodes (e.g. use z / \hat z / z' instead of z / \hat z' / z' as generic original / p.r. abstract / abstract terminal nodes).  C5) The mappings \phi_I and \phi_{I'_I}, introduced on line 134 and 149, are confusing. I suggest two changes: First, they have very different domains, so consider using different symbols for each of them (e.g. \phi and \psi, or \phi and \hat \phi). Second, drop the subscripts (e.g. \phi instead of \phi_I) -- the condition \phi(Z_{I}^a) = Z^{g(a)}_{f(I)} from line 136 has to be explicitly stated anyway).  C6) The logic behind the notation introduced on lines 104-109 is inconsistent and unintuitive. For a given information set I, the authors denote the set of terminal nodes somewhere below I as \mathcal Z_I, and the set of information sets directly below I as Z_I (without \mathcal). They also work with the set of information sets somewhere (resp. directly) below I, but denote these as \mathcal D_I and \mathcal C_I. Please either change the notation for terminal nodes to \mathcal Z_I and \mathcal Y_I, or the notation for information sets to \mathcal C_I and C_I, or choose some other consistent variant (e.g. Z_{I,a} and C_{I,a} for "somewhere below" and Z_{I,a}^{ch}, C_{I,a}^{ch} for "directly below").  C7) Explain the motivation behind the 'Z^\Rrightarrow'-like symbols (line 139).   After response:  I agree with reviewer #2 that the usefulness of the bounds can be better motivated, but I think the bounds give us insights into the relation of the error of the abstraction and the final strategy. Try to elaborate on the lessons learned.  Q2: No, there really is an undefined \delta in equation (6) in the submission I am looking at.


Improved Algorithms for Collaborative PAC Learning
We study a recent model of collaborative PAC learning where k players with k different tasks collaborate to learn a single classifier that works for all tasks. Previous work showed that when there is a classifier that has very small error on all tasks, there is a collaborative algorithm that finds a single classifier for all tasks and has O((ln(k))
2 ) times the worst-case sample complexity for learning a single task. In this work, we design new algorithms for both the realizable and the non-realizable setting, having sample complexity only O(ln(k)) times the worst-case sample complexity for learning a single task. The sample complexity upper bounds of our algorithms match previous lower bounds and in some range of parameters are even better than previous algorithms that are allowed to output different classifiers for different tasks.

This paper studies a PAC learning problem where there are k agents who each want to learn the same underlying concept. They are distinguished by the fact that they each have sample access to different distributions, and each requires the learned concept to have small expected error on their own distribution. This learning setting was introduced by Blum et al. [2017]. A bit more formally, suppose D_i is the distribution associated with agent i. The goal is to use a set of samples S (each of which is drawn from D_i for some i \in {1, ..., k}), labeled by an unknown function f*, to learn a concept f with low error. In particularly, with probability at least 1 – delta over the draw of S, across all D_i, the expected error of f should be at most epsilon.  Blum et al. show that if the VC dimension of the hypothesis class is d, then there is a learning algorithm for this problem which uses O(ln^2(k) / epsilon * ((d + k) * ln(1 / delta) samples. The goal of this paper is to change the dependence on ln(k) from ln^2(k) to ln(k). The authors give two algorithms. The first algorithm has a sample complexity of O(ln(k) / epsilon * (d * ln(1 / epsilon) + k * ln(k / delta))). This algorithm weights each distribution, starting off by placing equal weight on all distributions and drawing an equal number of samples from all distributions. It finds a hypothesis with zero error on the set of samples. It then draws some more samples from each distribution to estimate this hypothesis's error on each distribution. If the error associated with distribution D_i is high, it doubles that distribution's weight. In the next round, the number of samples it draws from each distribution is proportional to the distributions' weights. It repeats this procedure for log(k) rounds. It returns the majority function over all the classifiers it learned over all the algorithm’s rounds.  As the authors point out, the problem with this first algorithm is that when d = Theta(k), we again see a quadratic dependence on ln(k), as in the paper by Blum et al. [2017]. The authors get around this using another algorithm that runs for more rounds but uses fewer samples to test the error associated with each distribution. This second algorithm has a sample complexity of O(1 / epsilon * ln(k / delta) * (d * ln(1 / epsilon) + k + ln(1 / delta))). Finally, the authors also give algorithms for the non-realizable case, where there may not be a concept with zero test error, and the goal is to compete with the concept with the lowest test error.  I’d like to see more discussion about how the techniques from this paper compare to the techniques of the paper by Blum et al. [2017]. Both algorithms iteratively reweight each distribution, learn a classifier over the weighted sum of the distributions, estimate the error of this classifier on each distribution, and then use this error estimate to reweight the distributions and repeat. The differences in the two papers’ approaches seem to be in the reweighting scheme, and in the number of samples needed to estimate the classifier’s error at each round. It would be helpful if the authors could summarize where the ln^2(k) factor is coming from in Blum et al.’s paper. Why does using multiplicative weights help get around this?  As a smaller note, I understand the motivation of Blum et al.’s “personalized” variant of this problem, where each agent learns her own concept, but the goal is to avoid a sample complexity of O(dk). But in what scenario would the agents want to learn a single concept, which is the goal of this work? The authors give a little bit of motivation in the first paragraph, referring to AutoML, but it would be helpful to explain this connection more and provide more motivation.  Finally, the notation D_i is used in Section 2 before it is defined.  ===============After rebuttal================= Thanks, I like the bank loan example. I recommend adding more examples like this to motivate the paper.


Inequity aversion improves cooperation in intertemporal social dilemmas
Groups of humans are often able to find ways to cooperate with one another in complex, temporally extended social dilemmas. Models based on behavioral economics are only able to explain this phenomenon for unrealistic stateless matrix games. Recently, multi-agent reinforcement learning has been applied to generalize social dilemma problems to temporally and spatially extended Markov games. However, this has not yet generated an agent that learns to cooperate in social dilemmas as humans do. A key insight is that many, but not all, human individuals have inequity averse social preferences. This promotes a particular resolution of the matrix game social dilemma wherein inequity-averse individuals are personally pro-social and punish defectors. Here we extend this idea to Markov games and show that it promotes cooperation in several types of sequential social dilemma, via a profitable interaction with policy learnability. In particular, we find that inequity aversion improves temporal credit assignment for the important class of intertemporal social dilemmas. These results help explain how large-scale cooperation may emerge and persist.

Summary The authors consider the addition of inequity aversion to learning agents in social dilemmas. They show that inequity aversion as a heuristic helps systems converge to cooperation. They show this is true even when only a few agents are inequity averse.  Evaluation There is a lot to like about this paper. There are 2 main contributions: 1) Rather than considering end-to-end learning the authors consider simple heuristics that can be readily applied to solve a wide class of problems.  2) I also like the eligibility trace formulation of inequity aversion for Markov games, this opens, I think a wide way of experimenting with utility functions that include terms beyond one’s own reward in RL.  I recommend acceptance of this paper.   I have a few comments about the exposition for the authors that I think could improve the overall clarity of the paper.  First, I think it would be good to state in the introduction what the point of the research program is. Is it:  a) To design agents for environments where we control all agents, e.g. robot soccer (in which case, why not just optimize joint rewards?)  b) To design agents for environments where we control some but not all agents? (In this case, isn’t advantageous inequity aversion a bad idea since it makes one very exploitable?) c) To study how certain heuristics can evolve in a population? (In this case, it would be good to see the payoffs of each type of agent in games with mixed populations since that’s what would be favored by evolution) d) Something else?  I think nailing down more clearly in the introduction what the goal is and why people should care about this will make the paper even higher impact.


Compact Representation of Uncertainty in Clustering
For many classic structured prediction problems, probability distributions over the dependent variables can be efficiently computed using widely-known algorithms and data structures (such as forward-backward, and its corresponding trellis for exact probability distributions in Markov models). However, we know of no previous work studying efficient representations of exact distributions over clusterings. This paper presents definitions and proofs for a dynamic-programming inference procedure that computes the partition function, the marginal probability of a cluster, and the MAP clustering-all exactly. Rather than the N th Bell number, these exact solutions take time and space proportional to the substantially smaller powerset of N . Indeed, we improve upon the time complexity of the algorithm introduced by Kohonen and Corander [11]  for this problem by a factor of N . While still large, this previously unknown result is intellectually interesting in its own right, makes feasible exact inference for important real-world small data applications (such as medicine), and provides a natural stepping stone towards sparse-trellis approximations that enable further scalability (which we also explore). In experiments, we demonstrate the superiority of our approach over approximate methods in analyzing real-world gene expression data used in cancer treatment.

Post-rebuttal: thank you for clarifying / correcting the figure with experimental results.   Main idea:   Exactly compute the partition function of a multicut / correlation clustering problem using dynamic programming.   Strengths:   Signed graph partitioning is an extremely versatile tool, so any theoretical or practical progress is welcome and may (eventually) have large impact. Studying a Gibbs distribution over solutions rather than just the minimal energy / MAP solution is of fundamental interest with a view to passing meaningful uncertainty estimates on to downstream processing. In essence, I think this is meaningful work.   The paper is clearly written and thorough (though I did not check all mathematical details).   Overall, I think the work has more substance than many NIPS papers I see, which is why I count it among the better 50%.   Weaknesses:   The method is of limited practical usefulness, with exact computations limited to 25 elements or so.   The comparison with approximate methods (notably Kwikcluster) is not really fair. The state of the art to compare to is probably a set of multicut MAP solutions obtained on noisy versions of the original edge weights (ref. 7). This is not difficult to implement in practice and should give better results.   Comments:   I don't think the title summarizes the contribution well. There is not really much study or discussion of uncertainty, beyond the use of marginals. While marginals are a compact representation, the trellis certainly is not.   Similarly, section 2 is entitled "Uncertainty in clustering" even though that topic is not discussed there at all.   The core result which makes dynamic programming applicable is fact 1, proved in Appendix E. This being at the very heart of the contribution, I would recommend moving an extended version of Appendix E into the main text, at the expense of other parts. For instance, the MAP solution (section 3.2) which does not recur to branch & bound, is unlikely to be competitive with SOA ILP solvers in conjunction with efficient column generation. In fact, I wouldn't mind having the parts on sparse trellises omitted altogether. Without detailed analysis of what choice of trellis gives what kind of approximation quality, this section is not so useful. Bottom line: Fact 1 is all-important for this paper, and merits more extensive explanation / discussion in the main text.   To give a better feeling for the data used, it would be good to show both the MAP clustering as well as all the marginals (in an 11x11 matrix).   Fig 3: One would expect 11*(11-1)/2 = 55 entries, but there are fewer points in the scatter plot?    Minor comments:   Changing the definition of energy such that higher is preferable to lower (line 97) is not a good idea.   The data used in the experiments is not a very convincing example for correlation clustering (coordinates in some space, with repulsive interactions obtained "artifically"; in particular, this choice induces a spherical shape bias on the resulting clusters).  typo: elemnts line 241: unresolved reference    Note on my confidence score:  I did not check the maths; and I cannot guarantee that dynamic programming has not been used previously to compute the exact partition function of clusterings. I am confident of my other comments.