Blind Attacks on Machine Learners
The importance of studying the robustness of learners to malicious data is well established. While much work has been done establishing both robust estimators and effective data injection attacks when the attacker is omniscient, the ability of an attacker to provably harm learning while having access to little information is largely unstudied. We study the potential of a "blind attacker" to provably limit a learner's performance by data injection attack without observing the learner's training set or any parameter of the distribution from which it is drawn. We provide examples of simple yet effective attacks in two settings: firstly, where an "informed learner" knows the strategy chosen by the attacker, and secondly, where a "blind learner" knows only the proportion of malicious data and some family to which the malicious distribution chosen by the attacker belongs. For each attack, we analyze minimax rates of convergence and establish lower bounds on the learner's minimax risk, exhibiting limits on a learner's ability to learn under data injection attack even when the attacker is "blind".

This paper abstracts a problem of interest in (applied) adversarial machine learning, whereby an adversary manipulates a learner trained partially on benign stochastic data, with adversarially-chosen, stochastic data. Specifically the paper studies a learner intending to estimate parameter (theta) for distribution (F) from which training data is drawn, and a blind attacker – not knowing the distribution or the training set – who injects data with from a different distribution (G(phi)), in order to maximise the minimax risk for the learner. The authors discuss two settings – “informed learner” and “blind learner”, and investigate the effect of the relationship (KL divergence) between F(theta) and G(phi).
The paper presents an appropriate theoretical treatment of an interesting question that has persisted in adversarial machine learning: that of the effect of a blind attacker poisoning training data without access to benign iid training data. The paper's approach is one of minimax theory, with analysis variations given for a defender/learner aware of the attacker's poison distribution, and (the more realistic) case of a blind defender who is unaware of the attacker's choice. Results depend on the KL divergence between benign and malicious data distributions.  There have been relatively few theoretical treatments of adversarial machine learning settings. The authors suitably note the recent work of Duchi et al. on minimax rates and differential privacy (that appeared recently at NIPS), with this paper being complementary.  However some issues remain:  The work on Kearns and Li (1993) who looked at the effect of a fraction of contaminated data in the PAC learning framework should be cited and related briefly. Similarly the vast body of work on robust statistics: the presented framework seems to not be unlike the gross-error model (sampling distribution is a mixture of benign and a contamination distribution; the minimax approach there is to consider a contamination distribution that is worst case; this yields Huber estimators that is a robust class of estimators in the setting; etc.)  The paper could be better structured. For example, Section 1 spans half of the paper and would be better organised if divided into 2- 3 sections.  While the paper is written appropriately for a specialised COLT sub-audience present at NIPS, since NIPS is broader (than ever), and the (more applied) adversarial learning community should find the paper interesting, it would be beneficial to rework the exposition and background with such readers in mind. For example detailing the connection between problems at #89  Several notations are used before being properly defined, e.g., delta in #199, Vol(Z) in #292 (instead, these are defined in the supplementary material).  Not sure Fig1,2 strictly represent Markov chains (typically identical transition probabilities across finite states, etc.) but no significant matter. Also, consider shading nodes that are observed.  Quotations leading quoted phrases should be opening quotation marks. Something with the typesetting (#161) seems broken with the definition of theta in relation to theta_i.  **After author response: Thankyou for the thoughtful response. The assurances that connections with robust stats and PAC will be added as appropriate, are well received. The initial explanation on the differences between the blind model and the typical (non-agnostic) model typical in robustness appear to make sense.


Universal Correspondence Network
We present a deep learning framework for accurate visual correspondences and demonstrate its effectiveness for both geometric and semantic matching, spanning across rigid motions to intra-class shape or appearance variations. In contrast to previous CNN-based approaches that optimize a surrogate patch similarity objective, we use deep metric learning to directly learn a feature space that preserves either geometric or semantic similarity. Our fully convolutional architecture, along with a novel correspondence contrastive loss allows faster training by effective reuse of computations, accurate gradient computation through the use of thousands of examples per image pair and faster testing with O(n) feed forward passes for n keypoints, instead of O(n 2 ) for typical patch similarity methods. We propose a convolutional spatial transformer to mimic patch normalization in traditional features like SIFT, which is shown to dramatically boost accuracy for semantic correspondences across intra-class shape variations. Extensive experiments on KITTI, PASCAL, and CUB-2011 datasets demonstrate the significant advantages of our features over prior works that use either hand-constructed or learned features.
for example, SIFT or SURF for sparse structure from motion, DAISY or DSP for dense matching, SIFT Flow or FlowWeb for semantic matching. The Universal Correspondence Network accurately and efficiently learns a metric space for geometric correspondences, dense trajectories or semantic correspondences. explicit interpretation. In addition, patch similarity is inherently inefficient, since features have to be extracted even for overlapping regions within patches. Further, it requires O(n 2 ) feed-forward passes to compare each of n patches with n other patches in a different image.
In contrast, we present the Universal Correspondence Network (UCN), a CNN-based generic discriminative framework that learns both geometric and semantic visual correspondences. Unlike many previous CNNs for patch similarity, we use deep metric learning to directly learn the mapping, or feature, that preserves similarity (either geometric or semantic) for generic correspondences. The mapping is, thus, invariant to projective transformations, intra-class shape or appearance variations, or any other variations that are irrelevant to the considered similarity. We propose a novel correspondence contrastive loss that allows faster training by efficiently sharing computations and effectively encoding neighborhood relations in feature space. At test time, correspondence reduces to a nearest neighbor search in feature space, which is more efficient than evaluating pairwise patch similarities.
The UCN is fully convolutional, allowing efficient generation of dense features. We propose an on-the-fly active hard-negative mining strategy for faster training. In addition, we propose a novel adaptation of the spatial transformer [13], called the convolutional spatial transformer, desgined to make our features invariant to particular families of transformations. By learning the optimal feature space that compensates for affine transformations, the convolutional spatial transformer imparts the ability to mimic patch normalization of descriptors such as SIFT. Figure 1 illustrates our framework.
The capabilities of UCN are compared to a few important prior approaches in Table 1. Empirically, the correspondences obtained from the UCN are denser and more accurate than most prior approaches specialized for a particular task. We demonstrate this experimentally by showing state-of-the-art performances on sparse SFM on KITTI, as well as dense geometric or semantic correspondences on both rigid and non-rigid bodies in KITTI, PASCAL and CUB datasets.
To summarize, we propose a novel end-to-end system that optimizes a general correspondence objective, independent of domain, with the following main contributions:
• Deep metric learning with an efficient correspondence constrastive loss for learning a feature representation that is optimized for the given correspondence task.
• Fully convolutional network for dense and efficient feature extraction, along with fast active hard negative mining.
• Fully convolutional spatial transformer for patch normalization.
• State-of-the-art correspondences across sparse SFM, dense matching and semantic matching, encompassing rigid bodies, non-rigid bodies and intra-class shape or appearance variations.

Unlike previous patch similarity based methods, in this work the authors proposed a new model that discover visual correspondences using dense feature vector, correspondence contrastive loss function (a combination of mean squared loss and margin-based ranking loss), and CNN Spatial Transformer. The proposed method can successfully capture both geometric correspondence and semantic correspondence.
In this paper the authors proposed a universal correspondence network using CNN and pair-wise ranking loss. Overall this paper is well written and easy to follow. Below are the detailed review.  1. Equation(1) is the loss function used in the proposed model and s_i/(1-s_i) is acted as the identity function to select which part of the loss function will be used. In Line 121 the authors said they denote s = 0 for positive pairs and s=1 for negative pairs. In such way positive pairs will be trained by margin-based ranking loss and negative pairs will be trained by mean squared loss. This conflicts to Line 132 which claims that the mean squared loss is used for minimizing the distance between positive pairs and ranking loss is used to make sure negative pairs are at least m distance away.  2. In this work, all the key points are pre-selected so is it possible to run this algorithm without given any key points and let the model decide this? Basically this would become given a dense feature vector from left image, rank all vectors in the right image. Does training with hard negative pairs affect the performance in that case since a lot of vectors will not be updated during the training process?  3. The authors use a spatial transformer to mimic path normalization and show that by adding this the performance of the model increases. Does this differ from shortcuts? If so what is the advantage of spatial transformer over shortcut?  4. Figure 5 shows the Accuracy of PCK performance but no precision and recall. If I understand correctly, for a certain pixel threshold k, does the accuracy equal to #(actual correspondent pixels) / k? If so it would be interesting to see how the recall shape looks like.


Rényi Divergence Variational Inference
This paper introduces the variational Rényi bound (VR) that extends traditional variational inference to Rényi's α-divergences. This new family of variational methods unifies a number of existing approaches, and enables a smooth interpolation from the evidence lower-bound to the log (marginal) likelihood that is controlled by the value of α that parametrises the divergence. The reparameterization trick, Monte Carlo approximation and stochastic optimisation methods are deployed to obtain a tractable and unified framework for optimisation. We further consider negative α values and propose a novel variational inference method as a new special case in the proposed framework. Experiments on Bayesian neural networks and variational auto-encoders demonstrate the wide applicability of the VR bound.

Paper Summary ============= Conventional variational inference finds an approximate posterior that minimizes the KL divergence from the true (intractable) posterior. This paper shows how variational optimization objectives can use a wider class of Renyi's alpha divergences, which have a scalar parameter \alpha. Setting \alpha appropriately recovers many special cases of interest: \alpha -> 1 recovers the KL divergence while \alpha = 0 leads to exact posterior inference *if* the support of the approximating family contains the true posterior. Negative \alpha values provide an *upper* bound on marginal likelihood and tend to be "mass covering" as \alpha -> -infty, while postive \alpha values provide a lower-bound on the marginal likelihood and tend to be "mode seeking" as \alpha -> +infty.  Aside from connecting Renyi's alpha-divergences to variational objectives (which I haven't seen before), this paper presents a useful algorithm for optimization of these objective functions based on Monte Carlo (MC) approximations similar to those used by Stan and black-box VI [11,12].  The algorithm works by drawing a handful of samples from the current *reparameterized* approximate posterior, computing importance weights for each sample, selecting one sample with probability according to its weight, and then stepping in the direction of its gradient. Special care must apply to minibatch learning, because samples are biased for values of \alpha other than \alpha = 1. Experiments show that alpha values often need to be tuned in a dataset-specific way for best performance, but that values other than alpha=1 are sometimes much better.
Review Summary ============== Overall, I recommend accepting this paper for at least a poster presentation. It nicely unifies and extends various variational inference approaches, and provides a good mix of theoretical and experimental contributions with some solid explanatory figures. My biggest concerns is a lack of practical description of how to properly choose the alpha value for a specific dataset, and how much this can matter compared to other hyperparameters that might need tuning (# of MC samples, sophistication of posterior approximation family etc). I worry that the question of "which alpha to use" will cause concern for practioners and will often result in setting alpha=1 in practice, which is the special case of KL-divergence we already know a lot about. I'd also like to see discussion of how different choices impact computational speed.   Feedback on experiments ======================= My biggest concern is that choosing a good alpha value seems to need careful tuning for each dataset, and there isn't much guidance here on how to do it. It's not clear how much benefit we can get from properly tuning alpha, compared to say fixing alpha=1 (standard VI) and investing precious research time in other choices like using a larger number of samples for MC approximations, tuning minibatch sizes or learning rate schedules, or pursuing a slightly more sophisticated approximate posterior family. From a practical perspective, it would be nice to have a good default guess for alpha and some better intuition about how to tune it / when the default guess is "good enough".  I found the overall story of the first experiment (with Bayesian neural networks) a bit confusing. I would have expected a relatively smooth quadratic-like trend in how the chosen alpha value affects performance, at least within a single dataset. That is, for any dataset, there is an obvious "best choice" of alpha, with performance getting worse for alpha values further away.  owever, the Protein dataset has quite a weird story, where alpha=0.5 is noticeably worse than alpha = 0 or alpha = 1, and then alpha < 0 is worse again. Is there any explanation here? Might this be a local optima issue? In the supplement, the Energy and Yacht datasets have a similarly confusing story.  I wish there was more practical discussion about how various algorithmic choices (# of samples K, alpha value, etc) impact the runtime of the algorithm.I'd be interested in both big-oh and (more importantly) wallclock time comparisons in the experiments. How important is it to just pick one MC sample to do the back-propagation, instead of using all of them?    Feedback on Presentation ======================== I found Fig 2a (which gives intuition for the theorems quanitifying the bias in the MC approximations) quite hard to parse without lots of effort. It's not clear what the arrows mean. I'd suggest expanding this one figure into several smaller panels, each of which explores the influence of one parameter (# of samples K increasing, or alpha decreasing).  Also, some other labeling like alpha_{+} or alpha_{-} might be more useful than alpha_1 and alpha_2 to visually emphasize the domains of these alpha values better.  The technical discussion in Sec 4.3 explaining how to handle minibatches was quite confusing and hard to understand. I'd suggest a careful revision.  Typos, etc ========== Fig 1 caption was hard to parse.  Lines 192-202: Definitions of global and local as used here are a bit confusing. Could be clarified, since many readers will think of the global and local parameter classes described in SVI paper, which (I think) is a different sense of global/local.


Guided Policy Search via Approximate Mirror Descent
Guided policy search algorithms can be used to optimize complex nonlinear policies, such as deep neural networks, without directly computing policy gradients in the high-dimensional parameter space. Instead, these methods use supervised learning to train the policy to mimic a "teacher" algorithm, such as a trajectory optimizer or a trajectory-centric reinforcement learning method. Guided policy search methods provide asymptotic local convergence guarantees by construction, but it is not clear how much the policy improves within a small, finite number of iterations. We show that guided policy search algorithms can be interpreted as an approximate variant of mirror descent, where the projection onto the constraint manifold is not exact. We derive a new guided policy search algorithm that is simpler and provides appealing improvement and convergence guarantees in simplified convex and linear settings, and show that in the more general nonlinear setting, the error in the projection step can be bounded. We provide empirical results on several simulated robotic navigation and manipulation tasks that show that our method is stable and achieves similar or better performance when compared to prior guided policy search methods, with a simpler formulation and fewer hyperparameters.

The authors propose a new, simplified approach to guided policy search. Guided policy search consists in first improving a set of local policies starting from different initial states, second using supervised learning to "compile" them into a global policy, and then iterating those two steps towards convergence of local and global policies. In the previous method, a constraint was added through a surrogate cost function so that the local policies and global policies converge to each other. In this new approach, the surrogate cost function is removed and the process is simpler, based on minimizing the KL divergence between local and global policies. This simplification was made possible by recognizing the similarity between guided policy search and mirror descent. A very preliminary experimental study shows some advantages of the new approach.
Despite my initial lack of knowledge about guided policy search methods, which makes the paper hard to read for me, I believe this is a strong paper presenting state-of-the-art research. However, there are a few issues that I'm listing hereafter by order of decreasing importance.  My main point is with the way Algorithm 2 is written. In contrast to Algorithm 1, the C-step of Algorithm 2 requires to already have a global policy \pi_\theta so as to compute the KL divergence. Nothing is said about providing an initial policy \pi_\theta_0. Did I miss something here? Or shouldn't you split step 2 into a step 2 and a step 4 where you perform the re-optimization of local policies, as in Algo 1?  If I'm right, aren't there some constraints on the way \pi_\theta is initialized? In any case, this should be clarified.  The experimental evaluation is rather weak: - the description of the experiments is sketchy, one cannot reproduce them out of what is described (which simulator?, position of the holes?, size of the arm?, etc.) - the dimensions for x_t and u_t are not given - results are averaged over 3 runs only. Statistical significance is not checked. Worse than that, I doubt the validity of the success rates: how can you get a success rate of 62.50% over three runs with 4 holes? It means you succeeded 7,5 times (!). Something is not specified here (are there several instances of peg insertion for the same hole over one run?). - computational cost is not mentioned: it is high, to explain why averaging over 3 runs only? A comparison of computational cost and sample efficiency with respect to previous guided policy search methods or simpler baselines such as TRPO looks mandatory - the authors put scalability forward, but their policy is a 2 layers network with 40 neurons each, which is rather small. This should be discussed.  On one side, the paper is difficult and long. A good deal of the technical part of the paper is rejected into the supplementary material, and some sections of the paper can hardly be read without reading this supplementary part. I must say I didn't check many technical details in the methods nor the proofs in the supplementary material.  On the other side, I didn't like much Section 4.2, which tries to justify a rather elaborated policy for step size selection, but the message is ruined by footnote 3, which states that finally using a simple rule of thumb is enough.  So I would suggest the authors to remove Section 4.2 and use the resulting free space to clarify the experimental section and the methods.  Also, wouldn't Section 3.2 find a better place into Section 4?  It is never specified what is the domain of x_t and u_t. Are they in R^n? In a closed domain of R^n?  More local points:  Lines 55-58, this should be written more clearly. The first sentence states that policy gradient methods can be applied to huge policies (citing recent deep methods), and the second states that they are limited to small policies (citing older work). You may rather insist on number of episodes for training, as done later on.  Maybe you should not use casual writing such as "we'll, we've, we don't"... in scientific papers.  I just found one typo: In Algo 1, and each p(u_t|x_t) should be each p_i(u_t|x_t)


A Powerful Generative Model Using Random Weights for the Deep Image Representation
To what extent is the success of deep visualization due to the training? Could we do deep visualization using untrained, random weight networks? To address this issue, we explore new and powerful generative models for three popular deep visualization tasks using untrained, random weight convolutional neural networks. First we invert representations in feature spaces and reconstruct images from white noise inputs. The reconstruction quality is statistically higher than that of the same method applied on well trained networks with the same architecture. Next we synthesize textures using scaled correlations of representations in multiple layers and our results are almost indistinguishable with the original natural texture and the synthesized textures based on the trained network. Third, by recasting the content of an image in the style of various artworks, we create artistic images with high perceptual quality, highly competitive to the prior work of Gatys et al. on pretrained networks. To our knowledge this is the first demonstration of image representations using untrained deep neural networks. Our work provides a new and fascinating tool to study the representation of deep network architecture and sheds light on new understandings on deep visualization. It may possibly lead to a way to compare network architectures without training.

This paper performs 3 optimization tasks that are popular in deep visualization: 1) feature inversion, 2) texture synthesizing and 3) style transfer via a random network instead of a well trained network as often done in the literature. The result shows that a random network can be used to perform these 3 tasks well, and hints that random networks can be used to perform comparative analysis between different net architectures.
OVERALL: This is an interesting paper and I enjoy looking at the resultant images. The paper does raise an interesting idea that one could use random nets for inverting representation in order to understand architectures; but, did not actually show any result to back up that idea. The texture synthesis and style transfer experiments with a random net demonstrate a new and more efficient way (without having to train a net) to perform these tasks.  However, the paper merely shows the results, but does not offer thorough investigation and insights into using random nets. It'd be great to have some of these questions answered in the paper:   1. is it surprising that inverting a random function/network works here?  2. how easy it is to invert a random nets compared to a trained net? 3. what are the disadvantages of using a random vs trained net? 4. how would the results change if performing these tasks with other architectures, e.g. (shallow) AlexNet, and (very deep) ResNet?  TECHNICAL QUALITY 1. Inverting deep representations. - For a fairer comparison (qualitatively and quantitatively) with Mahendran et al, one should use the same setup (i.e. AlexNet and/or regularizer). Why was VGG used here? - Inverting from layer1-5 has been shown to be easy (Mahendran et al 2014, Dosovitskiy & Brox 2016). I wonder if random weights would still work similarly for higher layers?  - This section would be much more impactful if the authors indeed inverted and compared different net architectures.  - One way to improve the inversion quality further is to optimize an image to match both the code and the texture (Gram matrix) of a target image.  2. Textures synthesis: the results looks qualitatively great, and interesting as it's in contrast with the finding by Gatsy et al. - I wonder if it would produce a large variation (i.e. does starting from different random images produce very different samples)  3. Style transfer: Fig. 5 shows a recognizable artifact (rough, dark blobs of colors) in ranVGG compared to [13] which is smoother / qualitatively better. It'd be great to show more investigation into why it happened and see if the trained weights matter a lot here or it's just due to different hyperparam settings.  NOVELTY Inverting and/or harnessing deep representations extracted from random networks has been previously studied in these two papers: - Alexey Dosovitskiy and Thomas Brox. Inverting visual representations with convolutional networks. arXiv preprint arXiv:1506.02753, 2015. - Alexey Dosovitskiy and Thomas Brox. Generating images with perceptual similarity metrics based on deep networks. arXiv preprint arXiv:1602.02644, 2016.  Their approach is not the same as the one in this paper. But the general idea is the same, thus, their works in using random nets should be mentioned. Using random nets in texture synthesis and style transfer is novel/original, afaik. There is this concurrent texture synthesis work with similar ideas though: https://arxiv.org/pdf/1505.07376v3.pdf.  POTENTIAL IMPACT The texture synthesis result shows high practical impact to the community. However, the representation inversion result is not very surprising and the authors have not demonstrated how such a random net can be used to understand different net architectures (as stated in the Intro & Discussion).  CLARITY & PRESENTATION Overall, the paper is nicely written and easy to read. I see a few places that could be improved for clarity and presentation:  L61-63: "Could we do deep visualization using untrained nets": the authors did not describe the scope of "deep visualization", but if it includes synthesizing a preferred images for neurons (see Nguyen et al 2016), then a random net wouldn't work. L16-17: The authors state that this proposed method sheds light on new understandings of deep vis, but the paper did not really discuss this in depth. L56-57: The purpose of a visualization technique is to enhance some understanding, thus I'm not sure what it's meant here.


Minimizing Regret on Reflexive Banach Spaces and Nash Equilibria in Continuous Zero-Sum Games
We study a general adversarial online learning problem, in which we are given a decision set X in a reflexive Banach space X and a sequence of reward vectors in the dual space of X. At each iteration, we choose an action from X , based on the observed sequence of previous rewards. Our goal is to minimize regret. Using results from infinite dimensional convex analysis, we generalize the method of Dual Averaging to our setting and obtain upper bounds on the worst-case regret that generalize many previous results. Under the assumption of uniformly continuous rewards, we obtain explicit regret bounds in a setting where the decision set is the set of probability distributions on a compact metric space S. Importantly, we make no convexity assumptions on either S or the reward functions. We also prove a general lower bound on the worst-case regret for any online algorithm. We then apply these results to the problem of learning in repeated two-player zero-sum games on compact metric spaces. In doing so, we first prove that if both players play a Hannan-consistent strategy, then with probability 1 the empirical distributions of play weakly converge to the set of Nash equilibria of the game. We then show that, under mild assumptions, Dual Averaging on the (infinite-dimensional) space of probability distributions indeed achieves Hannan-consistency.
results in the literature, including algorithms for online learning on finite sets (Arora et al., 2012) and finite-dimensional online convex optimization (Hazan et al., 2007).
Given a convex subset X of a reflexive Banach space X, the generalized Dual Averaging (DA) method maximizes, at each iteration, the cumulative past rewards (which are elements of X * , the dual space of X) minus a regularization term h. We show that under certain conditions, the maximizer in the DA update is the Fréchet gradient Dh * of the regularizer's conjugate function. In doing so, we develop a novel characterization of the duality between essential strong convexity of h and essential Fréchet differentiability of h * in reflexive Banach spaces, which is of independent interest.
We apply these general results to the problem of minimizing regret when the rewards are uniformly continuous functions over a compact metric space S. Importantly, we do not assume convexity of either S or the rewards, and show that it is possible to achieve sublinear regret under a mild geometric condition on S (namely, the existence of a locally Q-regular Borel measure). We provide explicit bounds for a class of regularizers, which guarantee sublinear worst-case regret. We also prove a general lower bound on the regret for any online algorithm and show that DA asymptotically achieves this bound up to a √ log t factor.
Our results are related to work by Lehrer (2003) and Sridharan and Tewari (2010); Srebro et al. (2011). Lehrer (2003) gives necessary geometric conditions for Blackwell approachability in infinitedimensional spaces, but no implementable algorithm guaranteeing Hannan-consistency. Sridharan and Tewari (2010) derive general regret bounds for Mirror Descent (MD) under the assumption that the strategy set is uniformly bounded in the norm of the Banach space. We do not make such an assumption here. In fact, this assumption does not hold in general for our applications in Section 3.
The paper is organized as follows: In Section 2 we introduce and provide a general analysis of Dual Averaging in reflexive Banach spaces. In Section 3 we apply these results to obtain explicit regret bounds on compact metric spaces with uniformly continuous reward functions. We use these results in Section 4 in the context of learning Nash equilibria in continuous two-player zero sum games, and provide a numerical example in Section 4. All proofs are given in the supplementary material.

The paper considers the problem of regret minimization in reflexive Banach spaces, and the main contribution is showing that properly tuned variants of dual averaging/FTRL can guarantee sublinear regret bounds in this setting. This result is then applied to the special case of regret minimization in compact metric spaces, where the authors are able to show explicit near-optimal rates that hold uniformly for all comparators. Finally, the authors apply their techniques for proposing algorithms for finding epsilon-Nash equilibria in continuous zero-sum games. The results are illustrated with several simple experiments and examples.
POST-REBUTTAL COMMENTS  Thanks for the replies. I've found the example of X=L2(R) and \cal{X} = \Delta(X) to be convincing enough, so I understand now how the results are more general than those of Sridharan and Tewari (2010).  =============================  The paper is very technical and offers a rather demanding reading experience: the main paper is essentially a densely compressed collection of definitions and theorems for a number of different problem settings. The paper is made complete by a tremendous 17-page appendix that not only includes the full proofs, but adds several simulations, illustrative examples and technical results that are interesting in their own right. This structure itself is my main concern about the paper: I am not sure that NIPS is a suitable venue for such enormous papers with so much to say. Splitting papers so that the main file includes the main result and the supplement presents the proofs is considered normal by now; this current paper goes way beyond.  Despite my above comments, if the reader is familiar with the underlying concepts, the logic of the paper is reasonably easy to follow. The proofs are clean and appear to be correct. The techniques themselves are not very original: notably, the proof of the key result (Theorem 3) closely follows the analysis of Kwon and Mertikopoulos (2014). Other proof techniques are also familiar from related work. That said, I do appreciate the clarity and the generality of the new results in the current paper.  There is room for improvement in discussing the related literature. In particular, the authors write off the result of Sridharan and Tewari (COLT 2010) as being weaker than the new results because their bounds require a uniformly bounded strategy set. However, the results in the current paper also seem to make a similar assumption (Assumption S.2 in the appendix that seems to be necessary for proving all the results in the paper!), so I wouldn't consider this to be a serious distinction. Also, the authors fail to mention the follow-up work by Srebro, Sridharan and Tewari (NIPS 2011) who prove even more general results. I realize that some of the results in that work are way beyond the scope of this paper, but still, a short discussion would be useful to have. A discussion about how the bound of Corollary 3 relates to other results about continuous exponential weights would be even more useful.  Overall, while I'm not sure that the structure of the paper is really suitable for NIPS, I think that this is a solid technical paper that may warrant some interest.  Detailed comments ================= 020: Is Audibert and Bubeck (2009) really the best reference to illustrate this point? Maybe Cesa-Bianchi and Lugosi (2006) and Shalev-Shwartz (2011) are better references. 049: "existence" -> "existence of" 076: This is a strange way to define worst-case regret; worst-case is usually defined by taking the supremum over the sequence of losses, not the comparator.  086: "w.r.t. to" -> "w.r.t."; this occurs several times in the paper. 110: "strictly convex" was not defined formally, the notation A^{-1} for sets A likewise. Perhaps it would be good to give a reference to a convex analysis textbook for interested readers. 140: This theorem seems to require Assumption S.2 to hold. If so, please state this assumption in the main text. 149: Regret bounds that depend on the comparator x are sometimes the best one can hope for; see, e.g., McMahan and Orabona (COLT 2014). In particular, for the case of unbounded decision sets, it may not be reasonable to expect a bound that holds simultaneously for all x. 222: Also note that this choice of regularizer coincides with the entropic barrier of Bubeck and Eldan (COLT 2015) for Euclidean spaces. 237: "w is any function with modulus of continuity"---does the sequence depend on w? If so, please make it clear in the statement of the theorem. Appendix: 254, Eq. (13): Inconsistent notation for inner products in the first term. 256, display: shouldn't the inequality be an equality instead? 268, display: what is "d\tau"?


Unsupervised Learning from Noisy Networks with Applications to Hi-C Data
Complex networks play an important role in a plethora of disciplines in natural sciences. Cleaning up noisy observed networks poses an important challenge in network analysis. Existing methods utilize labeled data to alleviate the noise the noise levels. However, labeled data is usually expensive to collect while unlabeled data can be gathered cheaply. In this paper, we propose an optimization framework to mine useful structures from noisy networks in an unsupervised manner. The key feature of our optimization framework is its ability to utilize local structures as well as global patterns in the network. We extend our method to incorporate multiresolution networks in order to add further resistance in the presence of high-levels of noise. The framework is generalized to utilize partial labels in order to further enhance the performance. We empirically test the effectiveness of our method in denoising a network by demonstrating an improvement in community detection results on multi-resolution Hi-C data both with and without Capture-C-generated partial labels.
has been shown to have a fundamental impact on gene expression in biological processes [4]. For example, many genes are in 3D contact with genomic regions that are far away in the linear genome but close in 3D space. These genomic regions contain regulatory elements that control when the gene is active [5,6,34]. The rules by which regulatory elements come in contact with their target genes are still unclear [7]. While the exact mechanism for this selective interaction between the regulatory elements and the target genes is unknown, the 3D organization of the genome in domains of interaction seem to play a crucial role. Furthermore, topologically associated domains (TADs) [8], where local interactions are observed, are of potential biological interest as they have been shown to be conserved between mice and humans, which suggests an ancient root for higher order structures in the genome [8].
The interaction map of regulatory elements in a genome can be viewed as a network, where each node is a regulatory element and each link represents the interaction strength between two of the elements. In this context, we not only have prior knowledge about the types of structures in this interaction network, but we also have various types of noisy and incomplete observations of the links in this network based on recently-developed technologies.
One approach to observe these links is Hi-C, an experiment which uses high-throughput sequencing to construct a 2D contact map measuring the frequency with which pairs of genomic regions co-localize in 3D space. The results of Hi-C experiments can be summarized at multiple resolutions (lower resolutions are obtained by binning genomic regions together), ranging from 1 kb to 1 Mb [8][9][10]. Higher resolution maps capture more fine grained location interactions but the background noise generated by random collisions can be larger [11]. Lower resolution maps have less noise, but at the cost of losing the exact localization of genomic contacts. In addition to Hi-C, there are other experiment variants such as 4C, 5C and Capture-C technologies, which provide a new window into detection of a small number of interaction links with high confidence [12][13][14][15], by focusing sequencing resources on a selected subset of contacts. For these experiments the increased confidence comes at the cost of not measuring the full contact map. Thus, the integration of multi-resolution noisy Hi-C data with high-confidence Capture-C data is not only an interesting problem in the context of general network densoising but it is also biologically relevant.

The authors describe an optimization approach to identify clusters (communities) in a network while combining multiple noisy versions of the network. In addition their framework allows to incorporate high confidence evidence (though I’m not convinced that this part of their method is very effective). The authors apply their method to combine multiple resolutions of Hi-C data and analyze their approach on a simplified setting of disjoint blocks and in the presence of increasing amount of noise.
I believe the review of this paper should be done in 2 stages: 1) method; 2) application.  The method, as presented, is fairly general and could be applied to many different scenarios. It is a relatively novel method for network de-noising – combining multiple networks from noisy observations of the true underlying networks, in particular network that is made of more or less clear clusters. In this context the method is well described. I would be interested to know how well does it scale – the complexity and running time of the method on networks of various size. I understand that it probably doesn’t scale too well given that the genome had to be broken up into chromosomes and the chromosomes that were presented were not the largest. What is the resolution? How many nodes was the biggest network tested? Regularization in section 2.3 (to add high confidence edges) is not really fixing the edges – you could have the same number of completely different edges and it would still look ok (since you are only looking at sums here). Can you check how much it hurts you? You could regularize the edges directly… I think it would be interesting to see the effects of this regularization (the results in the experimental section support the fact that you are not using your high-confidence edges to the fullest…)  The application, I believe, is somewhat problematic. You start your paper talking about community detection. I’m not sure that’s the best or most important use of Hi-C data. You also keep talking about de-noising and community detection? Which is it? What is the biological motivation for community detection? Does one necessarily lead to the other? The link for me is missing. We don’t actually know whether true 3D genome data clusters nicely. Certainly, for many applications in biology, it would be nice to keep the 3D structure of this data preserved as a network rather than clusters. Going to the cluster level is really not the wholly grail of this data and in that, the method and the motivation of the work are misleading. In any case, I was not convinced by the motivation that cluster representation is the way to go (or even the right kind of embedding) for Hi-C data. The abstract was also misleading “we propose to mine useful structures from noisy networks”… - why are they useful? And also by structures here, I could imagine something more informative than relatively large clusters on which the method was ultimately tested.  In the same spirit, I’m not sure that in the method, as applied in this particular scenario, S should be low ranked, sparse yes, but low rank? I’m not sure that is true of the biology here.   Experiments To me, a more useful use of network denoising in the Hi-C data context would have been combining different resolutions of this data and testing the resulting denoised network with the Capture-C data – how well would your de-noising go? Also, there are other approaches combining multiple networks, such as Similarity Network Fusion (Wang et al, 2014) that this work could be compared to.  When you describe figure 2c in the text, you shouldn’t really be describing what is on the figure – that goes in the caption. In the caption of the figure, you shouldn’t really be describing that one method is performing better than the other, just what the figure is about, not the conclusions one would draw from it. The experimental section definitely requires some clarification, it is very confusing now. In figure 2a it’s not clear what you used as a baseline. In figure 2b – why is the \alpha_8 coefficient also going down?  Details: There are a lot of typos and strange use of long words, somebody has to read the manuscript very carefully and simplify/fix the English a bit. Section 3.1. Can you provide more intuition for J(R)? It’s not obvious what it captures. Line 123 L2 is not properly written In Figure 1 you first say that you combine (a,b,c) and then that you combine all 8 resolutions, so which is it? Please write what you actually have done and what is in the figure. If these are two different things, then please write that clearly. Figure 2c comes first, prior to Figures 2a and 2b, perhaps you could decouple these, so that they would be cited appropriately? They shouldn’t really be one figure


A primal-dual method for conic constrained distributed optimization problems
We consider cooperative multi-agent consensus optimization problems over an undirected network of agents, where only those agents connected by an edge can directly communicate. The objective is to minimize the sum of agentspecific composite convex functions over agent-specific private conic constraint sets; hence, the optimal consensus decision should lie in the intersection of these private sets. We provide convergence rates in sub-optimality, infeasibility and consensus violation; examine the effect of underlying network topology on the convergence rates of the proposed decentralized algorithms; and show how to extend these methods to handle time-varying communication networks.

The authors consider the problem of decentralized optimization when each local objective is a composition of a convex smooth non-smooth term. They consider the saddle point formulation of this constrained optimization problem, and by considering its Lagrangian relaxation, propose a primal dual scheme to solve it. This method is particularly useful for cases where projection onto the local constraint set is easily computable. By deriving a closed form expression for the Lagrange multiplier update of this problem, an efficient algorithm is developed. It is shown that this method converge a primal dual optimal pair at a rate of $1/K$ when the primal vector is recursively averaged. Then the authors consider the extension of this method to a time-varying graph by incorporating an inner-loop communications protocol between neighboring agents, which leads to a modified dual update with several consensus steps per objective gradient update. The convergence guarantees of the static network case are generalized to the switching network case. Then, numerical analysis on a support vector classification problem demonstrate's the proposed methods empirical convergence behavior.
Generally speaking, there is not enough explanation of key quantities of interest in between the presentation of main results or ideas, e.g., at the beginning of Algorithm DPDA-S the intialization variables should also be explained (initialize primal variables, dual variables, learning rate, etc); The derivation of the fact that the dual variable update may be written in terms of the sequence $s_i^k$ requires more explanation (perhaps a display equation relating $s$ and $lambda$); Theorem 2.2 has no discussion...all theorems and lemmas need discussion/interpretation.  The paper is not very readable as a result. The extension of the proposed method to time-varying topology could be omitted so that more explication and context for the algorithm derivation could fit. Nonetheless, the technical contributions of the paper are substantial in extending the primal-dual method to composite constrained convex programming in decentralized (time-varying network) settings. Numerical analysis for a decentralized support vector classification problem on a static network demonstrate the proposed method's practical utility, but are not very convincing for practitioners -- I find the multivariate Gaussian data example to be a mere toy problem. A switching network might arise in a cloud computing or computer network in which case a problem such as a playback buffer, resource allocation, or data packet routing would be more appropriate.     --The bounded linear operator $T$ in equation (4) is left undefined, and had to be looked up in reference [11]. This makes the initial presentation of the proposed algorithm confusing, and not self-contained. Must $T$ be as in Theorem 1.1, or could it be any linear operator?  --Also, why does a 2 appear in front of the inner-product term $\langle T( 2 x^{k+1} - x^k), y \rangle$ in equation (6)? This scaling does NOT appear in the update formula for (7) in reference [11]. This may be a mistake, but its carried through equations in the rest of the paper.   -- In equation (8) the Lagrange multipliers $\lambda_{ij}$ are included in the optimization problem as a domain constraint but don't actually effect the objective or the constraint functions. Thus, this is probably a typo and should be deleted until the Lagrangian relaxation presented in (9).


Measuring the reliability of MCMC inference with bidirectional Monte Carlo
Markov chain Monte Carlo (MCMC) is one of the main workhorses of probabilistic inference, but it is notoriously hard to measure the quality of approximate posterior samples. This challenge is particularly salient in black box inference methods, which can hide details and obscure inference failures. In this work, we extend the recently introduced bidirectional Monte Carlo [GGA15] technique to evaluate MCMC-based posterior inference algorithms. By running annealed importance sampling (AIS) chains both from prior to posterior and vice versa on simulated data, we upper bound in expectation the symmetrized KL divergence between the true posterior distribution and the distribution of approximate samples. We integrate our method into two probabilistic programming languages, WebPPL [GS]  and Stan [CGHL+ p], and validate it on several models and datasets. As an example of how our method be used to guide the design of inference algorithms, we apply it to study the effectiveness of different model representations in WebPPL and Stan.

The authors propose a generic method for evaluation of MCMC transition operators based on a recently proposed method known as bidirectional Monte Carlo (BDMC). The BDMC method performs two annealed importance sampling (AIS) sweeps, which are essentially the same as those used in the Tempered Transitions (TT) method [Neal Stat & Comp 1996], but reinterprets the accumulated probabilities as a lower and upper bound on the marginal likelihood. This paper proposes to use the gap between these two bounds as a generic performance measure for MCMC transition operators. The presented technical contributions are implementations of BDMC for the Stan and WebPPL probabilistic programming systems, and a protocol for evaluation of quality of posterior inference.
This paper has some strong points and some not so strong points. The main strong point is that using BDMC to assess convergence of MCMC operators is a beautifully simple idea, and easy to implement, which in my opinion means that this work is potentially high impact. This is particularly true in the context of probabilistic programming systems, which indeed are the envisioned use case here, and I think all such systems would do well to at least implement this method.   My main point of concern lie with the novelty and the depth of the technical contribution. The authors cite an arxiv submission on BDMC as existing work, but (I think wisely) choose to devote a relatively large amount of space to reiterating its description. Unfortunately this does mean that the main technical contributions presented in sections 3.1 and 3.2 are somewhat rushed, and it is unfortunately also here where the writing quality slips a bit. As a result some issues of clarity arise (see below).  Clarity issues aside, the technical contributions in this paper are somewhat modest. Relative to the existing work on BDMC, the main novelty appears to the realization that the difference between the lower and upper bound is itself provides an upper bound on the Jensen-Shannon divergence. This is indeed a very nice observation, though arguably not an entirely novel once.   This brings me a point about related work. The authors, somewhat curiously, do not cite Radford Neal's work on Tempered Transitions [Statistics and Computing (1996) 6, 353-366], which is highly relevant in this context. This citation is also missing from the manuscript on BDMC, which makes me wonder whether the authors are in fact aware of it. If so, this is unfortunate in the sense that there is a missed opportunity in pointing out connections between the two methods.  TT performs the same reverse "heating" and forward "annealing" sweep as BDMC, but with the interest of a "tempering" MCMC transtion move to improve mixing in multimodal distributions. In TT the final sample from the reverse sweep is used as the initial sample for the forward sweep, and the final sample from the forward sweep is then used as a proposal which is accepted or rejected according to the ratio of importance weights accumulated in both sweeps.   The acceptance ratio in TT is, as far a I can tell, precisely the ratio of the stochastic lower and upper bound. Section 3.3 of Neal's paper in fact presents an interpretation of this acceptance ratio in terms of two thermodynamic integration estimates that bound log Z_T - log Z_1 from above and below, noting that a TT sampler will accept with probability 1 when the two converge.   The analogy with TT also leads me to wonder why the authors do spend more attention on the choice of β_t values. In TT it is well known that too large steps in Δβ = β_{t+1}-β_t will lead to a poor acceptance ratio, wheras too small steps Δβ waste computation. The ratio of the stochastic lower and upper bound in BDMC will also depend on Δβ and this provides some very clear intuitions. If your two bounds are not close enough, you need to rerun BDMC with smaller Δβ. Similarly, I suspect that comparisons between MCMC transition operators are likely to be unreliable when the gap between the log upper and lower bounds is larger >1 nat. When this is the case, one should again lower Δβ.   Ιt hasn't quite become clear to me from reading the paper what the "auto" part in AutoBDMC is, but I think there is an opportunity here to come up with some schedule for iteratively decreasing Δβ until sufficiently reliable inference results are obtained. In this respect the experiments could be a little more thorough. It would be good to see a systematic study of how the lower and upper bound vary with the step size.   These criticisms aside, I think this is not a bad paper. As noted above, I think the method proposed by the authors is potentially quite useful. Depending on whether the author's response can convincingly argue that issues with clarity and discussion of related work can be addressed by camera ready, I could be inclined to argue for acceptance.     Clarity  - The definition of $\vec{v}$ as "all of the variables sampled in AIS before the final stage" is rather loose. The authors presumably mean ${z_{1:T-1}, \theta_{1:T-1}$, but this is not immediately unambiguous since there are two AIS sweeps in the algorithm. This is not helped by the use of inconsistent labels "rev" and "fwd" on the one hand and "for" and "back" on the other (is this difference significant, or a mere accident?)  - The authors do not define what estimator \hat{p}(y) they are taking the expectation of in equations (5-7) and (9-11), which makes it difficult to follow how these equations arise. Do the authors mean to write E[log \hat{R}_{rev}] and [log \hat{R}_{fwd}]?   - Equation (6), as written, makes no sense (the distributions inside the KL divergence are identical). Based on equations (4) and (10), I take it the right terms should read something like p(z, θ | y) q_back(v | z, θ, y)?  - As far as I can tell the authors never unambiguously define what they mean by AutoBDMC. This is presumably means (a) caculating the difference between the stochastic lower and upper bound in order to assess convergence and (b) using the fixed hyperparameter scheme described in section 3.2?   Experiments  - It is now clear how β_t is chosen. Looking at the code in the appendix I see `var step = 1/options.steps`. I assume that AutoBDMC is therefore parameterized by the number of steps T to go from β_T = 1 to β_1 = 1 / T? It would be helpful to state this somewhere. In figures 2 and 3, does "HMC / No-U-Turn steps" indicate that BDMC is performed with this number of steps?   - The acronym RAIS, which presumably stands for reverse AIS, is not defined in the caption, nor are the variants (u)c(R)AIS, which I am guessing refer to (un)collapsed variants?  - It would be helpful to increase the size of the axes and tick labels.   Typos  - Two closing parens in the D_KL terms are missing in the expression for D_JS - In the NUTS citation: Homan -> Hoffman


Maximal Sparsity with Deep Networks?
The iterations of many sparse estimation algorithms are comprised of a fixed linear filter cascaded with a thresholding nonlinearity, which collectively resemble a typical neural network layer. Consequently, a lengthy sequence of algorithm iterations can be viewed as a deep network with shared, hand-crafted layer weights. It is therefore quite natural to examine the degree to which a learned network model might act as a viable surrogate for traditional sparse estimation in domains where ample training data is available. While the possibility of a reduced computational budget is readily apparent when a ceiling is imposed on the number of layers, our work primarily focuses on estimation accuracy. In particular, it is well-known that when a signal dictionary has coherent columns, as quantified by a large RIP constant, then most tractable iterative algorithms are unable to find maximally sparse representations. In contrast, we demonstrate both theoretically and empirically the potential for a trained deep network to recover minimal 0 -norm representations in regimes where existing methods fail. The resulting system, which can effectively learn novel iterative sparse estimation algorithms, is deployed on a practical photometric stereo estimation problem, where the goal is to remove sparse outliers that can disrupt the estimation of surface normals from a 3D scene.

This paper investigates the problem of sparse recovery of a linear system with unknown dictionary. Given a set of training signal-response pairs generated from an unknown linear system, the authors introduce a predictive sparse recovery model based on deep neural network (DNN) training. The main idea is to assume that the sparse signal can be encoded by a deep network built over the observation. Inspired by iterative hard-thresholding (IHT), the hard-thresholding operation is applied as activation function on the hidden layers to induce the nonlinearity. Numerical results on synthetic data and photometric stereo applications are reported to show the actual performance of the proposed model.
The idea of using DNN to approximately unfold the IHT iteration procedure is interesting and novel as far as I am aware of. I have, however, the following major concerns on its content:  (1) Motivation is unconvincing: The training pairs (x, y) are assumed to obey a linear system y=Ax with unknown measurement matrix A. Given ample training data, why should we bother to use DNN-type models if A can simply be estimated via, e.g., least squared regression? Apparently, when A is recovered we may then apply IHT on a testing response vector with the estimated measurement matrix to approximately recover the sparse signal. Actually, I note the simulation study uses a training set of size 600,000 and the measurement matrix is only of size 20x100 << 600,000. In such a setting, it should not be uneasy to accurately estimate A from the data.   (2) Another major concern is its readability. The somewhat overcomplicated writing style in fact underscores the limited innovation of model and algorithmic contributions. Throughout the paper, I cannot find a clear description of the network structure and its optimization algorithm.   (3) There is a clear gap between theory and applications. The main sparse recovery result is about parameter estimation error in a noiseless linear system. The presented application of multi-resolution sparse estimation is defined as a multi-label classification problem to which it remains unclear whether the developed theory can apply.


FPNN: Field Probing Neural Networks for 3D Data
Building discriminative representations for 3D data has been an important task in computer graphics and computer vision research. Convolutional Neural Networks (CNNs) have shown to operate on 2D images with great success for a variety of tasks. Lifting convolution operators to 3D (3DCNNs) seems like a plausible and promising next step. Unfortunately, the computational complexity of 3D CNNs grows cubically with respect to voxel resolution. Moreover, since most 3D geometry representations are boundary based, occupied regions do not increase proportionately with the size of the discretization, resulting in wasted computation. In this work, we represent 3D spaces as volumetric fields, and propose a novel design that employs field probing filters to efficiently extract features from them. Each field probing filter is a set of probing points -sensors that perceive the space. Our learning algorithm optimizes not only the weights associated with the probing points, but also their locations, which deforms the shape of the probing filters and adaptively distributes them in 3D space. The optimized probing points sense the 3D space "intelligently", rather than operating blindly over the entire domain. We show that field probing is significantly more efficient than 3DCNNs, while providing state-of-the-art performance, on classification tasks for 3D object recognition benchmark datasets.

This paper focuses on the problem of object classification based on 3D data that one could acquire with the use of some sort of range sensor. The basic idea is that for 3D data one would ideally like to have 3D filters in a convolutional neural network. However, 3D filters are quite expensive to be optimized and used, and they are pretty wasteful since the majority of the input will probably be zeros (only near the object surface there are non-zero values). As a note here, the input is a 3D rectangle split into a 3D grid, where the original object point cloud is converted in a simplified, binarized form (you can think of it as building the surface of an object using LEGO blocks).  As such the paper proposes a field probing layer which replaces the convolutional layers with a combination of three "sub-layers": a) a sensor layer, b) a dot-product layer and c) a gaussian layer. The sensor layer learns to place sampling points/probes on the 3D rectangle space, which would be equivalent to learning the sampling/convolutional locations in a ConvNet. The gaussian layer implements a gaussian function that smoothens the sampling points from the sensor layer. And the dot-product is a standard inner product layer to compute the inner product between the sampling layer and some learnt weights. Interestingly, the weights are not shared in the field probing layer, meaning that in different locations different "probing filters" are learnt. A strong positive point is that the proposed approach can learn complex interaction between surface points, even if they are not neighboring ones. The reason is that the sensor layers are not constrained to place the probes within a confined space.
The paper is quite interesting, although not entirely novel or delivering impressive results. The list of positive and negative points is the following.  Positive points ------------------- + The problem is interesting and an approach that could deliver some good results would definitely have some impact. Indeed, relying on filters with cubic complexity for 3DCNN is unnecessarily intense, since most of the input space is composed of zeros. Learning more "surface-focused" filters make much more sense. + Modelling long-range surface dependencies is sensible, especially when scaling up to datasets with hundreds of categories. + The presentation of the paper is very good and the method is very clear.  Negative points ------------------- - In its current form a field probing can be viewed as some sort of sparse tensor encoding. As such it is not clear what is the novelty as compared to having a sparse encoding of the input space and the weight filters. This is also kind of supported by figure 5, where indeed the proposed method has a straight line (the same one would expect by a sparse encoding), whereas a 3D convolutional network naturally feature a cubic complexity. One difference I can identify is the fact that long-range dependencies can be learnt, namely that the the probing locations are learnt instead of just defined. Is that a difference and is it the only main difference? - It is not clear how can you learn meaningful long-range dependencies within such an unconstrained setting. With normal convolutional filters there exists a very strong constraint, namely the inputs should be nearby. In your case how can you make sure the network does not get stuck in a poor local optimum? - The fact that the weights in the field probing layers are not shared means that they are tied to the particular locations of the input. How generalizable is that? When you are presented with a random point cloud for which you cannot know the rotation as compared to the ground truth, how could these filters cope with the variance? - Extending on the point, how do you perform the rotation changes in Table 2? Do you apply the same, fixed rotation to all objects or do you rotate randomly per test object? If it is the former, does the experiment bring any insight, as I assume that the classification should be more or less similar for any random (but common across examples) viewpoint. If it is the latter, what is the difference between R and R_15? - Also, since the weights are not shared, does the network generalize for objects of different sizes, since in that case one would naturally need to increase the granularity so that to be able to maintain the appopriate level of detail for the recognition? If this cannot be guaranteed, doesn't this go against the point of having a more efficient network so that one can increase the grid resolution on demand?  Overall I believe the paper is quite interesting and although by itself does not excel, it could serve as a potential inspiration to other researchers.


Algorithms and matching lower bounds for approximately-convex optimization
In recent years, a rapidly increasing number of applications in practice requires optimizing non-convex objectives, like training neural networks, learning graphical models, maximum likelihood estimation. Though simple heuristics such as gradient descent with very few modifications tend to work well, theoretical understanding is very weak. We consider possibly the most natural class of non-convex functions where one could hope to obtain provable guarantees: functions that are "approximately convex", i.e. functionsf : R d → R for which there exists a convex function f such that for all x, |f (x) − f (x)| ≤ ∆ for a fixed value ∆. We then want to minimizẽ f , i.e. output a pointx such thatf (x) ≤ min xf (x) + . It is quite natural to conjecture that for fixed , the problem gets harder for larger ∆, however, the exact dependency of and ∆ is not known. In this paper, we significantly improve the known lower bound on ∆ as a function of and an algorithm matching this lower bound for a natural class of convex bodies. More precisely, we identify a function T : R + → R + such that when ∆ = O(T ( )), we can give an algorithm that outputs a pointx such thatf (x) ≤ min xf (x) + within time poly d, 1 . On the other hand, when ∆ = Ω(T ( )), we also prove an information theoretic lower bound that any algorithm that outputs such ax must use super polynomial number of evaluations off .
Trivially, one can solve the problem by constructing a -net and search through all the net points. However, such an algorithm requires Ω 1 d evaluations off , which is highly inefficient in high dimension. In this paper, we are interested in efficient algorithms: algorithms that run in time poly d, 1 (in particular, this implies the algorithm makes poly d, 1 evaluations off ).
One extreme case of the problem is ∆ = 0, which is just standard convex optimization, where algorithms exist to solve it in polynomial time for every > 0. However, even when ∆ is any quantity > 0, none of these algorithms extend without modification. (Indeed, we are not imposing any structure onf − f like stochasticity.) Of course, when ∆ = +∞, the problem includes any non-convex optimization, where we cannot hope for an efficient solution for any finite . Therefore, the crucial quantity to study is the optimal tradeoff of and ∆: For which , ∆ the problem can be solved in polynomial time, and for which it can not.
In this paper, we study the rate of ∆ as a function of : We identify a function T : R + → R + such that when ∆ = O(T ( )), we can give an algorithm that outputs a pointx such thatf (x) ≤ min xf (x) + within time poly d, 1 over a natural class of well-conditioned convex bodies. On the other hand, when ∆ =Ω(T ( )) 1 , we also prove an information theoretic lower bound that any algorithm outputs suchx must use super polynomial number of evaluations off . Our result can be summarized as the following two theorems:
Theorem (Algorithmic upper bound, informal). There exists an algorithm A that for any functionf over a well-conditioned convex set in R d of diameter 1 which is ∆ close to an 1-Lipschitz convex function 2 f , and
A finds a pointx such thatf (x) ≤ min xf (x) + within time poly d,

This paper studies the problem of minimizing approximately convex functions by zero order methods. This problem adds one extra layer of difficulty to the traditional exact minimization methods, given by the (uniform) distance ∆ of the objective to a convex 1-Lipschitz function. The main contribution is providing tight upper and lower bounds on the complexity of this problem for all regimes of inexactness ∆, accuracy ε, and dimension d. To the best of my knowledge, all lower bounds (except for Case 3, which is somewhat simpler) are new, and the same is true for the algorithm proposed for the regime ε/d ≼ ∆ ≼ ε^2/\sqrt{d}. This paper considerably extends the understanding of approximate convex optimization by some novel ideas: • The lower bound consists on hiding a linear function on a random slab. The improvement achieved in the paper relies on constructing a convex function outside the slab, which is key on the new results (previous extensions were only linear). • The upper bound is based on stochastic smoothing and gradient methods. By the divergence theorem, one can obtain an approximate gradient by estimating finite differences on random unit vectors, and this approximation is robust to the inexactness introduced by ∆. Although this idea is not entirely new (e.g., the cited work of Shamir), the analysis itself seems to be original, and it is studied in the inexact setup, which is certainly new. Overall, I believe this paper is very interesting, and therefore I strongly recommend its publication at NIPS. However, I have to mention some very mild criticism, particularly on the related literature and the writing style.
First I would like to justify my ratings for this paper. I believe the quality of the paper is indeed high: it resolves open questions from the literature, it brings new ideas in the construction of hard instances for convex programming, and can hopefully impact related areas. However, there are some aspects that deserve more attention, particularly I think writing requires some extra effort.   I would like to describe some high-level criticism for the work:  1. The paper emphasizes the differences between approximately convex minimization and various oracle models; however, it is not mentioned that approximately convex minimization is equivalent to minimization of convex functions with an inexact zero order method. This is somehow a more general setup (as the oracle does not need to be consistent with information among iterations) and it is arguably simpler to grasp.  3. The paper needs some writing revision. Aside from correcting typos, it would be nice to give some insights on the intriguing new regime ∆ = O( ε^2 / sqrt{d}), and perhaps expand the motivating discussions on proof techniques (Section 3.1) and the informal discussion on the lower bound, at the beginning of Section 4.  Below are some minor comments:  1. The Abstract and first lines of the Introduction stress the relevance of optimization with nonconvex objectives, mentioning applications such as neural networks, graphical models, clustering, etc. Algorithms for minimizing approximately convex functions are not applicable to these problems, so it seems inappropriate to mention these applications.  2. In the paper it should be mentioned the very first reference on inexact zero order methods from the Nemirovsky and Yudin monograph (Exercise 1 in page 360). This example provides the same superpolynomial lower bound when ∆ = Ω(ε/ sqrt{d}) (in line 77) for the unit Euclidean ball (instead of the hypercube), and it fits precisely within the model of the paper.  3. Line 145. The definition of the lower convex envelope is somewhat confusing. First, some form of differentiability is required so that the formula makes sense. Second, the remark: “When the input function F is convex, LCE(F) can be considered an extension of F to the entire R^d” is used incorrectly, as it is later used for a function defined on a nonconvex set (first line of proof of Lemma 4.2). I suggest to prove “by hand” that LCE and the function coincide in the proof, as it is easy and does not lead to ambiguities.  4. Line 282. “Thus, for every point \tilde u_1, there exists[..]”. Please add more details on this, as it is an important technical step in the proof.


Dynamic Network Surgery for Efficient DNNs
Deep learning has become a ubiquitous technology to improve machine intelligence. However, most of the existing deep models are structurally very complex, making them difficult to be deployed on the mobile platforms with limited computational power. In this paper, we propose a novel network compression method called dynamic network surgery, which can remarkably reduce the network complexity by making on-the-fly connection pruning. Unlike the previous methods which accomplish this task in a greedy way, we properly incorporate connection splicing into the whole process to avoid incorrect pruning and make it as a continual network maintenance. The effectiveness of our method is proved with experiments. Without any accuracy loss, our method can efficiently compress the number of parameters in LeNet-5 and AlexNet by a factor of 108× and 17.7× respectively, proving that it outperforms the recent pruning method by considerable margins. Code and some models are available at https://github.com/yiwenguo/Dynamic-Network-Surgery.
damage. Since the pruned connections have no chance to come back, incorrect pruning may cause severe accuracy loss. In consequence, the compression rate must be over suppressed to avoid such loss. Another issue is learning inefficiency. As in the paper [9], several iterations of alternate pruning and retraining are necessary to get a fair compression rate on AlexNet, while each retraining process consists of millions of iterations, which can be very time consuming.
In this paper, we attempt to address these issues and pursue the compression limit of the pruning method. To be more specific, we propose to sever redundant connections by means of continual network maintenance, which we call dynamic network surgery. The proposed method involves two key operations: pruning and splicing, conducted with two different purposes. Apparently, the pruning operation is made to compress network models, but over pruning or incorrect pruning should be responsible for the accuracy loss. In order to compensate the unexpected loss, we properly incorporate the splicing operation into network surgery, and thus enabling connection recovery once the pruned connections are found to be important any time. These two operations are integrated together by updating parameter importance whenever necessary, making our method dynamic.
In fact, the above strategies help to make the whole process flexible. They are beneficial not only to better approach the compression limit, but also to improve the learning efficiency, which will be validated in Section 4. In our method, pruning and splicing naturally constitute a circular procedure and dynamically divide the network connections into two categories, akin to the synthesis of excitatory and inhibitory neurotransmitter in human nervous systems [17].
The rest of this paper is structured as follows. In Section 2, we introduce the related methods of DNN compression by briefly discussing their merits and demerits. In Section 3, we highlight our intuition of dynamic network surgery and introduce its implementation details. Section 4 experimentally analyses our method and Section 5 draws the conclusions. The pipeline of (a) our dynamic network surgery and (b) Han et al.'s method [9], using AlexNet as an example. [9] needs more than 4800K iterations to get a fair compression rate (9×), while our method runs only 700K iterations to yield a significantly better result (17.7×) with comparable prediction accuracy.

This paper proposes a method for DNN compression which is essential for reducing the storage and processing power requirements of very deep networks.  The state-of-the-art method on deep network compression uses a variety of hashing and quantization mechanisms followed by a network pruning and retraining phase. In this paper, the authors propose a method for network pruning and retraining phase of the deep network compression method of Han et al. by introducing the option of restoring previously pruned connections. They show the efficiency of their method through experimental evaluation on several benchmark datasets.  Reported results suggest that their method performs better in terms of compression rate vs. accuracy trade-off in most of the benchmarks.
I think having the option of restoring the pruned connections is a very good idea and apparently it works well on the experimented benchmarks.  On the other hand, the argument that Han et al's method can cause permanent damage in the network does not seem exactly accurate. Because, actually their network performs better than yours in  LeNet-300-100 Ref vs LeNet-300-100 Pruned (with 0.05 loss whereas your network has 0.29 loss). This is also the case for AlexNet. Their accuracy gap is better than yours.  So, I don't think there is a permanent damage issue there. Secondly, they make multiple sessions of pruning and retraining so the network always has the opportunity to compensate for the erroneous pruning.  Your method is better in terms of run time because it requires less iterations, yet, it also brings additional space requirements that do not exist in Han et al.'s method. I think a small discussion on that issue would be great. You report results on all the benchmark datasets Han et al. used except for VGG-16. I wonder what is the reason for skipping VGG-16, which is the deepest and largest network.   In Line 100, you state "For the convolutional layers with learnable kernels, we unfold the coefficients of each kernel into a vector and concatenate all of them to Wk as a matrix." What is the reason for this?  I think the manuscript lacks bunch of details about the parameter selection which might be very relevant to the audience. In Algorithm 1, the parameter \alpha is used for updating the learning \beta. This parameter is not explained. Which learning rate update scheme do you use? For updating the parameters, you state that you have tested several candidates and decided upon magnitude-based elimination. Can you elaborate on what other schemes you have tested? On line 146, you say "the absolute value of the input". I think it should be the absolute value of the weight or parameter. The input is not the same as weights. How you decide on thresholds for parameter importance is left unclear. Can you provide more details on how you select a_k, b_k and t? Also, what scheme do you use to decay the update probability \sigma(iter) On line 188, you say three-layer network but according to your definition this is a two layer network. In Han et al. paper, they change the dropout probability during compression. How do you deal with dropout in your method? In Section 4.1, there is a toy example that depicts how your method works. It'd be great to see how Han et al.'s method would do on this example.  Minor comments: The figures look really nice in the paper but they are unreadable when printed BW.


Optimal spectral transportation with application to music transcription
Many spectral unmixing methods rely on the non-negative decomposition of spectral data onto a dictionary of spectral templates. In particular, state-of-the-art music transcription systems decompose the spectrogram of the input signal onto a dictionary of representative note spectra. The typical measures of fit used to quantify the adequacy of the decomposition compare the data and template entries frequency-wise. As such, small displacements of energy from a frequency bin to another as well as variations of timbre can disproportionally harm the fit. We address these issues by means of optimal transportation and propose a new measure of fit that treats the frequency distributions of energy holistically as opposed to frequency-wise. Building on the harmonic nature of sound, the new measure is invariant to shifts of energy to harmonically-related frequencies, as well as to small and local displacements of energy. Equipped with this new measure of fit, the dictionary of note templates can be considerably simplified to a set of Dirac vectors located at the target fundamental frequencies (musical pitch values). This in turns gives ground to a very fast and simple decomposition algorithm that achieves state-of-the-art performance on real musical data.

In this paper, spectral unmixing problem for music transcription is formulated as a matrix factorization problem like NMF. In contrast to existing approaches, e.g., PLCA that minimizes KL-divergence between an original spectrogram and multiplication of decomposed factors, authors take into account a small displacement in frequency domain and modeled this displacement by using a transportation matrix. The spectral unmixing is formulated as an optimization problem that minimizes a total transportation cost for a given dictionary matrix. This can be regarded as a natural extension of EMD-NMF which is special case of the proposed method when a cost matrix C is a symmetric matrix. The proposed cost function can be efficiently minimized by a closed-form solution if a set of Dirac vectors are used for a dictionary matrix. In the experiments, authors demonstrated that OST with regularization outperformed PLCA in terms of both accuracy and efficiency.
-Technical correctness and novelty The ideas of modeling frequency displacement by a transportation matrix and obtaining the optimal displacement by closed-form solution are very interesting. (But I am not sure if these ideas have already been proposed in preceding studies or not.) It is also interesting that a harmonic nature can be naturally modeled by a cost matrix C. As suggested by authors, much further investigation on training C from data is an interesting direction as a future work. From a critical point of view, authors do not show a certain cases where group regularization is beneficial for real data. As long as seeing section 6.2 and Table 1, OST with entropic regularization is sufficient to deal with real data. Much deeper investigation on group regularization for real-data is desired.  From another aspect, I am a little concerned about the fact that the dictionary matrix is completely hand-crafted and the frequency displacement model is also data-independent. Can OST outperform methods that use data-dependent dictionary that trained from labeled data?  -Usefulness It is very interesting that there exists closed form solution for OST. Thanks to this fact, the proposed method achieves significantly faster processing compared to PLCA. Reviewer thinks that this characteristic is very practical for real applications. On the other hand, I am not sure if OST really outperforms any other preceding works in terms of accuracy, because OST was compared with only a single method, PLCA, which was proposed in 2006 and may be outdated.  -Clarity This paper is quite well written. Although I do not confident in acoustic signal processing (instead, I confident in matrix factorization problem including NMF), it was very easy to understand technical details. The proposed procedure is correctly formulated by mathematical notations. I found a minor mistake in line 3-4 in Section 3. The variable 'N' in the following sentence, i,j=1, ..., N, must be 'M'.  %% After Rebuttal %%   The reviewer thanks to the authors for their detailed comments. I agree that this paper places importance on the theory part rather than experiments for the practical usefulness. This is a very good paper which should be published from NIPS.


Learning Sensor Multiplexing Design through Back-propagation
Recent progress on many imaging and vision tasks has been driven by the use of deep feed-forward neural networks, which are trained by propagating gradients of a loss defined on the final output, back through the network up to the first layer that operates directly on the image. We propose back-propagating one step further-to learn camera sensor designs jointly with networks that carry out inference on the images they capture. In this paper, we specifically consider the design and inference problems in a typical color camera-where the sensor is able to measure only one color channel at each pixel location, and computational inference is required to reconstruct a full color image. We learn the camera sensor's color multiplexing pattern by encoding it as layer whose learnable weights determine which color channel, from among a fixed set, will be measured at each location. These weights are jointly trained with those of a reconstruction network that operates on the corresponding sensor measurements to produce a full color image. Our network achieves significant improvements in accuracy over the traditional Bayer pattern used in most color cameras. It automatically learns to employ a sparse color measurement approach similar to that of a recent design, and moreover, improves upon that design by learning an optimal layout for these measurements.

Learning Sensor Multiplexing Design through Back-propagation  This paper proposes to learn jointly i) the color filter pattern of RGB camera sensors and ii) the color reconstruction of a final output image, using a neural network framework. Contributions have potential high impact in camera designing, since the concept of automatically learning the color filter pattern moves the industry beyond the arbitrary Bayer filter, and the jointly learned color reconstruction offers better image quality in terms of psnr. Experiments are following previous literature and evaluate noise distributions in various camera sensor designs.
Paper is nicely written, methodology is easy to follow, and validation shows the benefit of the proposed approach. Comments below:  One main question that could arise in learning automatically a color filter pattern, is the dependence on the dataset. Here benchmarking was performed with the Gehler-Shi dataset. Would the use of a different heterogeneous dataset produce a completely different sensor pattern?  If the pattern changes significantly between training sets, would it be relevant to question the robustness of a chosen pattern?   On the same note, how could this impact camera designs, ie., could a universal pattern exist, or would there be multiple patterns, each designed for particular situations (could patterns designed for outdoor, indoor, urban or nature settings, produce a much better image quality than using a universal pattern?)  In the experiments, the pattern was chosen to be learned with one particular noise level. Would learning in different noise levels produce drastically different patterns? If so, why one would choose one particular pattern to another? Would there be a trend in generated sensors across different noise levels that could be learned?   A temperature factor is used when learning the initial sensor layer. What would happen if this factor \alpha_t is sensibly slower/faster than the chosen warming rate?  Does the rate of increase of this factor condition the number of effective possible iterations during training? How is \gamma chosen?  In the reconstruction network, what is the motivation for learning the color combinations and their gating values in two separate paths?  What is the meaning of K, why is K chosen to be 24 in the experiments?  Is the log-domain used to ensure positiveness of the color channels?  In the second path, for learning the gating values \lambda, why the number of layers is chosen to be 128?   Perhaps a naive question: why choosing nips over a vision/photography conference?


MetaGrad: Multiple Learning Rates in Online Learning
In online convex optimization it is well known that certain subclasses of objective functions are much easier than arbitrary convex functions. We are interested in designing adaptive methods that can automatically get fast rates in as many such subclasses as possible, without any manual tuning. Previous adaptive methods are able to interpolate between strongly convex and general convex functions. We present a new method, MetaGrad, that adapts to a much broader class of functions, including exp-concave and strongly convex functions, but also various types of stochastic and non-stochastic functions without any curvature. For instance, MetaGrad can achieve logarithmic regret on the unregularized hinge loss, even though it has no curvature, if the data come from a favourable probability distribution. MetaGrad's main feature is that it simultaneously considers multiple learning rates. Unlike previous methods with provable regret guarantees, however, its learning rates are not monotonically decreasing over time and are not tuned based on a theoretically derived bound on the regret. Instead, they are weighted directly proportional to their empirical performance on the data using a tilted exponential weights master algorithm.

Online convex minimization is a fundamental problem in learning theory with various applications. Here, we are given a sequence of convex functions f1,...,fT in an online fashion and we seek to predict the minimizer of the sum so as to minimize the regret. This is a very well studied question with various guarantees known even when only given gradient information about the functions fi. For general functions this leads to a regret of sqrt(T) and for some special cases, we know better guarantees: O(log T) for strongly convex functions, O(d log T) for exp-convex functions etc. However, the algorithms that achieve the latter better guarantees are quite different and in some cases need to know bounds on the specific parameters of the functions (e.g., strong-convexity of the fi's, although this can now be avoided).   The paper seeks to present a single algorithm which with a small overhead, achieves the right dependence on T for all the above cases. As such, this is a very nice research direction.  Algorithm: The algorithm uses several techniques from previous works and while not very novel combines previous ideas in a clever way. The high level idea is to run several slave algorithms (that sort of correspond to using different parameter ranges for the base learning algorithm) and then use a master algorithm that combines the slave algorithms by using a modified exponential-weights method. This sort of approach of using two levels of learning algorithms is not particularly new to this approach, but the exact application is non-trivial.  Writing: The paper is written reasonably well but the high-level description of the algorithm can use more work. Specifically, it is too vague in some very important places (e.g., "tilted by learning rates") and assumes too much familiarity with previous work ("adaptive regularizer", "temporal ...") that muddles the presentation. It is important to connect to previous works, but it will be more helpful for the reader if such terms can be explained and if not they do not seem to help much.
The problem and algorithm are nice but there are two important drawbacks: the new algorithm does not seem to cover (i.e., obtain better than O(sqrt(T)) guarantee) any interesting new cases; the bound for strongly-convex is worse than what is known already. Even with these drawbacks, I recommend the paper be accepted as the problem is quite nice and the approch could be useful in the future.