Semi-Supervised Learning with Declaratively Specified Entropy Constraints
We propose a technique for declaratively specifying strategies for semi-supervised learning (SSL). SSL methods based on different assumptions perform differently on different tasks, which leads to difficulties applying them in practice. In this paper, we propose to use entropy to unify many types of constraints. Our method can be used to easily specify ensembles of semi-supervised learners, as well as agreement constraints and entropic regularization constraints between these learners, and can be used to model both well-known heuristics such as co-training, and novel domain-specific heuristics. Besides, our model is flexible as to the underlying learning mechanism. Compared to prior frameworks for specifying SSL techniques, our technique achieves consistent improvements on a suite of well-studied SSL benchmarks, and obtains a new state-of-the-art result on a difficult relation extraction task.

This paper proposes a method to combine (or ensemble) several SSL heuristics (regularizers) by using a Bayesian optimization approach. The basic idea of the proposed method borrowed from the previous method called D-Learner, which is declared in this paper. Therefore, the proposed method is basically a modification or extension of D-Learner, which seems not to be totally novel. In this perspective, this paper is rather incremental than innovative.  The experimental results look fairly well comparing with the methods in previous studies including the baseline D-Learner on the tasks of text classification and relation extraction examined in this paper.  The followings are my main concerns of this paper. If I misunderstand something, please provide rebuttals to my concerns. I am willing to change my score if I think the authors’ rebuttal is reasonable.   1, The proposed method seems to deeply depend on the TensorLog framework. From this perspective, I somehow feel that this paper is rather a technical report than a scientific paper. I would like authors to clearly explain that what the central theory of the proposed method is, or what the unique idea of the proposed method is. For example, the idea of incorporating many SSL regularizers together in a single leaner is an idea of the previous study.   2, For example, the DCE(Supervised) in Table 2(b), which is the baseline of the proposed method, already outperformed the performance for all the methods in Table 2(a). I wonder why the baseline of the proposed method got such higher performance. From this point, I concern the experimental settings used in this paper whether the comparison was performed in a fair condition.   3, The results for DCE (Ours) shown in Tables 1 (a) and (b) differs. I cannot figure out why this difference was induced from. Please clearly explain this.  After reading author's feedback: Thank you for answering my concerns/questions. Unfortunately, I am still not totally convinced the novelty of the proposed method. Moreover, the reason of the supervised DCE-Learner has already outperformed the most of previous methods is still not super clear for me. Is there any particular reason why any of previous methods did not use SL DCE-Learner (or similar strong baseline SL methods)? It is a bit hard for me to "strongly" recommend this paper for the acceptance. In addition, I would recommend authors to report the results of both splits to prevent the misunderstanding in the next version.


Data-Efficient Hierarchical Reinforcement Learning
Hierarchical reinforcement learning (HRL) is a promising approach to extend traditional reinforcement learning (RL) methods to solve more complex tasks. Yet, the majority of current HRL methods require careful task-specific design and on-policy training, making them difficult to apply in real-world scenarios. In this paper, we study how we can develop HRL algorithms that are general, in that they do not make onerous additional assumptions beyond standard RL algorithms, and efficient, in the sense that they can be used with modest numbers of interaction samples, making them suitable for real-world problems such as robotic control. For generality, we develop a scheme where lower-level controllers are supervised with goals that are learned and proposed automatically by the higher-level controllers. To address efficiency, we propose to use off-policy experience for both higherand lower-level training. This poses a considerable challenge, since changes to the lower-level behaviors change the action space for the higher-level policy, and we introduce an off-policy correction to remedy this challenge. This allows us to take advantage of recent advances in off-policy model-free RL to learn both higher-and lower-level policies using substantially fewer environment interactions than on-policy algorithms. We term the resulting HRL agent HIRO and find that it is generally applicable and highly sample-efficient. Our experiments show that HIRO can be used to learn highly complex behaviors for simulated robots, such as pushing objects and utilizing them to reach target locations, 1 learning from only a few million samples, equivalent to a few days of real-time interaction. In comparisons with a number of prior HRL methods, we find that our approach substantially outperforms previous state-of-the-art techniques.

Summary =======  The authors present a heirarchical reinforcement learning approach which learns at two levels, a higher level agent that is learning to perform actions in the form of medium term goals (changes in the state variable) and a low level agent that is aiming to (and rewarded for) achieving these medium term goals by performing atomic level actions.  The key contributions identified by the authors are that learning at both lower and higher level are off-policy and take advantage of recent developments in off-policy learning. The authors say that the more challenging aspect of this, is the off policy learning at the higher level, as the actions (sub-goals) chosen during early experience are not effectively met by the low level policy. Their solution is to instead replace (or augment) high level experience with synthetic high level actions (sub-goals) which would be more likely to have happened based on the current instantiation of the low level controller.  An additional key feature is that the sub-goals, rather than given in terms of absolute (observed) states, are instead given in terms of relative states (deltas), and there is a mechanism to update this sub-goal appropirately as the low level controller advances. As a side note, I suggest the authors look to the work of George Konidaris, in particular (Konidaris and Barto, 2007), in which agent centric formalisms are championed as most suitable for transfer learning (something the authors say they are interested in investigating in future work).  My concerns are relatively minor and a brief overview is as follows:   * I think that the cited paper [24] (though unpublished), is slightly closer to this work than the authors allow.   * There are some minor technical details, including that relating to prior work (e.g.  DDPG method [25], UVFA [34]), which could be made clearer and more explicit (see below).   * Some notation could be described more fully, and the reader guided a little to anticipate the meaning and use of certain constructions (e.g. the notation for subsequences of time series and sums over such series should be explicit, the fact that the raw tuples stored for the higher level controller will be later processed into conventional off-policy RL tuples).   * Some ideas are reiterated more than once or descriptions could be tightened up a bit, (e.g. Sections 3.2 and 3.3).  While I have a few comments on the details of the paper, I think the paper is well written and the work reproducible. The experiments, although in a limited domain, are varied and the results convincing. Moreover, I believe it presents a significant and interesting contribution to the field, and I am recommending it for acceptance.   Local references ---------------------  George Konidaris, Andrew G Barto (2007). Building Portable Options: Skill Transfer in Reinforcement Learning. IJCAI.  Details ====== ### p3, line 104, minor point. > ...for fixed σ...  I assume this is standard deviation, but it is fairly common practice to parametrise normal distributions by variance.  ### p3, line 111 > ...specify a potentially infinite set of lower-level policies...  To be precise, I would suggest *unbounded* rather than *potentially infinite*.  ### p3, line 122 > a fixed goal transition function $g_t = h(s_{t−1} , g_{t−1 }, s_t )$  This is quite an unusual feature and is necessary here because goals (being relative) are not static. It would help to tell the reader that here.  ### p4, line 130-1 > the higher-level transition (s t:t+c−1 , g t:t+c−1 , a t:t+c−1 , R t:t+c−1 , s t+c )  The reader might be guided here too.  ### p4, line 139-40 > To maintain the same absolute position of the goal regardless of state change, the goal transition model h is defined as  Being a little picky maybe, but...What are the assumptions on your state space that ensures a relative goal can be (always) defined? And that when updated in the way suggested, will always be meaningful/consistent?   ### p5, lines 154-8, clarity > Parameterized rewards are not a new concept, and have been studied previously [34, 19]. They are a natural choice for a generally applicable HRL method and have therefore appeared as components of other HRL methods [43, 22, 30]. A significant distinction between our method and these prior approaches is that we directly use the state observation as the goal, and changes in the state observation as the action space for the higher-level policy...  The (unpublished) work [24], cited elsewhere does use states as sub-goals, and also as actions in the higher level policy. But not changes in the state as done here.    ### p5, line 159, typo > This allows the lower-level policy to begin receiving reward signals immediately, even before the lower-level policy has figured out how to reach the goal...  I think the second *lower-level*, should be *higher-level*.  ### p5, line 164, citation needed > While a number of prior works have proposed two-level HRL architectures  ### p5, line 169-70, citation needed > off-policy algorithms (often based on some variant of Q-function learning) generally exhibit substantially better sample efficiency than on-policy actor-critic or policy gradient variants.  ### p5, line 191 and Equation (5) > log probability...computed as...  I think the proportional to sign should be an equals, and this relies on an assumption about the sampling policy that should be made explicity, e.g. as a mean action with isotropic gaussian noise.  This part of the method seems the most tied to a specific formulation of the learner, and I wonder whether it could defined in more general terms first, with this as the example (I realise there are other approaches in the supplementary file). Also, authors should be clear at the outset that this is an approximate optimisation.  ### p5, line 194 > ...eight candidate goals sampled randomly from a Gaussian...  What is the Gaussian's covariance matrix?  ### p6, line 223-4, clarification when referring to ref [24] > ...[24]. In this work, the multiple layers of policies are trained jointly, while ignoring the non-stationarity problem.  The work cited is unpublished (arXiv), but I happen to have read it. In the cited work, the authors adapt a published method Hindsight Experience Replay (HER), which in their own work is used to replace the goals at lower levels with the actually achieved states. This contrasts with the currently submitted paper, in that the action of the higher level agent is the replaced entity (rather than the lower level goal), but in other respects there is some degree of similarity which could be remarked upon.  ### p6, line 231 > In our experiments, we found this technique to under-perform compared to our approach, which uses the state in its raw form. We find that this has a number of benefits.  This could be clearer.  ### p7, Figure 4 caption > extremely fast...  Use "quickly" or "rapidly".  Post Rebuttal ============  Reviewer #3's question regarding the significance of one of the results in Table 1 is valid. I still think the paper is interesting and worthy of publication, but I have moved my recommendation down to a 7 to reflect this concern.  If accepted for publication, I would encourage the authors to make clear in the camera ready version whether the values are standard deviations or standard errors or something else. The authors should also make clear their justification for the significance claim.


A probabilistic population code based on neural samples
Sensory processing is often characterized as implementing probabilistic inference: networks of neurons compute posterior beliefs over unobserved causes given the sensory inputs. How these beliefs are computed and represented by neural responses is much-debated (Fiser et al. 2010 . Here, we show that these alternatives -contrary to common assumptions -are not mutually exclusive and that the very same system can be compatible with all of them. As a central analytical result, we show that modeling neural responses in area V1 as samples from a posterior distribution over latents in a linear Gaussian model of the image implies that those neural responses form a linear Probabilistic Population Code (PPC, Ma et al. 2006). In particular, the posterior distribution over some experimenter-defined variable like "orientation" is part of the exponential family with sufficient statistics that are linear in the neural sampling-based firing rates.

REPLY TO THE AUTHORS' FEEDBACK ============================== Thank for very much for the response and the very concrete description of changes. I can picture how the updated manuscript looks. With the presentation issues out of the way, I raised my score to 9 points, i.e., a strong vote for acceptance.  Just for completeness, allow me to clarify the "Strictly speaking the statement is correct” (L163ff). In the idealized experiment considered in Fig 3, there are two sources of stochasticity: The neural samples x|I and the images I|T(s). For the main result ( eqs (6)-(8) ), the images I have been marginalized out. Thus, I understand the discussion around L163ff such that the network can describe the _average_image_T(s)_ well in case of a complete model A. The term "image" alone (without an attribute such as "average") is defined as the noisy realization I, according to line 76. Therefore, the sentence "Ax̄ will exactly reconstruct the input image" [...the limit can only refer to the x-samples.] describes the network's ability to capture also the noise-realization in I. For a complete model A, this is indeed the case. So, strictly speaking the statement is correct. But the discussion, as I understand it, refers to the average image, or prototype, T(s).  Anyhow, "Congratulations!" to a great piece of science. In light of the topic's relevance, I recommend this paper for a talk.  SUMMARY =======  The authors investigate the relation between sample-based codes and probabilistic populations codes (PPCs), two prevalent -- and often competing -- theoretical interpretations of neural activity for representing Bayesian computations in brain circuits. As the main result, the authors reconcile the two interpretations by showing analytically and in computer simulations how a sample-based code can yield a PPC code in the limit of many samples. For this, the authors set out with two independent, widely accepted modeling assumptions: (1) A linear Gaussian generative model of the input with neural sampling activations as latent variables. (2) An almost unrestricted Gaussian generative model of the input conditioned on a relevant stimulus parameter. Marginalizing over inputs followed by some mathematical massaging leads to the main result: a PPC form for p( stimulus parameter | neural activations ). The paper then discusses some often debated topics (multimodality, nuisance variables, linear vs log-probabilities) from the unified standpoint of their model.   QUALITY: 6  ==========  + The introduction is very thoughtfully written: precisely carving out the assumptions and differences of the two compared interpretations. + This is one of the few papers on this topic that do not intend to bolster up one interpretation over the other. - It would be interesting to see if the derivation holds for more general (exponential family) generative models beyond linear Gaussian.  + I think that the derivation is correct, once the reader... - corrected the many small mistakes in the equations, - skipped over copy&paste ruins (entire lines could be removed), - ignored unused and false assumptions, - guessed the definition of multiple undefined important quantities.  These issues impact both quality and clarity of the manuscript. Please see below for details.   CLARITY: 3 ==========  The presentation of the central derivation in sections 2 and 3 leaves (politely speaking) the impression of a very hasty writing process. Besides the above-mentioned errors, there is no high-level outline of the argumentation. Figure 1 could help explaining this, but the figure is never referred to in the main text. And the argumentation would be so simple:  [A] From eq. (2) and assuming conditionally independent samples (x^i ⊥ x^j | I) follows eq (4). [B] From eq. (4) and the stimulus model ( what should be eq. (3); currently unnumbered ) follows eq. (5) in the limit of many samples. [C] Eq (5) can be cast into the canonical PPC form eq (6) - (8) by identifying r with x^[1:t].  The last step, identifying r with x^[1:t], is completely omitted in the manuscript, but crucial since this connects the quantities of the two interpretations.  Overall, the clarity of presentation is the major weakness of the manuscript. Should the manuscript be rejected for this reason (which is not unlikely), I would like to encourage the authors to get the maths correct and resubmit to a leading computational neuroscience journal (e.g., PLoS CompBiol or PNAS).   ORIGINALITY: 8 ==============  The investigated question is around already for several years. The paper is original in that someone finally worked out the details of the maths of both representations. This is a very important contribution which requires a fine understanding of both theories. A big "Thanks" to the authors for this!  Yet, I think that two relevant papers are missing:  - Savin, C., Deneve, S. Spatio-temporal representations of uncertainty in spiking neural networks, Advances in Neural Information Processing Systems 27, 2014.   This paper relaxes the assumption of a one-to-one correspondence between neurons and latent variables for sampling.  - Bill J, Buesing L, Habenschuss S, Nessler B, Maass W, et al. (2015) Distributed Bayesian Computation and Self-Organized Learning in Sheets of Spiking Neurons with Local Lateral Inhibition. PLOS ONE   In the discussion section, a similar relation between sampling and PPCs is outlined, for the case of a generative mixture model instead of the linear Gaussian model considered here.   SIGNIFICANCE: 10 ================  Sampling vs PPC representations is a hotly debated topic for years, staging the main track talks of the most important computational neuroscience conferences. Reconciling the two views is highly relevant for anyone who studies neural representations of uncertainty.   CONCLUSION ==========  This is one of the papers, I feel thankful that it has finally been written (and researched, of course). Unfortunately, the paper suffers from an unjustifiable amount of typos and errors in the maths. If accepted to NIPS, I would suggest a talk rather than a poster. Overall, I tend to recommend acceptance, but I could perfectly understand if the quality of presentation does not merit acceptance at NIPS.  Due to the very mixed impression, I assigned a separate score to each of the four reviewing criteria. The geometric mean leads to the suggested overall score 6/10.    SPECIFIC COMMENTS =================  - L163ff: Strictly speaking the statement is correct: A complete basis A permits to perfectly model the noise in I \sim N(T(s), noise). But I think this is not what the authors mean in the following: The example addresses how well Ax approximates the template T(s) without a noise sample. If so, the discussion should be corrected.   Errors in the derivation / equations ------------------------------------  Since I walked through the derivation anyway, I also include some suggestions.  - Eq (1): The base measure is missing: ... + f(s); alternatively write p(s|r) \propto exp(...) to match eq (6) of the main result. - L66: Missing def. of PF_i; required e.g. in line 137 - L90: p(s|x) not r - L90 and general: The entire identification of r  multiple samples x^[1:t] is missing, but crucial for the paper. - Eq after L92: This should be eq (3). These are the "forward physics" from stimulus generation to spikes in the retina. The name p_exp(I|s) is used later in L109ff, so it should be assigned here. - Eq (3): Copy of the previous eq with wrong subscript. Could be deleted, right? - Eq after L106: Missing def of \bar x = 1/t \sum x^i (important: the mean is not weighted by p(x^i) ) - Eq after L107: p_exp(x|s) is not defined and also not needed: delete equation w/o replacement - L108 and L109: The equality assumption is undefined and not needed. Both lines can be deleted. The next eq is valid immediately (and would deserve a number). - L117 and eq 6: At some point r (from eq 1) and x^[1:t] must be identified with another. Here could be a good point. Section 4 could then refer to this in the discussion of binary x^i and longer spike counts r.   Minor Points ------------  - Figure 1 is not referred to in the text. - L92: clarify "periphery" (e.g. retinal ganglion cells). This is an essential step, bringing the "pixel image" into the brain, and would deserve an extra sentence. - L113: The Gaussian converges to a dirac-delta. Then the integration leads to the desired statement.   Typos -----  - Figure 1, binary samples, y_ticks: x_n (not N) - Caption Fig 1: e.g.,~ - L48: in _a_ pro... - L66: PF_n (not 2) - L71: ~and~ - L77: x^(i) not t  [general suggestion: use different running indices for time and neurons; currently both are i] - Almost all the identity matrices render as "[]" (at least in my pdf viewer). Only in the eq after L92, it is displayed correctly. - Footnote 1: section_,_ valid for infinitely _many_ samples_,_ ... - L106: Dr - Caption Fig 2: asymptotic - L121: section 3 (?, not 4) - L126: neuron~s~ - Figure 3C: ylabels don't match notation in 3B (x and x'), but are illustrations of these points - L165: ~for~ - L200: invariant - L220: _is_ - L229: _easy_ or _simple_ - L230: correspond _to_ - L232: ~a~ - L242: ~the~


Towards Robust Interpretability with Self-Explaining Neural Networks
Most recent work on interpretability of complex machine learning models has focused on estimating a posteriori explanations for previously trained models around specific predictions. Self-explaining models where interpretability plays a key role already during learning have received much less attention. We propose three desiderata for explanations in general -explicitness, faithfulness, and stability -and show that existing methods do not satisfy them. In response, we design self-explaining models in stages, progressively generalizing linear classifiers to complex yet architecturally explicit models. Faithfulness and stability are enforced via regularization specifically tailored to such models. Experimental results across various benchmark datasets show that our framework offers a promising direction for reconciling model complexity and interpretability.

Summary: The paper proposes an alternative approach to obtaining explanations from complex ML algorithms by aiming to produce an explainable modle from the start.  Recently there has been a number of works on interpretability. This work is most similar to concept-based explainability where some of the more recent ones include  • Bau, David, et al. "Network Dissection: Quantifying Interpretability of Deep Visual Representations." 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017. • Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions in AAAI 2018  Conditions on interpretability, as they are mentioned in this paper, have been explored earlier by f.e. Lipton, Zachary C. "The mythos of model interpretability." arXiv preprint arXiv:1606.03490 (2016)., It starts out with a linear regression model and replaces the parameters of the model with a function dependent on the input, adds an optional transformation of the input into a more  low-dimensional space and a generalization of the aggregration into the output. The main novelty of this paper is the idea to start out with an intrinsically interpretable model and extending it. Review:  This  paper is below the acceptance threshold.  They propose training an autoencoder to transform a high-dimensional input into a more interpretable feature space. To construct an appropriate distance metric for the autoencoder (i.e., one that disregards noise and compares the preservation of interpretable features), we would have to have extremely detailed knowledge of the interpretable features for the entire dataset. Otherwise there is no guarantee that the learned features are any more interpretable than the input.  One of their main points is the replacements of the parameter of the linear regression model theta with a function dependent on the input. This function theta(x) is realized with a neural network, a notoriously uninterpretable algorithm. This just moves the uninterpretability but does not provide anymore insight, since the output will still depend on the (uninterpretable) NN.  The authors do propose a difference boundary on theta  for small differences in x.  However, this restricts the interpretable part of the model to a very small input space around the current input. For interpretability we are interested in comparison to the entire data space to gain insight into why this input produces a particular output.   Furthermore, they do not provide precise results or comparisons for their experiments. As an example, for MNIST it is only written that they achieve <3% error rate, which is not competitive for MNIST. For UCI they only mention ‘competitive performance’, when the results could easily have been included in the supplementary.  As such, I would consider the experiments insufficient. Additionally, harder tasks are not included in the experiments.  Some ideas in the paper are promising and I recommend the authors submit this paper to a workshop.  The paper is written  clearly and concisely. The idea of generalizing an interpretable model into a more complex one is interesting and to my knowledge novel.  In the same vein, the proposed conditions for interpretability have not been published in this form before. In my opinion, more experiments with  more complex datasets would greatly benefit this paper as it is unclear to me whether the approach generalizes to more complicated tasks. Minor comments: L. 179: wrong ref approach ?? L. 213 wrong ref


Automating Bayesian optimization with Bayesian optimization
Bayesian optimization is a powerful tool for global optimization of expensive functions. One of its key components is the underlying probabilistic model used for the objective function f . In practice, however, it is often unclear how one should appropriately choose a model, especially when gathering data is expensive. We introduce a novel automated Bayesian optimization approach that dynamically selects promising models for explaining the observed data using Bayesian optimization in model space. Crucially, we account for the uncertainty in the choice of model; our method is capable of using multiple models to represent its current belief about f and subsequently using this information for decision making. We argue, and demonstrate empirically, that our approach automatically finds suitable models for the objective function, which ultimately results in more-efficient optimization.

PAPER SUMMARY:  This paper investigates a new Bayesian Optimization (BO) approach that automatically learns the correlation structure (i.e., kernel function) of the GP model of the black-box function while collecting observation to optimize it.  The key idea is to run a second BO on the model space (i.e., a collection of complex kernels generated by summing and multiplying base kernels) at each iteration to update the list of model candidates before choosing the next observation.   This 2nd BO routine casts the model evidence as a random function distributed by another GP. The implementation is adopted from a previous work (BOMS) on model selection via BO.  SIGNIFICANCE & NOVELTY:  This is an interesting problem & the approach taken is fairly new to me. I do, however, have the following concerns regarding the significance & technical soundness of the proposed solution:  The authors propose to treat the data collected at earlier stages as noisier observations to avoid recomputing the model evidence (as new data arrives) & claim that it would make the search better informed. This is, however, a strange intuition: adding (implicitly) more noise to earlier observations only increases the prediction variance & while it might help preventing the model from being overly confident in its prediction, it does not make the model more informed because the quality of information remains the same if the model evidence is not recomputed.  Section 4 (line 150-151) seems to suggest that the noise variance of a sample candidate depends on the previously collected samples. If this is indeed the case, the integration in line 179 is no longer analytically tractable. The description of Algorithm 1 is somewhat confusing: the update model sub-routine is said to recompute the corresponding model evidence before running the 2nd BO sub-routine on the model space but I thought the entire point here is to exploit heterogeneous noise scheme and avoid recomputing them? What exactly does the update model sub-routine do?  How do we set the model prior p(M)? At the beginning of a new iteration, the pool of model candidate is expanded & the prior will need to be changed. This does not seem to be discussed in the main text. Also, another issue is the large no. of approximation layers involved in computing the acquisition function for the main BO routine (see Section 4.1). This might affect the solution quality and I am raising this concern because I think the experiments are not very conclusive in this regard (please see my comments below).  EXPERIMENT:  If I understand correctly, the whole point of adding an extra model-searching BO sub-routine is to boost the performance of the main BO process. To demonstrate this conclusively, besides the experiments that the authors have presented, the proposed approach should also be compared with other state-of-the-art BO algorithms (e.g., predictive entropy search (PES) [1], upper-confidence-bound (UCB) ant etc., [2]) implementing the best model among a set of pre-selected candidates.   However, in this regard, the proposed approach is only compared with a baseline including a standard EI implementing the simple non-isotropic SE kernel. Is this SE model among the proposed approach’s pre-selected pool of candidates & is it the best among those? Also, the experiment description seems to suggest that this baseline is subject to the same amount of approximations in Section 4 (see lines 185-186). This is somewhat strange because EI is analytically tractable and such approximations are not necessary and only serve to weaken the baseline.  For a more conclusive demonstration, I would suggest the authors to compare their method with other state-of-the-art BO algorithms (without model selection) implementing (a) the worst and (b) the best model among those in the pre-selected pool. If the model selection machinery is efficient, we should at least see ABO outperforms (a) and approaching (or surpassing) (b) & this would resolve the last issue that I mentioned in the previous section (regarding the stable quality of the resulting solution after being subjected to multiple approximations).  CLARITY:  The paper is clearly written.  REVIEW SUMMARY:   I find the problem investigated in this paper novel & interesting but I have a few concerns and doubts regarding the proposed solution as well as its empirical evaluations, which are not very conclusive.  ---  [1] https://arxiv.org/abs/1406.2541 [2] https://arxiv.org/pdf/1206.2944  ---  Thank you for the rebuttal.   My first comment in the significance & novelty section is about the inner BO routine (not the outer BO routine as interpreted in your response). I do understand that by adding noises to the evidence observation to discourage poor model & at the same time, acknowledge that some of the old model evaluations might be out of date.   This however does not really explain how this would make the inner BO routine more informed in model search (e.g., how would a previously discarded model get reconsidered in light of new data? If the model search only uses its model evaluation + artificial noise, how could the predicted score of the discarded model be improved to make the search more informed?).   Secondly, part of my concern is whether the SE model is competitive against other model candidates in the pool. If it is less competitive than the others then the benefit from considering more model is not shown conclusively. I still think that the authors should compare with other models in the pool to demonstrate the efficiency of the developed model selection machinery.


Visualizing the Loss Landscape of Neural Nets
Neural network training relies on our ability to find "good" minimizers of highly non-convex loss functions. It is well-known that certain network architecture designs (e.g., skip connections) produce loss functions that train easier, and wellchosen training parameters (batch size, learning rate, optimizer) produce minimizers that generalize better. However, the reasons for these differences, and their effect on the underlying loss landscape, are not well understood. In this paper, we explore the structure of neural loss functions, and the effect of loss landscapes on generalization, using a range of visualization methods. First, we introduce a simple "filter normalization" method that helps us visualize loss function curvature and make meaningful side-by-side comparisons between loss functions. Then, using a variety of visualizations, we explore how network architecture affects the loss landscape, and how training parameters affect the shape of minimizers.

Overview: Visualizing the loss landscape of the parameters in neural nets is hard, as the dimension of parameters in neural network is huge. Being able to visualize the loss landscape in neural net could help us understand how different techniques are helping to shape the loss surface, and how the loss surface "sharpness" vs "flatness" are related to generalization error.  Previous works include interpolating a 1D loss path between the parameters of two models to reveal the loss surface along the paths. However, 1D visualization sometimes can be misleading and losing critical information about the surrounding loss surface and local minimums. For 2D visualizations, usually 2 directions are selected for interpolating the loss surface along the 2 directions from an origin. However, as the neural net has a "scale-invariant" problem, which means the weights in different layers can appear in different orders but remain the same effect.  The author proposed a new way of visualizing neural network loss surface in 2D with consideration of scale-invariant in neural network parameters. The method is to sample two Gaussian random vectors that normalized by the filter norms in the neural network, and then compute the loss across different combinations of the two filter-normalized Gaussian random vectors from the minimizer. The author conducted experiments to show that the loss surface in 1D could be misleading, and their 2D filter-normalized plot is revealing more information about minima comparing to non-filter-normalized plot. The experiments on resnet also visualized the network trained under-different settings to show that certain techniques do help flatten the loss surface, and flattened surface is helping the generalization error.  Quality: The method used in the paper is technically sound and the experiments are well organized.   Clarity: The writing of the paper is mostly clear and easy-to-understand. There are two suggestions: 1) When introducing the method to generate filter-normalized plot, it is a little bit ambiguous when the author mention the term "filter". My understanding is that each part of the elements in the gaussian vector should be normalized by the corresponding convolutional filter in the neural network. Probably a simple toy example will be better explaining this idea. 2) The organization of the sections is a little bit ambiguous, as the author firstly introduced the method to generate 2D filter-normalized plot, but then in the first parts of section 3, the author showed the result for 1D plot to reveal the constraint of 1D plot. This is a little bit confusing. Moving the 1D plot to the front and introducing the method to generate 2D-filter-normalized plot could probably be better in the reading flow.  Originality: The major difference from the previous method is the 2D filter-normalized plot. However, the author also claimed that without filter-normalization, the plot could still reveal some of the local-minima of the two different methods. The other originality is to apply the visualization on the network trained by different methods.   Significance: The visualizations of the loss landscapes are very impressing and very informative. However, the method itself is a little bit simple. The observations and conclusions are very helpful, but stayed on an eyeball based analysis. If somehow the author could bring a more quantitative study of the sharpness/flatness of the landscape, or even relate the sharpness/flatness of the surface to the regularizer in training, or relate that to the sensitivity analysis of the network could be more impactful. Also, this method is still computational heavy, as one needs to compute the value of each interpolation block. I don't see high-resolution with an efficient implementation in their study._x000c_


Learning convex polytopes with margin
We present an improved algorithm for properly learning convex polytopes in the realizable PAC setting from data with a margin. Our learning algorithm constructs a consistent polytope as an intersection of about t log t halfspaces with margins in time polynomial in t (where t is the number of halfspaces forming an optimal polytope). We also identify distinct generalizations of the notion of margin from hyperplanes to polytopes and investigate how they relate geometrically; this result may be of interest beyond the learning setting.

The paper considers the problem of PAC learning of fat convex polytopes in the realizable case. This hypothesis class is given by intersections of t fat hyperplanes, i.e., hyperplanes with margin gamma.   Using standard results, the authors derive that the VC dimension of this class is quadratic in the inverse margin size and thus that the sample complexity is polylogerithmic in this quantity. As their main result, they provide two algorithms for finding with high probability a consistent fat polytope: one with exponential runtime in t and one polynomial greedy algorithm that, however, only guarantees to find a (t log n)-polytope. Complementary, the paper states two hardness of approximation results: one for finding an approximately consistent fat hyperplane, i.e., one with the minimum number of negative points on wrong side (and all positive correctly classified), and one for finding a consistent fat polytope with the minimum number of hyperplanes. Finally, the authors also show how their results relate to the alternative class of polytopes that separate points outside of their gamma-envelope (area with Euclidean distance less or equal to gamma from the polytope boundary).  The topic is without a doubt highly relevant to NIPS and the results potentially of high impact. However, I do have some concerns regarding how the given results fit together to back the main claim. Ultimately this may be mostly an issue of the presentation as much could come down to a more conservative description of the main results including a more explicit statement of the learning problem (in particular the role of the parameter t).  - The claimed main contribution is an algorithm for learning t-polytopes in time polynomial in t. However, the polynomial time algorithm does only guarantee to find a polytope with potentially many more sides (O(t log t) for constant epsilon and delta). In contrast, the other algorithm, which indeed finds a consistent t-polytope, is exponential in t. To what degree is it appropriate to say that either algorithm efficiently (in terms of t) learns t-polytopes?  - The hardness of approximation result of Lemma 6 appears to relate to the relevant problem of finding an approximately consistent hyperplane (however, it is insufficient to conclude hardness for t larger than 1, for some of which the problem could turn out to be easier). In contrast, Theorem 7 is dealing with the problem of approximating a consistent polytope with the minimum number of sides. How does this relate to the realizable PAC learning problem? Here, we are given a dataset and know that there is a consistent t-polytope. In any case, the proof of Theorem 7 is unclear / too brief and in particular it seems Lemma 6 is not actually used in it (it might be intended as independent complementary result; the question remains, how either relates to the problem of finding a consistent t-polytope knowing that such a polytope exists).  In any case, the final two pages of the paper with the main contributions are very dense and hard to follow. In contrast, the non-essential discussion of margins versus envelopes takes up a lot of space. I believe the extension to envelops would have been fine as a brief afterthought (with proof potentially delegated to supplementary material). Instead, the actual main results should be presented in a more detailed and didactically pleasing form. So I strongly recommend to shift paper real estate accordingly.  Some additional minor comments:  - In Lemma 2 we probably need min instead of max since we combine two valid upper bounds to the VC dimension - In the proof correct "hyperplances" - Figures 1 and 2 are very similar and probably only one is needed given the tight space limitation (see comments above) - Probably, it would be more straightforward to define a linear embedding f to be a JL transform if it satisfies the reconstruction properties (with high probability) instead of talking about "linear embeddings that satisfy the guarantees of _the_ JL transform".  - In the proof of Observation 1 two times "at least" should probably be "at most" - Last line of proof of Lemma 9: "contains within _it_ a delta-net" - Theorem 10: the first part of the bound of Lemma 2 is not reflected here


Geometrically Coupled Monte Carlo Sampling
Monte Carlo sampling in high-dimensional, low-sample settings is important in many machine learning tasks. We improve current methods for sampling in Euclidean spaces by avoiding independence, and instead consider ways to couple samples. We show fundamental connections to optimal transport theory, leading to novel sampling algorithms, and providing new theoretical grounding for existing strategies. We compare our new strategies against prior methods for improving sample efficiency, including quasi-Monte Carlo, by studying discrepancy. We explore our findings empirically, and observe benefits of our sampling schemes for reinforcement learning and generative modelling.

Summary: --------------  This paper proposes a number of expectation estimation strategies which strive to attain lower error than naive methods relying on iid sampling. This is especially important in settings where evaluating the objective function at the sampled variates is expensive (e.g. RL problems.) The advantage of Monte Carlo over deterministic methods is that theoretical assurances are more readily obtained.  The approach is based on (numerically) finding optimal couplings, i.e. joint distributions over an augmented state space marginalizing to the expectation distribution of interest. While a uniformly optimal coupling does not exist over generic function classes, as is appreciated in statistical decision theory, the problem is well-defined in both expected and minimax variants. The optimizer to the K-expected loss is shown to minimize a multi-marginal transport problem, with the nice result that the solutions are space-filling, bringing to mind the behavior of QMC methods. In Proposition 2.7, a relaxation of the minimax problem is obtained when F is the unit ball in a RKHS associated with K, which interestingly turns out to solve the expected loss under the GP prior with covariance K.  Algorithm 2 shows how to optimally couple m antithetic pairs in accordance with Theorem 2.10, which states that sequential orthogonal sampling achieves this under an isotropic marginal.  Section 3 connects geometrically-coupled sampling with low-discrepancy sequences, and Section 4 provides some instructive concentration bounds for both vanilla ES gradient estimators and orthogonal estimators.  This paper provides a number of valuable theoretical contributions with important practical implications, and I recommend its acceptance to NIPS. While the topic is not my field of research, the ideas in the paper were in my view sufficiently well-explained and motivated for me to appreciate the problems and solution.   Issues/Questions: ----------------- The intended domain of application is when evaluation time of f dominates the process. While Algorithm 1 is clearly efficient, is it possible to give a sense of the cost generating conditionally orthogonal samples as in Algorithm 2?   Clarity: -------------- The paper is very well-written and clear;.   Significance: ------------  In addition to its algorithmic contributions, this paper provides several explanations of the success of currently used approximation methodologies.  The restriction of the marginal to be isotropic is strong and prohibits applicability in other contexts, however the case is convincingly made that in ES-based policy learning and VAE ELBO optimization, the method is practically relevant.   Technical Correctness: ---------------------  I did not painstakingly verify each detail of the proofs, but the results are technically correct as far as I could tell.  Minor: ------- - Why does "s" refer to number of samples in the experiments while "m" in the rest of the paper? - l186: sampes -> samples


Differentially Private Uniformly Most Powerful Tests for Binomial Data
We derive uniformly most powerful (UMP) tests for simple and one-sided hypotheses for a population proportion within the framework of Differential Privacy (DP), optimizing finite sample performance. We show that in general, DP hypothesis tests can be written in terms of linear constraints, and for exchangeable data can always be expressed as a function of the empirical distribution. Using this structure, we prove a 'Neyman-Pearson lemma' for binomial data under DP, where the DP-UMP only depends on the sample sum. Our tests can also be stated as a post-processing of a random variable, whose distribution we coin "Truncated-Uniform-Laplace" (Tulap), a generalization of the Staircase and discrete Laplace distributions. Furthermore, we obtain exact p-values, which are easily computed in terms of the Tulap random variable. We show that our results also apply to distribution-free hypothesis tests for continuous data. Our simulation results demonstrate that our tests have exact type I error, and are more powerful than current techniques.

Short summary: In this paper, the authors derive uniformly most powerful differentially private simple and one-sided tests for the parameter $\theta$ of the Binomial distribution with parameters $(n,\theta)$. In this context, the requirement of differential privacy translates to certain regularity conditions on the test as a function. The authors explicitly establish the necessary form of a test which fulfils these conditions (i.e. the in some sense strongest version of them) and express it as the distribution function of a novel distribution they call Tulap distribution. The UMP-property is then asserted through a Neyman-Pearson-type lemma. Later on, the close connection between the test and the Tulap and Binomial distributions leads to a convenient representation of the associated p-value and therefore the test as such. The paper closes with possible applications to situations without a priori Binomial distributions and simulations comparing the empirical type-I error probability and power when using the new test, the classical non-private UMP test and the test based on Normal approximation.   Comments: Since the differentially private UMP tests for the considered problem were unknown and the authors introduce a new distribution as an essential tool, I consider the paper significant and original. Also, this work appears to be technically sound. A final proof-reading would be advisable nonetheless; for instance, there are several typos in the paragraph on the nearest integer function (lines 98-100) and there is a factor $\exp(\epsilon)$ missing in the second equation on page 11. In fact, I would like to mention as my primary concern that I found sections 2 though 4 very difficult to read - fortunately, sections 5 through 7 are well-written and helped clarify my earlier problems. More specifically, Definition 2.1. clearly creates the impression that $X$ is a parameter. Then, in Definition 2.3., $X$ ist the random object and there is a new parameter $\theta$; however, only a few lines after that, $X$ appears as the index of the test, which is very confusing since it is actually the input variable (i.e. you would typically write $\phi(X)$). After section 3, there is no motivation yet to introduce the Tulap distribution and on top of that, you are again confused about the role of $X$ since it seems to be a parameter. I understand that each consideration in these sections is an important preparation for what follows, but the current structure is unsettling, at least for me: For instance, would it not be more natural to place Lemma 5.2 at the end of section 3 and let this be the motivation for Definition 4.1? Furthermore, the notion of post-processing (where the interpretation of $X$ as a parameter is warranted) is only needed in section 7, so would it not be more natural to discuss it only in that section? I realise that many people would dismiss all this as being a question of taste and hence it does not affect my opinion that this significant and original paper should be published, but I would still appreciate a more accessible version (through a revised structure or helpful remarks).    Regarding the rebuttal: I have read the other reviews and the authors' response. Even in light of the points raised by the other reviewers, I still vote for accepting the paper since the authors' response is quite convincing; I particulary appreciate how seriously the authors took my review and which adjustments they announced.


M-Walk: Learning to Walk over Graphs using Monte Carlo Tree Search
Learning to walk over a graph towards a target node for a given query and a source node is an important problem in applications such as knowledge base completion (KBC). It can be formulated as a reinforcement learning (RL) problem with a known state transition model. To overcome the challenge of sparse rewards, we develop a graph-walking agent called M-Walk, which consists of a deep recurrent neural network (RNN) and Monte Carlo Tree Search (MCTS). The RNN encodes the state (i.e., history of the walked path) and maps it separately to a policy and Q-values. In order to effectively train the agent from sparse rewards, we combine MCTS with the neural policy to generate trajectories yielding more positive rewards. From these trajectories, the network is improved in an off-policy manner using Q-learning, which modifies the RNN policy via parameter sharing. Our proposed RL algorithm repeatedly applies this policy-improvement step to learn the model. At test time, MCTS is combined with the neural policy to predict the target node. Experimental results on several graph-walking benchmarks show that M-Walk is able to learn better policies than other RL-based methods, which are mainly based on policy gradients. M-Walk also outperforms traditional KBC baselines.

This work tackles the problem of knowledge base completion (KBC) by walking through a knowledge graph. The graph walk is treated as an MDP with a known state transition model. MCTS is used to plan a trajectory through the graph, and these trajectories are used to train a value function with Q-learning. The Q-function is used to form a policy prior for the MCTS. A KBC-specific RNN structure is also proposed. Results on a range of KBC benchmarks show small but consistent gains over several baselines, including on-policy RL approaches and non-MCTS ablations.  Quality MCTS is a strong planning algorithm for delayed-reward problems and seems a good fit for KBC, so this is an at the least an interesting application paper. The authors also extend previous work on combining MCTS with learned value functions and policy priors by leveraging the MCTS trajectories to learn the optimal value function of the KBC graph-traversal policy with Q-learning. This seems like a reasonable approach to improve the sample efficiency of learning the prior, compared to just updating the policy prior to match the MCTS visitation frequencies at the root node. The experiments seem to present competitive baselines and show some promise on a number of tasks.  However, I would appreciate some more discussion or clarity of the exact roles of pi, Q, and V. Firstly, the background section 2 does not make the import distinction between an arbitrary policy and the optimal policy (and their corresponding value functions). In particular, note that Q-learning does not estimate Q^pi but Q* for the MDP.  In M-Walk, Q is trained off-policy, and so learns about the optimal state-action value function Q* for graph-traversal. However, V is trained on-policy (wrt the MCTS tree policy, not pi_theta), but only on terminal MCTS states (not terminal MDP states, as I understand it). Could the authors elaborate on this choice (as opposed to for example using V = max_a (Q) )?  Meanwhile, pi is simply a soft transformation of Q into a probability distribution (the authors note that simply using a softmax works similarly to their final implementation). Traditionally, Q-learning with Boltzmann exploration as the behaviour policy would anneal its entropy (ie temperature) to converge to a good solution. I am concerned that a fixed temperature would restrict the ability of the algorithm to develop strong confidence in its prior policy, even if it were justified by the data. In this particular setting with strongly bounded rewards and sufficient MCTS samples to correct for even an uncertain prior, it seems to work fine, but the exact mechanism by which the Q-learning transfers to the MCTS tree policy deserves some further discussion.   Clarity The paper is quite clearly written on the whole. Some of the details of the architecture and intermediate state representation are a little hard to follow, and some design decisions regarding the neural architecture (s3.1) are a bit opaque. It should also be emphasised that the policy pi_theta is neither used to traverse the MDP (except indirectly as a prior) nor is its value learned by any function, which is a departure from the expectations set by section 2.  Originality The combination of MCTS with learned policy and value priors has been explored in a number of domains, but this application to KBC appears novel and includes a number of algorithmic and architectural innovations.  Significance This work addresses a significant and well-studied problem, as well as a significant class of solutions (MCTS with learned value/policy priors) with methods that may transfer to other problem settings. The experimental results are promising.  However, more care with the formulation and explanation of the approach wrt the exact roles of the learned value functions and consequent policy prior would be very useful for solidifying the theoretical foundations of the approach.


Submodular Maximization via Gradient Ascent: The Case of Deep Submodular Functions
We study the problem of maximizing deep submodular functions (DSFs) [13,3] subject to a matroid constraint. DSFs are an expressive class of submodular functions that include, as strict subfamilies, the facility location, weighted coverage, and sums of concave composed with modular functions. We use a strategy similar to the continuous greedy approach [6], but we show that the multilinear extension of any DSF has a natural and computationally attainable concave relaxation that we can optimize using gradient ascent. Our results show a guarantee of max 0<δ<1 (1− − δ − e 2 ) plus time for pipage rounding [6] to recover a discrete solution, where k is the rank of the matroid constraint. This bound is often better than the standard 1 − 1/e guarantee of the continuous greedy algorithm, but runs much faster. Our bound also holds even for fully curved (c = 1) functions where the guarantee of 1 − c/e degenerates to 1 − 1/e where c is the curvature of f [37]. We perform computational experiments that support our theoretical results.

The submodular maximization is an important problem with many real-world applications. One powerful way of tackling these problems is to consider their contentious relaxations, then maximize those relaxations, and finally rounding the resulting solutions into discrete answers. In this paper, the authors consider maximizing a special class of submodular functions called  deep submodular functions (DSFs) under a matroid constraint.  The (almost) standard approach to solve a submodular maximization problem in the continuous domain is to consider the multilinear extension of the original set function. The contribution of this paper is to show that the natural concave extension of a DSF function provides an acceptable approximation for the multilinear extension. Because this natural extension is concave, a gradient ascent method will find the maximum value. The idea of bounding multilinear extension by a concave function for coverage functions is already introduced in [1]. In this paper, authors provide a tighter bound for a more general class of functions.  I have read the paper in details and I believe it is technically correct.  Next you can find my detailed comments.  1. This works is along the new progresses on maximizing DR-continuous functions (and in particular multi-linear extension). For  example, Hassani et al. [2] recently have proven that stochastic gradient ascent achieves 1/2 approximation for DR-continuous functions. Furthermore, Mokhtari et al. [3] have developed practical algorithms for stochastic conditional gradient methods which mitigates the need for subsampling while achieves the  (1-1/e)-approximation guarantee. The computational complexity of the algorithm from [3] is much better than that of O(n^7). The references (and comparison) to these closely related papers are missing.   2. The bound in Theorem 3 is a function of k, |V^1|, w_min and w_max. DSFs are a narrow family of submodular functions. The dependence on |V^1|, w_min and w_max might makes the coverage of these DSFs even narrower (for the cases their algorithm provides improvements for the approximation guarantee). For example, it might be the case that |V^1| should be very large in order to cover a broad family of submodular functions. I believe that the authors should have elaborated (more) on the dependence on these parameters. Considering these points and the fact that the result is only for a subclass of submodular functions with a matroid constraint, I find the contribution of this paper somewhat marginal.  3. The big O notation is not used correctly. Note that O(k) could be 0, and the approximation guarantee could be arbitrarily bad. Also, it is not mathematically sound to write \Delta(x) \geq O(k) (line 243).  4. The Experiments section of the paper is weak as it only considers synthetic datasets. I think adding experiments over the real world datasets would improve the quality of this paper.  5. Minor comments:  a) The presentation of paper could be improved. Next you can find several suggestions:  In page 1, "A" is not defined. Figure 2 is discusses before Figure 1. Font of equations (2) and (5) is very small. It is better to write the equation (2) in an iterative formulation. Different notations are used for definitions: \equiv and \triangleq. The fractions in the text are sometimes written by \nicefrac{}{} and sometimes \frac{}{}. Some sentences starts with numbers, e.g., line 80. The punctuation for "i.e.," and "e.g.," is not consistent.  b) line 40: benefit  c) line 75: \epsilon^{-2}  d) line 178: "we are focusing" or "we focus"  e) line: it is better to say "in continuous domain"  f) line 204: the definition of concave closure seems unnecessary  g) line 289: proposed  References: [1] Mohammad Reza Karimi, Mario Lucic, S. Hamed Hassani, Andreas Krause. Stochastic Submodular Maximization: The Case of Coverage Functions. NIPS 2017. [2] S. Hamed Hassani, Mahdi Soltanolkotabi, Amin Karbasi. Gradient Methods for Submodular Maximization. NIPS 2017. [3] Aryan Mokhtari, Hamed Hassani, Amin Karbasi. Conditional Gradient Method for Stochastic Submodular Maximization: Closing the Gap. AISTATS 2018.


With Friends Like These, Who Needs Adversaries?
The vulnerability of deep image classification networks to adversarial attack is now well known, but less well understood. Via a novel experimental analysis, we illustrate some facts about deep convolutional networks for image classification that shed new light on their behaviour and how it connects to the problem of adversaries. In short, the celebrated performance of these networks and their vulnerability to adversarial attack are simply two sides of the same coin: the input image-space directions along which the networks are most vulnerable to attack are the same directions which they use to achieve their classification performance in the first place. We develop this result in two main steps. The first uncovers the fact that classes tend to be associated with specific image-space directions. This is shown by an examination of the class-score outputs of nets as functions of 1D movements along these directions. This provides a novel perspective on the existence of universal adversarial perturbations. The second is a clear demonstration of the tight coupling between classification performance and vulnerability to adversarial attack within the spaces spanned by these directions. Thus, our analysis resolves the apparent contradiction between accuracy and vulnerability. It provides a new perspective on much of the prior art and reveals profound implications for efforts to construct neural nets that are both accurate and robust to adversarial attack.

The reason behind adversarial vulnerability of high dimensional classifier is still an open question. There are a few works trying to address this question by providing various and sometimes conflicting hypotheses. This work attempts to address the problem from a geometric perspective. The main hypothesis is that the “adversarial” directions are the ones which the classifier uses to distinguish between different classes. They provide some empirical evidence to support this idea.  The paper raises some interesting points regarding adversarial robustness. It shows that the subspace spanned by the “curved” directions of the classifier in the image space captures the most relevant features for the classification task. Hence, projecting images on this subspace does not degrade the classification performance, while by projecting on the complement (flat subspace), the classifier accuracy drops significantly (Figure 4). Then, the authors argue that various adversarial perturbations are captured in the subspace of curved directions. Therefore, they conclude that adversarial directions are nothing but the discriminative features of different classes. This is indeed what happens in linear classifiers.  I have to admit that I have not understood completely all the experiments. Honestly, I had hard time to follow some of the arguments as they were not rigorous enough. I feel that most of the arguments could be conveyed more clearly by equipping them with suitable math equations. Though this work has a lot of potential, it lacks the sufficient clarity and quality. For example,  - The quality of plots are quite low, axes are missing, some numbers are even not rendered correctly. E.g., the numbers of the y-axes for CIFAR10 networks are missing. - Many definitions are missing and it is basically relied on the reader to guess them, e.g. N_f, N_p, N_n, etc. - Where does QR decomposition come from? I can guess but please be explicit. - What is Q_d? - What are the axes in Figure 1? x-axis? y-axis? To what does each image correspond? - How Alg 1 differs from that in [3]?  why? - How do you compute the subspaces for ImageNet?  One of the main drawbacks of the paper is that half of it is dedicated to prior works while it skips many important details regarding the contributions. Also, it is sometimes difficult to recognize their contribution from prior works.  Though I really like the approach they take to study the adversarial vulnerability of deep classifiers, I guess the paper can heavily benefit from a complete rewriting. The most important change could be to shorten the prior works and to elaborate the contributions by taking the time to explain things using math equations.  Anyway, the paper ends with the following sentence: “…nets’ favourite features are their worst adversaries”. This would be a great discovery if the authors could take a bit more time to polish their work.


Extracting Relationships by Multi-Domain Matching
In many biological and medical contexts, we construct a large labeled corpus by aggregating many sources to use in target prediction tasks. Unfortunately, many of the sources may be irrelevant to our target task, so ignoring the structure of the dataset is detrimental. This work proposes a novel approach, the Multiple Domain Matching Network (MDMN), to exploit this structure. MDMN embeds all data into a shared feature space while learning which domains share strong statistical relationships. These relationships are often insightful in their own right, and they allow domains to share strength without interference from irrelevant data. This methodology builds on existing distribution-matching approaches by assuming that source domains are varied and outcomes multi-factorial. Therefore, each domain should only match a relevant subset. Theoretical analysis shows that the proposed approach can have a tighter generalization bound than existing multiple-domain adaptation approaches. Empirically, we show that the proposed methodology handles higher numbers of source domains (up to 21 empirically), and provides state-of-the-art performance on image, text, and multi-channel time series classification, including clinical outcome data in an open label trial evaluating a novel treatment for Autism Spectrum Disorder.

Title: Extracting Relationships by Multi-Domain Matching  Summary  Assuming that a corpus is compiled from many sources belonging to different to domains, of which only a strict subset of domains is suitable to learn how to do prediction in a target domain, this paper proposes a novel approach (called Multiple Domain Matching Network (MDMN)) that aims at learning which domains share strong statistical relationships, and which source domains are best at supporting to learn the target domain prediction tasks.  While many approaches to multiple-domain adaptation aim to match the feature-space distribution of *every* source domain to that of the target space, this paper suggests to not only map the distribution between sources and target, but also *within* source domains. The latter allows for identifying subsets of source domains that share a strong statistical relationship.  Strengths  Paper provides a theoretical analysis that yields a tighter bound on the weighted multi-source discrepancy.  Approach yields state-of-the-art performance on image, text and multi-channel time series classification tasks.  Weaknesses  Tighter bound on multi-source discrepancy depends on the assumption that source domains that are less relevant for the target domain have lower weights. While intuitively, this may seem obvious, there is no guarantee that in practice, the irrelevant source domains can reliably be identified.  No commitment that source code may get released.  Questions  L 96: Is it intended that the sum runs over all domains but s, including the target domain S? L120: Why is the Wasserstein distance not affected by a varying feature scale in practice?  There is a shift in notation in Section 3 where the target domain is not denoted by T while the source domains are denoted by s=1...S. In Section 2, the source domains were defined as s=1,...,S-1 while the single target domain was defined as S.  Theorem 3.3 shows that weighting yields a tighter bound given that irrelevant domains are assigned small weights. However, what happens if the algorithm fails to assign small weights to irrelevant domains or, in the most adverse case, if the least relevant domains get assigned the highest weights? More general: for which weight distributions does Theorem 3.3 provide tighter bounds?  To what number of source domains does the provided method scale? A total of 21 domains may still be a small number if this method were to be applied to other tasks. What potential does MDMN have to be extended or applied to fuzzy domains, i.e., where the source data set does not induce a canonical partitioning into domains?  Comments  Editorial Notes L 36 is be -> is L 219 develop -> development L 287 An -> A L 320 [add reference] L 321 domains is large L 342 state-of-the0art -> state-of-the-art  ---------------  Thanks to the authors for their detailed responses and addressing my questions. I am looking forward to the release of the code.


Found Graph Data and Planted Vertex Covers
A typical way in which network data is recorded is to measure all interactions involving a specified set of core nodes, which produces a graph containing this core together with a potentially larger set of fringe nodes that link to the core. Interactions between nodes in the fringe, however, are not present in the resulting graph data. For example, a phone service provider may only record calls in which at least one of the participants is a customer; this can include calls between a customer and a non-customer, but not between pairs of non-customers. Knowledge of which nodes belong to the core is crucial for interpreting the dataset, but this metadata is unavailable in many cases, either because it has been lost due to difficulties in data provenance, or because the network consists of "found data" obtained in settings such as counter-surveillance. This leads to an algorithmic problem of recovering the core set. Since the core is a vertex cover, we essentially have a planted vertex cover problem, but with an arbitrary underlying graph. We develop a framework for analyzing this planted vertex cover problem, based on the theory of fixed-parameter tractability, together with algorithms for recovering the core. Our algorithms are fast, simple to implement, and out-perform several baselines based on core-periphery structure on various real-world datasets.
1 Partially measured graphs, data provenance, and planted structure Datasets that take the form of graphs are ubiquitous throughout the sciences [4,23,49], but the graph data that we work with is generally incomplete in certain systematic ways [28,33,34,36,39]. A common type of incompleteness comes from the way in which graph data is generally measured: we observe a set of nodes C and record all the interactions involving this set of nodes. The result is a measured graph G consisting of this core set C together with a a potentially larger set of additional fringe nodes-the nodes outside of C that interact with some node in C. For example, in constructing a social network dataset, we might study the employees of a company and record all of their friendships [54]. From this information, we now have a graph that contains all the employees together with all of their friends, including friends who do not work for the company. This latter group constitutes the set of fringe nodes in the graph. The edge set of the graph G reflects the construction: each edge involves at least one core node, but if two nodes in the fringe have interacted, it is invisible to us and hence not recorded in the data.
E-mail and other communication datasets typically have this core-fringe structure. For example, the widely-studied Enron email graph [24,37,43,55] contains tens of thousands of nodes; however, this graph was constructed from the email inboxes of fewer than 150 employees [35]. The vast majority of the nodes in the graph, therefore, belong to the fringe, and their direct communications are not part of the data. The issue comes up in much larger network datasets as well. For example, a phone service provider has data on calls and messages that its customers make both to each other and to non-customers, but it does not have data on communication between pairs of non-customers. Another example would be a massive online social network may get some information about the contacts of its users with a fringe set comprised of of people not on the system-often including entire countries that do not participate in the platform-but generally not about the interactions taking place in this fringe set. Similarly, Internet measurements at the IP-layer from individual service providers give 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montréal, Canada.
1 Partially measured graphs, data provenance, and planted structure Datasets that take the form of graphs are ubiquitous throughout the sciences [4,23,49], but the graph data that we work with is generally incomplete in certain systematic ways [28,33,34,36,39]. A common type of incompleteness comes from the way in which graph data is generally measured: we observe a set of nodes C and record all the interactions involving this set of nodes. The result is a measured graph G consisting of this core set C together with a a potentially larger set of additional fringe nodes-the nodes outside of C that interact with some node in C. For example, in constructing a social network dataset, we might study the employees of a company and record all of their friendships [54]. From this information, we now have a graph that contains all the employees together with all of their friends, including friends who do not work for the company. This latter group constitutes the set of fringe nodes in the graph. The edge set of the graph G reflects the construction: each edge involves at least one core node, but if two nodes in the fringe have interacted, it is invisible to us and hence not recorded in the data.
E-mail and other communication datasets typically have this core-fringe structure. For example, the widely-studied Enron email graph [24,37,43,55] contains tens of thousands of nodes; however, this graph was constructed from the email inboxes of fewer than 150 employees [35]. The vast majority of the nodes in the graph, therefore, belong to the fringe, and their direct communications are not part of the data. The issue comes up in much larger network datasets as well. For example, a phone service provider has data on calls and messages that its customers make both to each other and to non-customers, but it does not have data on communication between pairs of non-customers. Another example would be a massive online social network may get some information about the contacts of its users with a fringe set comprised of of people not on the system-often including entire countries that do not participate in the platform-but generally not about the interactions taking place in this fringe set. Similarly, Internet measurements at the IP-layer from individual service providers give only a partial view of the global Internet network [58,60]. Therefore, graph data often takes the form illustrated in Figure 1a: the nodes are divided into a core and a fringe, and we only see edges that involve a member of the core. Thus, the core set is a vertex cover of the underlying graph-since a vertex cover, by definition, is a vertex set incident to all edges.
Often, a graph dataset is annotated with metadata about which nodes belong to the core, and this is crucial for correctly interpreting the data. But there are a number of important contexts where this metadata is unavailable, and we do not know which nodes constitute the core. In other words, at some point, we have "found data" that has core-fringe structure, but the labels identifying the core nodes are missing. One reason for this scenario is that metadata is lost over time for a wide range of reasons: data is repeatedly shared and manipulated, managers of datasets change jobs, and hard drives are decommissioned. This is a central theme in data provenance, lineage, and preservation [11,44,56,59], an especially challenging issue in modern digitization efforts [38] and large-scale data management [32]. A concrete example is the following. Suppose a telecommunications company shares an anonymized dataset of telephone call records with a university but does not include on which nodes were customers and which were the fringe set of non-customers. By the time a student begins to analyze the data and realizes the metadata is missing and important for analysis, the researchers at the company who assembled the dataset have left. At this point, there is no easy way to reconstruct the metadata.
These issues also arise in current research on counter-surveillance [29]. Intelligence agencies may intercept data from adversaries conducting surveillance and build a graph to determine which communications the adversaries were recording. In different settings, activist groups may petition for the release of surveillance data by governments or infer it from other sources [47]. In these cases, the "found data" consists of a communication graph in which an unknown core subset of the nodes was observed, and the remainder of the nodes in the graph (the fringe) are there simply because they communicated with someone in the core. In such situations, there generally is not any annotation to distinguish the core from the fringe. In this case, the core nodes are the compromised ones, and identifying the core from the data can help to warn the vulnerable parties, hide future communications, or disseminate misinformation.
Planted Vertex Covers. Here we study the problem of recovering the set of core nodes in found graph data, motivated by the range of settings above. We can view this as a planted vertex cover problem: we are given a graph G in which an adversary knows a specific vertex cover C. We do not know C, but we want to output a set that is as close to C as possible. The property of being "close" to C corresponds to a performance guarantee that we will formulate in different ways. We may want to output a set not much larger than C that is guaranteed to completely contain it, or we may want a small set that is guaranteed to have substantial overlap with C. A simple instance of the task is depicted in Figure 1b, after the explicit labeling of the core nodes has been removed from Figure 1a.
Generically, planted problems arise when some hidden structure (like the vertex cover in our case) has been "planted" in a larger input, and the goal is to find the structure. Planted problems are typically core fringe (a) Graph dataset built from a small core.
(b) The dataset without the core labeled. The resulting data contains these core nodes together with a potentially much larger fringe, consisting of all other nodes that had an interaction with some member of the core. (b) Knowing which nodes are in the core is important for interpreting the dataset. However, in many cases, the graph is "found data" and this metadata is not available. This can arise from challenges in data provenance that lead to the loss of the metadata or when only partial information is available in contexts such as counter-surveillance. We study how accurately we can recover the core, despite limited information about how the dataset was constructed. Algorithmically, this leads to a planted vertex cover problem.
based on formal frameworks in which the input is generated by a highly structured probabilistic model. This is true in some of the most studied planted problems, such as the planted clique problem [5,20,25,46] and the recovery problem for stochastic block models [1,2,3,7,19,48].
It might seem inevitable that planted problems should require such probabilistic assumptions-how else could an algorithm guess which part of the graph corresponds to the planted structure, if there are no assumptions on what the "non-planted" part of the graph looks like? But the vertex cover problem turns out to be different, and it is possible to solve what, surprisingly, can be described as a "worstcase" planted problem. With extremely limited assumptions, we can provide provable guarantees for approximately recovering unknown vertex covers. Specifically, we make only two assumptions (both necessary in some form, though relaxable): (i) the planted vertex cover is inclusion-wise minimal and (ii) its size is upper-bounded by a known quantity k. We think of k as small compared to the total number of nodes, which is consistent with many datasets. Among other results, we use fixed-parameter tractability to show that there is an algorithm operating on arbitrary graphs that can output a set of f (k) nodes (independent of the size of the graph) guaranteed to contain the planted vertex cover. We obtain further bounds with additional structural assumptions.
We pair our theoretical guarantees with experiments showing the efficacy of these methods on realworld networks with core-fringe structure. Using our theory, we develop a natural heuristic based on constructing a union of minimal vertex covers from random initializations. Our simple algorithm provides superior recovery performance and running time compared to a number of competitive baselines. Among these, we show improvements over a line of well-developed heuristics for detecting core-periphery structure [15,31,53,63]-a sociological notion in which a network has a dense core and a sparser periphery, generally for reasons of differential status rather than measurement effects.

This paper considers graphs that are partitioned into two separate components called a "core" (C) and a "fringe" (F), where all edges have at least one endpoint in C. The partitioning is supposed to be unknown, and the goal of the paper is to come up with an algorithm that recovers it.  Strong aspects of the paper are:  - The paper first performs a theoretical analysis, deriving various conditions for the set C and upper and lower bounds for its size under various assumptions. Based on the theory, it then proposes an algorithm for recovering C, and shows that this algorithm outperforms various baselines.  - The theoretical analysis is sound and intuitive. The theory nicely and naturally leads to an algorithm that is indeed surprisingly simple and performs well empirically.   - The paper is very well written, the motivation is clearly explained, the theoretical results are clearly structured and nicely followable. Most of the theoretical results are relatively straightforward with a few exceptions (Lemma 3 and Theorem 2).   - The considered problem is a special case of "core-periphery" structure identification in networks (e.g. Zhang et al, 2015, Phys Review E; their ref 61). The contribution rests on the assumption that no links at all exist between nodes in the periphery. This assumption leads to an elegant algorithm that can convincingly outperform others when the assumption holds. The techniques are potentially of interest for other researchers in the networks field.  I have three main criticisms:  - The restriction to no edges in F seems very restrictive and I am not convinced by the given examples that this is a problem that would be of interest in the real world. In the used datasets, the authors dropped the available labels of C and F. In the motivation given in the beginning, they say that in some applications these labels are lost, but can they point to any real datasets that have this property or even better, analyze one and present the results?   - It is perhaps not too surprising that the proposed algorithm outperforms baseline given that the baseline methods were developed for a more general scenario and thus do not take advantage of the absence of edges in F. It would have been interesting to see how the proposed algorithm would fare on more general community structure detection problems such as the blog link data shown in Zhang et al, 2015, Phys Review E. Would the algorithm still provide a useful approximation in this case, or would it break down completely?  - Finally, I wonder if NIPS is the appropriate avenue for this research area. Networks has traditionally not been a topic with a very strong presence at NIPS as far as I am aware, and more appropriate venues that would might to mind would be KDD, ICDM. A possible sign of mismatch is that the paper does not cite any paper from NIPS or any other ML/AI venue. Perhaps the authors can explain in their rebuttal why they chose this specific conference?   Minor points: - I would have liked to see more justification for the choice of the number of clusters to be generated (fixed here at 300). Is this important for performance? Under which circumstances would I need to set this higher or lower?  - It was hard to compare the numbers in Table 2 due to different units, can this be standardized to one unit or alternatively, can at least the fastest algorithm be highlighted in each row?   = After author response =  The authors have presented some new data that addresses my second criticism and since no other reviewer had any doubts about suitability for NIPS, I retract that criticism as well. The authors cite personal communications as evidence for the importance of the considered setting, which still does not convince me entirely. However, I am satisfied with the response overall and will revise my score up by 1.


End-to-End Differentiable Physics for Learning and Control
We present a differentiable physics engine that can be integrated as a module in deep neural networks for end-to-end learning. As a result, structured physics knowledge can be embedded into larger systems, allowing them, for example, to match observations by performing precise simulations, while achieves high sample efficiency. Specifically, in this paper we demonstrate how to perform backpropagation analytically through a physical simulator defined via a linear complementarity problem. Unlike traditional finite difference methods, such gradients can be computed analytically, which allows for greater flexibility of the engine. Through experiments in diverse domains, we highlight the system's ability to learn physical parameters from data, efficiently match and simulate observed visual behavior, and readily enable control via gradient-based planning methods. Code for the engine and experiments is included with the paper.

This paper proposed a way of implementing a differentiable physics engine based on Linear Complementarity Problem (LCP) where analytical gradients can be computed based on the primal-dual interior point method. As a result, it can be integrated as a module in an end-to-end learning pipeline, such as deep neural networks.  Existing popular physical simulators, such as MuJoCo and Bullet, have also cast contact dynamics simulation as an optimization problem and resorted to techniques similar to LCP for solving the motion equations. The main (but limited) contribution of this work is to make analytical gradient computation more efficient with the interior point method.  The problem of having a differentiable simulator is well motivated. It enables gradient-based learning/planning methods to backpropagate through dynamics. The authors illustrated the benefits of such a differentiable physical simulator in three types of experiments: 1) system identification, i.e., inferring the physical parameters from behaviors by backpropagating gradients to these parameters (e.g., mass); 2) using the physical simulator as a neural network layer for future prediction from simple visual observations; and 3) using it as a model to build controllers via iLQR.   The proposed simulator adopted similar optimization techniques as in popular simulators, such as MuJoCo. Thus, it would suffer from the same issues caused by the “soft” contact model as in other LCP-based simulators, where contact is triggered when the distance of two contacting objects is below a threshold. Such approximation would lead to dynamics mismatch between the simulated contact physics and the real contact physics.  However, the writing and analysis of this work can be significantly improved:  A great amount of technical detail has been missing from the paper. Many of the symbols and notations in equations (e.q., Eq (1), (2), (3)) are undefined, and Sec. 3 deserved an expanded description of the LCP formulation as well as the primal-dual interior point method. For the sake of improving understanding, I would suggest the authors move some detailed motion equations to the supplementary material and instead provide more intuitive explanations of the method.  In addition to the technical details, more analysis and comparison would be beneficial for understanding the speed and scalability of the proposed simulator. I would like to understand how fast the simulator can perform forward steps in some benchmarking tasks, compared to popular choices, e.g., MuJoCo, Bullet, and DART. Another question is the scalability of this simulator, such as how many contact points, objects, collision meshes the simulator can afford. The experiments have been conducted in tasks with a small number of articulated primitive shapes. The merit of such a physical simulator might be hindered by the limited size of problems it can simulate efficiently.  Questions: Fig. 6: what is the main cause of the lower asymptotic performance of the model-based method in comparison to Double Q-Learning? Is it due to the approximate models built for the Cartpole and Breakout, or the iLQR solver used by the controller?  In summary, this work introduces a way of implementing differentiable simulator with LCP techniques that allows efficient compute of analytical gradients. The technical novelty is a bit limited, but the experiment shows preliminary successes of such a differentiable physical simulator in several tasks. The major weakness of this paper is the lack of technical clarity and detailed analysis of this simulator, which has caused difficulty in judging the merit of this simulator in comparison to existing ones. As this paper can be significantly benefited from another round of revision, I am leaning towards rejecting this paper for now.


Estimating Learnability in the Sublinear Data Regime
We consider the problem of estimating how well a model class is capable of fitting a distribution of labeled data. We show that it is possible to accurately estimate this "learnability" even when given an amount of data that is too small to reliably learn any accurate model. Our first result applies to the setting where the data is drawn from a d-dimensional distribution with isotropic covariance, and the label of each datapoint is an arbitrary noisy function of the datapoint. In this setting, we show that with O( √ d) samples, one can accurately estimate the fraction of the variance of the label that can be explained via the best linear function of the data. For comparison, even if the labels are noiseless linear functions of the data, a sample size linear in the dimension, d, is required to learn any function correlated with the underlying model. Our estimation approach also applies to the setting where the data distribution has an (unknown) arbitrary covariance matrix, allowing these techniques to be applied to settings where the model class consists of a linear function applied to a nonlinear embedding of the data. In this setting we give a consistent estimator of the fraction of explainable variance that uses o(d) samples. Finally, our techniques also extend to the setting of binary classification, where we obtain analogous results under the logistic model, for estimating the classification accuracy of the best linear classifier. We demonstrate the practical viability of our approaches on synthetic and real data. This ability to estimate the explanatory value of a set of features (or dataset), even in the regime in which there is too little data to realize that explanatory value, may be relevant to the scientific and industrial settings for which data collection is expensive and there are many potentially relevant feature sets that could be collected.

When I started reading the paper, I actually liked it. But when I got to page 5, I was very disappointed and annoyed. It might not be intentional, but it felt almost deceptive that the authors did not mention very relevant prior art till then and made it look like many of the previous literature as their own contribution.  Authors claim that they propose an algorithm that uses O(sqrt(d)) samples for estimating the variance of noise.  In fact, after a read of this paper and previous papers, what I got was: O(sqrt(d)) was essentially known before (for most cases) and new results are in fact much weaker.   Dicker et al: Maximum Likelihood for Variance Estimation in High-Dimensional Linear Models (http://www.stat.rutgers.edu/home/ldicker/papers/variance.pdf), showed that O(sqrt(d)) samples are sufficient for learning sublinearity when additional noise (eta) is Gaussian i.i.d. and data points X are Gaussian and either have identity covariance or any general covariance (Lemma 2 and proposition 1 respectively). Further, for the convergence proof, the properties of Gaussianity they used were only that the moments are bounded. Hence their results and proofs more or less extend to non-Gaussian cases with similar moment bounds.  What the authors showed:  a. Extend the results of Dicker et al (with a very similar algorithm) to non-Gaussian but isotropic and bounded moments. But (mostly) the exact proof of Lemma 2 in Dicker et al holds for any isotropic distributions with bounded moments already.  b. Theorem 1: They propose a weaker version of the result when the X is not from a Gaussian and is non-isotropic. Here the result is O(d^(1-g(epsilon)), which is much weaker than sqrt(d). Further the convergence rate is weak 1/poly(log(d/n)) as opposed to the polynomial convergence rates of Dicker et al. The dependence of epsilon is also bad and super linear. (exponential in 1/epsilon in certain cases).  c. Extend isotropic results to the agnostic case, but here again, I am not sure what is the novelty compared to Lemma 2 in Dicker et al.  Also authors did not compare Dicker et al algorithms with the “new” one in experiments.  I request the authors to  a. Exactly state what is new in Algorithm 1 compared to the algorithm of Dicker et al. b. Why are Theorem 1 results much weaker than the Proposition 1 results of Dicker et al.? c. What is the novelty in Proposition 1 and Theorem 2 compared to the proof of Lemma 2 in Dicker et al. d. State the convergence rate of the algorithm (Theorem 1).  e. If the results are different than Dicker et al, include their algorithms in your experimental plots and compare experimentally which one works better.