In this paper we investigate an artificial agent's ability to perform task-focused tool synthesis via imagination. Our motivation is to explore the richness of information captured by the latent space of an object-centric generative model -and how to exploit it. In particular, our approach employs activation maximisation of a task-based performance predictor to optimise the latent variable of a structured latent-space model in order to generate tool geometries appropriate for the task at hand. We evaluate our model using a novel dataset of synthetic reaching tasks inspired by the cognitive sciences and behavioural ecology. In doing so we examine the model's ability to imagine tools for increasingly complex scenario types, beyond those seen during training. Our experiments demonstrate that the synthesis process modifies emergent, task-relevant object affordances in a targeted and deliberate way: the agents often specifically modify aspects of the tools which relate to meaningful (yet implicitly learned) concepts such as a tool's length, width and configuration. Our results therefore suggest, that task relevant object affordances are implicitly encoded as directions in a structured latent space shaped by experience.

This paper proposes an architecture for synthesizing tools to be used in a reaching task. Specifically, during training the agent jointly learns to segment an image of a set of three tools (via the MONet architecture) and to classify whether one the tools will solve the given scene. At test time, one of the three tools is selected based on which seems most feasible, and then gradient descent is used to modify the latent representation of the tool in order to synthesize a new tool to (hopefully) solve the scene. The paper demonstrates that this approach can achieve ok performance on familiar scenes with familiar tools, but that it fails to generalize when exposed to unfamiliar scenes or unfamiliar tools. The paper reports a combination of the quantitative results showing that optimizing the latent space can lead to successful synthesis in some cases, and qualitative results showing that the synthesized tools change along interpretable dimensions such as length, width, etc. The combination of these results suggest that the model has learned something about which tool dimensions are important for being able to solve the types of reaching tasks given in the paper.
While I think this paper tackles a very interesting, important, and challenging problem, I unfortunately feel it is not ready for publication at ICLR and thus recommend rejection. Specifically, (1) neither the particular task, results, or model are not very compelling, (2) there are no comparisons to meaningful alternatives, and (3) overall I am not quite sure what conclusions I should draw from the paper. However, given the coolness of the problem of tool synthesis, I definitely encourage the authors to continue working on this line of work!
1. The task, results, and model are not very compelling. Any of these three things alone would not necessarily be a problem, but given that all three are true the paper comes across as a bit underwhelming.

- First, while the task can be construed as a tool synthesis task, it doesn’t come across to me as very ecologically valid. In fact, the task seems to be more like a navigation task than a tool synthesis task: what’s required is simply to draw an unbroken line from one part of the scene to another, rather than actually generate a tool that has to be manipulated in an interesting way. Navigation has been studied extensively, while synthesis of tools that can be manipulated has not, which makes this task both not very novel and disappointing in comparison to what more ecologically-valid tool synthesis would look like. For example, consider a variation of the task where you would have to start the tool at the red region and move it to the green region. Many of the tools used here would become invalid since you wouldn’t actually be able to fit them through the gaps (e.g. Figure 2E).

- Second, given that the “synthesis” task is more like a navigation task, the results are somewhat disappointing. When provided with a feasible solution, the model actually gets *worse* even in some of the in-sample scenes that it has seen during training (e.g. scene types C and D) which suggests that it hasn’t actually learned a good generative model of tools. Generalization performance is pretty bad across the board and is only slightly better than random, which undermines the claim in the abstract that “Our experiments demonstrate that the synthesis process modifies emergent, task-relevant object affordances in a targeted and deliberate way”. While it’s clear there is successful synthesis in some cases, I am not sure that the results support the claim that the synthesis is “targeted” or “deliberate” given how poor the overall performance is.

- Third, the model/architecture is a relatively straightforward combination of existing components and is highly specialized to the particular task. As mentioned above, this wouldn’t necessarily be a problem if the task were more interesting (i.e. not just a navigation task) and if the results were better. I do think it is cool to see this use of MONet but I’m skeptical that the particular method of optimizing in the latent space is doing anything meaningful. While there is prior work that has optimized the latent space to achieve certain tasks (as is cited in the paper), there is also a large body of work on adversarial examples which demonstrate that optimizing in the latent space is also fraught with difficulty. I also suspect this is the reason why the results are not particularly good.

2. While I do appreciate the comparisons that are in the paper (to a “Random” version of TasMON that moves in a random direction in the latent space, and to “FroMON” agent which is not allowed to backpropagate gradients from the classification loss into MONet), these comparisons are not particularly meaningful. The difference between FroMON performance and TasMON tool imagination performance (I didn’t test tool utility) across tasks is not statistically significant (z(520, 544)=-0.8588, p=.38978), so I don’t think it is valid to claim that “a task-aware latent space can still provide benefits.” The Random baseline is a pretty weak baseline and it would be more interesting to compare to an alternative plausible architecture (for example, which doesn’t use a structured latent space, or which doesn’t have a perceptual frontend and operates directly on a symbolic representation of the tools/scene).

3. Overall, I am not quite sure what I am supposed to get out of the paper. Is it that “task relevant object affordances are implicitly encoded as directions in a structured latent space shaped by experience”? If so, then the results do not support this claim and so I am not sure what to take away. Is it that the latent space encodes information about what makes a tool feasible? If so, then this is a bit of a weak argument---of *course* it must encode this information if it is able to do the classification task at all. Is it that tool synthesis is a challenging problem? If so, then the lack of strong or canonical baselines makes it hard to evaluate whether this is true (and the navigation-only synthesis task also undermines this a bit).
Some additional suggestions:

It would be good to include a discussion of other recent work on tool use such as Allen et al. (2019) and Baker et al. (2019), as well as on other related synthesis tasks such as Ha (2018) or Ganin et al. (2018).

The introduction states that “tool selection and manufacture – especially once demonstrated – is a significantly easier task than tool innovation”. While this may be true, it is a bit misleading in the context of the paper as the agent is doing something more like tool selection and modification rather than tool innovation (and actually the in-sample scenes are more like “manufacture”, which the agent doesn’t always even do well on).

It would be helpful to more clearly explain scene types. Here is some suggested phrasings: in-sample = familiar scenes with familiar tools, interpolation = novel scenes with familiar tools, extrapolation = novel scenes with novel tools.

I was originally confused how psi’ knew where to actually place the tool and at what orientation, and whether the background part of the rendering process shown in Figure 1. I realized after reading the supplemental that this is not done by the agent itself but by separate code that tries to find the orientation and position of the tool. This should be explained more clearly in the main text.

In Table 1 it would be helpful to indicate which scene types are which (in-sample, interpolation, extrapolation).

Allen, K. R., Smith, K. A., & Tenenbaum, J. B. (2019). The Tools Challenge: Rapid Trial-and-Error Learning in Physical Problem Solving. arXiv preprint arXiv:1907.09620.

Baker, B., Kanitscheider, I., Markov, T., Wu, Y., Powell, G., McGrew, B., & Mordatch, I. (2019). Emergent tool use from multi-agent autocurricula. arXiv preprint arXiv:1909.07528.

Ganin, Y., Kulkarni, T., Babuschkin, I., Eslami, S. M., & Vinyals, O. (2018). Synthesizing programs for images using reinforced adversarial learning. arXiv preprint arXiv:1804.01118.

Ha, D. (2018). Reinforcement learning for improving agent design. arXiv preprint arXiv:1810.03779.


An important goal in deep learning is to learn versatile, high-level feature representations of input data. However, standard networks' representations seem to possess shortcomings that, as we illustrate, prevent them from fully realizing this goal. In this work, we show that robust optimization can be re-cast as a tool for enforcing priors on the features learned by deep neural networks. It turns out that representations learned by robust models address the aforementioned shortcomings and make significant progress towards learning a high-level encoding of inputs. In particular, these representations are approximately invertible, while allowing for direct visualization and manipulation of salient input features. More broadly, our results indicate adversarial robustness as a promising avenue for improving learned representations.

###  Summary
1. The paper proposes robustness to small adversarial perturbations as a prior when learning representations.
2. It demonstrates that representations that satisfy such a prior have non-trivial properties -- they are easier to visualize, are invertible (i.e. optimizing an input that produces the desired activation leads to reasonable images), and allows for direct manipulation of input features (by changing a feature in the representation space and then optimizing the image to satisfy this new representation.)
### Non-blind review
THIS IS NOT A BLIND REVIEW
Reviewing this paper reminded me of a recent NeurIPS paper I read.
I went back to that NeurIPS paper (to better compare the similarities and differences) only to found out:
1- The NeurIPS19 paper cites an earlier arxiv version of this paper as an inspiration for its approach.
2- It is from the exact same authors.
This, unfortunately, means I know who the authors are (however, there is no conflict of interest).
More importantly, this paper is too similar to the NeurIPS paper and It's hard to review without taking into account the NeurIPS paper. In this review, I will treat the said NeurIPS19 paper as published work, and evaluate if this work adds more to the discourse. (I've refrained from naming the neurips paper so the anonymity is maintained for other reviewers; the authors, I presume, would immediately know which paper I'm referring to).
### Decisions with reasons
Even though I think the idea introduced in this paper is interesting, I would argue for rejecting this paper for the simple reason: It doesn't add much to the existing discourse.
Using the proposed framework (i.e. learning robust representations), it demonstrates two phenomena.
First, it shows that robust models allow feature inversion. Second, it shows that it's easily possible to directly visualize and manipulate features for such a model. (Both of these are achieved using the same idea: treating input to the model as parameterized, and optimizing for a target activation)
These are interesting observations and show that robust models learn features that rely on salient parts of the input image. However, the NeurIPS19 paper shows this even more cleary.
As a result, I'm not convinced that demonstrating the same phenomena with different examples is sufficient for this to be a standalone paper. (Perhaps the two papers could have been one single paper).
### Questions
What is the rationale behind dividing examples showing robust models rely on salient parts of input into two papers? Is there a semantic meaning to the grouping i.e. showing feature inversion, feature manipulation, and visualization in one paper and Generation, inpainting, translation, etc in another?
If I understand correctly, all of these examples exist because the robustly learned representation relies on the salient parts of the input and not on the non-robust features. If that is the case, it makes more sense to show all of these examples in a single paper.
### Update after Author's response
Since the authors have added and discussed the pertinent NeurIPS paper in this submission, I'm updating my score.
I still think that the two papers are more similar than they might seem (See Re: Response 3 for more details).
### Update 2
I pointed out the similarities between the three contributions in this paper and the NeurIPS paper in "Re: Response #3" below. The authors replied to my concerns. I'm summarizing the author's position to my concerns followed by my response.
#### Author's Position
The authors agreed that features manipulation and feature visualization is similar, but pointed out that the chain of dependency is this paper -> NeurIPS paper and not the other way around. They mentioned that the NeurIPS paper cites this paper and acknowledges this. Moreover, they argued that even if we consider NeurIPS paper to be prior work, feature visualization is explored in much more detail in this paper.
#### Response
I think the direction of the chain of dependency is not that important since neither paper clearly builds on top of the other. The NeurIPS paper is published work now, and it makes sense to consider it prior work (Especially since it is from the same authors).
Moreover, during the NeurIPS review period, the authors did not cite this paper; they only added the citation in the camera-ready version. This means that during the NeurIPS review period, they did, in fact,  take credit for the ideas used in feature painting. (The authors mention that they somehow did not, and just stated the method and showed the pictorial result in the NeurIPS paper. However, I don't see how it is possible to present a method and a pictorial result without citing other work and not take credit for the method and result.)
I would agree with the authors that this paper does go into more detail for feature visualization. More specifically, this paper also looks at visualizing individual features in the representation (The NeurIPS feature painting restricts the visualization using a mask) and demonstrates that the same feature can be used to visualize similar semantic concepts (such as red limbs) on multiple images. This is definitely interesting, but still very related to the feature-painting result. It would have made more sense to include these feature visualization results in the NeurIPS paper instead of adding them in a separate paper.
#### Author's Position 2
They disagreed that feature inversion (this paper) is similar to image generation (NeurIPS paper). I did acknowledge in my initial response that feature inversion is slightly more general than image generation, however, the authors suggest that they are completely different."
#### Response
I think representation inversion is more similar to generation than it might seem. Representation for an in-distribution image would correspond to a class with high probability. Maximizing a class probability would indirectly optimize for a representation (Say R_0) that maximizes that class probability. Image generation, as presented in NeurIPS paper, can be seen as inverting R_0.

Moreover, the qualitative results for feature inversion, as presented in this paper, are not extra-ordinary. In the majority of the inverted images, I can not classify the inverted image correctly. That shows the model is still not paying attention to the correct aspects of the input to do classification. As a result, this paper certainly does not solve the feature inversion problem (Ideally, inverted features would highlight parts of the input necessary for making predictions and ignore other parts. Robust models, on the other hand, seem to be uniformly retaining all information of the image including the background and not highlighting the parts important for making predictions. As a result, many inverted images can not be classified by humans).
#### My current position
At the end of the day, both this and the NeurIPS paper are demonstration papers (They are empirically demonstrating an unintuitive phenomenon). Both papers are demonstrating that robust models learn features that correspond to salient parts of the input. Even though both papers are nice, either one is sufficient to demonstrate the phenomenon. For this to be a stand-alone paper, the authors would have to do more in my opinion. One option would be to explore and compare different forms of adversarial robustness as priors (The paper is called "Adversarial Robustness as a Prior for Learned Representations" and not "L2 Adversarial Robustness as a Prior for Learned Representations," after all). Another option would be to see if such representations are 'quantitatively' better in some settings (Such as for transfer learning).
In its current form, I feel that the two papers are too similar to recommend acceptance.


ABSTRACT Self-supervision, in which a target task is improved without external supervision, has primarily been explored in settings that assume the availability of additional data. However, in many cases, particularly in healthcare, one may not have access to additional data (labeled or otherwise). In such settings, we hypothesize that self-supervision based solely on the structure of the data at-hand can help. We explore a novel self-supervision framework for time-series data, in which multiple auxiliary tasks (e.g., forecasting) are included to improve overall performance on a sequence-level target task without additional training data. We call this approach limited self-supervision, as we limit ourselves to only the data at-hand. We demonstrate the utility of limited self-supervision on three sequence-level classification tasks, two pertaining to real clinical data and one using synthetic data. Within this framework, we introduce novel forms of self-supervision and demonstrate their utility in improving performance on the target task. Our results indicate that limited self-supervision leads to a consistent improvement over a supervised baseline, across a range of domains. In particular, for the task of identifying atrial fibrillation from small amounts of electrocardiogram data, we observe a nearly 13% improvement in the area under the receiver operating characteristics curve (AUC-ROC) relative to the baseline (AUC-ROC=0.55 vs. AUC-ROC=0.62). Limited self-supervision applied to sequential data can aid in learning intermediate representations, making it particularly applicable in settings where data collection is difficult.

This paper propose an approach for self-supervised learning on time series.
Three datasets are considered (simulation and 2 healthcare datasets).
The gist of the contribution is to both optimize prediction loss
of the true task and at the same time do a good job for a family
of auxiliary tasks. 4 auxiliary tasks are considered. While
the first 3 auxiliary tasks are quite common, the 4th tasks
called piecewise-linear autoencoding appears novel. The idea
is that the hidden representation of the LSTM should be a good predictor
of the past using a piecewise-linear approximation.
The author coin the term "limited self-supervision" for their approach
although it's not clear why it is fundamentally not just self-supervised
learning as it has been proposed in the past.
The paper is overall well written and addresses the relevant issue
of learning from limited annotated data.
Major concerns
- It is yet another way to do self-supervised learning on time series
and no clear benchmark with alternatives is provided (time contrastive
learning (TCL) or Contrastive Predictive Coding (CPC) https://arxiv.org/pdf/1807.03748.pdf
etc.)
- On any of the applied problem it is not clear if the proposed
approach brings an improvement on the state-of-the-art or if it's
just an illustration of the method disconnected from the literature
of the application.


The field of Deep Reinforcement Learning (DRL) has recently seen a surge in the popularity of maximum entropy reinforcement learning algorithms. Their popularity stems from the intuitive interpretation of the maximum entropy objective and their superior sample efficiency on standard benchmarks. In this paper, we seek to understand the primary contribution of the entropy term to the performance of maximum entropy algorithms. For the Mujoco benchmark, we demonstrate that the entropy term in Soft Actor Critic (SAC) principally addresses the bounded nature of the action spaces. With this insight, we show how streamlined algorithms without entropy maximization can match the performance of SAC. We also propose a simple non-uniform sampling method for selecting transitions from the replay buffer during training. We further show that the streamlined algorithm with the simple non-uniform sampling scheme outperforms SAC and achieves state-of-the-art performance on challenging continuous control tasks.

The main contribution of this paper is a normalization scheme to avoid saturating the squashing function typically used to constrain actions within a bounded range in continuous control problems. It is argued that algorithms like DDPG and TD3 suffer from such saturation, which prevents proper exploration during training, while maximum entropy algorithms like Soft Actor-Critic (SAC) avoid it thanks to their entropy bonus. The main reason behind the success of SAC would thus be its ability to keep exploring throughout training, by avoiding saturation. A second contribution is a new experience replay sampling scheme, named Emphasizing Recent Experience (ERE), based on the idea that most recently added transitions should be given higher weights when sampling mini-batches from the replay bufffer. Combining both ideas leads to the SOP (Streamlined Off-Policy)+ERE algorithm, which is shown to consistently outperform SAC on Mujoco tasks.
Although this paper presents interesting insights and very good empirical results, I am currently leaning towards rejection mostly due to missing some important empirical comparisons, which hopefully can be added in a revised version.
The first key missing comparison (IMO) is to the Inverting Gradients approach from Hausknecht & Stone (2016), which the authors know about since it is cited in the related work section. Note that in that paper, the problem of saturating squashing functions preventing proper exploration was already mentioned, although not investigated in as much depth as in this submission («(…) squashing functions quickly became saturated. The resulting agents take the same discrete action with the same maximum/minimum parameters each timestep »). Their proposed Inverting Gradients technique was found to work significantly better than squashing functions, which is why I believe it should be an obvious baseline to compare to.
The other important experiments which I think need to be added are simply to implement the proposed normalization scheme within DDPG & TD3 to demonstrate its usefulness as a standalone improvement over existing algorithms. This would strengthen the claim that « algorithms such as DDPG and TD3 based on the standard objective with additive noise exploration can be greatly impaired by squashing exploration ». Without this comparison on the same benchmark, it is difficult to fully grasp the impact of this normalization.
Finally, regarding the ERE sampling scheme, I would appreciate to see SAC+ERE as well, to (hopefully) show that it can benefit SAC too (since this second contribution is orthogonal to the SOP algorithm).
Minor points:
•	I would tone down a bit the claims for « the need to revisit the benefits of entropy maximization in DRL », since better exploration has always been put forward as a major benefit (« the maximum entropy formulation provides a substantial improvement in exploration and robustness », as written in « Soft Actor-Critic Algorithms and Applications »). To me, what this submission shows is essentially that naive implementation of additive noise exploration in e.g. DDPG is very bad for exploration, more than uncovering some novel properties of SAC.
•	Below eq. 1: « the optimal policy is deterministic » => should be replaced with « there exists an optimal policy that is deterministic »
•	« principle contribution » => principal
•	The normalization scheme does not appear in Alg. 1
•	In Alg. 1 there are a Q_phi,i and a Q_phi,1 that should probably be Q_phi_i and Q_phi_1
•	The results from section E in the Appendix should be mentioned in the main text
•	In Fig. 4f the y axis’ label is a bit clipped
Update based on new revision: thank you for adding more results. From what I can see, it is difficult to conclude on the benefits of SOP over IG, which I find really problematic. It seems to me that the most impactful result is related to the improvements brought by the ERE sampling scheme, which could probably be worth a paper on its own (by showing its benefits over a wider range of algorithms, e.g. TD3 & DQN+variants), but this would be a different paper. As a result I am sticking to "Weak Reject".


We propose a method to automatically compute the importance of features at every observation in time series, by simulating counterfactual trajectories given previous observations. We define the importance of each observation as the change in the model output caused by replacing the observation with a generated one. Our method can be applied to arbitrarily complex time series models. We compare the generated feature importance to existing methods like sensitivity analyses, feature occlusion, and other explanation baselines to show that our approach generates more precise explanations and is less sensitive to noise in the input signals.

--- Overall ---
This paper proposes a method for evaluating the influence of individual observations on the output of a time series prediction model by replacing each (discrete time) observation with its conditional expectation given the other observations. They evaluate this method qualitatively on synthetic, healthcare, and climate datasets. I reviewed this paper for NeurIPS and was happy to see that the authors have made substantial improvements to the presentation and evaluation of the method. With that said, I think that the methodological contribution is incremental (sampling from a conditional rather than marginal distribution), there is at least one major correctness issue that needs to be addressed, and the analysis of the experiments fails explain why the models perform differently.
--- Major comments ---
1. The Montecarlo approximation in Algorithm 1 does not approximate Imp(i,t). Specifically, because the averaging is done before the absolute value, Algorithm 1 approximates |F(X_{0:t}) - E[F(X_{0:-t},x_{-i,t},\hat{x}_{i,t}]|. This is also a valid measure of feature importance and it is not clear from the paper why we should prefer one over the other.
2. I think the paper needs to do a better job explaining why sampling from the conditional leads to better explanations than sampling from the marginal. The second paragraph makes an argument based on variance, but it is not clear that low variance translates to better explanations. In particular, using mean imputation has very low variance, but I would expect it to give poor explanations. I recommend using a toy example to make this point. For example, in a healthcare context, doctors are reacting to changes relative to a particular patient's baseline. A conditional model can capture this baseline but a marginal model cannot.
3. In general, I thought that the experiments were well done, but stops short of explaining *why* the methods perform differently. Put differently, I think it is really important to clearly explain why certain methods fail while others succeed. For example, the authors demonstrate the sensitivity analysis fails on the synthetic data, but never explain why. I am looking for a statement of the form: "Sensitivity analysis fails on this data because... FFC solves this weakness by doing... which is reflected in the experimental results."
4. In 4.2.1, it is very unsurprising to me that a model that samples from an approximation of the conditional has higher likelihood under the conditional than samples from the marginal, but why should we expect this to lead to better explanations?
5. I thought the idea of looking at feature importance just before clinician intervention was a very clever evaluation, but I wanted the qualitative evaluation to go one step further. That is, does bicarbonate being the most important feature just before administration norepinephrine and fluids make clinical sense? Is this picking up on a specific condition and if so what condition? A clinician could tell you what they are typically reacting to when they administer fluids or vasopressors and you can compare what they say to what the model says. I was surprised to see the top features all being lab measurements as opposed to vital signs. In particular, in a vacuum, I would expect systolic blood pressure to be the most important feature in both of these cases. Is it possible that the frequency of measurements affects which features are selected as important?
6. I thought GHG experiment was *much* better and clearer in this version of the paper. Well done.
7. I recommend moving the notation from the appendix to the main paper. I don't think a reader should have to reference another document to follow notation.
--- Minor comments ---
1. Pg. 3 "The magnitude of our...": I call the authors' definition of feature importance absolute not relative. I would expect a "relative importance" to be a ratio of some sort (e.g. relative risk).
2. Pg. 5: Figure 8 --> Figure 3


We focus on solving the univariate times series point forecasting problem using deep learning. We propose a deep neural architecture based on backward and forward residual links and a very deep stack of fully-connected layers. The architecture has a number of desirable properties, being interpretable, applicable without modification to a wide array of target domains, and fast to train. We test the proposed architecture on several well-known datasets, including M3, M4 and TOURISM competition datasets containing time series from diverse domains. We demonstrate state-of-the-art performance for two configurations of N-BEATS for all the datasets, improving forecast accuracy by 11% over a statistical benchmark and by 3% over last year's winner of the M4 competition, a domain-adjusted hand-crafted hybrid between neural network and statistical time series models. The first configuration of our model does not employ any time-series-specific components and its performance on heterogeneous datasets strongly suggests that, contrarily to received wisdom, deep learning primitives such as residual blocks are by themselves sufficient to solve a wide range of forecasting problems. Finally, we demonstrate how the proposed architecture can be augmented to provide outputs that are interpretable without considerable loss in accuracy.

This goal of this paper is to present a strong empirical result showing that a "pure" machine learning based method can outperform all known methods on some of the most challenging time series forecasting benchmarks (TOURISM, M3 and especially M4). Since I am not from the field of forecasting, I can not be sure of this, but from my understanding these benchmark datasets are indeed challenging and the cited references back up the claims of the paper related to these datasets being important in the field.
On the most challenging dataset (M4), the best known performing method combines RNNs with a traditional smoothing algorithm. The model proposed in this paper outperforms it without being combined with any classical approach, though it does utilize ensembling.
The experimental setup is sound in my opinion, and the result appears to be of high potential significance.
However, despite trying to go through Section 3 multiple times, the exact model architecture is not clear to me. Due to this reason, my current decision is a weak rejection since the model is a central contribution of the paper. I will be happy to increase my score if the authors can make the model description crystal clear.
Even though my expertise is deep neural network architectures, I find it hard to follow the descriptions in Sec. 3. I faced the most difficulty understanding section 3.1, which obviously made the rest of subsections even harder to follow. Here are my main points of confusion:
- One big issue is that the paper uses an illustration (Fig. 1) to explain the architecture instead of equations, but then uses symbols in the main text that do not appear on Fig. 1 at all such as g_theta. Is the "FC Stack (4 layers)" g_theta?
- Where are (uppercase) phi functions? I could infer that these are the "FC" blocks but they should be labeled.
- The Figure has the symbols g^b_theta and g^f_theta that do not appear in the text description. What exactly do they do? And is the theta that parameterizes each of them the same theta that parameterizes g_theta? How is this possible if g_theta is the "FC Stack (4 layers)"?
- The description in second and third paragraph of Section 3.1 is very confusing and unclear. It should be replaced or augmented with equations using clearly defined symbols that match Figure 1.
- More confusion stems from the use of the term "parameters" in (I believe) a different context than is used in neural networks, where "parameters" refers to connection weights. But here parameters are outputs of some functions, so either they are not connection weights or this is a fast-weight style architecture where outputs are weights [1], in which this should be made clear.
- Design of the doubly residual architecture in Section 3.2 makes sense to me at a high level, but I feel it is still very hard to clearly understand and implement it. Again, use of equations to clearly define the computation would be very helpful.
[1] Schmidhuber, Jürgen. "Learning to control fast-weight memories: An alternative to dynamic recurrent networks." Neural Computation 4.1 (1992): 131-139.
--- Update after rebuttal ---
I am happy to see the paper greatly improved by the authors in their updates. My concerns related to the presentation of the model have been addressed, and I find the architecture much easier to understand. I also appreciate the detailed supplementary material that is likely to help readers interested in the area. Related to the areas I work in, I noticed the following missing references:
Densenets: Lang, K.J. and Witbrock, M.J., 1988, June. Learning to tell two spirals apart. In Proceedings of the 1988 connectionist models summer school (No. 1989, pp. 52-59).
Metalearning: Schmidhuber, J., 1987. Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-... hook (Doctoral dissertation, Technische Universität München).
While improving papers is generally the objective of the rebuttal phase, I suggest that the authors to not take this as an opportunity to submit unpolished papers in the first phase. That said, I have increased my rating to reflect my satisfaction with the current version of the paper.


Despite the eminent successes of deep neural networks, many architectures are often hard to transfer to irregularly-sampled and asynchronous time series that occur in many real-world datasets, such as healthcare applications. This paper proposes a novel framework for classifying irregularly sampled time series with unaligned measurements, focusing on high scalability and data efficiency. Our method SEFT (Set Functions for Time Series) is based on recent advances in differentiable set function learning, extremely parallelizable, and scales well to very large datasets and online monitoring scenarios. We extensively compare our method to competitors on multiple healthcare time series datasets and show that it performs competitively whilst significantly reducing runtime.

Summary:
The work is focused on classification of irregularly sampled and unaligned multi-modal time series. Prior work has primarily focused on imputation methods, either end-to-end or otherwise. This paper approaches the problem as a set function mapping between the time-series tuples to the class label. The proposed method is uses a set encoding of a multi-modal time series input, followed by mode-specific encoding of the tuples which are then aggregated in multiple ways prior to classification. An attention mechanism is attached in order for the model to automatically weigh the relevance of tuples for the classification. The model is compared to imputation based baselines on clinical ICU time series classification tasks. The performance mostly appears comparable across baselines but the proposed method has much better run-times.
The paper is for the most part well written, and related work well characterized. The formulation is interesting and clinically relevant as well so the choice of data-sets makes some sense. I have a few concerns about the architecture formulation and lack of clarification and intuition in what appears to be the main contribution of the paper (Sec 3.2 and 3.3) which I will detail below:
a. In the evaluation, I really want to see a decoupling between the "time encoding step" and "attention based aggregation" on the performance to figure out to isolate different sources of performance improvements. That is can there be a SEFT without time encoding? If not, why not? I encourage more ablation like studies that look at different sources of performance gains and demonstrate them in experiments.
b. The description of Sec 3.3. is really missing key motivation for the choices made around how the attention formulation is designed. For example why does the dot produce include the set elements? What if it doesn't? What is Q supposed to capture?
c. Is a_{j,i} shared across instances? Then irrespective of the number of observations per instance, the $j^{th}$ tuple gets similar weights? If not appropriate indexing will help clarify this.
d. It would be useful to provide how exactly a label is inferred for a *new* test instance.
I have some minor additional feedback (just for presentation and motivation purposes):
1. Authors make a claim in the introduction which should likely be qualified with a citation - "Furthermore, even though a decoupled imputation scheme followed by classification is generally more scalable, it may lose information that is relevant for prediction tasks". How does decoupled imputation imply loss of relevant information? By losing information about which observations are missing and relying on that for prediction? Does this clinically make sense? Or even generally?
2. In Sec 3.3, you probably mean $W_i \in R^{(im(f') + |s_j|) \times d}$. That is parenthesis are missing?
3. What are the +- std errors indicating? Is it cross validation error on a held-out test set?
4. Initially $i$ is indexing samples and by equation (3), (4) $i$ indexes time(?) and in Sec 3.3 $i$ indexes observations? How are observations defined here? is it measurement of specific modality at a specific time instance? Can you clear this in the introduction itself?
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
I have read the authors updated draft and response. The experiments section looks much better now.
1. The overall contribution has less clinical utility in my opinion as generally a patient likely deteriorates over time before an adverse outcome and therefore -- to give the model too much flexibility w.r.t. time ordering doesn't make quite as much sense. This is reflected in the fact that experimental results are not drastically better than other baselines. The authors might be able to show the utility of the method on other time series classification datasets where this is not a limitation of the data itself. However in those settings, it may be a bit hard to beat transformers. Do the authors have a sense of where the benefits of this method really are?
2. Mortality tasks are generally on the simpler side of clinical prediction problems as well. Nonetheless I think the contribution has some utility to the community. I do encourage the authors to try non--clinical datasets for a comparison
3. Please have a discussion that includes limitations and to discuss where the benefits of your methods really lie. A clear and thoughtful discussion is currently missing in your conclusions.
With that said, I am updating my score to a 6.


Convolution plays a crucial role in various applications in signal and image processing, analysis and recognition. It is also the main building block of convolution neural networks (CNNs). Designing appropriate convolution neural networks on manifold-structured point clouds can inherit and empower recent advances of CNNs to analyzing and processing point cloud data. However, one of the major challenges is to define a proper way to "sweep" filters through the point cloud as a natural generalization of the planar convolution and to reflect the point cloud's geometry at the same time. In this paper, we consider generalizing convolution by adapting parallel transport on the point cloud. Inspired by a triangulated surface based method Schonsheck et al. (2018), we propose the Narrow-Band Parallel Transport Convolution (NPTC) using a specifically defined connection on a voxelized narrow-band approximation of point cloud data. With that, we further propose a deep convolutional neural network based on NPTC (called NPTC-net) for point cloud classification and segmentation. Comprehensive experiments show that the proposed NPTC-net achieves similar or better results than current stateof-the-art methods on point clouds classification and segmentation.

The paper introduces a new geometric convolution that can be applied to point clouds approximating a 2D manifold in R^3. Similar to previous work, the continuous mathematical description of the method involves a kernel that is a function on R^2, which is pushed to the tangent spaces of the manifold via a choice of frame u. A local neighbourhood around the kernel is parameterized by the tangent space via the exponential map and matched against the kernel. The frame u, which determines the orientation of the kernel, is chosen by first picking an origin and then setting u1 to the direction of steepest ascent of the function that measures geodesic distance to the origin. The other basis vector u2 is computed as u1 x n where n is the normal vector. In a practical implementation, the point cloud is voxelized in order to compute the distance function and it's gradient. The gradient is then interpolated back to the point cloud. If I understood correctly, a convolution is then directly applied to the point cloud, though I could not find a description of how this is done.
The problem of defining geometric convolutions on point clouds is interesting and relevant, and something like what is presented in this paper might work well. Indeed the experiments demonstrate that the method produces decent scores on modelnet and S3DIS. The method is not very fast, due to the voxelization preprocessing which is required, which at .5s precludes real-time applications. Also, as discussed below, there are some issues related to singularities in the vector fields u1,u2 that are not adequately discussed. My main concern, and the reason I have given a "weak reject" rating, is that the clarity and completeness of the paper is insufficient and I find the paper hard to follow (see below). I would actually like to see an improved version published, and if the authors can address the concerns I would increase my score.
# Writing
I find that the paper is hard to read because of the way it is structured, as well as numerous small mathematical and expository issues (discussed below) and missing information.
The present paper builds on the PTC method of Schonsheck et al., but this method is not properly explained in the paper. PTC is mentioned in 1.1, but only described at a very high level. Since I had not read this paper before (I have now) the present paper was hard to follow on a first reading.
Section 1.2 is a large sub-section of the related work section that actually discusses the proposed NPTC. I think it's better not to spend so much of the related work section on this. Moreover, section 3.1.1. "General idea of NPTC", contains another high level description, which is somewhat redundant (though at the same time, both sections leave lots of questions unanswered - see below). The following sections, 3.1.2 and 3.1.3 explain the computation of the distance function and the vector fields, but contain no clear explanation of the whole algorithm.
Many important questions about how the method is actually implemented are left unanswered, even though the main contribution of this paper is at the implementation level rather than the theoretical level.
- How is voxelization done? Count the number of points per voxel? How is the voxel size chosen? How do you make sure that you obtain a "narrow band" around the surface defined by the point cloud? What kind of datastructure do you use to store a sparse voxel grid?
- How is the filter parameterized? In the theory, k is a function on R^2. How do we evaluate it at an arbitrary point in R^2?
- How is convolution done? I think it is done on points not voxels, similar to eq. 4, but this is never stated in the paper. How do you approximate the integral over the tangent space in Eq 2, if eq 2 is what the conv is based on?
I would remove section 2.1 where the notion of a connection is formally introduced. The reader will either know this concept, in which case it is unnecessary, or not, in which case the description is far too compact (one must read a textbook to really understand this notion). It may be better to include only a short intuitive introduction to parallel transport, refering the reader to other texts for mathematical definitions of basic notions like connections.
The fact that the voxelization of the point cloud is sparse is only mentioned very late in the paper. For a while I was thinking the idea would be to construct a dense voxel grid, which would be extremely memory intensive. Also, the phrase "narrow-band" is often used with a very different meaning, as in a small part of the spectrum. It was not clear to me what this was referring to until halfway through the paper. Better to explain the general idea of the algorithm in the introduction.
# Singularities
I looked at the papers citing the main related work (the PTC paper by Schonscheck et al., 2018), and of the four listed in google scholar, one seems relevant in that it prominently cites Schonscheck et al. and claims to improve upon it, but this work is not cited in the present paper:
Cohen, Weiler, Kicanaoglu, Welling, Gauge Equivalent Convolutional Networks and the Icosahedral CNN.
This work highlights one potential downside of the PTC approach, which is that it is not possible to find a global frame u (called in w in their paper) without singularities on manifolds like the sphere. Moreover, the choice of frame is fundamentally arbitrary and should not affect the results. In the PTC approach, a certain choice of frame with singularities is made, which leads to filter orientations that are 1) essentially arbitrary, and 2) change smoothly in most places but abruptly near the singularities. The results are heavily dependent on the choice of frame.
The present paper dismisses these issues by saying that "We remark that possible singularities will lead to no convolution operation at those points. These are isolated points on a closed manifold and do not effect experiment results."
However, this is not actually demonstrated in the paper, and one might reasonably believe that this problem affects more than a small finite set of points. Near a singularity (e.g. the poles of the sphere in fig 2d), nearby filters will be oriented very differently. If we draw a small circle around the singularity, the orientation of the filter changes 360 degrees as we travel around the circle. As a result, the meaning of the patterns in the output feature map is very different near the poles than it is near the equator. Nevertheless, the next layer will try to match these patterns with the same set of filters / perform weight sharing across different positions. In my opinion this issue should be more frankly discussed.
Fortunately, in some applications with point clouds this issue can be avoided. For point clouds coming directly from a (RGB+)depth sensor, the manifold is essentially like a wrinkeled sheet (homeomorphic to a disk), and one can easily find vector fields / frames without singularities on such manifolds. (this is not necessarily true for point clouds that represent a whole scene, e.g. in SLAM). I think this is what is mentioned in paragraph 3 of 3.1.1. However, it is not clear to me if the frame used in this paper is without singularities even when it is possible to find such a frame. The reason is that the frame is defined in terms of the distance function rho to an arbitrary origin. If I understand correctly, one will always have a singularity at the origin. This could in principle easily be solved though by simply using the x and y direction of the camera plane as a frame (projecting it from R^3 to each tangent space). However, if the authors decide to go that route, one would also like to see a comparison to simply applying 2d convolutions to the RGB+D images.
# Experiments
The experiments on Modelnet40 show that, when compared with other point-cloud methods, the proposed method outperforms. I do not know if there are better point-based methods than those listed in the paper, but on the official Modelnet40 leaderboard (which is incomplete) there are several methods that score better (97%+). Since this difference could possibly be attributed to a loss of information due to the discretization into a point cloud, I still think these results are quite good. Similarly on S3DIS the method scores quite well, though does not reach state of the art.
I could not find the architecture for the classification network, and only a high-level description of the segmentation architecture. More details on the training and evaluation procedures could be added for both experiments.
# Misc
Some more things I noticed below. Many of these are not serious problems, just things that can be improved / fixed.
- p2:
-- "Spatial mesh-based methods are more intuitive" - this is subjective
-- f(v)  where v is a tangent vector in T_xe M is not defined. One would expect f to be a function on the manifold. Typically one writes f(exp_x v) instead.
-- script M is not defined (though it is intuitive that this is the manifold). Later in 2.1 it is written "Let M be a 2 dimensional differential manifold". Later still (3.1) the notation is switched to P without notice.
-- TxM is not defined. Later it is explained this is the tangent plane. Neither is g_x (it is obvious this should be the metric, but only if one knows differential geometry).
-- "and do not effect experimental results" -> affect
-- "singularities from a given vector field can be overcome ...." I don't understand what this is saying. How can we avoid the issues with singularities?
-- consider writing log instead of exp^{-1}
-- "while uses tangent planes" -> using
- p3:
-- You have f(z) in the subscript of the indicator I_argmax ...
-- In the explanation of pointnet it is not clear where there are any parameters in the kernel k if k(xi, xj) = delta(xi, xj)
-- I don't understand the explanation of edge convolution. If f(x_j) = 1 (for all j(?)) then k(xi, xj) = MLP(f(xi), f(xi) - f(xj)) would equal MLP(1, 0) which makes no sense.
-- Description of figure 1: "are in parallel": need to say along what curve they are parallel. If I understand correctly, they are only in parallel if they are attached to different points on a minimizing geodesic emanating from the origin.
- p4:
-- "embed in R^3" -> embedded
-- In 2.1 it is explained that T_x M is the tangent plane, but you've already discussed T_x M several times before.
-- A vector field is not just any assignment M -> TM. It is a section of TM, ie a map M -> TM that, when composed with the projection TM -> M, yields the identity on M.
-- The interval "I" mentioned below eq 5 has not been defined. "I" was used as indicator function before.
-- "X is the unique section of Gamma(TM)". The concept of "section" has not been introduced. Moreover, one could say "section in Gamma(TM)" or "section of TM" but not "section of Gamma(TM)" since Gamma(TM) *is* the set of sections of TM. Probably best to say "unique vector field".
-- "there will be a geodesic connecting x0 and x1" - only true if M is connected.
- p5:
-- "how a distance functions are" -> function
-- "easliy"
-- below eq 6 "f(x) is a strictly positive function", but f(x) does not appear in eq 6
-- in 3.1., script P has not been defined. I first thought it is just a new letter for M, but I think it is the point cloud? Needs to be made explicit.
-- "Geodesic curve represents, in some snse, the shortest path". This is false. A geodesic is a *straightest* curve, but a geodesic need not be distance minimizing in any sense.
-- "ascend direction" -> ascent
-- In the first paragraph of 3.1.1 it is not mentioned explicitly that y plays the role of an "origin". This makes the last sentence confusing.
-- "The value f(v) is computed by f(v) = f(z) where z in P is the closest point to v". Firstly, this sentence is in the middle of a paragraph about the vector fields u1,u2, but seems unrelated. Secondly, what are v and z? I guess v is a point in M and z in P? Or is v a tangent vector? In that case, do you mean closest in the embedding space R^3?
- p6:
-- "consider point clouds in R^3". This assumption was already made before, right?
-- "it is not straightforwardly compute distance function" -> "straightforward to compute the distance function"
-- "Here Lambda is chosen as certain point on the point cloud" -> "a certain point". Also, Lambda was used before as a subset of Omega-bar (below eq 6), but here it is a point. I guess this point is the origin. In other places (earlier) you also refer to the origin, e.g. by y. Needs to be made consistent.
-- "Show that the directly" - remove "the"
-- "Finally, we interpolate the distance function from the voxels to point cloud" -> "the point cloud". On first reading, this (page 6!) was the point where I realized (guessed) that you are going to define the convolution on the point cloud not voxel grid. This needs to be explained much earlier.
-- "it is nature to" -> natural
- p7
-- "in our implement" -> implementation
-- "Appendix." -> "Appendix A" or "the Appendix."
- p8
-- "Scene semantics segmentation" -> semantic


Obtaining policies that can generalise to new environments in reinforcement learning is challenging. In this work, we demonstrate that language understanding via a reading policy learner is a promising vehicle for generalisation to new environments. We propose a grounded policy learning problem, Read to Fight Monsters (RTFM), in which the agent must jointly reason over a language goal, relevant dynamics described in a document, and environment observations. We procedurally generate environment dynamics and corresponding language descriptions of the dynamics, such that agents must read to understand new environment dynamics instead of memorising any particular information. In addition, we propose txt2π, a model that captures three-way interactions between the goal, document, and observations. On RTFM, txt2π generalises to new environments with dynamics not seen during training via reading. Furthermore, our model outperforms baselines such as FiLM and language-conditioned CNNs on RTFM. Through curriculum learning, txt2π produces policies that excel on complex RTFM tasks requiring several reasoning and coreference steps.

First of all, I should acknowledge that this is a last-minute emergency review. Second, I should also acknowledge that I haven’t actively followed the specific research topic handled in this paper, although I work quite a lot in reinforcement learning and dialogue systems. The last paper I read on the topic was Branavan et al’s “Learning to win by reading manuals in monte-carlo framework” (ACL 2011).
This paper is about language-conditioned reinforcement learning, where the agent is required to perform machine reading of documents to learn policies to solve a task in new environments. The contribution of this paper is seemingly two fold: first, a new benchmark test suite is proposed (named “read to fight monsters” funny acronym), and second, an improved neural model (named “txt2pi”) which extends FiLM proposed for related tasks. Various experiments on txt2pi are conducted to show how the model performs against baselines, as well as it behaves under different learning settings.
Overall, this paper makes a good contribution to the research on deep models for language-conditioned reinforcement learning, but I see a number of things I miss in this paper.
The first one is about the test suite RTFM. The authors claim that it poses a new challenge that the agent has to understand the language specification of the goal, the environment dynamics, and the observations *all together* (compared to prior work that misses one or two), I am not confident that it is not challenging enough.
- The main concern I have is that they are all machine-generated, and soon or later, this kind of test suite could be beaten-to-death. A more ideal test suite would be the texts collected by humans, aiming for more diversity, ambiguity, and sometimes incorrectness.
- The RTFM would combinatorily generate a large number of environments, but still, they all look pretty simple. It would be interesting to have “partially specified” documents, such as “lightning monsters are known to be weak against grandmaster’s or soldier’s weapons, but it is not specifically known”. It would be interesting to have partially observable environments, where the agent has to do some information gathering in the environment to behave optimally.
The second one is about txt2pi model proposed in the paper.
- txt2pi adequately extends FiLM to handle multiple sources of information, but what is the lesson being learned?
I find the discussions in page 15 very enlightening. Could you move the content to the main text and defer the curriculum learning details to the appendix?
In summary, this paper proposes an interesting test suite for language-conditioned reinforcement learning, but I wish it is more realistic in the language data and more complex in the RL tasks.
---
I have updated my rating based on the reviewer response. Thank you for clarifying some of my concerns. I am still a bit reserved because of the use of synthetic corpus. However, at the same time, I think there could be a lot of follow-up papers based on this work.


Training neural networks with verifiable robustness guarantees is challenging. Several existing approaches utilize linear relaxation based neural network output bounds under perturbation, but they can slow down training by a factor of hundreds depending on the underlying network architectures. Meanwhile, interval bound propagation (IBP) based training is efficient and significantly outperforms linear relaxation based methods on many tasks, yet it may suffer from stability issues since the bounds are much looser especially at the beginning of training. In this paper, we propose a new certified adversarial training method, CROWN-IBP, by combining the fast IBP bounds in a forward bounding pass and a tight linear relaxation based bound, CROWN, in a backward bounding pass. CROWN-IBP is computationally efficient and consistently outperforms IBP baselines on training verifiably robust neural networks. We conduct large scale experiments on MNIST and CIFAR datasets, and outperform all previous linear relaxation and bound propagation based certified defenses in ∞ robustness. Notably, we achieve 7.02% verified test error on MNIST at = 0.3, and 66.94% on CIFAR-10 with = 8/255.

This work proposes CROWN-IBP - novel and efficient certified defense method against adversarial attacks, by combining linear relaxation methods which tend to have tighter bounds with the more efficient interval-based methods. With an attempt to augment the IBP method with its lower computation complexity with the tight CROWN bounds, to get the best of both worlds. One of the primary contributions here is that reduction of computation complexity by an order of \Ln while maintaining similar or better bounds on error. The authors show compelling results with varied sized networks on both MNIST and CIFAR dataset, providing significant improvements over past baselines.
The paper itself is very well written, lucidly articulating the key contributions of the paper and highlighting the key results. The method and rationale behind it quite easy to follow.
Pros:
> Show significant benefits over previous baseline with 7.02% verified test error on MNIST at  \epsilon = 0.3, and 66.94% on CIFAR-10 with \epsilon = 8/255
> The proposed method is computationally viable, with up to 20X faster than linear relaxation methods with similar. better test errors and within 5-7X slower than the conventional IBP methods with worse errors
Cons:
> Extensive experiments with more advanced networks/datasets would have been more convincing, esp. given the computation efficiency that enables such experiments
> More elaborate insights into the choice of the training config/hyper-params esp. with the choice of \K_start, \K_end across the different datasets
Other comments:
> For the computational efficiency studies, it would be helpful to provide a breakdown of the costs between the different layers and operations, to better asses/confirm that benefits of CROWN-IBP method
> Impact of other complementary techniques such a lower precision/quantization? One fo the references compared against is the Gowal et al. 2018 for the as a baseline, however, it seems to be those results were obtained on a different HW platform (TPUs - motioned in Appendix-B), with potentially different computational accuracies (BFLOAT16 ?). So, this bears to question of the impact of precision on these methods and also the computation complexity.
> Since I'm not very well versed with the current baseline and state-of-art for variable robust training of DNN, it would be good to get an additional confirmation on the validity of the used baselines.


High-performance Deep Neural Networks (DNNs) are increasingly deployed in many real-world applications e.g., cloud prediction APIs. Recent advances in model functionality stealing attacks via black-box access (i.e., inputs in, predictions out) threaten the business model of such applications, which require a lot of time, money, and effort to develop. Existing defenses take a passive role against stealing attacks, such as by truncating predicted information. We find such passive defenses ineffective against DNN stealing attacks. In this paper, we propose the first defense which actively perturbs predictions targeted at poisoning the training objective of the attacker. We find our defense effective across a wide range of challenging datasets and DNN model stealing attacks, and additionally outperforms existing defenses. Our defense is the first that can withstand highly accurate model stealing attacks for tens of thousands of queries, amplifying the attacker's error rate up to a factor of 85× with minimal impact on the utility for benign users.

This paper proposed an effective defense against model stealing attacks.
Merits:
1) In general, this paper is well written and easy to follow.
2) The approach is a significant supplement to existing defense against model stealing attacks.
3) Extensive experiments.
However, I still have concerns about the current version.
I will possibly adjust my score based on the authors' response.
1) In the model stealing setting, attacker and defender are seemingly knowledge limited. This should be clarified better in Sec. 3.  It is important to highlight that the defender has no access to F_A, thus problem (4) is a black-box optimization problem for defense. Also, it is better to have a table to summarize the notations.
Additional questions on problem formulation:
a) Problem (4) only relies on the transfer set, where $x \sim P_A(x)$, right?
b) For evaluation metrics, utility and non-replicability, do they have the same D^{test}? How to determine them, in particularly for F_A?
c) One utility constraint is missing in problem (4). I noticed that it was mentioned in MAD-argmax, however, I suggest to add it to the formulation (4).
2)  The details of heuristic solver are unclear. Although the authors pointed out the pseudocode in the appendix, it lacks detailed analysis.
3) In Estimating G, how to select the surrogate model? Moreover, in the experiment, the authors mentioned that defense performances are unaffected by choice of architectures, and hence use the victim architecture for the stolen model. If possible, could the author provide results on different architecture choices for the stolen model as well as the surrogate model?
############## Post-feedback ################
I am satisfied with the authors' response. Thus, I would like to keep my positive comments on this paper. Although the paper is between 6 and 8, I finally decide to increase my score to 8 due to its novelty in formulation and extensive experiments.


Bayesian optimization is an effective tool to optimize black-box functions and popular for hyper-parameter tuning in machine learning. Traditional Bayesian optimization methods are based on Gaussian process (GP), relying on a GP-based surrogate model for sampling points of the function of interest. In this work, we consider transferring knowledge from related problems to target problem by learning an initial surrogate model for warm-starting Bayesian optimization. We propose a neural network-based surrogate model to estimate the function mean value in GP. Then we design a novel weighted Reptile algorithm with sampling strategy to learn an initial surrogate model from meta train set. The initial surrogate model is learned to be able to well adapt to new tasks. Extensive experiments show that this warm-starting technique enables us to find better minimizer or hyper-parameters than traditional GP and previous warm-starting methods.

POST-REBUTTAL FEEDBACK
Thanks for your response.
The justifications provided in the response have not convinced me to improve my score. They are at times hard to understand: For example, the authors have claimed that while their design choice is not reasonable, it is less unreasonable than the other.
SUMMARY OF REVIEW
The authors have proposed the use of a neural surrogate model in place of the GP posterior mean and a weighted Reptile algorithm to meta-learn the initial weights of the neural surrogate model. This approach appears interesting. However, there seems to be multiple highly restrictive (at times impractical) assumptions in this work that are atypical of the BO setting adopted by other meta BO algorithms and not discussed, as detailed below. Justifications are required.
Clarifications are also needed with regards to how they exactly run their algorithm in the experiments and whether the prior/initial information from related problems/meta tasks provided to the tested algorithms is fair.
DETAILED COMMENTS
The authors say that "We still use the variance in Eq. (4) to measure uncertainty, because the estimation uncertainty should be independent in individual problems." This does not seem to hold true. If a meta task or train set is indeed correlated (or provides information) to the new problem, the posterior variance/uncertainty at a point depends on the observations in the meta task or train set near to this point (see, for example,
Feurer et al. (2018)). Can the authors discuss the implications of such an assumption in their work?
Q and Q_i have always been referred to as problems. In Algorithm 3, Q is suddenly referred to as meta train set. On page 4, you have said that x^*_i is the minimizer of i-th problem Q_i(x) in meta train set. Based on these information, I assume that the authors consider x^*_i as the global minimizer and that x^*_i is known in order to compute the rewards. Can the authors discuss why is this a reasonable assumption?
In their proposed weight Reptile algorithm (Algorithm 3), the authors have also assumed access to the black-box functions of the related problems or meta tasks, which is not typical of other meta BO works that only require the existing observations or datasets of the related problems/meta tasks. As a result, compared with the existing meta BO algorithms, their proposed weighted Reptile is considerably more expensive due to the need to additionally evaluate the black-box functions of the related problems or meta tasks many times during execution. Can the authors discuss the practical implications of such an assumption and how it affects the types of problems/applications that can be considered by this work?
The authors have not provided any justification for their choice of reward on page 4. If the black-box function is indeed complex and highly varying, the distance between points may not work well at all. Can the authors provide a justification and discuss the practical implications and limitations with such a choice?
Isn't it more natural to consider a single Bayesian neural network instead of using a neural network for the mean and a GP for the variance?
For the experiments, it would be good to see two other variants of the proposed algorithm to understand the individual contributions of the neural surrogate model and weighted Reptile algorithm: one without neural surrogate model and the other with simply the use of neural surrogate model.
Can the authors explain in greater detail how they run their algorithms (Algorithms 2 and 3) in the experiments? For example, the authors say that "WRA-N starts with learned initial surrogate model". I assume that WRA-N refers to Algorithm 3 based on its acronym. Isn't the learned initial surrogate model the output of WRA-N in the first place? Also, the graphs in Fig. 2 seem to show iteration 1 to 13 in NOE (Algorithm 2). However, Algorithm 3 accepts N_T = 13 and executes NOE for N_T = 13 (and not 1, 2, or 3, ...) for each problem in each epoch. How do the authors generate the plot of WRA-N for iterations 1 to 12?
What seems to make more sense to me is that the authors in fact run Algorithm 2 instead of Algorithm 3 for each experiment and they initialize w in Algorithm 2 to the output of Algorithm 3. In any case, a clarification is needed here.
It is not clear to me whether the initial/prior information from related problems/meta tasks provided to WRA-N, TST-R, AND TSR-M is fair. Can the authors provide a justification?
To clarify, for each related problem/function, only N_T number of datapoints are used to train a corresponding neural network with 1 hidden layer of 15 hidden units?
The authors say that "Since TST-R needs base models for combination, we sample 20 points from uniform distribution in (−10, 10) to construct base models." Is this sampling procedure the same as that in (Wistuba et al., 2016)?
Can the authors explain the comparable performance of WRA-N and TST-R in Fig. 5? Why are the error bars missing?
How does the proposed approach compare with that of Feurer et al. (2018)?
Minor issues
Page 1, 3: adapt well to new tasks.
Page 2: The author says "depends on a GP-based surrogate model fitting function values without learnable parameters". This is not true: The GP hyperparameters need to be learned and they adapt to new problems.
Page 4: descent order?
Pages 4, 5: Why is there an input x to Q_i?
Algorithm 2: t^* should be at the superscript of x.
Equation 7: What is N?
Page 5: Does it make a difference in the performance when delta is set to 0?
Page 5: well define meta-features?


Generative Adversarial Networks (GANs) are powerful framework for modeling complex and high dimensional data. The training of GANs is difficult because the optimization is a min-max problem. This paper understands GANs from the perspective of duality gap and shows that the duality gap can be a good metric to evolution the difference between the true data distribution and the distribution generated by generator. Training GANs using the duality gap can provide competitive results. Furthermore, we establish the generalization error bound of the duality gap to help design the neural network architecture and select the sample size.

This paper has problems with clarity/polish and experimental design that are sufficiently severe
to merit rejection by themselves.
Regarding clarity/polish:
I am generally not super picky about these things, but there does have to be some standard.
This paper looks very hastily put together, especially pages 7 and 8.
There are many typos and unclear statements.
Just a few examples:
> Generative Adversarial Networks (GANs) are powerful framework for (in the abstract)
> be a good metric to evolution the difference (in the abstract)
> In the past few years, Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) are impactful because it has shown lots of great results for many AI tasks, (first sentence)
> It means that there is no an unanimous metric to represent the difference between the true data distribution and the generated distribution
What does this mean? People have mostly settled on using FID for this.

> It is also difficult to know whether the generated distribution is close to the true distribution, and this is often observed by human eyes.
Isn't this just restating the point made in the first sentence?
Regardless, nobody really uses human evaluation anymore - so this is just not correct.
>  It means that if the original generator and discriminator are random, it is difficult to confirm that the generator and discriminator can converge to the ideal conclusion by training with given data.
But this paper doesn't propose a way to solve that problem, so it's strange to mention this here in this way.
These issues would maybe be excusable if not for the totally inadequate experimental validation.
A non-exhaustive list of methodological problems with the (single) experiment:
1. The experiment uses a single run each of the baseline and DG-GAN, when it's well-known that GAN training runs
have inter-run variance larger than the difference in score reported in Fig 1 and 2.
2. The models have not been trained for long enough.
3. The architecture of the neural networks used for the Generator and Discriminator is very non-standard, which
probably leads to:
4. The scores achieved by the baseline are very far from state of the art, making the comparison mostly useless,
and rendering the third claim from the introduction ("We propose an new algorithm with the new metric which demonstrates better results than state-of-the-art algorithms.") completely untrue.
In light of these other issues, I haven't checked the proofs.


System identification is the process of building a mathematical model of an unknown system from measurements of its inputs and outputs. It is a key step for model-based control, estimator design, and output prediction. This work presents an algorithm for non-linear offline system identification from partial observations, i.e. situations in which the system's full-state is not directly observable. The algorithm presented, called SISL, iteratively infers the system's full state through nonlinear optimization and then updates the model parameters. We test our algorithm on a simulated system of coupled Lorenz attractors, showing our algorithm's ability to identify high-dimensional systems that prove intractable for particle-based approaches. We also use SISL to identify the dynamics of an aerobatic helicopter. By augmenting the state with unobserved fluid states, we learn a model that predicts the acceleration of the helicopter better than state-of-the-art approaches.

#######################
Rebuttal Response:
Thanks for these clarifications and updating the paper. I adapted my score to weak accept. To increase my score to accept, I would like to see evaluations of control experiments rather than just comparing the MSE.
#######################
Review:
Summary:
The paper introduces an iterative learning scheme to perform system identification on partially observable systems. The proposed approach iteratively optimizes the unobserved state and the model parameters. The proposed approach is demonstrated on a 'high'-dimensional Lorenz oscillator and existing quadcopter data. Furthermore, the authors show that existing approaches don't scale to 'high' dimensions.
Conclusion;
The paper introduces a simple iterative scheme to learn a model with a hidden state. The approach is sound and reasonable and the paper is well written but I am uncertain whether this paper fits the scope of ICLR. Maybe R:SS would be a better fit? Furthermore, the experiments can be improved, which is especially important as this paper does not propose a fundamental new approach. It would be really interesting to see this model controlling an actual quadcopter. My current vote is borderline reject but can be improved by further highlighting why quadcopter dynamics are only partially obsevable and further experiments.
Experiments:
My main concern with the paper is the experiments of more complex systems. The initial experiments demonstrating that the approach works and previous methods don't scale to high dimensions are very good and highlight important points. However, the more complex experiments might be not indicative of the performance. Especially for the quadcopter experiments, the proposed method only learns a non-linear noise model while the state transition is still linear and compares it against the purely linear case. Both methods seem qualitatively comparable and the quantitative improvement is only marginal. Therefore, I am doubtful whether the quadcopter data really requires to incorporate the partially observable model and I would prefer to see an evaluation of the proposed iterative scheme using a linear model. Within your evaluation you should compare the different optimization approaches (i.e., naive vs iterative) using the same model representation as your proposed algorithm only proposes a new optimization scheme but is not fixed to a representation. Furthermore, it would be interesting to see a non-linear state model and a non-linear observation model. Does the iterative approach also learn this model? Learning the robot dynamics model with contacts would be the best experiment to validate your approach, as this is only partially observable using the joint measurements and is non-linear. Could you setup an experiment with a simulated two-link robot, generate data containing samples with & without contact and learn the model just using q, qd, qdd, and tau. This experiment would clearly highlight that your iterative scheme can identify a partially observable model.
Minor Comments:
"We let ̃yt=yt−NNθn(ut)be the prediction errors of the trained naive model." -> it is a bit unclear whether the linear model approximates just the noise or the complete model? From the following text I assume that the complete model is modelled as linear model. Then the introduction of the naive model seems unnecessary as the naive model is not used. Could you please clarify this point?


Machine learning models, especially deep neural networks have been shown to reveal membership information of inputs in the training data. Such membership inference attacks are a serious privacy concern, for example, patients providing medical records to build a model that detects HIV would not want their identity to be leaked. Further, we show that the attack accuracy amplifies when the model is used to predict samples that come from a different distribution than the training set, which is often the case in real world applications. Therefore, we propose the use of causal learning approaches where a model learns the causal relationship between the input features and the outcome. An ideal causal model is known to be invariant to the training distribution and hence generalizes well to shifts between samples from the same distribution and across different distributions. First, we prove that models learned using causal structure provide stronger differential privacy guarantees than associational models under reasonable assumptions. Next, we show that causal models trained on sufficiently large samples are robust to membership inference attacks across different distributions of datasets and those trained on smaller sample sizes always have lower attack accuracy than corresponding associational models. Finally, we confirm our theoretical claims with experimental evaluation on 4 datasets with moderately complex Bayesian networks. We observe that neural network-based associational models exhibit upto 80% attack accuracy under different test distributions and sample sizes whereas causal models exhibit attack accuracy close to a random guess. Our results confirm the value of the generalizability of causal models in reducing susceptibility to privacy attacks.

Summary:
   The authors consider a transfer learning problem where the source distribution is P(X,Y) while the target distribution is P*(X,Y) and classifier is trained on data from the source distribution. They also assume that the causal graph generating the data (X and Y) is identical while the conditional probabilities (mechanisms) could change between the source and the target. Further, they assume that if X_C is the Markov Blanket for variable Y in P and P*,  then P(Y|X_C) = P*(Y|X_C). Therefore the best predictor in terms of cross entropy loss for both distributions is identical if it focuses on the variables in the Markov Blanket. Authors define "causal hypothesis" as the one that uses only variables in the Markov Blanket (X_C) to predict Y.
 In this setting, the authors show two sets of results: a) Out of Distribution Generalization Error is less for the optimal causal predictors from any class of causal hypotheses under any loss function. b) If we tweak the definition of differential privacy where neighboring datasets are defined by replacing one of the variables in the training set by a sample from the test distribution (I have lots of questions about this definition later on), then the authors show that causal classifiers that depend only on the Markov Blanket has lower sensitivity to the change than that of associative classifiers (that use all features). This is used to prove that the causal ones have tighter differential privacy guarantees than the associative ones
 c) Using the differential privacy results, they also show that optimal causal classifiers are more resistant to membership attacks.
The authors demonstrate the results through membership attack accuracies on causally trained and associative models on 4 datasets where the causal graph and the parameters (conditional probabilities) of the Bayesian Network are known apriori.
Major Issues:
    I have lots of issues with the theory in the paper. Thats the main reason for my recommendation.
  1. Why is h_{c,P}^{OPT} = h_{c,P*}^{OPT} ?? (Equation 23 in Page 11) - Authors say that since the markov blanket is the only thing used to predict Y for causal predictors and P (Y|X_C) = P *(Y|X_C), this should be true. But I have problems with this statement/argument. First of all, this is claimed for for any loss function L (not just Cross Entropy Loss) and particularly for a generic hypothesis class H_C (that depends only on Markov Blanket).
Consider the following counter example - Suppose all the features are in the Markov Blanket of Y (even a simpler case where all features are causal parents of Y). Suppose the true P (Y|X) is a logistic model with weights w_1 on one part of the domain D_1 and with weights w_2 for another part of the domain D_2. Suppose P is a mixture distribution of D_1 and D_2. P* is another mixture distribution (mixed differently) of D_1 and D_2.
Suppose I consider the class of logistic classifiers (but a single logistic model with one weight w) to be my hypothesis class and I use the standard logistic loss (logistic regression) on P, since a single logistic model with one slope cannot match the different slopes in different parts of the domain, it will result in some weight vector w^{opt}_1. Now, since the mixtures of D_1 and D_2 are different in the P* (but P (Y|X ) is identical), the optimal w^{opt}_2 for the P* will be different.
So for arbitrary hypothesis classes (that do not capture the true P(Y|X_c)) and for a non cross entropy loss - clearly this does not hold at all !! Covariate shifts amongst X_C alone with create a different classifier for an arbitrary loss (even if P (Y|X_C) is the same across both). In fact, the only way I see to salvage this is to assume Cross Entropy loss and talk about all soft classifiers (without restrictions to hypothesis class). But even if thats the case, then the best associational model will be the one that uses the Markov Blanket too ! .
This claim about h_C is crucially used in proof of theorem 1 (Page 11) and Proof of Corollary 1 (Page 13). This is fundamental to all theorems later. That brings into question the validity of many (if not all) the theoretical results in the paper. Authors must address this.
2. Issues regarding definition of certain quantities.
a) In equation 5 a quantity max_{x,x'} L_{x sampled from P} (h_c,f) - L{x' sampled from P*} (h_c,f) is defined - the inner quantity is a random quantity that depends on the samples x and x', Then what is the max operator over ?? - What does it mean to have worst case over samples from a distribution ??? Does it mean samples from two different domains ??
Even the quantity does not seem to be well defined.
b) Similar issue occurs in Lemma 1 - Neighboring datasets S and S' are created by first sampling S from P and then S' is obtained by replacing an arbitrary point in S by a random point from P*. Then sensitivity is defines as a max over pair of neighboring datasets - again S and S' are random samples, so what is the max over ?? If it is the worst case - why is the sampling coming in there ? Since it uses Corollary 1 - the main result inherits the same fundamental issues the has been pointed out above.
c) In theorem 2, {cal F}_a is an algorithm. What does it mean to add noise ? - Does it mean you add noise to the model parameters ?? - This is confusing at best.
Minor Issues:
1. Authors claim that connection between causality and privacy has not been explored (Page 2). Pls refer to https://arxiv.org/pdf/1710.05899.pdf where differential privacy itself is related to interventional effects in a system. This connection is very different from the scope of the current paper. However, the statement by the authors is strictly not true.
2. Why is the ground truth function f:X->Y (Section 2.2.) relevant when clearly you have distribution P (X,Y) and P*(X,Y) ?? We might as well define Loss with L (h(x), y) where (x,y) is drawn according to P. What f is never defined anywhere. Authors seems to mean the suggestion I just said in the paper. Authors could clarify. This confuses stuff in the proofs too.
3. Markov Blanket is not causal by any means in my opinion. It is just a minimal set of features conditioned on which Y does not depend on anything else. This only requires conditional independence tests to determine - a purely observational notion - in fact the markov blanket only depend on the moralized graph which does not change across the members of the equivalence class. So calling it causal is a bit confusing. If the features referred to least causal Parents - then still it would be consistent with the invariance in the Invariant Causal Prediction Literature (Peters et al 2016.) and would be causal.


Unsupervised learning is an important tool that has received a significant amount of attention for decades. Its goal is 'unsupervised recovery,' i.e., extracting salient factors/properties from unlabeled data. Because of the challenges in defining salient properties, recently, 'contrastive disentanglement' has gained popularity to discover the additional variations that are enhanced in one dataset relative to another. Existing formulations have devised a variety of losses for this task. However, all present day methods exhibit two major shortcomings: (1) encodings for data that does not exhibit salient factors are not pushed to carry no signal; and (2) introduced losses are often hard to estimate and require additional trainable parameters. We present a new formulation for contrastive disentanglement which avoids both shortcomings by carefully formulating a probabilistic model and by using non-parametric yet easily computable metrics. We show on four challenging datasets that the proposed approach is able to better disentangle salient factors.

Contributions:
1. This paper fixes two problems that appeared in common contrastive disentanglement methods.
2. The paper evaluates its method in multiple experiments.
Overall, this paper does a nice contribution to improving existing methods of contrastive disentanglement. My major concern is the novelty in this paper since all the contributions can be summarized into adding two additional loss terms, which I would like to discuss below.
1. [The missing term KL[q(s|y)||p_y(s)]]. One major claim of this paper is to add back the additional term KL[q(s|y)||p_y(s)] in order to push the encoder q(s|y) to converge to a point mass \delta\{s=0\} for background images y. According to the mathematical formula, this term is natural, but the major concern is the non-overlapping support between q(s|y) and p_y(s), since in the paper they assume p_y(s)=\delta\{s=0\}. This paper claims they fix this problem. But I do not think the solution is satisfactory, because they use another term E_q[\mu_{\phi,s}(y)^2+\sigma2_{\phi,s}(y)] to replace KL[q(s|y)||\delta\{s=0\}]. If I get it correct, the replacing term E_q[\mu(y)^2+\sigma2(y)]=H(q(s|y)) is the entropy of q(s|y). But if we check the true divergence term, KL[q(s|y)||\delta\{s=0\}]=-E_q[\log\delta\{s=0\}]-H(q(s|y)) that consists of a negative entropy term. This means the behavior of the proposed loss tries to minimize the entropy while the original term tries to maximize the entropy, regardless of it is ill-defined. Thus, the replacement of the loss term seems reasonable to me at first glance but does not quite directly fix the problem of an ill-defined KL-divergence. A more reasonable solution is to replace the KL divergence with the Wasserstein distance. Is there any difficulty if we do that?
2. [The additional distributional matching term W_2^2(q_t(z),q_b(z))]. This term makes perfect sense to me, even though it is just added artificially. I'm more curious about the way to evaluate the gradient. Normally, computing Wasserstein distance is hard due to a hard linear programming matching algorithm. In this paper, the authors seem to use an existing library to compute the optimal transportation matrix. Is that a heavy computational burden? Or one could resort to some entropy regularization methods with sinkhorn iterative solvers. I'm curious about how these two methods compare with each other.
3. Another concern is the numerical result seems not as good as the qualitative results.
Overall, I think this paper does make some changes regarding previous works on contrastive disentanglement. But their main contributions need some more explanation and justification.