We introduce two approaches to topic modeling supervised by survival analysis. Both approaches predict time-to-event outcomes while simultaneously learning topics over features that help prediction. The high-level idea is to represent each data point as a distribution over topics using some underlying topic model. Then each data point's distribution over topics is fed as input to a survival model. The topic and survival models are jointly learned. The two approaches we propose differ in the generality of topic models they can learn. The first approach finds topics via archetypal analysis, a nonnegative matrix factorization method that optimizes over a wide class of topic models encompassing latent Dirichlet allocation (LDA), correlated topic models, and topic models based on the "anchor word" assumption; the resulting survival-supervised variant solves an alternating minimization problem. Our second approach builds on recent work that approximates LDA in a neural net framework. We add a survival loss layer to this neural net to form an approximation to survival-supervised LDA. Both of our approaches can be combined with a variety of survival models. We demonstrate our approach on two survival datasets, showing that survival-supervised topic models can achieve competitive time-to-event prediction accuracy while outputting clinically interpretable topics.

The paper addresses the problem of survival analysis (predicting time until, e.g., death) using topic modeling. The point of introducing topic modeling here is to gain better insight into what helps predict survival times of unseen test subjects. This contrasts with much of the earlier work in survival analysis, which is mainly or only concerned with getting the best prediction without concern with the “thematic structure” of features. Such a thematic structure could be useful to clinicians, but analysis of the learned topic structure by clinical experts is apparently left for future work. Empirical results on pancreatitis and metabric datasets show that the methods of the paper are comparable to several established baselines.
I think the goals of the paper are well motivated, as time-to-death is not the only measure of interest and clinicians and other domain experts may want to get better insight into why a given model makes certain predictions. The two approaches presented in the paper are mostly on par with (but do not outperform) the best baselines considered in the paper, which would be fine in the approaches were particularly novel and/or the learned “thematic structures” would have demonstrated uses. Unfortunately, I feel the latter conditions are not met, as I have the following concerns:
(1) I find the paper to be relatively incremental in terms of ML methods, and I consider it to be more of an applied ML paper. The paper builds largely upon survLDA (Dawson and Kendziorsky, 2012), using Car et al. (2018)’s way of approximating LDA in a neural framework. Much of the exposition of Survival-Scholar (i.e., section 4) is based on Scholar (Card et al.). The paper presents another method based on archetypal analysis, which adds a supervised term to the loss. The archetypal approach of the paper merely consists of an “algorithmic modification” and otherwise borrows extensively from Javadi and Montanari (2019) and Bolte et al. (2014). In both cases (survLDA extension and archetypal analysis), most of the equations/algorithms can be traced back to these previous papers.
(2) The main point of the paper is to go beyond simply predicting time-to-event outcomes, and instead gaining some insight into the features helping predict time of death. The authors rightly “note that for both datasets, the vast majority of words we used require clinical expertise to interpret”, but it doesn’t seem they have given the learned topics to clinicians to judge whether they make sense or are helpful in any ways. Instead, appendix F provides lots of tables that are difficult to interpret for non-domain experts with little medical background.
(3) I have some more minor concerns (or questions) about the experimental results. See detailed comments.
Overall, I found the paper to be clear and well written, but algorithmic and empirical contributions are I think rather small (small or no empirical gains, incremental ideas).
Detailed comments/questions:
- Table 1: I don’t think it makes sense to label a system as either first or second. With differences of less than 1% and the small size the test set, most of the top systems are probably within the same confidence intervals, which I think the authors should compute and add to the tables. Many of the earlier works cited in the paper (e.g., DeepSurv, DeepHit) provide such confidence intervals, which are crucial in the case of the submission as its dataset is relatively small compared to e.g. SEER used with DeepHit.
- “For all methods, if the method does not already have a hyperparameter selection procedure, we use 5-fold cross-validation on the training data to select hyperparameters prior [..]”. Shouldn’t cross-validation for hyperparameter selection be used with all the methods of the paper, as the paper doesn’t define a validation set? I’m a bit concerned results may not be fully comparable if the selection procedure is not consistent across algorithms (e.g., 5-fold vs other methods). I presume “already have a hyperparameter selection procedure” refers to a procedure applied to the MIMIC dataset (as original hyperparameters borrowed from previous work might not work well). If so, I think the paper should say this explicitly.
- Experimental results consist of single table that leaves much to be desired in terms of understanding why a method works well or not. For example, what about indicating the number of model parameters (for the parametric methods) and learned hyperparameters. With such a small dataset, cross-validation could still be noisy and prone to selecting a bad set of hyperparameters (e.g., significant differences in terms of number of parameters). An indication of how sensitive to hyperparameters the methods are would be useful too.


Search space is a key consideration for neural architecture search. Recently, [32] found that randomly generated networks from the same distribution perform similarly, which suggests we should search for random graph distributions instead of graphs. We propose graphon as a new search space. A graphon is the limit of Cauchy sequence of graphs and a scale-free probabilistic distribution, from which graphs of different number of nodes can be drawn. By utilizing properties of the graphon space and the associated cut-distance metric, we develop theoretically motivated techniques that search for and scale up small-capacity stage-wise graphs found on small datasets to large-capacity graphs that can handle ImageNet. The scaled stage-wise graphs outperform DenseNet and randomly wired WattsStrogatz networks, indicating the benefits of graphon theory in NAS applications.

The authors propose a new search space based on graphons and explore some of its benefits such as certain theoretical properties. The architecture search shares similarities with DARTS. An important difference is that the network parameters are not shared.
The paper is well-written and the authors consider that the typical reader will not be familiar with graphons. I agree that their proposed model allows for more architectures but in practice it is not much stronger than WS-G. The argumentation with respect to parameters is unclear to me. On one hand, you manually influence the number of parameters, on the other you argue that you use less parameters. Obviously, you chose that your baselines have more parameters. How do results for WS-G look like if you reduce its parameters to match yours? In fact, you were searching for an architecture on CIFAR-10 but you did not report your results here. Instead you only report your transferred results to ImageNet. Is it possible that you also report results on CIFAR-10? Finally, you do not discuss that your graph contains only one kind of node. In many NAS methods the search space contains various types of operations. Do you think this is a problem? Is there a trivial way to extend your method to cover this as well?


Deep convolutional neural networks are now widely deployed in vision applications, but the size of training data can bottleneck their performance. Transfer learning offers the chance for CNNs to learn with limited data samples by transferring knowledge from weights pre-trained on large datasets. On the other hand, blindly transferring all learned features from the source dataset brings unnecessary computation to CNNs on the target task. In this paper, we propose attentive feature distillation and selection (AFDS) that not only adjusts the strength of regularization introduced by transfer learning but also dynamically determines which are the important features to transfer. When deploying AFDS on ResNet-101, we achieve state-of-theart computation reduction at the same accuracy budget, outperforming all existing transfer learning methods. On a 10× MACs reduction budget, transfer learned from ImageNet to Stanford Dogs 120, AFDS achieves an accuracy that is 12.51% higher than its best competitor.

This paper proposes a method called attentive feature distillation and selection (AFDS) to improve the performance of transfer learning for CNNs. The authors argue that the regularization should constrain the proximity of feature maps, instead of pre-trained model weights. Specifically, the authors proposes two modifications of loss functions: 1) Attentive feature distillation (AFD), which modifies the regularization term to learn different weights for each channel and 2) Attentive feature selection (AFS), which modifies the ConvBN layers by predicts unimportant channels and suppress them.
Overall, this is a good work in terms of theory and experimentation, thus I would recommend to accept it. The approach is well motivated, and the literature is complete and relevant. The author's argument is validated by experiments comparing the proposed AFDS method and other existing transfer learning methods.
To improve this paper, the authors are suggested to address the following issues:
1.  Section 3.5 is not well organized. Besides, it is not mentioned what value the threshold hyper-parameter delta_m is set.
2. In page 9, "MACs" is missing in the sentence "In Figure 3, ...the number of vs. the target..."


A structured understanding of our world in terms of objects, relations, and hierarchies is an important component of human cognition. Learning such a structured world model from raw sensory data remains a challenge. As a step towards this goal, we introduce Contrastively-trained Structured World Models (C-SWMs). CSWMs utilize a contrastive approach for representation learning in environments with compositional structure. We structure each state embedding as a set of object representations and their relations, modeled by a graph neural network. This allows objects to be discovered from raw pixel observations without direct supervision as part of the learning process. We evaluate C-SWMs on compositional environments involving multiple interacting objects that can be manipulated independently by an agent, simple Atari games, and a multi-object physics simulation. Our experiments demonstrate that C-SWMs can overcome limitations of models based on pixel reconstruction and outperform typical representatives of this model class in highly structured environments, while learning interpretable object-based representations.

This paper tackles the problem of learning an encoder and transition model of an environment, such that the representation learnt uses an object-centric representation which could favor compositionality and generalisation. This is trained using a contrastive max-margin loss, instead of a generative loss as previously explored. They do not consider RL or follow-up tasks leveraging these representations and transition models yet.
They perform an extensive assessment of their model, with many ablations, on 2 gridworld environments, one physical domain, and on Atari.
The paper is very well motivated, easy to follow, and most of its assumptions and decisions are sensible and well supported. They also provide interesting assessments and insights into the evaluation scheme of such transition models, which would be of interest to many practitioners of this field.
Apart from some issues presented below, I feel that this work is of good quality and would recommend it for acceptance.
1.	The model is introduced in a very clear way, and most decisions seem particularly fair. I found the presentation of the contrastive loss with margin to be clear, and the GraphNet is also well supported (although see question below).
However, two choices are surprising to me and would deserve some clarification and more space in the main text, instead of the Appendix:
	a.	Why does the object extractor only output a scalar mask? This was not extremely clear from reading the main text (and confused me when I first saw Figure 1 and 3a), but as explained in the Appendix, the CNN is forced to output a sigmoid logit between [0, 1] per object channel.

	This seems overly constraining to me, as this restricts the network to only output “1 bit” of information per “object”.
	However, maybe being able to represent other factors of these objects might be necessary to make better predictions?

	This also requires the user to select the number of output channels precisely, or the model might fail. This is visible in the Atari results, where the “objectness” is much less clear.

Did you try allowing the encoder to output more features per objects?
	Obviously this would be more complicated and would place you closer to a setting similar to MONet (Burgess et al. 2019) or IODINE (Greff et al. 2019), but this might help a lot.
	b.	It was hard to find the dimensionality D of the abstract representation $z_t$. It is only reported in the Appendix, and is set to $D=2$ for the 2D gridworld tasks and $D=4$ for Atari and the physics environments.
These are quite small, and the fact that they exactly coincide with your assumed sufficient statistics is a bit unfortunate.

	What happens if D is larger? Could you find the optimal D by some means?
2.	The GraphNet makes sense to me, but I wondered why you did not provide $a_t^j$ to $e_t^{(i, j)}$ as well? I could imagine situations where one would need the action to know if an interaction between two slots is required.
3.	Similarly, the fact that the action was directly partitioned per object (except in Atari where it was replicated), seemed slightly odd. Would it still work if it was not directly pre-aligned for the network? I.e. provide $a_t$ as conditioning for the global() module of the GraphNet, and let the network learn which nodes/edges it actually affects.
4.	In your multi-object contrastive loss, how is the mapping between slot k in $z_t$ and $\tilde{z}_t$ performed? Do you assume that a given object (say the red cube) is placed in the same $k$ slot across different scenes/timesteps?
This may actually be harder to enforce by the network than expected (e.g. with MONet, there is no such “slot stability”, see [1] for a discussion).
5.	It was unclear to me if the “grid” shown in Figure 3 (b) and 5 is “real”? I.e. are you exactly plotting your $z_t$ embeddings, and they happen to lie precisely along this grid? If yes, I feel this is a slightly stronger result as you currently present, given this means that the latent space has mirrored the transition dynamics in a rather impressive fashion.
6.	Related to that point, I found Figure 3 b) to be slightly too hard to understand and parse. The mapping of the colours of the arrows is not provided, and the correspondence between “what 3D object is actually moving where” and “which of the coloured circles correspond to which other cubes in the image” is hard to do (especially given the arbitrary rotation).
Could you add arrows/annotations to make this clearer?
Alternatively, presenting this as a sequence might help: e.g. show the sequence of real 3D images, along with the trajectory it traces on the 2D grid.
7.	Figure 4 a) was also hard to interpret. Seeing these learnt filters did not tell much, and I felt that you were trying too hard to impose meaning on these, or at least it wasn’t clear to me what to take of them directly. I would have left this in the Appendix. Figure 4 b) on the other hand was great, and I would put more emphasis on it.
8.	There are no details on how the actual test data used to generate Table 1 was created, and what “unseen environment instances” would correspond to. It would be good to add this to the Appendix, and point forward to it at the end of the first paragraph of Section 4.6, as if you are claiming that combinatorial generalization is being tested this should be made explicit. I found Table 1 to be great, complete, and easy to parse.
9.	It would be quite interesting to discuss how your work relates to [1], as the principles and goals are quite similar.
On a similar note, if you wanted to extend your 2D shape environment from a gridworld to a continuous one with more factors of variations, their Spriteworld environment [2] might be a good candidate.
References:
[1] Nicholas Watters, Loic Matthey, Matko Bosnjak, Christopher P. Burgess, Alexander Lerchner, “COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration”, 2019, https://arxiv.org/abs/1905.09275
[2] Nicholas Watters, Loic Matthey, Sebastian Borgeaud, Rishabh Kabra, Alexander Lerchner, “Spriteworld: A Flexible, Configurable Reinforcement Learning Environment”, https://github.com/deepmind/spriteworld/


Edge intelligence especially binary neural network (BNN) has attracted considerable attention of the artificial intelligence community recently. BNNs significantly reduce the computational cost, model size, and memory footprint. However, there is still a performance gap between the successful full-precision neural network with ReLU activation and BNNs. We argue that the accuracy drop of BNNs is due to their geometry. We analyze the behaviour of the full-precision neural network with ReLU activation and compare it with its binarized counterpart. This comparison suggests random bias initialization as a remedy to activation saturation in full-precision networks and leads us towards an improved BNN training. Our numerical experiments confirm our geometric intuition.

Summary: This paper tries to improve the training for the binary neural network.
Weaknesses:
[-] A lack of related works. There have been many related works about BNN in these years (after 2017), but the authors do not have a quick summary of them.
[-] More reference. e.g, when authors mention 'many related works require to store the full-precision activation map during the inference stage',  some reference is necessary.
[-] Weak Motivation: The authors argue 'We analyze the behaviour of the full-precision neural network with ReLU activation' in the abstract. However, in Section 3, I cannot find any analysis. Only writing down the backward and forward cannot be called analysis. Initialization is different from the training dynamics. Assumptions and theorems should be highlighted.
[-] Poor writing: A lot of typos. Only in the last paragraph in Section 2, I find many typos,  e.g. 'replaced replacing ReLU activation', 'any relaated works'.
Questions:
[.] In experiments, what structure is used for ResNet? ResNet-18-like or ResNet-110-like? (The results for these two kinds of structure are totally different for binary neural network, as the difference in the number of channels)
[.] In experiments, the performance of the baselines seems lower than related papers? Do the authors increase the number of channels in each layer as the other people do? It can improve the result a lot, and I wonder whether the improvement still exists in this setting.
[.] In experiments, only CIFAR10 results have been reported, but I wonder what is the error bar looks like? (Do the authors run the experiments several times and calculate the variance?)


We propose a novel defense against all existing gradient based adversarial attacks on deep neural networks for image classification problems. Our defense is based on a combination of deep neural networks and simple image transformations. While straight forward in implementation, this defense yields a unique security property which we term buffer zones. In this paper, we formalize the concept of buffer zones. We argue that our defense based on buffer zones is secure against state-of-the-art black box attacks. We are able to achieve this security even when the adversary has access to the entire original training data set and unlimited query access to the defense. We verify our security claims through experimentation using FashionMNIST, CIFAR-10 and CIFAR-100. We demonstrate < 10% attack success rate -significantly lower than what other well-known defenses offer -at only a price of a 15-20% drop in clean accuracy. By using a new intuitive metric we explain why this trade-off offers a significant improvement over prior work.

This paper proposes a defense against black box adversarial attack. The authors train an ensemble of deep networks, and output a null label when the ensemble disagree. Success of adversarial attack is defined as the ensemble outputs an incorrect label that is not null. The authors experimentally show improved robustness to adversarial attack.
The idea is itself new, but very similar ideas are well known in the literature, and it is difficult to conclude that the proposed approach is superior. Several examples are:
Defense by majority vote with ensembles has appeared several times in the literature (e.g. Pang et al 2019). The pro is that this paper proposes a novel way to create an ensemble by applying random linear transformations and rescaling to the input. But it is not clear this is superior compared to existing methods.
Randomized smoothing (Cohen et al, 2019) guarantees smoothness of the classifier (and thus robustness to perturbation attack under certain norms). Note that randomized smoothing  provides certified guarantee against a stronger attack model (white box); it also guarantees the size of the margin (or buffer as the authors call it). The intuition of this paper is based on similar ideas so it seems necessary to at least compare with randomized smoothing.
Outputting a null label is a major workhorse of adversarial defense. For example, previous work use a generative model to detect out of distribution samples; use a calibrated classifier to output null when low confidence.
Because the idea is only mildly interesting, good experimental support becomes crucial. However, I think there are several short-comings with the experiments:
The experiment contains only one (fairly old) attack method. Several recent alternatives such as SimBA (Guo et al 2019) can make the experiments more convincing.
The architecture is no longer the same for the target model (which is an ensemble with an additional random transformations) and surrogate model. It is unclear if the improvement is simply because of the difference in architecture.
The comparison to baselines seem unfair because it seems that the compared baselines do not have the option of outputting the null label. For example, a simple baseline of randomized smoothing + output the null label if the logit scores are below a threshold can make the story much stronger.

Minor comments:
Several suggestions on writing: the introduction contains much technical detail and even experimental results, and these are repeated again in later sections. The experimental section has many minor implementation details that could go into the appendix.


We propose a feature extraction for periodic signals. Virtually every mechanized transportation vehicle, power generation, industrial machine, and robotic system contains rotating shafts. It is possible to collect data about periodicity by measuring a shaft's rotation. However, it is difficult to perfectly control the collection timing of the measurements. Imprecise timing creates phase shifts in the resulting data. Although a phase shift does not materially affect the measurement of any given data point collected, it does alter the order in which all of the points are collected. It is difficult for classical methods, like multi-layer perceptron, to identify or quantify these alterations because they depend on the order of the input vectors' components. This paper proposes a robust method for extracting features from phase shift data by adding a graph structure to each data point and constructing a suitable machine learning architecture for graph data with cyclic permutation. Simulation and experimental results illustrate its effectiveness.

This paper proposes a machine learning framework for periodic data. The authors note that representing input data in vector form does not encode input coordinates relationship to one. Capturing this structure can be especially important for periodic signals. The authors address this by adding graph structure to each data point to encode relative structure about each coordinate. They then apply a graph neural network to the resulting structured data. They evaluate the method on an anomaly detection in a low data setting.
This paper would be greatly improved by an addition of a related work section. It is unclear where the novelty comes in precisely in this work because it is not very well situated within previous work on (i) anomaly detection w/ periodic signals, (ii) temporal and periodic signal processing and (iii) graph neural network approaches. Contextualized this work within these related areas would improve the clarity and readability of the work and also help frame the results.
The method is evaluated on synthetic and real datasets, comparing a couple variants of the model (one that can deal with phase shifts). Results support the main claims of the paper.
It is hard for me to assess the significance of this work since this is a very specific application of known techniques. I think the application is an important one, and also one that requires some domain knowledge, so there does appear to be a useful contribution in terms of adapting graph-based methodologies here. However, the methodology and application is outside my area of expertise.

Detailed suggestions / questions / comments:
- the ws-dimensional  vector v_i is defined as v_i = (x_{(i−1)∗ss+1}, x_{i+1}, . . . , x_{(i−1)∗ss+ws}). Is there a typo here? It's not obvious to me how the second index relates to the sequence or how this sequence is specified? Perhaps it should say  v_i = (x_{(i−1)∗ss}, x_{(i-1)*ss+1}, . . . , x_{(i−1)∗ss+ws})?
 - The authors mention alternative methods of capturing structured temporal information in the input features. For example, they suggest concatenating the original signal with the cross correlated signal. They also suggest time-frequency analysis methods (such as a Fourier transform and the wavelet transform) and applying a CNN to the time-frequency signal. They authors mention that these methods would require much more data than their graph convolution approach. I agree this is probably the case, but this would still be a useful empirical result to show the degree of data required for these alternative.
- What are previous approaches to detecting the properties explored in this work? In addition to discussing previous approaches in a related work section, some empirical analysis comparison would help contextualize this work as well.
Overall, I think this paper is a useful application of graph-based methods. The claims are verified empirically on real and synthetic data. I think it could be significantly improved with a discussion of related work and better situating of the methods / more comparisons in the results. As a result of these significant weaknesses I'm really on the fence with my recommendation -- the work is sensible but the paper has a lot of room for improvement and I'm not quite sure its ready for publication. However, it is possible underestimated the significance/impact of this work because I am not very familiar with the topic.


Deep reinforcement learning agents are known to be vulnerable to adversarial attacks. In particular, recent studies have shown that attacking a few key steps is effective for decreasing the agent's cumulative reward. However, all existing attacking methods find those key steps with human-designed heuristics, and it is not clear how more effective key steps can be identified. This paper introduces a novel reinforcement learning framework that learns key steps through interacting with the agent. The proposed framework does not require any human heuristics nor knowledge, and can be flexibly coupled with any white-box or black-box adversarial attack scenarios. Experiments on benchmark Atari games across different scenarios demonstrate that the proposed framework is comparable and could be superior to existing methods for learning key steps.

This paper proposes to learn the ‘key-steps’ at which to to apply an adversarial attack on a reinforcement learning agent. The framing of this problem is a Lagrangian relaxation of a constrained minimization problem which takes the form of an RL problem itself, where the attacking agent’s reward is the negative reward of the target agents plus a penalty (lambda, hyper-parameter) for choosing to attack. The attacking agent’s action space is binary, attack or no attack.
The RL approach is compared with a random attack policy and two heuristic methods for attacking agents in games on the Atari benchmark.
The setting addressed in this work, where the attacker only learns whether/when to attack or not is a greatly simplified version of the full problem. It is (in my opinion, but feel free to correct me) likely that the type of perturbation also being learned has even greater potential, but is (obviously) much harder.
Additionally, although the authors mention this as future work, I think the co-training setting is particularly interesting. As the authors suggest, this could lead to more robust target agents, but additionally I wonder if the type and difficulty of attacks would vary as the target agent trains.
Besides the (picky) complaints, I think the problem formulation is quite reasonable. Again, I find the larger problem extremely interesting, but perhaps this is far too intractable right now. The formulation is straightforward, so while I recognize it as a contribution, it is not a particularly large one.
On the other hand, the experimental results appear somewhat weak to me.
If you vary the lambda parameter (as in Table 1, but for all games) you should be able to get similar line plots in Figure 5&6 for the RL approach, which would give a much better comparison for the trade-offs as you get for the heuristic methods. I think this comparison would be very interesting and strengthen the existing results in those figures.
The results, currently, do not appear very significant because (1) the gap between the RL solution and the heuristics is very small and (2) these *appear* to be single runs without standard deviations displayed. Can you argue for why these results *are* in fact significant (statistically or otherwise)?
Another question raised by the results is how performance of the attacking policy varies with training. The authors point out that the training is quite small compared to the target policy, but is that because it has already found the best solution it can in that time? How would it improve with more training?
Small note on Figure 7, I think the point for these would be better made by normalizing the respective histograms.
Update:
Thank you for your responses and updating with new results. I think these provide a much better picture of the performance of the RL-based system. Although I am revising my score upward, I still think this is generally a rejection. Obviously I still think the full problem is interesting, but even the key step identification problem would be publishable if the performance was improved or further analysis helped me understand why RL is not doing better (since the heuristics are after all just heuristics). Good luck on future versions of this work.


In many machine learning applications, we are faced with incomplete datasets. In the literature, missing data imputation techniques have been mostly concerned with filling missing values. However, the existence of missing values is synonymous with uncertainties not only over the distribution of missing values but also over target class assignments that require careful consideration. In this paper, we propose a simple and effective method for imputing missing features and estimating the distribution of target assignments given incomplete data. In order to make imputations, we train a simple and effective generator network to generate imputations that a discriminator network is tasked to distinguish. Following this, a predictor network is trained using the imputed samples from the generator network to capture the classification uncertainties and make predictions accordingly. The proposed method is evaluated on CIFAR-10 image dataset as well as three real-world tabular classification datasets, under different missingness rates and structures. Our experimental results show the effectiveness of the proposed method in generating imputations as well as providing estimates for the class uncertainties in a classification task when faced with missing values.

This is a nice piece of incremental work on top of previously published GAN imputation methods. It seems to work well in the limited evaluation and is at least claimed to be easier to use for practitioners. This paper could benefit tremendously from both better evaluation and discussion. The paper would be much clearer if GI contextualized itself relative to GAIN on the one hand (which is the most similar GAN method) and multiple imputation on the other hand (of which this is almost, but not quite, an instance of).
Suggestions for improving the introduction & discussion:
* The purpose of this paper is to model uncertainties about missing values — you really should say more about probabilistic methods than "A few exceptions exist such as Bayesian models”.  At least give some motivation for why certain imputations problems couldn’t be feasibly solved by modeling the missing values in a probabilistic programming framework.
* Other GAN methods for imputation (GAIN and MisGAN) are dismissed as "often very complicated to be applied in practical setups by practitioners”. Given that the described method resembles GAIN, is it really much simpler? If so, can you be more specific when characterizing related work?
* "This is different from approaches such as multiple imputation where several predictors are trained on different imputed versions of a dataset.” — the main difference between this approach and MI is that you’re interleaving imputation and training a downstream model. Emphasize this earlier on, since it will make the whole technique easier to understand.
Suggestions for improving the evaluation:
* You’re imputing missing rectangles from an image dataset — please show us the resulting images. I would greatly prefer this to Section 4.5 — which is a very low sample size, low dimensionality example and it’s really unclear how well it generalizes to real data.
* "We also considered using root means squared error (RMSE); however, we decided not to use this measure as we observed an inconsistent behavior using RMSE in our comparisons as RMSE favors methods that show less variance rather than realistic and sharp samples from the distribution.” — I think this is a mistake, likely motivated by the proposed method doing worse under the RMSE metric. Show us several relevant metrics and then discuss their tradeoffs afterward.
* "We run each experiment multiple times (at least 4)” — please report how often each experiment was run, even better if you standardize this number.
* For Table 2, please provide accuracy without missing values as a baseline.
* Add MICE or some other “standard” imputation method as a baseline.
Suggestions for improving readability:
* Many sentences start with “in this” (e.g. “in this case”, “in this setting”, &c). Sometimes these sentences even co-occur within the same paragraph. Try to switch up the phrasing and move away from repetition.
* Not a complete sentence: "For instance, jointly training multiple generator/discriminator networks, tuning objective functions with multiple hyper-parameters, etc."
Update: I think the latest draft of the paper is a big improvement, the inclusion of a "classical" baseline, improved language and additional appendices are all welcome. I'm leaving the rating as a "weak accept" since the paper still feels rough and could use additional editing/streamlining.


To continuously improve quality and reflect changes in data, machine learningbased services have to regularly re-train and update their core models. In the setting of language models, we show that a comparative analysis of model snapshots before and after an update can reveal a surprising amount of detailed information about the changes in the data used for training before and after the update. We discuss the privacy implications of our findings, propose mitigation strategies and evaluate their effect.

This paper provides an empirical evaluation of the privacy implications of releasing updated versions of language models. The authors show how access to two sequential snapshots of a trained language model can reveal highly specific information about the content of the data used to update the model, even when that data is in-distribution.
The paper contains easy to understand, concrete experiments and results, but seems altogether a little underdeveloped. The methodology is sound, but the synthetic experiments around which much of the paper is based may not be sufficiently novel and give little indication of broader implications. It would have been more convincing if these results replicated with two splits of the same dataset, rather than identical datasets with one augmented by canary tokens.

The qualitative evaluation of subject-specific updates is also not sufficiently informative. It would have been useful to define a specific attack and see under what circumstances such an attack would succeed. In the current results, I am not convinced that any of the phrases in Table 3 represent a privacy violation.
The differential privacy experiment seems to be missing many details: what dataset was this trained on? Are the accuracy values for the training set or a separate testing set? Other works have shown that it is possible to train a differentially private language model without large sacrifices in accuracy, so it would be helpful to know what differentiates this experiment.
I would also note that the motivation, a predictive keyboard, is not a situation in which maximizing accuracy is generally desirable: users tend to find this creepy rather than helpful.
This is a nice idea but would benefit from some more polishing and more extensive testing.


We present a method that trains large capacity neural networks with significantly improved accuracy and lower dynamic computational cost. We achieve this by gating the deep-learning architecture on a fine-grained-level. Individual convolutional maps are turned on/off conditionally on features in the network. To achieve this, we introduce a new residual block architecture that gates convolutional channels in a fine-grained manner. We also introduce a generally applicable tool batch-shaping that matches the marginal aggregate posteriors of features in a neural network to a pre-specified prior distribution. We use this novel technique to force gates to be more conditional on the data. We present results on CIFAR-10 and ImageNet datasets for image classification, and Cityscapes for semantic segmentation. Our results show that our method can slim down large architectures conditionally, such that the average computational cost on the data is on par with a smaller architecture, but with higher accuracy. In particular, on ImageNet, our ResNet50 and ResNet34 gated networks obtain 74.60% and 72.55% top-1 accuracy compared to the 69.76% accuracy of the baseline ResNet18 model, for similar complexity. We also show that the resulting networks automatically learn to use more features for difficult examples and fewer features for simple examples.

Summary: This paper studies conditional channel gated networks. The network is designed to disable certain channels depending on the inputs. This can be used to save computation. The idea is built on top of “Convolutional Networks with Adaptive Inference Graphs.” The authors propose the technique of batch shaping to encourage the marginal statistics of the gating to be selective to different inputs. With similar inference time, the gated network can achieve better accuracy it can afford adding more layers in the network.
Detailed comments:
- The results show quite significance difference compared to ConvNet-AIG, which demonstrates that batch-shaping is very helpful. Table-1 show 3% increase in accuracy compared ResNet-18 by using similar inference time. It is good that the paper reports wall-clock time measurements.
- It’s good to see some visualizations in the paper, including the image samples and gate locations. I recommend to move Figure 7 to the main paper. A regular neural network can also be used to visualize the sensitivity of patterns of specific neurons. What would be the qualitative differences?
- 1-2x reduction in MAC is not super impressive, especially taking into consideration of the overhead for gathering the active channels for convolution.
- Figure 3a) plot is cut off on the right. The baselines only have a single point in the plot, I guess it is also valid to simply add/remove layers in the baseline models to generate a curve in the plot.
- ResNet-50-L0 is missing in Figure 3b). It would be better if the plots can be grouped better. Currently there are too many lines and it is hard to understand the differences.
- It would be good to see comparisons to some other alternatives to batch shaping. For example, one can penalize so that the average value is around 0.5 by using a L1 loss |E(x) – 0.5|.
- The ImageNet experiment has a very complicated set-up, where L0 loss is applied in the middle of the training. Is this necessary? How important is this step? What would happen if L0 loss is not applied in ImageNet? And what would happen if L0 loss is applied from the beginning? Why is L0 loss not applied in other experiments (e.g. CIFAR or Cityscapes), will L0 loss be beneficial on these benchmarks as well?
- There are a number of related works on adaptive spatial attention for faster inference, which can be included in the related work section.
1) M.  Figurnov,  M.  D.  Collins,  Y.  Zhu,  L.  Zhang,  J.  Huang,D.  P.  Vetrov,  and  R.  Salakhutdinov. Spatially  adaptive computation  time  for  residual  networks. CVPR, 2017.
2) X. Li, Z. Liu, P. Luo, C. C. Loy, and X. Tang.  Not all pixelsare equal:  Difficulty-aware semantic segmentation via deeplayer cascade. CVPR, 2017.
3) M. Ren, A. Pokrovsky, B. Yang, R. Urtasun. SBNet: Sparse Blocks Network for Fast Inference. CVPR, 2018.
Conclusion: The batch shaping technique introduced in this paper has significant improvement on networks that exploit conditional inference. Further understanding of the effect of L0 loss and other alternative loss function is recommended. My overall rating is weak accept.


Effective defense of deep neural networks against adversarial attacks remains a challenging problem, especially under white-box attacks. In this paper, we develop a new generative cleaning network with quantized nonlinear transform for effective defense of deep neural networks. The generative cleaning network, equipped with a trainable quantized nonlinear transform block, is able to destroy the sophisticated noise pattern of adversarial attacks and recover the original image content. The generative cleaning network and attack detector network are jointly trained using adversarial learning to minimize both perceptual loss and adversarial loss. Our extensive experimental results demonstrate that our approach outperforms the state-of-art methods by large margins in both white-box and black-box attacks. For example, it improves the classification accuracy for white-box attacks upon the second best method by more than 40% on the SVHN dataset and more than 20% on the challenging CIFAR-10 dataset.

This paper developed a method for defending deep neural networks against adversarial attacks based on generative cleaning networks with quantized nonlinear transform. The network is claimed to recover the original image while cleaning up the residual attack noise. The authors developed a detector network, which serves as the dual network of the target classifier network to be defended, to detect if the image is clean or being attacked. This detector network and the generative cleaning network are jointly trained with adversarial learning so that the detector network cannot find any attack noise in the output image of generative cleaning network. The experimental results demonstrated that the proposed approach outperforms the state-of-art methods by large margins in both white-box and black-box attacks.
A few comments:
1. It does not provide theoretical reasons why the prosed method can defend against those attacks.
2. The experiments are a bit messy and the attacks' setup need to improve.
3. The proposed defense showed only empirical results against the target attack. It seems to provide no theoretical / provable guarantees.


Recent work has studied the emergence of language among deep reinforcement learning agents that must collaborate to solve a task. Of particular interest are the factors that cause language to be compositional-i.e. express meaning by combining words which themselves have meaning. Evolutionary linguists have found that in addition to structural priors like those already studied in deep learning, the dynamics of transmitting language from generation to generation contribute significantly to the emergence of compositionality. In this paper, we introduce these cultural evolutionary dynamics into language emergence by periodically replacing agents in a population to create a knowledge gap, implicitly inducing cultural transmission of language. We show that this implicit cultural transmission encourages the resulting languages to exhibit better compositional generalization.

This paper studies whether language composition may emerge by partially re-sampling new agents inside a pool of language agents. They set up a consistent experimental setting for assessing compositionality,  assess different agent architectures, e.g., memory vs. memoryless agents,  and explore how the language remains close to each other by re-sampling new agents.
The paper is well-motivated with substantial background literature on the cognitive science and emergent communication side. The claim is clear, the hypotheses are well-stated, and the experiments look solid (I particularly appreciated the paragraph on shortcoming evaluation). In the end, I enjoy reading the paper despite its density, and I could see that the authors made quite some effort in that direction.
Improvement direction, questions:
 - The authors made were careful not to take ownership of Kottur et al. 's works. Yet, the writing sometimes gives the feeling that their work is solely an extension of Kottur's work, which is not the case. It also gives the feeling that Kottur et al. is the only valid experimental setting, which is not the case (at the authors pointed out in the related work section). Thus, I would recommend to summarise at some point the similarity/difference between the two papers, or at least stop referring the paper every two lines!
 - Replacement strategy: the authors use simple replacement strategies, and conclude that it has little impact. Althought It sounds reasonable in the current setting, the conclusion may be a bit premature. I would recommend to discuss further this result with complementary experiments could be the following: see the impact of epsilon, trying tournament strategies, why 8 populations (this sounds a bit arbitrary too). I would also like to put those observations in perspective with the evolutionary literature [1], and even provide a full paragraph in the related work section.
 - Population dynamics: I am missing a key element in the paper: an analysis of the population dynamics. Although the paper deals with generational transmissions, there are no experiments that analyze the evolution of language generations after generations. Most of the experiments deal with the final convergence state. Again, I would recommend having a look at the evolutionary literature to see which protocol they use to analyze such behavior.
 - Literature side: The authors did an excellent job on the emergent communication and cognitive science side. I think that it is worth extending the comparison further. For instance:
    * generational transmission can be studied in the light of game theory [2] where compositionality can be seen as a Nash Equilibrium between agent.
    * generational transmission is a form of dynamic distillation [3]
    * and evolutionary algorithms!
 - I had some difficulties in understanding Fig4, and the final-take away correctly. Would it be possible to give me one or two examples to correctly parse the table? More generally, I would recommend to add a few lines with some concrete and cherry-picked examples from the experiments to help the reader to have more intuition).
 - In a similar spirit, it is hard to interpret the distance in Figure 3. What would correspond to an increase of 1pt of distance? Having said that, the experiment is sound, and it is insightful.
 - reproducible: having a final table in the array in the appendix could be very helpful
 - crazy experiment: even if I am also a DRL addict, I would be curious to train one of the models with evolutionary algorithms (CMA-ES over parameters, for instance) to assess whether RL has an impact on compositionality (or it is solely the experimental protocol that matters).
 - I may have missed this point, but how many seeds did you use to run your experiments?
 - I may have also missed this point, what is the average length of the dialogue. Can you upload (non-understandable) dialogue example?
Last point... but it does not undermine the soundness of the experimental protocol!
 - In the end, Is 25 accuracy points really compositionality? What would be the score of simple strategies with overcomplete tokens? What is the score of the minimal vocab if we are only correct with one modality, two modalities?
Remarks:
 - in the introduction, you mention that previous old agents have grounded language, I am not sure whether we can speak of grounded language here, they have a predefined language, but it is not grounded.
 - Please remove the bold sentence in the introduction :) The claim is clear!
 - P11: Alg undefined
 - P12: the legend cannot be read

Conclusion
I am familiar with this type of experimental protocols, and I am well aware that they are never-ending works. There are always more experiments to do, more parameters to analyze. The final question is the following: is this paper have enough of these never-ending experiments? I think that this paper is just above this threshold by a short margin, and I vouch for weak accept.
However, I am missing at least one dynamic figure (to see the impact of the population along time, which is one of the core concepts of the paper), and there are several links with other ML communities that still have to be highlighted (especially evolutionary algorithms).
Besides, I somehow feel that the authors pursue two different goals in this paper: they both analyze memory/memoryless complete/overcomplete agents, which is somehow orthogonal to the general transmission hypothesis. Maybe, It would have made more sense to focus on one (or two) of the models and change the experimental setting on them (population size, training time, etc.)
In the end, I would favor a weak accept.
I am open to discussion regarding this scoring.
[1] Bäck, Thomas, and Frank Hoffmeister. "Extended selection mechanisms in genetic algorithms." (1991).
[2] Lanctot, Marc, et al. "A unified game-theoretic approach to multiagent reinforcement learning." Advances in Neural Information Processing Systems. 2017.
[3] Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. "Distilling the knowledge in a neural network." arXiv preprint arXiv:1503.02531 (2015).


An object can be described as the combination of primary visual attributes. Disentangling such underlying primitives is the long-term objective of representation learning. It is observed that categories have the natural hierarchical characteristics, i.e. any two objects can share some common primitives in a particular category level while they may possess their unique ones in another. However, previous works usually operate in a flat manner (i.e. in a particular level) to disentangle the representations of objects. Though they may obtain the primitives to constitute objects as the categories in that level, their results are obviously not efficient and complete. In this paper, we propose the hierarchical disentangle network (HDN) to exploit the rich hierarchical characteristics among categories to divide the disentangling process in a coarse-to-fine manner, such that each level only focuses on learning the specific representations and finally the common and unique representations in all levels jointly constitute the raw object. Specifically, HDN is designed based on an encoder-decoder architecture. To simultaneously ensure the disentanglement and interpretability of the encoded representations, a novel hierarchical generative adversarial network (GAN) is elaborately designed. Quantitative and qualitative evaluations on four object datasets validate the effectiveness of our method.

This paper proposes a method for solving the following problem: given an Image I from a labelled dataset with a label hierarchy as a tree of depth L, produce a set of vector representations {F_1, F_2 ... F_L}, such that a) the set can be used to reconstruct I as well as possible and b) each representation in the set only contains information about the label at the corresponding level in the label tree.
While I am not extremely familiar with work in disentangled representation learning, the authors claim is true to my knowledge that most work on disentangling factors does not explicitly take into account hierarchical structure as this work does. Therefore, this work appears novel and interesting to me. I will leave the assessment of the degree of novelty to other reviewers/AC who may be more familiar with the literature.
The approach is also effective, and the authors demonstrate through visualizations and experiments that the proposed model can be trained and accomplishes its objectives reasonably well. My overall decision is to accept this paper, but there are some improvements I'd like to see since I found it difficult to understand in some places.
- There is repeated use of the term "granularity" in the abstract and Sec. 1 which is undefined. What, according to the authors, is the difference between having a hierarchical structure and multi-granularity? I suggest clarifying what is meant by this, or avoid using the term (used hierarchy instead).
- In Sec. 3.2, it appears that what is meant by R_l is actually the set {R_1, ..., R_L}. This would imply that the R's from different levels are randomly combined, and the number of representations combined is always L. Is this correct? In either case, what happens here should be made much clearer. It took me several readings to arrive at this interpretation.
- It took me a while to infer how the results in Figure 3 are generated. There is a sudden switch in Sec. 4.1 from model training details to its use for semantic translation, which was not explained.
Minor suggestions:
- Please use parenthetical citations throughout the paper where appropriate (use \citep{}) to avoid breaking the flow of reading.
- Pg. 1, last line: "us human" -> "humans"
- Pg. 2, line 1: "with others" -> "to others"
- Pg. 2, line 2: "hierarchy structure" -> "hierarchical structure"
- Pg. 2: "multi-granularity nature" -> "multi-granular nature" or "hierarchical nature"
- Pg. 7, line 2: "an significant" -> "a significant"
- Pg. 7, line 4: "At the last" -> "At the end"
- Pg. 7, "it can be reached" -> "it can be seen that"
- Pg. 7: "Table.4.1 gives the evaluation results" -? "Table 1 gives ..."
- Pg. 7, Please revise: "which is deserved to make more efforts"
- Pg. 7: "to applied" -> "to be applied"
- Pg. 8, line 1: Remove "quite"; typo in "leraning"


We introduce a framework for Continual Learning (CL) based on Bayesian inference over the function space rather than the parameters of a deep neural network. This method, referred to as functional regularisation for Continual Learning, avoids forgetting a previous task by constructing and memorising an approximate posterior belief over the underlying task-specific function. To achieve this we rely on a Gaussian process obtained by treating the weights of the last layer of a neural network as random and Gaussian distributed. Then, the training algorithm sequentially encounters tasks and constructs posterior beliefs over the task-specific functions by using inducing point sparse Gaussian process methods. At each step a new task is first learnt and then a summary is constructed consisting of (i) inducing inputs -a fixed-size subset of the task inputs selected such that it optimally represents the task -and (ii) a posterior distribution over the function values at these inputs. This summary then regularises learning of future tasks, through Kullback-Leibler regularisation terms. Our method thus unites approaches focused on (pseudo-)rehearsal with those derived from a sequential Bayesian inference perspective in a principled way, leading to strong results on accepted benchmarks.

The paper develops a continual learning method based on Gaussian Processes (GPs) applied in the way introduced by prior work as Deep Kernel Learning (DKL). The proposed method summarizes tasks as sparse GPs and use them as regularizers for the subsequent tasks in order to avoid catastrophic forgetting. Salleviating the instability resulting from the representation drift.
Employing inducing point training for task memorization is a novel and interesting idea, which could be useful for the continual learning community. The fact that this approach also captures the uncertainty of the replays contributes fairly to robustness. Lastly, performing knowledge transfer by inheriting the KL term of the ELBO is also interesting, however, its theoretical implications deserve a close look. It would be enlightening to analyze which true posterior the learned model then corresponds to. Would not it be a slightly more principled Bayesian approach (i.e. one that has stronger grounds at first principles) to perform the knowledge transfer to assign the posterior of one task as the prior of the other, alternatively to keeping the entire KL term intact which employs the q(u_i) as the surrogate for q(u_j), i.e. the way introduced by Nguyen et al., 2017?
The presentation clarity of the paper is open for improvement. For instance, the abstract is written in a sort of convoluted way. I do not get how the KL divergence suddenly kicks in and for what exact purpose. Is it variational inference or a hand-designed regularization term?
I find the argumentation from Eq. 1 downwards until the end of Sec 2.1 on BNNs with stochastic weights and their relation to GPs a bit unnecessary complication. These are very well known facts. It would suffice to state briefly that the task learner is a vanilla DKL used within a vanilla sparse GP.
Figure 1 is also not so descriptive. I do not get what the GP here is exactly doing. What is input to and for which output modelity does it find a mapping? What is the calligraphic L in the figure? Is it a neural net loss or an ELBO?
In general I could not grasp why it makes sense to treat the the output layer params of a neural net treated for continual learning? They will not be sufficient to encode a task anyway, as an expressive enough neural net will leave only a linear mapping to the final layer. What happens if the intermediate representations of the input observations require a domain shift as the tasks evolve?
Overall, the presented ideas are fairly interesting and the experimental results are good enough for proof-of-concept, though not groundbreaking (behind the close relative VCL on MNIST and no comparison against VCL on Omniglot). Hence, this is a decent piece of work that lies somewhere around the borderline. My major concern is that the proposed method is conceptually not novel enough compared to Nguyen et al., 2017. My secondary concern is that the presentation is very much open to improvement in points hinted above.
--
Post-rebuttal: Thanks to authors for their tremendous effort to alleviate my concerns. The fact is that the conceptual novelty of the paper is too slim compared to VCL. As mentioned above, I even find the VCL approach more principled. I could have viewed the outcome of the paper as a slightly bigger news for the community if there was something unexpectedly positive on the reported results. However, as it appears from the comment below, the authors propose a super close variant of VCL that combines a few well-known techniques in a rather straightforward way and achieves in the end a model that performs on par with it. Under these conditions, I have hard time to find a reason to champion this paper for acceptance. That being said, I view this paper a tight borderline case due to its technical depth, hence I will not object to a reject decision either.


To reach human performance on complex tasks, a key ability for artificial systems is to understand physical interactions between objects, and predict future outcomes of a situation. This ability, often referred to as intuitive physics, has recently received attention and several methods were proposed to learn these physical rules from video sequences. Yet, most these methods are restricted to the case where no occlusions occur, narrowing the potential areas of application. The main contribution of this paper is a method combining a predictor of object dynamics and a neural renderer efficiently predicting future trajectories and explicitly modelling partial and full occlusions among objects. We present a training procedure enabling learning intuitive physics directly from the input videos containing segmentation masks of objects and their depth. Our results show that our model learns object dynamics despite significant inter-object occlusions, and realistically predicts segmentation masks up to 30 frames in the future. We study model performance for increasing levels of occlusions, and compare results to previous work on the tasks of future prediction and object following. We also show results on predicting motion of objects in real videos and demonstrate significant improvements over state-of-the-art on the object permanence task in the intuitive physics benchmark of Riochet et al. (2018).

This paper proposes a method to predict future trajectories by modeling partial and full occlusions. Although it is well-written and the topic sounds interesting, I failed to catch why this approach is required for this setting. So, to strengthen the message of this paper, I listed a couple of suggestions and comments below (from the most important to the least important):
1. It is a bit hard to catch how this model handles "diversity." Specifically, when predicting the futures, it should be able to generate stochastic outputs. However, I failed to find how diverse the output of the model is. If the output is not that stochastic, then it would be tough to believe that the model can "predict" the future; instead, it may "extrapolate" the current condition only. To reassure such concerns, I recommend reporting how diverse your output is. (One easy way is to report the variance of the predicted center mass values between multiple samples while reporting the l2 distance.)
2. For the future prediction task, it would be much better if it is compared with various state-of-the-art future prediction approaches [1, 2, 3, 4, 5, 6]. For some of the models, it could not be able to compare directly with this approach (e.g., lack of 'center of mass' information). However, it would be still okay once it is compared with other state-of-the-art results without feeding some 3D information (e.g., provide projected 2D video as an input). By doing so, I believe the readers can easily catch (1) why it is better to predict physical interaction in 3D space (instead of directly predicting from a 2D space), and (2) also why predicting occlusion is essential in this problem setting.
3. Minor comments:
(a) It is a bit hard to catch how the author computes the "aggregate pixel reconstruction error" in Table S1. I recommend adding an equation number there to make it clear.
(b) There are a couple of missing references: the last sentence on page 4, the first paragraph in Supplementary, the last sentence in Supplementary page 3, etc.
(c) \citep is often misused. Please replace some inappropriate \citep with \citet.
(d) Please check the format of the reference, as well; currently, it has various styles even for the same source/conference.

------------------------------------------------------------------
[Some comments based on the authors' rebuttal]
I thank the authors for their thorough comments and detailed explanations for each question. I carefully read the whole (not just my part), but it didn't change my mind; it would be much better if the claim comes with a more directly comparable result.
Some additional comments:
Q1-comment) I think the limitation of "learning to extrapolate"-style video prediction approach is partially presented in Reviewer #2's claim as well. Therefore, in this context, I recommend the author to show a better result to reassure the reader's concern.
Q2-comment) I at least strongly recommend to add more experiments with other baselines, rather than relying mainly on the original model of the dataset. Although the input condition of a model could be different, I at least do believe that it will help the readers to catch the benefit of your setting more clearly.
I hope this review phase would make your paper more powerful.

[1] Liang et al., Dual Motion GAN for Future-Flow Embedded Video Prediction, in ICCV, 2017
[2] Denton and Fergus, Stochastic Video Generation with a Learned Prior, in ICML, 2018
[3] Wichers et al., Hierarchical Long-term Video Prediction without Supervision, in ICML, 2018
[4] Wang et al., Video-to-Video Synthesis, in NeurIPS, 2018
[5] Heish et al., Learning to Decompose and Disentangle Representations for Video Prediction, in NeurIPS, 2018
[6] Minderer et al., Unsupervised Learning of Object Structure and Dynamics from Videos, in NeurIPS, 2019


The availability of large datasets is essential for progress in coreference and other areas of NLP. Crowdsourcing has proven a viable alternative to expert annotation, offering similar quality for better scalability. However, crowdsourcing require adjudication, and most models of annotation focus on classification tasks where the set of classes is predetermined. This restriction does not apply to anaphoric annotation, where coders relate markables to coreference chains whose number cannot be predefined. This gap was recently covered with the introduction of a mention pair model of anaphoric annotation (MPA). In this work we extend MPA to alleviate the effects of sparsity inherent in some crowdsourcing environments. Specifically, we use a nonparametric partially pooled structure (based on a stick breaking process), fitting jointly with the ability of the annotators hierarchical community profiles. The individual estimates can thus be improved using information about the community when the data is scarce. We show, using a recently published large-scale crowdsourced anaphora dataset, that the proposed model performs better than its unpooled counterpart in conditions of sparsity, and on par when enough observations are available. The model is thus more resilient to different crowdsourcing setups, and, further provides insights into the community of workers. The model is also flexible enough to be used in standard annotation tasks for classification where it registers on par performance with the state of the art.

This paper extends the unpooled mention pair model of annotation (MPA) (Paun et al., 2018b) with the hierarchical priors (e.g., mean and variance) on the ability of the annotators. The proposed method was evaluated on Phrase Detectives 2 corpus, which was annotated by players in a game with a purpose setting for coreference resolution. To control the sparsity of the dataset, the authors split annotations from larger player workloads into smaller batches, and assumed that each batch was produced by a different player. The experimental results show that, when the data is sparse, the proposed method (CommunityMPA) worked better than MPA on Phrase Detectives 2 corpus in terms of mention-pair accuracy, silver-chain quality, and the performance of the state-of-the-art method trained on the aggregated mention pairs. This paper also includes a discussion about the inferred community profiles. The comparison with the traditional approaches that consider communities showed that the proposed method is comparable to the traditional approaches.
I am wondering of the connection between community and sparsity. This study assumes that knowledge of communities (spammers, adversarial, biased, average or high-quality players) would allow to regularize the ability of the annotators towards. In P2, this paper wrote, "This partially pooled structure can prove effective in conditions of sparsity where there are not enough observations to accurately estimate the ability of the annotators in isolation." I have two questions here: why is it effective to consider community for reducing the problem of the sparsity? If the knowledge of communities is useful, why did the advantage of the proposed method disappear in Figures 1, 2, and 3 when we used all data?
In addition, I'm not convinced with the idea of "breaking the larger player workloads into smaller batches" for simulating the sparsity and communities. This treatment introduces quite a few shadow users whose capabilities are exactly the same, and deviates from the reality of the user community. Does this treatment favor the proposed method over MPA more than necessary? I am also wondering why evaluating portions of the dataset where annotations were made by 'sparse' users would not work to highlight the effectiveness of the proposed method for sparse users.
The impact of this paper would be greater if the experimental results could support the importance of modeling user community on the real data. The authors may justify the simulation procedure of the sparsity because 98.67% of Phrase Detectives 2 corpus was annotated by those who produced more than 40 annotations. However, I also think that it is important to show how the proposed method is effective on the real data with sparse annotators. Currently, Table 1 showed no improvement over the conventional methods.
Minor comment
In Section 2.1: It was difficult to separate which part is the base model (MPA) and the novel proposal without reading Paun et al. (2018b).


In the industrial field, the positron annihilation is not affected by complex environment, and the gamma-ray photon penetration is strong, so the nondestructive detection of industrial parts can be realized. Due to the poor image quality caused by gamma-ray photon scattering, attenuation and short sampling time in positron process, we propose the idea of combining deep learning to generate positron images with good quality and clear details by adversarial nets. The structure of the paper is as follows: firstly, we encode to get the hidden vectors of medical CT images based on transfer Learning, and use PCA to extract positron image features. Secondly, we construct a positron image memory based on attention mechanism as a whole input to the adversarial nets which uses medical hidden variables as a query. Finally, we train the whole model jointly and update the input parameters until convergence. Experiments have proved the possibility of generating rare positron images for industrial non-destructive testing using countermeasure networks, and good imaging results have been achieved.

This work proposes a method to transfer information from PET imaging data from the medical domain, where data is highly available, to the industrial one, where data is scarce, in the context of non-destructive material quality evaluation. The basic idea seems interesting, but unfortunately in the present form he paper is very difficult to appreciate, as it lacks of important details concerning methodology, experimental results, and comparison with respect to the state-of-the-art. Moreover, the manuscript still appears in a draft form. Sentences are often broken, the text presents many typos and grammar mistakes, and the citations are not understandable.
Missing methodological details are grouped in the following parts of this review.
Section 3.2 Encoder
Paragraph 2: The authors claim that they introduce the knowledge of migration learning without explaining it. Where does this concept come from? Is there a literature about it or is this a new concept?
Medical images are fitted though a variational auto encoder (VAE) (citation missing). The encoder description is minimal and lacks of implementation details (see Eq. 3), while the decoder description is missing throughout the paper.
Eq. (2): the authors write that they obtain a distribution p(x) according to Eq. (2), but this equation is just the formula for the sample mean, where p(x) is sampled and not computed.
Eq. (3): f1 and f2 are never made explicit in the paper so we do not know if they are linear or non-linear functions. The prior p(Z) is decomposed as a summation of posteriors p(Z|X) and the choice to have these posteriors equal to N(0,1) (1-dimensional, which is unusual) regardless of the data point X is not explained.
Section 3.3 Feature extraction memory module
Feature extraction from industrial images is done through principal component analysis (PCA). In the same paragraph is written that features are extracted through convolution neural networks (CNN). So it is not clear if there is an arbitrary choice to use PCA or CNN, and what are the conditions when this happens.
Eq. (6): the score between z_t (medical image distribution) and y_s (industrial image feature vector) is computed as scalar product dot(z_t , W_a * y_s). The key link is the linear mapping W_a, which is never made explicit in the paper. How do the authors compute W_a ?
Section 4.1 Implementation details
the networks called “identification network” and “front-end network” are not well defined. They may refer to the  VAE, the CNN, the Adversarial Nets. There is too much ambiguity. A captioned figure can help in resolving the ambiguities.
Section 4.2 Dataset
In the first paragraph the authors cite a dataset of CT images, while the main focus of the paper is on PET images. How the CT images comes into play?
Section 4.3 Evaluation
The authors compare their method with respect to other three methods, namely VAE, DCGAN, and WGAN.
The implementation details of the competing methods are not described so we cannot be sure about the fairness of the comparison.
Other considerations
Introduction, 4th paragraph: “imaging quality is higher” With respect to what? Usually PET images have the worst quality in the medical domain.
Introduction, 3rd-to-last paragraph: “We use the medical CT image ...”. Should be PET images
Related work: This section is a list of works and the relation with the current paper is not highlighted.
“lung cancer cakes” what are they?
Paragraph 3.2: “excessive pursuit of quality” why is it bad? “migration learning” do they mean transfer learning? Equation 2 and 3 in relation to a clustering problem never pointed out before in the paper.
Paragraph 4.1. What is the meaning of “Adam algorithm(=0.5)” ?
Figures 1 and 2 have very minimal captions. What do they represent?
Citation formatting problem: name and surname are switched. Journal/conferences often omitted.


The ability to learn new concepts with small amounts of data is a critical aspect of intelligence that has proven challenging for deep learning methods. Meta-learning has emerged as a promising technique for leveraging data from previous tasks to enable efficient learning of new tasks. However, most meta-learning algorithms implicitly require that the meta-training tasks be mutually-exclusive, such that no single model can solve all of the tasks at once. For example, when creating tasks for few-shot image classification, prior work uses a per-task random assignment of image classes to N-way classification labels. If this is not done, the meta-learner can ignore the task training data and learn a single model that performs all of the meta-training tasks zero-shot, but does not adapt effectively to new image classes. This requirement means that the user must take great care in designing the tasks, for example by shuffling labels or removing task identifying information from the inputs. In some domains, this makes meta-learning entirely inapplicable. In this paper, we address this challenge by designing a meta-regularization objective using information theory that places precedence on data-driven adaptation. This causes the meta-learner to decide what must be learned from the task training data and what should be inferred from the task testing input. By doing so, our algorithm can successfully use data from non-mutually-exclusive tasks to efficiently adapt to novel tasks. We demonstrate its applicability to both contextual and gradientbased meta-learning algorithms, and apply it in practical settings where applying standard meta-learning has been difficult. Our approach substantially outperforms standard meta-learning algorithms in these settings.

################################################################################
Summary:
This paper illustrates, identifies, and formally defines a memorization problem in meta-learning -- the model can simply memorize meta-training tasks and ignore meta-training train sets. The paper proposes to optimize the mutual information between testing predictions and the training data (given input and meta model), and upper bound it by imposing a information bottleneck between output and input+model. Unlike related work, this paper specifically is able to generalize to meta-test even when the meta-train dataset is not made confusing enough (i.e. even when model can learn well from test data in meta-train alone), making it applicable to use cases where it is hard to make the dataset confusing.
################################################################################
Decision
I vote for accepting this paper, since as far as I know this paper gives a novel insight to the overfitting problem in meta learning, and has formulated the problem formally with theoretical insight, and given a working solution with strong experiment results and clean comparative studies. Despite somewhat narrow experiments and sometimes confusing writing, the paper should provide new insight to meta-learning.
################################################################################
Pros:
! DISCLAIMER: I am not an expert in this field, so take my novelty judgements with a grain of salt.
+ Novel view into meta-learning's overfitting problem
(1) Large models can simply memorize which input data corresponds to which task, and memorize the meta-training tasks, without being able to generalize into meta-test tasks.
(2) Formulates this into a low mutual information between meta-train training data and predictions.
+ Easy to implement and quite widely applicable as a regularization loss addon to multiple existing meta-learning methods
+ Impact-wise, the paper takes meta-learning further from memorization, making methods more capable of operating on less-carefully designed, more natural datasets (rather than permutation of datasets)
+ Experiments are clean with ablation studies and hyperparameter sensitivity tests, and method performs well across real and toy datasets
+ Motivation part is easy to read
################################################################################
Cons:
- I'm still skeptical of the novelty of the paper, since the conclusions are, in hindsight, very straightforward.
- Sometimes sentences are very confusing to readers.
(1) The term "mutually-exclusive" is confusing because the view-point example the paper gives seems to be mutually exclusive (each task has its own kind of data, hence "exclusive"). It is unclear whether the task data is exclusive, or the task function is exclusive, and not straightforward to see its relationship with memorization. Consider renaming it to e.g. "mutually-confusing" or "mutually-contradictory mapping".
(2) Can you please rename "information-theoretic meta-regularizer" to "meta-regularizer using information theory"? It is hard to read for non-native speakers.
(3) Paragraph under definition 1 is confusing and has redundancies.
(4) Section 4.1, not very clear how the logic goes from the decomposition to adding the upper bound to the loss, and how the other term comes in.
- For the motivation, it is better to give examples of use cases when it is impossible to make meta-train "mutually-exclusive". I'm sure even in the patient example you can shuffle classes or input dimensions.
- An experiment comparing to others in mutually-exclusive datasets would be nice to have, in order to judge how much this compensates a badly-designed meta-learning dataset.
################################################################################
Improvements:
- Clarify each point in the "Cons" section.
- Please also clarify if all methods in all experiments are hyperparameter-tuned separately, i.e. that the experiments are not favoring the MR-* model in any way. (the paper only clarifies it in one of the experiments)
- For future work, does recent developments in mutual information modeling (e.g. MINE https://arxiv.org/abs/1801.04062) help this method in any way? e.g. try increasing mutual information between some representation of the meta-train training data and some feature vectors before the prediction?
- First parenthesis in Section 4.1 has a misplaced space
################################################################################
Post rebuttal
################################################################################
It seems that reviewers agree that the contributions are novel (regardless of whether each reviewer thinks it is trivial). So that addresses my main concern. I think the contributions are novel enough since it gives theoretical guidance as well. Other concerns are mostly addressed by the rebuttal. I will keep my rating.
Although I do urge the authors to reconsider the name choice "mutual-exclusive tasks" since it is not very informative and quite confusing to readers.


Label shift refers to the phenomenon where the marginal probability p(y) of observing a particular class changes between the training and test distributions, while the conditional probability p(x|y) stays fixed. This is relevant in settings such as medical diagnosis, where a classifier trained to predict disease based on observed symptoms may need to be adapted to a different distribution where the baseline frequency of the disease is higher. Given estimates of p(y|x) from a predictive model, Saerens et al. (2002)  proposed an Expectation Maximization (EM) algorithm to efficiently correct for the difference in class proportions between training and test distributions. A limiting assumption of this EM algorithm is that the probabilities p(y|x) are calibrated, which is not true of modern neural networks. Recently, Black Box Shift Learning (BBSL) (Lipton et al., 2018)  and Regularized Learning under Label Shifts (RLLS) (Azizzadenesheli et al., 2019) have emerged as state-of-the-art techniques to cope with label shift when a classifier does not output calibrated probabilities. In this work, we show that while EM in the absence of calibration achieves weak performance with modern neural networks, the combination of EM with appropriate calibration typically outperforms BBSL and RLLS. In particular, we show that the popular calibration strategy of Temperature Scaling (TS) is often insufficient to completely correct miscalibration; rather, EM achieve state-of-the-art results primarily when used with a calibration approach that includes class-specific bias parameters that TS lacks. We further introduce a theoretically principled strategy for estimating source-domain priors in EM-based domain adaptation that allows the EM algorithm to behave more robustly when the calibrated probabilities contain systematic bias. Colab notebooks reproducing experiments on CIFAR10, CIFAR100, MNIST and diabetic retinopathy detection are available at (anonymized link): https://github.com/blindauth/labelshiftexperiments

This paper builds upon recent work on detecting and correcting for label shift.
They explore both the BBSE algorithm analyzed in Detecting and Correcting for Label Shift (2018)
and another approach based on EM where the predictive posteriors and test set label distributions
are iteratively computed, each an update based on the estimate of the other.
Crucially, while the former method requires only that the confusion matrix be invertible,
the  latter method only appears valid under strong assumptions including the calibration fo the classifier.
Thus the authors propose an approach for “bias-corrected calibration”
and shows that bias-corrected calibration can improve the performance of BBSE and EM.
The method is crucial for EM and with it, the results seem to show that EM,
in the large sample (8000 examples) regime and with good initial classifiers
 (on the relatively easy CIFAR10 task with a strong baseline)
that EM outperforms BBSE.
The paper is easy to follow an the authors should also be credited for releasing
code anonymously with which we could reproduce their results.
I have as few specific concerns/questions about the paper that I would like the authors to address:
 * They consider JS divergence as a metric for evaluation. But they don’t consider other metrics
  like the error in weight estimates which is considered in most of the prior work
 * They don’t compare their results with regularizations suggested on top of BBSE, particularly Azizzadenesheli et. al. https://arxiv.org/abs/1903.09734.
 * They compare methods for particularly limited ranges of Dirichlet shift (\alpha=0.1,1.0).
  * What happens when the \alpha increases to have less severe shifts?
Optimizing ELBO with EM can lead to local convergence to the likelihood function when the likelihood is not unimodal.
 *  Is this likelihood function unimodal?  Does the EM approach converges to MLE under some appropriate initialization and assumptions?

A small presentation note: many of the papers are reporting the same metric and ought to be grouped as a large table, not as many tables. Also every table should state clearly what it is reporting in the caption, not just referring to earlier tables.
=======Update
I have read the rebuttal and appreciate that the authors took the time to establish the concavity of the likelihood function for EM. Overall this paper makes an interesting contribution in establishing the usefulness of the likelihood formulation (here optimized by EM) of label shift estimation and its apparent benefits over BBSE in some settings. I am happy to keep my score despite apparent disagreement from the other reviewers. I must say that some other reviews were disappointingly lacking in thoroughness.

The paper still leaves open some serious questions, e.g. --- why is this bias correction heuristic so effective vis-a-vis EM and is this explained by its performance at the calibration task itself? The original temperature scaling paper reported a similar heuristic yet didn't see such a benefit wrt their metrics. Why is it so useful here?
Still, while this paper can be improved in some key ways, it does make an interesting contribution.


In the Information Bottleneck (IB), when tuning the relative strength between compression and prediction terms, how do the two terms behave, and what's their relationship with the dataset and the learned representation? In this paper, we set out to answer these questions by studying multiple phase transitions in the IB objective: IB β [p(z|x)] = I(X; Z)−βI(Y ; Z) defined on the encoding distribution p(z|x) for input X, target Y and representation Z, where sudden jumps of dI(Y ;Z) dβ and prediction accuracy are observed with increasing β. We introduce a definition for IB phase transitions as a qualitative change of the IB loss landscape, and show that the transitions correspond to the onset of learning new classes. Using secondorder calculus of variations, we derive a formula that provides a practical condition for IB phase transitions, and draw its connection with the Fisher information matrix for parameterized models. We provide two perspectives to understand the formula, revealing that each IB phase transition is finding a component of maximum (nonlinear) correlation between X and Y orthogonal to the learned representation, in close analogy with canonical-correlation analysis (CCA) in linear settings. Based on the theory, we present an algorithm for discovering phase transition points. Finally, we verify that our theory and algorithm accurately predict phase transitions in categorical datasets, predict the onset of learning new classes and class difficulty in MNIST, and predict prominent phase transitions in CIFAR10.

This paper contributes theoretically to the information bottleneck (IB) principle. In particular, the author(s) provided theoretical reasoning on the phase transition phenomenon: when the beta parameter of IB varies, the generalization performance changes in a stepwise manner rather than continuously. The core result is given by theorem 1: the phase transition betas necessarily satisfy an equation, where the LHS is expressed in terms of an optimal perturbation of the encoding function X->Z.
Overall, the reviewer believes that this work is a solid contribution to the IB principle, and should be accepted. Remarkably, this work gives the first formulation on the phase transition of IB, which was an empirical observation. Furthermore, the author(s) give an algorithm to find the transition betas, which agrees with empirical studies on CIFAR and MNIST datasets.
The reasons for the weak acceptance instead of a strong acceptance are explained by the following weaknesses.
As a theoretical contribution, it is important to have good formulations and clear statements. The quality of the mathematical statements is not satisfactory and can be largely improved.
Definition 1: introduce the concept of relative perturbation first. As a definition, it must be self-contained. Therefore introduce the second-order variation first (don't point to the appendix).
Lemma 0.1: inf_r(z|x) \mathcal{G} \ge \beta, is equivalent to \forall r(z|x), \mathcal{G}\ge\beta, the latter is more elegant. What is \mathcal{G}, anyway?
Lemma 0.2:  State the condition on the scale of the $\Delta\theta$. The reviewer suspects that if $\Delta\theta$ is large enough, the statement can be violated.  This lemma feels like a digress. Why the Fisher information is needed for explaining the phase transition?
Theorem 1: the statement is fine. It is better to have all conditions/assumptions listed as this is the core theorem. To be more complete, the authors can discuss the number of roots after the theorem and what happens if there is no root. In general, there have to be a few remarks after theorem 1.
Definition 2: \rho_r(X,Y;Z) is defined with respect to what? For example, state "Given a joined distribution p(X,Y), the representation maximum correlation....". Use a different equivalent symbol for definition, such as ":=".
Theorem 2: Again, \forall{f(x,z)} is with respect to what? In the "if" part of the "if-then" statement, "\forall{f(x,z}}, it can be decomposed as .." is a false statement, change to "If f{x,z} can be decomposed"
In the abstract, avoid citations, and explain X, Y, Z, p(z|x), for completeness.
As the paper is over the recommended length,  the reviewer is asked to be harsher in the assessment.


This paper considers the growth in the length of one-dimensional trajectories as they are passed through deep ReLU neural networks, which, among other things, is one measure of the expressivity of deep networks. We generalise existing results, providing an alternative, simpler method for lower bounding expected trajectory growth through random networks, for a more general class of weights distributions, including sparsely connected networks. We illustrate this approach by deriving bounds for sparse-Gaussian, sparse-uniform, and sparse-discrete-valued random nets. We prove that trajectory growth can remain exponential in depth with these new distributions, including their sparse variants, with the sparsity parameter appearing in the base of the exponent.

In this paper, how the length of the trajectory in the input space is amplified by a ReLU neural network is analyzed. Specifically, the paper studied the case when the weights and biases are sparse random matrices. Some theoretical lower bounds are derived and are also empirically checked.
Though the results are interesting, I am slightly lean to the rejection side. The main reason is that the motivation is not strong enough and it makes the entire work somehow incremental.
As described in the paper, the analysis of the trajectory length of NN has been initiated by Raghu et al. (2017). Although Raghu et al. considered the case of densely connected NNs, this work extended the notion to the sparsely connected NNs. I feel the extension is reasonable but it is not a big "jump" from the original work, and the originality is less significant. A good point of the derived results is that they are simple and easy to understand the dependency of the variance of the weights and the sparsity level. However, I am not so excited about these results because I cannot find a practical value from them. For example, (the current form of) Corollary 1 does not tell us how to control the sparsity level \alpha to maintain some accuracy-sparsity tradeoff.
From the technical side, it is nice the proof is written in line by line. However, it looks everything is done in a normal manner and nothing special happens. Also, the condition of Theorem 2 is unclear. What does the "restriction" of \hat{w}_i mean? Does the condition E[] >= M ||u|| hold for any u and M?
Finally, I have a simple question about the relationship between the scale of the output and the trajectory length. Let W be a weight and x be an input. When the smallest singular value of W is m, x is amplified at least by m, i.e., ||Wx|| >= m||x|| for any x. This means that (a part of) the trajectory growth is explained by the smallest singular value of the sparse random matrix W. Can you clarify the difference?