Clustering is the central task in unsupervised learning and data mining. k-means is one of the most widely used clustering algorithms. Unfortunately, it is generally non-trivial to extend k-means to cluster data samples beyond Gaussian distribution, particularly, the clusters with non-convex shape (Beliakov & King, 2006). To this end, we introduce Extreme Value Theory (EVT) to improve the clustering ability of k-means. Particularly, the Euclidean space was transformed into a unique probability space denoted as extreme value space by EVT. We thus propose a novel algorithm called Extreme Value k-means (EV k-means), including GEV k-means and GPD k-means. Besides, we also introduce the tricks to accelerate Euclidean distance computation in improving the computational efficiency of classical k-means. Furthermore, our EV k-means is extended to an online version, i.e., online Extreme Value k-means, in utilizing the Mini Batch k-means to cluster streaming data. Extensive experiments are conducted to validate our EV k-means and online EV k-means on synthetic datasets and real datasets. Experimental results show that our algorithms outperform competitors in most cases.

The paper considers extending the k-means algorithm to allow for finding clusters with non-convex shapes. Particularly, it uses an existing theoretical framework (Extreme Value Theory) to maps a Euclidean space into what it calls the extreme value space, and proposes two Extreme Value k-means algorithms: GEV k-means and GPD k-means. It then provides some empirical results demonstrating their approach.
I have some concerns with the paper's claimed novelties, its empirical evaluation, and its overall presentation, and thus am initially recommending a weak reject. To raise my score, the following concerns should be clarified or addressed:
1) An initial concern is with the claimed novelty of the work. A paper by Li et al. (2012) also uses Extreme Value Theory (EVT) to improve k-means using the Generalized Extreme Value (GEV) distribution. Their algorithm is also called GEV k-means, and is based on an observation that the squared distance from a point to its closest center follows the GEV distribution for large numbers of clusters. From this, it doesn't appear to be the first time that EVT has been used to improve k-means, and it would be good for the authors to contrast their methods in the context of existing work in this direction.
2) Is it generally applicable to measure similarity based on the probability of being an extreme value, compared with classic metrics like Euclidean distance? In other words, would it always be better to do this or are there clear counterexamples where you would not want to measure similarity based on this?
3) In the empirical evaluation, it is said that 10 independent runs were performed, and the maximum result of the 10 runs was reported. I believe it would be more informative to report the mean, or expected performance of the algorithm, as well as some statistic about the mean to ensure any differences are significant. It is not clear whether this maximum can be expected or reproduced, and can negatively be interpreted as the algorithm having considerably higher variability- in other words, it could be the case that the minimum of the 10 runs for the EV methods was also lower than the minimums of classic k-means. Were any statistical tests done to ensure that the larger maximum over the runs was not by chance?
4) The paper has many frequent, but minor, grammatical and spelling errors. As such, it is possible to get the overall message (and didn't strongly impact my score), but it does detract from the paper's overall presentation and quality.
-----
Post-rebuttal:
Thank you for your clarifications regarding 1) and 2), and for now reporting the mean. However, there still does not seem to be any statistical significance testing. Further, after reporting the mean and comparing with more methods, the method doesn't seem to perform as well as previously reported. This makes the concern in 2) more prevalent- if the method is not generally applicable, and are likely to help on a problem-specific basis, it would be more informative to characterize *when* one might expect the method to perform better, and support this with empirical results. Based on this, I am maintaining my score, but think the work is interesting and encourage the authors to improve on the paper!


The Transformer architecture is popularly used in natural language processing tasks. To train a Transformer model, a carefully designed learning rate warm-up stage is usually needed: the learning rate has to be set to an extremely small value at the beginning of the optimization and then gradually increases in some given number of iterations. Such a stage is shown to be crucial to the final performance and brings more hyper-parameter tunings. In this paper, we study why the learning rate warm-up stage is important in training the Transformer and theoretically show that the location of layer normalization matters. It can be proved that at the beginning of the optimization, for the original Transformer, which places the layer normalization between the residual blocks, the expected gradients of the parameters near the output layer are large. Then using a large learning rate on those gradients makes the training unstable. The warm-up stage is practically helpful to avoid this problem. Such an analysis motivates us to investigate a slightly modified Transformer architecture which locates the layer normalization inside the residual blocks. We show that the gradients in this Transformer architecture are well-behaved at initialization. Given these findings, we are the first to show that this Transformer variant is easier and faster to train. The learning rate warm-up stage can be safely removed, and the training time can be largely reduced on a wide range of applications.

1. Specific problem tackled by the paper:
Moving the LayerNorm layer to be inside the residual connection in a stack of transformers can remove the need for learning rate warm up. This paper provides a theoretical motivation for doing this.
2. Motivation of the paper:
The authors motivate the problem clearly, performing experiments to demonstrate the problem. Their experiments provide good context for the problem they are solving and acts as a solid reference point for their (and other peoples' future) work. To be more convincing  the authors should have performed multiple runs and shown the standard deviations across runs.
3. Claims of the paper:
The authors claim that layer norm should be placed inside the residual connection (Pre-LN) rather than outside (Post-LN) it. The authors show, theoretically, that in a Post-LN transformer the magnitude of the gradients in a transformer scale with the number of layers and that the magnitude of the hidden states scale linearly with the layers that that they are output by. While in a Pre-LN transformer the magnitudes of the gradients and states are independent of the number of layers.
i.e. During training a Post-LN transformer is likely to have weaker gradients in the lower (closer to input layers) than at the output layers.
This theory is backed by an empirical study. It is good that these experiments are repeated ten times however the authors should show the standard deviations too.
The authors show machine translation results, demonstrating that using Pre-LN rather than Post-LN leads to faster convergence, however the models converge to the same result. However, there is benefit in not having to design a training schedule. Wang et al. have also shown the benefits of Pre-LN rather than Post-LN transformers for machine translation.
4. Decision (accept or reject) with one or two key reasons for this choice and reasons for the decision.
Weak accept.
Pros:
(1) The authors provide theory that supports the use of Pre-LN rather than Post-LN transformers. Using Pre-LN rather than Post-LN transformers may save a lot of time by avoiding hyper-parameter tuning, without loss in performance and this looks very easy to implement.
(2) The paper is well organised and the problem is very well motivated.
(3) The experiments are sufficient. They could be improved with repeats and by showing the standard deviations (as mentioned above).
Cons:
(1) This work is incremental. Wang et al. have already shown that Pre-LN transformers are better that Post-LN transformers when the network has many transformer layers, they explain theoretically why this is the case.
(2) While the organisation of the paper is good, the paper is not well written. The gramma is poor.
(3) The paper is very long for an incremental improvement.
5. Additional feedback with the aim to improve the paper.
[a] Ideally, it would be good to see repeats for Figure 2 and the standard deviations for Figure 3.
[b] Without reading the appendix it is not clear where the assumption that W^Q and W^K are zero is used. Making some connection with how this assumption relates to the lemmas would be useful. Additionally, you should explain what this means qualitatively, because this makes the assumption more acceptable. I am assuming that it means that the attention is uniform.
[c] In Lemma two you are comparing the magnitudes for the input in the Pre-LN and the output in the Post-LN transformer according to how x_{l,i}^post and x_{l, i}^pre are defined in Table 1.
[d] In Figure 3(b) the gradients are clearly decreasing with the number of layers, are there any comments on this? In the limit this could cause vanishing gradients?
[e] On the surface Figure 2 and 4 appear to contradict. Is the difference a result of using RAdam? If so, this should be made very clear. If not, why are the results contradictory?
More minor comments:
[1] Many gramma errors.
[2] Figure 1 is referenced before explaining what an FFN is. Figure 1 could also be enhanced by labelling Post-LN as previous work and Pre-LN as current work.
[3] MultiHeadAtt is not well defined. Multi-Head( ), Attention( ) and Head( ) each take three arguments, while MutliHeadAtt( ) takes two. It would be worth connecting these.
[4]"sub-layer --> ..." here sub-layer is not defined. Should this say self-attention sub-layer?
[5] Where does equation (4) come from? There should be a citation and/or explanation.
[6] The BLEU score is not defined (this could be with a footnote).
[7] Top of page 7: "As most of the parameters are initialized by Gaussian distributions" --> Using the word "most" is very vague. The authors should be specific about which parameters they are referring to.
[8] There is a good balance of equations in the main text with most of the proofs in the appendix.
[9] Inconsistent use of LN and LayerNorm, both are used.


Neural conditional text generation systems have achieved significant progress in recent years, showing the ability to produce highly fluent text. However, the inherent lack of controllability in these systems allows them to hallucinate factually incorrect phrases that are unfaithful to the source, making them often unsuitable for many real world systems that require high degrees of precision. In this work, we propose a novel confidence oriented decoder that assigns a confidence score to each target position. This score is learned in training using a variational Bayes objective, and can be leveraged at inference time using a calibration technique to promote more faithful generation. Experiments on a structured data-to-text datasetWikiBio (Lebret et al., 2016) -show that our approach is more faithful to the source than existing state-of-the-art approaches, according to both automatic metrics and human evaluation.

This paper presents a method for conditional text generation that has higher factual precision, minimizing hallucination of facts. The method involves predicting confidence of generation at each time step and using this confidence measure to skip tokens during generation and calibrate output probabilities in test time. Their method achieves SoTA performance on automatically measured precision and human evaluated "faithfulness." However their method does see a drop in recall (automatic metric and human evaluation).
Comments and issues,
- The intuitive explanation for the confidence score is a little confusing. In Section 4, page 3, you say that "If a token is likely a content word (i.e. when its generation probability by the encoder-decoder is much higher than the unconditioned language model), but the attention score is low, then the token might not be predicted based on the source, and could be hallucination." However, this doesn't seem like an airtight conclusion. Isn't it possible that the base-LM and enc-dec model have similar probabilities for a content word with the enc-dec attention being low? This seems possible given your observation that low attention to the source is what may be causing content hallucination. This same thing is essentially restated in section 4.1 "we expect P(y_t |y_<t, x) to be higher than P(y_t | y_<t) for content words so the confidence score will largely depend on the attention score", which seems more tangled up since P(y_t |y_<t, x) inherently depends on the attention score. This is all clarified when you explain the alteration made to the base-LM. I would recommend rewording/rearranging some of the earlier explanation for the efficacy of the confidence score since it seems that the alteration to the base-LM is an essential part of the explanation.
- Need some explanation for Equation 6. I don't really get the intuition behind it.
- The presented results are pretty good! However, I would like to see some numbers on average score across a few runs.
- It would also be good to see results on one more dataset like E2E.
- Provide a little more detail on human evaluation, you don't even mention if the evaluation was done with crowd-workers or another pool of people like grad students. How many annotators? What is the inter-annotator agreement? What was the prompt/structure? Human evaluation of models is notoriously difficult, more details would give some more weight to the results.
I think this is a well written paper with thought out experiments. I recommend it be accepted to ICLR. I'd also be curious to see some future work that improves, or at least maintains recall, while keeping the higher precision.
Minor requests/recommendations:
- Include more examples of generations. Could be an appendix.


Stagnant zone segmentation with U-net
Abstract-Silo discharging and monitoring the process for industrial or research application depend on computerized segmentation of different parts of images such as stagnant and flowing zones which is the toughest task. X-ray Computed Tomography (CT) is one of a powerful non-destructive technique for cross-sectional images of a 3D object based on X-ray absorption. CT is the most proficient for investigating different granular flow phenomena and segmentation of the stagnant zone as compared to other imaging techniques. In any case, manual segmentation is tiresome and erroneous for further investigations. Hence, automatic and precise strategies are required. In the present work, a U-net architecture is used for segmenting the stagnant zone during silo discharging process. This proposed image segmentation method provides fast and effective outcomes by exploiting a convolutional neural networks technique with an accuracy of 97 percent.

- This paper simply proposes to use UNet for the segmentation of stagnant zones in X-ray CTs. While the applicability of this model may represent an advance in the particular field of the authors, the technical contribution of this paper is far from the level expected in this conference.
- As the paper reads, the main contribution of the paper is the modified version of UNet 'proposed' by the authors, which major modification consists on replacing SGD by Adam. Nevertheless, this cannot be considered a contribution, as changing the optimizer in a deep model is a marginal change, from a methodological point of view.
- Overall, the quality of the paper is below the standards of ICLR (content, technical contribution, length).
- The submission is not anonymized (authors included their names and affiliations).


Federated Learning (FL) refers to learning a high quality global model based on decentralized data storage, without ever copying the raw data. A natural scenario arises with data created on mobile phones by the activity of their users. Given the typical data heterogeneity in such situations, it is natural to ask how can the global model be personalized for every such device, individually. In this work, we point out that the setting of Model Agnostic Meta Learning (MAML), where one optimizes for a fast, gradient-based, few-shot adaptation to a heterogeneous distribution of tasks, has a number of similarities with the objective of personalization for FL. We present FL as a natural source of practical applications for MAML algorithms, and make the following observations. 1) The popular FL algorithm, Federated Averaging , can be interpreted as a meta learning algorithm. 2) Careful fine-tuning can yield a global model with higher accuracy, which is at the same time easier to personalize. However, solely optimizing for the global model accuracy yields a weaker personalization result. 3) A model trained using a standard datacenter optimization method is much harder to personalize, compared to one trained using Federated Averaging, supporting the first claim. These results raise new questions for FL, MAML, and broader ML research.

Update: I thank authors for the rebuttal. I agree that direction of exploring personalization in FL is interesting. With a stronger methodological contribution, this could become a good paper.
----------------------------------------------------------------------------------------------------------------
The main contribution of this paper is to notice the connection between Federated Averaging (FedAvg) and Model Agnostic Meta Learning algorithms (MAML). Authors also consider an algorithm that first trains with FedAvg and then continues training using Reptile.
Pros:
Interpretation of FedAvg as a meta-learning algorithm is interesting.
Cons:
Very limited methodological contribution. Proposed algorithm is essentially two existing algorithms applied one after another.
Experiments are not conducted rigorously enough. There are many arbitrary hyperparameter choices which may bias the conclusions made in the paper. Statement "We tried SGD with a range of other learning rates, and in all cases we observed this value to work the best." is alarming suggesting that authors tried a variation of settings observing test data performance and reported a few selected runs. Although "each experiment was repeated 9 times with random initialization", the train/test split of the clients was fixed. Randomizing over client train/test split could help to improve the reliability of the results.
EMNIST-62 is the only dataset analyzed in some detail. This dataset has drastically varying P(y|x) across clients, i.e. some people write zeros as some others write 'o's. This suggests that it is very hard to train a good global model and personalization is necessary. However this doesn't mean that Shakespeare dataset "does not provide any interesting insights". Perhaps, it is indeed more interesting and challenging, demanding more advanced methodology.
In Figure 1, number of communication rounds may be impractical for FL (considering also addition 200 Reptile rounds). On Shakespeare, FedAvg paper reports 54% accuracy achieved in under 50 communication rounds in one of the settings. There are also recent works on improving communication efficiency that were not discussed or studied for personalization quality, e.g. FedProx from "Federated Optimization in Heterogeneous Networks" and PFNM from "Bayesian Nonparametric Federated Learning of Neural Networks".
Questions about Figure 2 experiments:
1. Fine-tuning requires 200 extra epochs over the initially trained model. What's the initial model accuracy when FedAvg is further trained with Adam optimizer for 200 extra communication rounds?
2. The personalized test accuracy with FedAvg and Reptile fine-tuning reaches the same value in 10 update epochs, even when Reptile fine-tuning gets 200 extra initial training epochs. Does Reptile fine-tuning provide additional benefits to the initial model as compared to running FedAvg for more number of epochs?


Biological intelligence can learn to solve many diverse tasks in a data efficient manner by re-using basic knowledge and skills from one task to another. Furthermore, many of such skills are acquired without explicit supervision in an intrinsically driven fashion. This is in contrast to the state-of-the-art reinforcement learning agents, which typically start learning each new task from scratch and struggle with knowledge transfer. In this paper we propose a principled way to learn a basis set of policies, which, when recombined through generalised policy improvement, come with guarantees on the coverage of the final task space. In particular, we concentrate on solving goal-based downstream tasks where the execution order of actions is not important. We demonstrate both theoretically and empirically that learning a small number of policies that reach intrinsically specified goal regions in a disentangled latent space can be re-used to quickly achieve a high level of performance on an exponentially larger number of externally specified, often significantly more complex downstream tasks. Our learning pipeline consists of two stages. First, the agent learns to perform intrinsically generated, goal-based tasks in the total absence of environmental rewards. Second, the agent leverages this experience to quickly achieve a high level of performance on numerous diverse externally specified tasks.

This paper proposes to pre-train policies on some goal-reaching tasks, and then leverage the associated successor features to improve the learning of a new task. The method heavily draws from the Generalized Policy Evaluation/Improvement framework without adding much to it. The only relevant point would be showing (as the title indicates) how to obtain disentangled cumulants, and whether they help transfer to new tasks. Nevertheless, both the definition, the full method, and the claimed benefits are quite ambiguous.
Among other concerns showing that the theory needs more formal treatment, the pillar definition of “Optimal independent controllability” is very confusing because it seems to depend on “a trajectory generated by following \pi_i^*”. But what If the environment is stochastic? Then following that policy might give different trajectories! This definition needs to be revisited. More concerning examples are given at the end of this review.
On the experimental side, Fig. 4 is the only reported result, and it has an x-axis that is not clearly explained. What are the “steps (min)”?
It is also not clear what they mean by the “off-diagonal trick”, which seems so important for the good performance.
Furthermore, it seems that their method doesn’t really learn anything new in most of the tasks, it just stays at the same performance that is started with after the whole pre-training steps. It is not clearly stated how much computation effort is required to obtain the desired cumulants, and this invalidates quite strongly any result they report. Even if there’s no “reward” needed during the pre-training, which arguably is not even true because you do need the rewards related to whether you have achieved a specific change in a feature!
In fact, it would be greatly appreciated if the “final” tasks could be expressed in a similar notation than the rest of the pre-training tasks, or vice-versa. As far as I understand, the pre-training tasks consist of making a certain feature fall into a certain subset of its possible values. Can’t the final tasks, like “move the agent to the top right” be also expressed in that form. The link between the two kinds of tasks needs to be much more explicit to be able to assess the relevance of this work.
Finally, they only test their algorithm on Spritworld, which is a small discrete state-action space environment. Even if they try different kinds of tasks in this environment, more detailed analysis or more extensive experiments are needed to assess the benefits of the proposed approach.
This is particularly timely because their method relies on a discretization of some given features that represent the state, which will probably not be very practical in higher dimensional environments.
Finally, I would like a comment on how this method interacts with discrete versus continuous action-state spaces.
Misc comments:
- Why do the authors introduce the terminology “Endogenous RL”, and then say it’s the same as doing RL with intrinsic motivation? This seems like introducing a new name for the same concept, which seems pointless and confusing.
- The connection with “latent learning” of Tolman 1984 is very unclear.
- There’s a “Representation Learning” section, but it’s not clear at all whether any features are actually learned, or whether the features are actually hand-defined. Is the number of features n also hand-defined?
- There might be a typo in the first sentence after equation (9): “While \phi_w is not guaranteed to be optimal with respect to \phi_w”.
Because of all these concerns, I suggest the paper to be weakly rejected.


Adversarial training has been demonstrated as one of the most effective methods for training robust models so as to defend against adversarial examples. However, adversarially trained models often lack adversarially robust generalization on unseen testing data. Recent works show that adversarially trained models are more biased towards global structure features. Instead, in this work, we would like to investigate the relationship between the generalization of adversarial training and the robust local features, as the robust local features generalize well for unseen shape variation. To learn the robust local features, we develop a Random Block Shuffle (RBS) transformation to break up the global structure features on normal adversarial examples. We continue to propose a new approach called Robust Local Features for Adversarial Training (RLFAT), which first learns the robust local features by adversarial training on the RBS-transformed adversarial examples, and then transfers the robust local features into the training of normal adversarial examples. To demonstrate the generality of our argument, we implement RLFAT in two currently state-of-the-art adversarial training frameworks. Extensive experiments on STL-10, CIFAR-10, CIFAR-100 datasets show that RLFAT significantly improves both the adversarially robust generalization and the standard generalization of adversarial training. Additionally, we demonstrate that our models capture more local features of the object on the images, aligning better with human perception.

In this paper, the authors proposed a new approach to improve the robustness of CNNs against adversarial examples.
The recent studies show that CNNs capture local features, which can be easily affected by the adversarial perturbations.
Thus, in the paper, the authors proposed to train CNNs so that they can capture local features that are robust against the adversarial perturbations.
The difficulty here is that existing adversarial training algorithms tend to bias CNNs to ignore local features and to capture only global features.
To avoid this unfavorable property of the adversarial training, the authors proposed the random block shuffle (RBS) that intensionally destroys the global feature of the images.
The authors demonstrated that combining RBS with the existing adversarial training algorithms can lead to robust CNNs.
I found the paper well-written and the idea is easy to follow.
Especially, the use of RBS seems to be an interesting idea.
As a small downside, the proposed approach looks rather straightforward, and I expect to see any theoretical foundations if possible.
### Updated after author response ###
In summary, the contribution of this study is in twofolds.
1. Proposed an algorithm for learning robust local features.
2. Demonstrated that learning robust local features is effective to improve the robustness of the model.
The possible downside is
3. The proposed approach looks straightforward.
Overall, I like the paper (especially for the reason 2 above), and therefore keep my score.


The inference of models, prediction of future symbols, and entropy rate estimation of discrete-time, discrete-event processes is well-worn ground. However, many time series are better conceptualized as continuous-time, discrete-event processes. Here, we provide new methods for inferring models, predicting future symbols, and estimating the entropy rate of continuous-time, discrete-event processes. The methods rely on an extension of Bayesian structural inference that takes advantage of neural network's universal approximation power. Based on experiments with simple synthetic data, these new methods seem to be competitive with state-ofthe-art methods for prediction and entropy rate estimation as long as the correct model is inferred.

The paper focuses on the problem of modeling, predicting and estimating entropy information over continuous-time discrete event processes. Specifically, the paper leverages unifilar HSMM's for model inference and then uses the inferred states to make future predictions. The authors also use inferred model with previously developed techniques for estimating entropy rate. The authors describe the methods and provide the evidence of the effectiveness of their method with experiments on a synthetic dataset.
The paper should be rejected as it doesn't appear to be fit for this conference. While the paper does build techniques that leverages classical statistical learning and combine them with deep learning, it falls short on various aspects. Specifically, the paper neither provides any rigorous analysis of the developed methods so as to be useful to theoretical learning community nor does it support its approaches with strong empirical evidence so as to be of practical importance.
- While the authors cite lack of work in such data setting (which is actually incorrect as described below), they do not motivate or justify why such an approach of using either unifilar HSMM or Bayesian inference techniques are good starting point to model such information. The authors claim that unifilar HSMM is more of a restriction to make the three proposed tasks easier. But such a model assumption is very strict and will likely lead to large model mis-specification issues when used in real-world. The author does not describe what can be done in this scenario.
- Page2, Section 3: the statistics of {(x_i, t_i)} are unchanging in time. What does this mean?
- As stated by authors, to make the approach tractable, they have to adopt a slew of approximations which seems to put severe restriction on what kind of data can be modeled and inferred on using these approaches.
- It is not clear why one wants to estimate the entropy rate in such settings and authors avoid discussing it in Section 4.2
similarly the authors explicitly avoid discussing bias-variance tradeoff of such method, however, this makes the paper
devoid of any insights into the proposed approach.
- How was k chosen to be 3 in Eq 10?
- Another major miss for the paper is that there has been lot of work at the intersection of stochastic processes (e.g. temporal point processes) and machine learning techniques that aim to model exactly the continuous-time discrete event information and the paper does neither cites any of them nor discusses them or compare with them. We encourage the authors to look at  the relevant literature and position their work in comparison to these existing approaches. I provide a few references here as starting point, however, lot of follow-up work are available:
Point Processes and Machine Learning:
------------------------------------
Modeling the Dynamics of Learning Activity on the Web
Smart Broadcasting: Do You Want to be Seen?, Karimi et. al.
Uncovering the Temporal Dynamics of Diffusion Networks, Gomez-Rodriguez et. al.
Hidden semi-markov models:
-------------------------
Recurrent Hidden Semi-Markov Model, Dai et. al.
Deep learning and temporal modeling:
-----------------------------------
Recurrent Marked Temporal Point Processes: Embedding Event History to Vector, Du et. al.
The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process, Mei et. al.
- Finally, the authors show only one synthetic experiment. This is far from providing convincing evidence of the efficacy of the method and certainly needs a lot more work for acceptance in any machine learning conference.  The authors, at the least, must use their approaches on couple of real-world dataset (e.g. earthquake information, animal behavior as they mention or others) and also compare with the above stochastic processes techniques to test how their methods fare.
- Also, the analysis of efficiency of their method is required. It appears that their methods may not be scalable for instance to a multi-dimensional discrete label case.
- Figure 4 is not correctly referenced in the paragraph right above Section 4.3.


Continuous embeddings of tokens in computer programs have been used to support a variety of software development tools, including readability, code search, and program repair. Contextual embeddings are common in natural language processing but have not been previously applied in software engineering. We introduce a new set of deep contextualized word representations for computer programs based on language models. We train a set of embeddings using the ELMo (embeddings from language models) framework of Peters et al (2018). We investigate whether these embeddings are effective when fine-tuned for the downstream task of bug detection. We show that even a low-dimensional embedding trained on a relatively small corpus of programs can improve a state-of-the-art machine learning system for bug detection.

The paper proposes to use ELMO embeddings to improve the precision on the first step of the DeepBugs tasks defined by Pradel and Sen (2018). This first step is an artificial problem created by taking real programs (with and without bugs, but assuming almost all of them are correct) and introducing bugs of certain type into the programs. Then, a classifier needs to distinguish between the real and the artificial set. This classifier is then to be used as a checker for anomalies in code and the anomalies are reported as bugs, however the paper skips this second step and only reports results on the first classification problem.
Technically, the paper improve this first step of DeepBugs by using a standard variant of ELMO. The evaluation is detailed, but the results are unsurprising. The paper simply tech-transfers the idea from NLP to Code. If this work is accepted at the conference, I cannot imagine an interesting presentation or a poster that simply cites the changed numbers. Did we expect ELMO to be worse than more naive or random embeddings?
The work and its results heavily peg on the DeepBugs and increases the precision of its first step by a significant margin, but does not show getting any more useful results.  In fact, on one task (Wrong Binary Operator), SCELmo gets to 100% accuracy. This means it will never report any bugs, whereas DeepBugs seems to be performing best on exactly this kind of reports with its weaker model.
I would recommend the authors to either work on showing practical usefulness of the technique, showing something for the full bugfinding task (not merely the first, artificial part), or to investigate if (or how) the idea to add bug-introducing changes to a code dataset is conceptually flawed for bugfinding (as this idea is widely used by several other works like Allamanis et al 2018b or by https://arxiv.org/abs/1904.01720 which also don't get to practical tools ). There seems to be some indication of this by the reported 100% accuracy, but right now this remains completely uninvestigated.
Minor issues:
Listing 3: Opernad -> Operand
Page 5. There is no Table 6.1


We propose a reinforcement learning agent to solve hard exploration games by learning a range of directed exploratory policies. We construct an episodic memorybased intrinsic reward using k-nearest neighbors over the agent's recent experience to train the directed exploratory policies, thereby encouraging the agent to repeatedly revisit all states in its environment. A self-supervised inverse dynamics model is used to train the embeddings of the nearest neighbour lookup, biasing the novelty signal towards what the agent can control. We employ the framework of Universal Value Function Approximators (UVFA) to simultaneously learn many directed exploration policies with the same neural network, with different trade-offs between exploration and exploitation. By using the same neural network for different degrees of exploration/exploitation, transfer is demonstrated from predominantly exploratory policies yielding effective exploitative policies. The proposed method can be incorporated to run with modern distributed RL agents that collect large amounts of experience from many actors running in parallel on separate environment instances. Our method doubles the performance of the base agent in all hard exploration in the Atari-57 suite while maintaining a very high score across the remaining games, obtaining a median human normalised score of 1344.0%. Notably, the proposed method is the first algorithm to achieve non-zero rewards (with a mean score of 8,400) in the game of Pitfall! without using demonstrations or hand-crafted features.

-after rebuttal:
I read the replies from the authors and re-read the modified version of the paper and I believe there has been a noticeable improvement in the presentation. I still think it could be improved more (in terms of wording and better exposition of the results) but due to the in-place improvements I increase my score to weak accept.
----------------------------------------------------------------------------------------------------------------------------------------------------
In this paper, the authors present a methodology for generating intrinsic rewards for reinforcement learning agents targeting hard exploration environments. The intrinsic reward is generated using an episodic memory module and a lifelong novelty module.  A state representation is learnt in such a way that the novelty signals are biased towards what the agent can control. A single neural network learns the q-values of exploratory policies with different degrees of exploration. Several experiments on a simple domain and on Atari domains are conducted to evaluate and compare the performance of the proposed method against the baselines.
In my opinion, this is an interesting idea as it present a novel combination of methods (Random Network Distillation and Episodic Memory) that works and can inspire other researchers in the field.  However, I would like to see clearer explanations in the experimental section before acceptance. For this reason I rate this paper as a weak reject but if clarity is improved I will increase my score.  See below for more general comments and detailed explanation about the experimental section.
General comments:
- The authors state that a novel contribution is to disentangle exploration and exploitation. This is not true: see [1] for a recent paper on the topic. I believe the authors should cite this paper.
- In page 3: “To determine the bonus, the current observation is compared to the content of the episodic memory. Larger differences produce larger episodic intrinsic rewards” When computing the intrinsic reward for a new state, it must be compared to the episodic memory, which is composed by recent states (and old). Therefore, in this situation the intrinsic reward is always high? Specially because it is compared with the k-nearest neighbours?  Maybe the authors could elaborate on this?
Experiment Section:
This section should be tidied up in my view.  In general, I feel that there are too many fine-grained results / experiments. I think some aggregation of results would be good for clarity. Without this aggregation there are statements made from the authors that are difficult to believe, e.g. a general result statement is valid for one or two games but not for the rest (but still the statement is formulated in a general way). This can lead to misunderstandings and overstatements.  This together with lack of some experimental details makes the section very dense and difficult to parse. Below I make my points. They are in order of appearance in the main manuscript. I marked with (*) the ones I consider the most important.
Section 4.1:
- Why the exploration policy in this section is set to have $\beta = 0.3$ ? Is my understanding correct that any value of $\beta$ would produce similar results since this is just a scaling of the intrinsic reward (i.e. it does not balance exploration vs exploitation)?
- The parameter $\beta$ appears in the construction of the reward r = r^e + \beta r^i , but also the output of Algorithm 1 scales the similarity by $\beta$ does that mean that the final contribution of $\beta$ is squared? Is this intended? If so, why?
- At the end of this section: “However, staying still is enough: staying still every state will produce ….” Since the agent can only take 4 action {left, right, up, down}, what do the authors mean by “staying still”, is really the agent doing some sort of cyclic policy e.g. left right left right … ?
Section 4.2:
- In the appendix A it is stated “use last 5 frames of the sampled sequences to train the action prediction network… “ Does this refer to frame-stacking? I assume it is not since at the beginning section 4.2 it is stated that there is no frame-stacking. If it is not frame-stacking, the authors could explain in more detail what do they refer to.
- In the paragraph “Architecture” it is stated that 8 games were selected to choose the hyperparemeters and that the results are in Appendix B. However, appendix B only shows 2 games (Pitfall and Montezuma’s Revenge). Is this a typo?
- (*) In paragraph “NGU Agent”: This is the most dense paragraph and the most difficult to parse. First of all Table 1 shows all the results, but as one can see, the different ablations have very similar performance on most games with only a few exceptions. Note that most of the mean performances with error bars, have actually overlapping error bars for many combinations of games and methods. Therefore, further statements about this table are difficult to believe since they could have been just the result of random seeds. I think it is fine to show these fine-grained results on the appendix, but I would say it would be better to aggregate them in the main paper and show that the statements made by the authors still hold for this aggregation.
Specific comments about NGU agent paragraph:
- “ we observe an improvement from increasing the number of mixtures (with diminishing returns) on hard exploration games.” I would say the authors cannot claim this since the hard exploration games are the ones in table 2, which are different from the ones in table 1 (only Pitfall, MR and Private Eye coincide). Also the statement is only true for 2 the three that are hard exploration games (Pitfall and Private Eye).
- (*)  “for smaller $\beta$ we observe better performance on Pong and Beam Rider, but worse performance on all hard exploration games” This is a strange result in my opinion. If I understood correctly, the base agent has $\beta = 0.3$, which supposedly has been selected to be good on hard-exploration games. However, here only changing $\beta$ slightly, reduces the performance on hard-exploration games (Pitfall and Private Eye) significantly. Is this due to the parameter being very sensitive? If so, I believe the authors should report how sensitive are the results to this parameter, specially, on the full hard-exploration games.
- “superhuman performance on 3 games”: which ones?
Comments on paragraph on “hard exploration games”:
- “with NGU(N=1)-RND ….” what do the authors mean by this? This seems to be the best setting for Pitfall but it is actually not using mixture of explorations and (I guess) neither RND?
- Table 2 first row “best base” what does this mean?
Minor Comments:
-In Equation 3 the squared distance is normalized by a running average. Why?
- Right after Equation 3: “… episodic reward can be found in Alg. 14” . Probably meant Alg 1.
- Similarly why the errors are normalized by a running average when computing \alpha_t ?
[1] MULEX: Disentangling Exploitation from Exploration in Deep RL ( Lucas Beyer et al )


Conservation laws are considered to be fundamental laws of nature. It has broad application in many fields including physics, chemistry, biology, geology, and engineering. Solving the differential equations associated with conservation laws is a major branch in computational mathematics. Recent success of machine learning, especially deep learning, in areas such as computer vision and natural language processing, has attracted a lot of attention from the community of computational mathematics and inspired many intriguing works in combining machine learning with traditional methods. In this paper, we are the first to view numerical PDE solvers as a MDP and to use (deep) RL to learn new solvers. As a proof of concept, we focus on 1-dimensional scalar conservation laws. We deploy the machinery of deep reinforcement learning to train a policy network that can decide on how the numerical solutions should be approximated in a sequential and spatial-temporal adaptive manner. We will show that the problem of solving conservation laws can be naturally viewed as a sequential decision making process and the numerical schemes learned in such a way can easily enforce long-term accuracy. Furthermore, the learned policy network is carefully designed to determine a good local discrete approximation based on the current state of the solution, which essentially makes the proposed method a meta-learning approach. In other words, the proposed method is capable of learning how to discretize for a given situation mimicking human experts. Finally, we will provide details on how the policy network is trained, how well it performs compared with some state-of-the-art numerical solvers such as WENO schemes, and how well it generalizes. Our code is released anomynously at https://github.com/qwerlanksdf/L2D.

This paper proposes to use reinforcement learning for constructing discretziation stencils of numerical schemes. More specifically, the method focuses on the widely used WENO schemes, which are an established class of finite difference schemes. Within this context, the method aims for training models to infer the weighting for a specific stencil with eight flux terms.
For RL this task requires a continuous action space, and the DDPG algorithm is used for training the policy. The network itself is an MLP with 6 layers, and ca. 20000 weights in total. This is a significant number, given the focus on 1D problems.
The tests are quite thorough and interesting, while at the same time being limited in scope. The paper targets 1D cases, which make the problem very low-dimensional. Despite the simplicity, only a single data set (Burgers) is used, and a single modified target function with a u^4 term. Targeting 1D casesl, I would have expected a broader range of tests and model equations.
Despite the limited scope of the models, table 1 and 2 assess a nice range of different timestep and discretization parameters. I found it very interesting to see that the method consistently outperforms the regular WENO scheme. The gains are relatively small, with 4-5%, but WENO already represents a quite accurate scheme, so it's surely not easy to outperform it.
While reading the paper, I was wondering about the bigger picture, i.e. using RL in the context of discretization stencils. We have model equations, and discretized versions of all operators involved in training. Why employ a "brute force" approach like RL here? Wouldn't it be better in terms of efficiency and potentially also accuracy to train the stencils in a supervised manner, e.g., with a more accurate discretization as reference? One could argue that it would be expensive to pre-compute such data, but I think RL scales even worse to higher dimensional problems.
What's also missing in the current version is a more thorough discussion of inference and training performance. I guess that despite the small model problems, the training takes a substantial amount of time. And due to the large size of the trained model, which has to be evaluated for every single node in the 1D mesh, it's probably also quite slow. I think this is worth a discussion in the text. One could even estimate the number of operations necessary to evaluate the model, and run a higher-order WENO scheme for a "fair" comparison.
Minor, but in equation (1), I guess the t subscript should indicate a material derivative, and just just a time derivative, right? This could be clarified in the text (or written out).
I am somewhat on the edge with this paper - the 1D case for the two equations is carefully evaluated in the submission, and it's great to see the trained model can improve the accuracy across a fairly wide range of settings. As such, it's definitely a good and interesting first step. On the other hand, there are a range of open questions, as outlined above, and it's not clear whether the approach could be easily translated to higher dimensions. I hope the authors can clarify some of these points in the rebuttal, right now I'm leaning towards the positive side.


Non-autoregressive models are promising on various text generation tasks. Previous work hardly considers to explicitly model the positions of generated words. However, the position modeling is an essential problem in non-autoregressive text generation. In this study, we propose PNAT, which incorporates positions as a latent variable into the text generative process. Experimental results show that PNAT achieves top results on machine translation and paraphrase generation tasks, outperforming several strong baselines.

This work proposes an alternative approach to non-autoregressive translation (NAT) by predicting positions in addition to the word identities, such that the word order in the final prediction doesn't matter as long as the positions are correct. The length of the translation is predicted similar to Gu et al 2017, as well as smoothly copying the source sequence to decoder input. However, since the positions are unknown, this paper employs a heuristic search method to find the nearest neighbors in the embedding space to obtain position supervision.
Experiments are conducted on a machine translation task and a paraphrase generation task. For WMT14 DE-EN and IWSLT16 DE-EN, this method gets superior performance compared to baselines. For paraphrase generation, this approach with an AR position predictor gets even better performance than a normal AR transformer. Further analysis shows that this approach degrades less without knowledge distillation, or without removing repetitions.
Pros:
1. This approach gets better performance compared to baselines.
2. The idea of modeling positions as a latent variable is interesting and might generalize to other tasks beyond NAT.
Cons:
1. This work should compare to later baselines such as FlowSeq (https://arxiv.org/pdf/1909.02480.pdf) which gets better performance with flow.
2. In table 1, although the proposed approach outperforms imitate-NAT, the speedup is lower, making it hard to judge which is actually better.
3. In table 2, why is AR predictor used? What's the performance of NAR predictor? (in general why consider NAR position predictor at all?)
4. It is not clear why heuristic search would work here. Is any pretraining required? Otherwise, since there's no gradient signal for the positions, I'm not sure how the model figures it out.
Questions:
1. How many samples are used in table 1 LPD? Or is it argmax decoding for each length?
2. Is it possible to include a few examples showing predicted positions in an appendix to help better understand the model's behavior?
3. Why do you think positions can be predicted in a NAR manner? Isn't it just shifting the burdens to the position predictor? (Since in transformers if it's able to learn positions then it should be trivial to reorder based on those positions)
Minor details:
Some typos need to be fixed.
Overall, it is an interesting idea to predict the positions and word identities separately, and with Gumbel-Softmax and VAE we might be able to optimize the true marginals instead of relying on heuristic search. However, empirically there's been better performance achieved by flow-based models, so I am inclined to reject this work.
--------updates after reading the authors' rebuttal-----
I really appreciate the substantial experiments the authors conducted post-review. However, I still have some concerns regarding the baseline flowseq. Flowseq-large gets better results but is not reported in the main paper. Besides, it is unclear how is the speedup computed since Ma et al 2019 reported that the speedup depends on batch size, and at batch size 120 the speedup is ~X6 for flowseq-base.
Minor issue: in table 1 which dataset are the speedup numbers based on?


Clustering high-dimensional data, such as images or biological measurements, is a long-standing problem and has been studied extensively. Recently, Deep Clustering gained popularity due to the non-linearity of neural networks, which allows for flexibility in fitting the specific peculiarities of complex data. Here we introduce the Mixture-of-Experts Similarity Variational Autoencoder (MoE-Sim-VAE), a novel generative clustering model. The model can learn multi-modal distributions of high-dimensional data and use these to generate realistic data with high efficacy and efficiency. MoE-Sim-VAE is based on a Variational Autoencoder (VAE), where the decoder consists of a Mixture-of-Experts (MoE) architecture. This specific architecture allows for various modes of the data to be automatically learned by means of the experts. Additionally, we encourage the lower dimensional latent representation of our model to follow a Gaussian mixture distribution and to accurately represent the similarities between the data points. We assess the performance of our model on synthetic data, the MNIST benchmark data set, and a challenging real-world task of defining cell subpopulations from mass cytometry (CyTOF) measurements on hundreds of different datasets. MoE-Sim-VAE exhibits superior clustering performance on all these tasks in comparison to the baselines and we show that the MoE architecture in the decoder reduces the computational cost of sampling specific data modes with high fidelity.

Summary:
The paper proposes to expand the VAE architecture with a
mixture-of-experts latent representation, with a
mixture-component-specific decoder that can specialize in a specific
cluster.  Importantly, the method can take advantage of a similarity
matrix to help with the clustering.
Overall, I recommend a weak accept.  The method seems reasonable, and
the paper is well-written, but the results are only marginally better
than other methods, and there are several weaknesses with the proposed
architecture and experimental setup.
Positives:
* The idea of a more expressive variational distribution seems good,
  although it is not novel.
* The ability to have multiple decoder networks seems reasonable.
* The ability to incorporate domain knowledge (in the form of a
  similarity matrix S) is a plus.
* The experiments are thorough, although the method is generally only
  slightly better than competing methods.
Negatives:
* It's not clear if the similarity matrix S is already solving the
  clustering problem - in which case, why do we need the rest of the
  model?  For example, in your experiments you often used UMAP to
  cluster data.  How does using UMAP by itself work?  (Along these
  lines, it was not clear if your GMM experiments clustered data in
  the original space, or in the UMAP'd space - please clarify this).
  A good ablation would be to somehow remove the S matrix, to see if
  the model can accurately cluster samples.
* There is little variance in the generated samples.
* There is not a one-to-one mapping of clusters to labels, so it is
  hard to use this method to generate a specific type of data (for
  example, it is hard to generate a specific digit).  This is a big
  difference from, say, a conditional sampler as learned by a GAN.
  This also arises in Fig. 3, where it is clear that latent cluster
  assignments do not match human-interpretable cluster assignments.  I
  suppose this is to be expected, but taken with the previous point
  (little variance in generated samples) I think it seriously weakens
  the paper's claim that this is an "accurate an efficient data
  generation method."
* The method does not do well when the number of clusters is large.
  Regular GMMs seem to outperform it.
* I felt that this paper made excessive use of the appendix.  The
  paper is not self-contained enough, effectively violating the length
  restrictions.  Please make an effort to move key results back in to
  the main body of the paper.
Experiments to run:
An ablation regarding the similarity matrix S.
Clarification of whether GMM experiments are run in data-space, or
UMAP'd space.
MIXAE features prominently in your related works, but is not compared
to in your experiments.  It sounds like a natural comparison.  Please
run this experiment, or explain why it is not a comparable method.


Nash equilibrium has long been a desired solution concept in multi-player games, especially for those on continuous strategy spaces, which have attracted a rapidly growing amount of interests due to advances in research applications such as the generative adversarial networks. Despite the fact that several deep learning based approaches are designed to obtain pure strategy Nash equilibrium, it is rather luxurious to assume the existence of such an equilibrium. In this paper, we present a new method to approximate mixed strategy Nash equilibria in multi-player continuous games, which always exist and include the pure ones as a special case. We remedy the pure strategy weakness by adopting the pushforward measure technique to represent a mixed strategy in continuous spaces. That allows us to generalize the Gradient-based Nikaido-Isoda (GNI) function to measure the distance between the players' joint strategy profile and a Nash equilibrium. Applying the gradient descent algorithm, our approach is shown to converge to a stationary Nash equilibrium under the convexity assumption on payoff functions, the same popular setting as in previous studies. In numerical experiments, our method consistently and significantly outperforms recent works on approximating Nash equilibrium for quadratic games, general blotto games, and GAMUT games.

This paper proposes a novel algorithm for finding mixed-strategy Nash equilibria in games with continuous action spaces. The paper proves convergence to a stationary Nash equilibrium under the assumption of convex utility functions. In quadratic games, general blotto games and GAMUT games, the proposed algorithm (MC-GNI) significantly outperforms its competitors. This experimental finding is not surprising, as its competitors were designed to compute pure-strategy Nash-equilibria.
This paper investigates an important, relatively unexplored problem. Its proposed algorithm appears theoretically justified and to achieve nontrivial performance in practice. However, I feel that there are some issues with this paper.
There has been a significant amount of recent work on differentiable games. It would benefit readers for the paper to discuss how this work and those works are related. In particular, the latter are motivated by designing gradient-based algorithms for finding parameters of neural networks corresponding to local equilibria. These neural networks may themselves be playing nondeterministic strategies in some underlying game (e.g., GANs). This closely resembles the paradigm of the algorithm proposed in this paper, which finds deterministic parameters for a network that itself induces a nondeterministic strategy. This leads to a number of questions:
1. In the experiments, the paper’s language suggests that SGA is constrained to pure strategies. Why is it not parameterized by a network producing a stochastic strategy? This seems like the sensible comparison.
2. How are the convergence guarantees discussed in this paper related to the capacity of the network being used to approximate the policy? Are they lost if the network has insufficient capacity?
3. The paper mentions that SGA outperforms other existing algorithms applicable to continuous game settings. What about stable opponent shaping??
The writing in the paper would benefit from revision. While the paper generally gets the point across, much of it feels sloppy. There are also many typos, some of which are listed below.

Typos:
- Raghunathan et al. (2019) introduces
- In precise
- drown
- for $\forall$
- we prepare each distribution πj a corresponding pushforward function
- Therefore, a common algorithm of computing
- On the one hand, global infimum can
- We develop a solution significantly improves the status-quo.
- of Mixed strategy on Continuous game
Other:
- The paper alternates between his and it when referring to players
Overall, I think this work is an outline for a nice paper but would like to see
1) Clarification regarding its relationship to existing work
2) Experimental baselines in which SGA and stable opponent shaping are parameterized to produce nondeterministic strategies
3) Cleaner writing


Designing the architecture of deep neural networks (DNNs) is a cumbersome task that requires significant human expertise. There have been attempts to automatize this task, mainly by considering DNN architecture parameters such as the number of layers, the number of neurons and the activation function of each layer as hyper-parameters, and using an external method for optimizing them. This framework is computationally too expensive since each hyper-parameter optimization iteration required a whole model training. Here we propose Farfalle Neural Networks, a novel recurrent structure in which neurons connect each other through their trainable embeddings. Hence, important architecture features such as the number of neurons in each layer and the wiring among the neurons are automatically learned during the training process. We theoretically prove that the proposed model can replace a stack of dense layers, which is used as a part of many DNN architectures. By comparing the performance of dense networks and FNNs we show FNNs achieve comparable or higher accuracy using significantly fewer parameters.

This paper introduces a new neural network architecture, in which all neurons (called "floating neurons") are essentially endowed with "input" and "output" embedding vectors, the product of which defines the weight of the connection between any two neurons. The authors discuss two network architectures employing floating neurons: (a) multi-layer floating neural networks and (b) farfalle neural network (FNN), in which there is one hidden layer, but additional recurrent connections are introduced between the hidden neurons.
As mentioned by the authors, the proposed architecture is similar to architectures employing low-rank weight matrix factorization. In my opinion, the main novelty lies in: (a) "floating neuron" interpretation, (b) additional weight matrix normalization, and (c) FNN architecture similar to that of a "floating neuron" RNN network with additional restrictions.
I find the proposed idea to be promising and quite intriguing, but I think that the paper has some room for improvement and provided empirical evidence might be insufficient (including for understanding the importance of individual model components), which in turn makes the claims of potential practical attractiveness less justified. I will be happy to update the final score provided with more compelling arguments or empirical analysis of the proposed architecture.
Addressing the following issues might greatly improve the quality of the paper:
1. In Section 4.1, the authors compare FNN and DNN on MNIST and CIFAR10 datasets. My concern is that the authors pick a seemingly arbitrary DNN architecture (just a single one) and restrict comparison to it. One issue is that ~50% accuracy on CIFAR10 can be easily demonstrated by a variety of 5-layer DNN architectures including those much smaller, with just ~600k parameters (!) and possibly even lower. This makes the 90% parameter reduction claim not particularly meaningful. And why were models matched based on the total number of neurons, but not, say, the total number of parameters, or other measures? I believe that these questions require additional discussion and empirical evidence. Just as an example, if it was possible to sample (potentially randomly) different DNN architectures (with a reasonable parameter prior) and compare them with FNNs on a 2D accuracy-parameters plot (or using other important metrics), it would provide much more information to the reader.
2. Another important point that I would like to make is that there is much more that can be done to explore the hyper-parameter space of FNN to isolate which particular factors play a decisive role in its superior accuracy. The authors present us with a specific choice of the normalization function, and values of k and d, but it would be very informative to study how results change when different choices are considered. FNNs differ from DNNs in at least three aspects: usage of low-rank factorization, weight normalization and recurrent structure. How important are these individual aspects? Are some of them redundant, or almost redundant, or do FNNs require all of these components to achieve their peak performance? In other words, I believe that a careful ablation study would greatly improve this publication.
3. As a minor note, I think that the statement that FNNs "are more general" than floating neural networks is only partially correct. If I am not mistaken, FNN can also be "unrolled" and represented as a multi-layer floating neural network with additional parameter sharing. Also, the computational complexity of the constructed FNN (in Theorem 1) appears to be significantly higher than that of the floating neural network (especially for high l). This would imply that FNNs do not necessarily supersede multi-layer floating neural networks, at least when the computational complexity is of importance.
4. There are a few minor misprints throughout the text. For example, in "0<j<=j" in the proof of Theorem 1, or in "R output floating neurons for the final deduction from hidden neuron" (output should be S). Also, I could not find information about the value of d used in the described experiments (which I estimated to be 256; is this correct?).
5. In Section 4.3, the authors propose to use FNNs for the final layers of conventional CNN architectures. The issue is that the VGG16 network chosen for experiments was probably picked because it uses several large fully-connected (FC) layers in its tail whereas all more recent and efficient CNN architectures actually gravitate towards a smaller single FC layer. It is possible that FNNs could still be used in FC layers of these modern networks as well (especially with a large number of classes). But additional empirical results for these architectures would, in my opinion, be much more convincing.
Updated: The authors updated the text and addressed many of my questions. In my opinion, this improved the paper and made some of its claims much better justified. I change the rating to "Weak Accept".


The search-based reinforcement learning algorithm AlphaZero has been used as a general method for mastering two-player games Go, chess and Shogi. One crucial ingredient in AlphaZero (and its predecessor AlphaGo Zero) is the two-head network architecture that outputs two estimates -policy and value -for one input game state. The merit of such an architecture is that letting policy and value learning share the same representation substantially improved generalization of the neural net. A three-head network architecture has been recently proposed that can learn a third action-value head on a fixed dataset the same as for two-head net. Also, using the action-value head in Monte Carlo tree search (MCTS) improved the search efficiency. However, effectiveness of the three-head network has not been investigated in an AlphaZero style learning paradigm. In this paper, using the game of Hex as a test domain, we conduct an empirical study of the threehead network architecture in AlpahZero learning. We show that the architecture is also advantageous at the zero-style iterative learning, producing neural network models stronger than those from the two-head counterpart in the same MCTS.

The paper applies three-head neural network (3HNN) architecture in AlphaZero learning paradigm. This architecture was proposed in [1] and the paper builds upon their work. In AlphaGo and AlphaZero 2HNN is used, which predicts policy and value for a given state. 3HNN also predicts action-value Q function. In [1], the three new terms are added to the loss to train such a network, and the network is trained on a fixed dataset. The paper utilizes the same 3HNN idea with the same loss, and the contribution is that 3HNN is trained synchronously with MCTS iterations on an updating dataset (“AplhaZero training style”). Learning speed of 3HNN is shown to be higher than that of 2HNN. The special attention is drawn to varying the threshold expansion parameter, as the 3HNN architecture allows to set it above zero, while 2HNN does not. The approach is demonstrated on the game of Hex. Results are presented on two test datasets: positions drawn from a strong agent’s games and random positions. Labels in both datasets are perfect, obtained by a special solver.
I tend to reject the paper, because the demonstrated results suggest that the models were not tuned well enough. Indeed, the paper claims that the parameters were not tuned. The paper claims using threshold expansion > 0 to be one of the main advantages of 3HNN. However, the best model in the experiment is the one with the parameter equals zero. Overall, for a purely experimental paper, the experiments are too crude.
Main argument
1.	It seems that some of NN models didn’t learn at all:
           • 	Figure 2, 2HNN model. MSE on both the left and right plots is not improving. Moreover, on the right plot it
                fluctuates around 1, which is the performance of a random guess.
           •	Figure 3, right. MSE of the 3HNN model is not improving. Probably, random positions are too unnatural and
                nothing similar is presented in the dataset drawn from MCTS iterations.
2.	Supmat reveals, that some of the models are in fact learned using the different loss than it is said in the paper. In particular, data augmentation term is sometimes on and sometimes off. A disabling scheme is suggested, depending on the dithering threshold and the number of the moves played before the state s. Some models use the scheme, for others the term is always on. This should be clearly stated in the main text, not in the supmat.
Also, there is an experiment in the supmat, when the data augmentation term is always off. The influence of this term is itself interesting, as it is one of the reasons 3HNN is learning q-function at all. However, introduction of this term itself is the contribution in [1]. I suggest to add to the main part of the paper the experiment, comparing three regimes: 1) with the scheme, 2) term always on and 3) term always off. In fact, it is almost done, as all three regimes are used in different figures, but somewhy the final comparison (with other parameters fixed) is not shown and partly concealed in the supmat. It could become methodological improvement of the paper over [1].
3.	When the leaf node s is expanded, the v function of the new node s’ = s ∪ a is initialized to predicted q(s, a). It is one of the advantages of 3HNN and allows node expansion threshold. When s’ itself is expanded, how do you merge q(s,a) with backup values during mcts iterations?
4.	Nothing is said about the architecture of NNs. If the representation of the state is the same as in [1], it should at least be mentioned.
5.	How exactly does figure 2 shows that one iteration AlphaZero-2HNN is 5-6 times slower, than AlphaZero-3HNN. Probably the definition of data point in figure 2 is missing (e.g. one data point corresponds to one MCTS iteration).
6.	Figures 3 and 4 basically shows the same experiment, but with different models: AlphaZero-2HNN versus AlphaZero-3HNN with threshold 0 on Figure 3 and AlphaZero-2HNN versus AlphaZero-3HNN with threshold 1 on Figure 4. They could be united in one figure.
7.	Results from Figures 3 and 4 suggest, that 3HNN with threshold = 0 is better, than with threshold = 1. However, the paper claims setting threshold > 0 as an advantage. If it allows to save time (as each mcts iteration is faster), maybe they should be compared plotting time on x-axis (like in Figure 2)?
8.	Please provide error bounds in table 2.
Additional arguments
Argumentation presented in this section didn’t affect the score. However, it might improve the paper.
1.	3HNN predicts q(s,a), which should be equal to v(s’), where s’ = s ∪ a (state after action a is taken in state s). It would be interesting to see how condition q(s,a) == v(s ∪ a) holds during  3HNN models learning. It can be checked on a first dataset (drawn from games), or a special random dataset, containing consecutive positions, could be generated. Probably, this condition could potentially be an additional loss term.
2.	Also, it would be interesting to see how the condition between p and q holds. The higher q(s, a), the lower should be p(a). There is an interesting illustrating figure 7 in supmat, it may be presented in the main text. Also, it could be interesting not only for the first move.
3.	The supmat claims that the motivation for turning off data augmentation term is that it assumes that in the selfplay game both players are selecting the best actions produced by search. Is it connected with the fact, that in the game of Hex all states are either winning or losing, according to theorem proved by Nash? How does this term would work for games with a draw trend, for example chess? In chess, for a lot of states the “ground truth” v(s) would be close to zero and there is no action guaranteeing the win.
4.	The paper claims “Elo (Elo, 1978) improvement does not always correspond to real playing improvement—the monotonic increase of Elo score did not capture the fluctuation in strength”. Citation needed, what fluctuation in strength is not captured? Is it specific to game of Hex? For example, Elo is used as the main measure of total agent strength in AplhaZero papers, as well as by chess community (both chess programs and human players).
Minor comments
1.	Page 4, section 2.2: Even though ... . Our summarization -> Even though ... , our summarization.
2.	Page 4, table 1: mvoe -> move.
3.	Page 7, bullet point above section 4.4: perfect -> perfectly.
4.	Page 7, the lowest paragraph. “For the value learning, however, due to fast search, the AlphaZero-3HNN learned much faster than AlphaZero-2HNN both on games states produced by strong player (T1) as well as examples produced by random play (T2).”
It is confusing, it seems that 3HNN on datasets T1 and T2, however, it was only tested on these datasets.
5.	Page 8: imposing an an auxiliary task -> imposing an auxiliary task.
6.	Page 9: “produce playing strength significantly stronger than” – reformulate.
[1] Chao Gao, Martin Müller, and Ryan Hayward. Three-head neural network architecture for monte carlo tree search. In IJCAI, pp. 3762–3768, 2018b.
=====Post Rebuttal=====
Score updated from 3 to 6.


Generative adversarial networks (GANs) have made significant progress on realistic image generation but often suffer from instability during the training process. Most previous analyses mainly focus on the equilibrium that GANs achieve, whereas a gap exists between such theoretical analyses and practical implementations, where it is the training dynamics that play a vital role in the convergence and stability of GANs. In this paper, we directly model the dynamics of GANs and adopt the control theory to understand and stabilize it. Specifically, we interpret the training process of various GANs as certain types of dynamics in a unified perspective of control theory, which enables us to model the stability and convergence easily. Borrowed from control theory, we adopt the widely-used negative feedback control to stabilize the training dynamics, which can be considered as an L2 regularization on the output of the discriminator. We empirically verify our method on both synthetic data and natural image datasets. The results demonstrate that our method can efficiently stabilize the dynamics and converge to better results.

This paper gives a perspective of GANs from control theory, where it is well established in which cases a dynamical system (which can be described analytically as a function of time) is stable or not (by looking at the roots of the denominator of the so-called transfer function in control theory). It is interesting that the analysis using this framework on simple examples is in line with known results in the GAN literature (Dirac GAN).
Although I personally enjoyed reading the results that from control theory perspective are inline with GAN literature, the paper does not provide novel surprising results. For e.g. the results on the oscillating behavior of Dirac-GAN are described in related works (e.g. Mescheder et al. 2018), and in practice, WGAN with no regularization is not used (as well as GAN with momentum, as normally beta1=0 in practice). In my understanding, the authors present these results to justify the validity of the approach. However, this limits the novelty of the results relative to existing literature. The authors do not focus (in the main paper) on GAN variants used currently, and it is not clear if the proposed approach provides improvement relative to the current state of the art implementations (see next paragraph). Moreover, if I understand correctly the WGAN analysis does not take into account that G and D are non-linear, and it is unclear if these can be done.
I am also wondering if the comparison with the baselines is fair. In other words, although in the present results, the proposed NF-SGAN/WGAN outperforms the baseline, the reported performance of the baselines is worse than in related works on CIFAR10. In particular, FID of ~30 on CIFAR10 for the baselines is notably higher then current reported results on this dataset (e.g. Miyato et al. 2018; Chavdarova et al. 2019). In my opinion, the authors could start from the existing state of the art implementations on this dataset, and report if negative feedback (NF) improves upon.
As the approach uses NF is derived specifically for unstable dynamics, it is not clear to me how adding it would affect the training if the dynamics *is stable*. In this context, I consider that for example, the work by Balduzzi et al. 2018 may be relevant as it describes that the dynamics of games (the Jacobian) has two components, one of which describes the oscillating behavior (Hamiltonian game); whereas most games are a mix of oscillating and non-oscillating dynamics. It is not clear to me if NF would improve stability/performances in general games. As the authors’ main claim is improved stability I am curious to see more detailed analysis on real-world datasets (e.g. multiple seed runs, 2nd-moment estimates over iterations as in Chavdarova et al. 2019).
In summary, although the proposed perspective seems promising given the presented results and it is interesting, in my opinion, it does not provide novel insights nor obtains current state-of-the-art results on CIFAR10, or guarantee that if pursuing it would allow for solving current issues of GAN training.
--- Minor ---
- Abstract: ‘converge better’ it is not clear to me in what sense (faster/better final performances)
- Page 3, Eq. 5: as D and G are functions of time here, eq 5 should maybe be written in more detail to include t
-  Page 3, Sec. 3: I think citing works that also focus on Dirac-GAN would motivate better why you focus on Dirac-GAN in this paper (e.g. writing `as in Mescheder et al. 2018`)
- Page 4: infinity - infinite
- Page 5: can also.. explains ->  explain


This paper proposes a novel procedure for training multiple Transformers with tied parameters which compresses multiple models into one enabling the dynamic choice of the number of encoder and decoder layers during decoding. In sequenceto-sequence modeling, typically, the output of the last layer of the N -layer encoder is fed to the M -layer decoder, and the output of the last decoder layer is used to compute loss. Instead, our method computes a single loss consisting of N × M losses, where each loss is computed from the output of one of the M decoder layers connected to one of the N encoder layers. A single model trained by our method subsumes multiple models with different number of encoder and decoder layers, and can be used for decoding with fewer than the maximum number of encoder and decoder layers. We then propose a mechanism to choose a priori the number of encoder and decoder layers for faster decoding, and also explore recurrent stacking of layers and knowledge distillation to enable further parameter reduction. In a case study of neural machine translation, we present a cost-benefit analysis of the proposed approaches and empirically show that they greatly reduce decoding costs while preserving translation quality.

This work proposes a way to reduce the latencies incurred in inference for neural machine translation. Basic idea is to train a model with softmax attached to each output of decoder layers, and computes a loss by aggregating the cross entropy losses over the softmaxes. During inference, it could either use one of the softmax or train an additional model which dynamically selects softmaxes given an input string. Experimental results show that it is possible to reduce latencies by trading off the translation qualities measured by BLEU. Dynamically selection did not show any gains in latencies, though, this work empirically shows potential gains in oracle studies. This work further shows that the model could be compressed further by knowledge distillation.
I have several concerns to this work and I'd recommend rejecting this submission.
- One of the problems of this paper is presentation. This work basically combines three work together as a single paper, i.e., section 3 for the basic model, section 4 for dynamic selection and section 5 for distillation, with each section describing a separate experiment. I'd strongly suggest the author to focus on the main point, e.g., dynamic selection, and present the basic model and dynamic selection. Experiments should be presented in a single section for brevity.
- Similarly, this work should have been submitted when meaningful gains were observed in the dynamic selection method, given that the proposal is somewhat new. Otherwise, I don't find any merits to see this accepted in ICLR, given the rather negative results in section 4.
- The description in section 4.2 is totally messed up. x^i and y^i_k are strings since they are an input sentence and an output sentence, respectively,. However, they are treated as scalars in Equation 3 by multiplied with \delta_k, subtracted from 1 and taking sigmoid through \sigma. I strongly suggest authors to carefully check variables used in the equations and the description in the section.
- The authors claim that the use of knowledge distillation is novel. However, it is already widely known in the research community and I don't think it is worthy to keep it as a single section. It could have been described as a yet another experiment in a single experimental section.
Other comment:
- Although this paper claims that attaching a softmax for each output layer is new, there is a similar work in language modeling, though the motivation is totally different.
  Direct Output Connection for a High-Rank Language Model, Sho Takase, Jun Suzuki and Masaaki Nagata, EMNLP 2019.
- In section 3.4, this paper claims that the training of all 36 models took 25.5 more time, but took 9.5 more time for a tied-model when compared with a basic 6-layer Transformer. It is not clear to me whether this comparison is meaningful given that it might be possible to employ multiple machines to train 36 models.


In societies with well developed internet infrastructure, social media is the leading medium of communication for various social issues especially for breaking news situations. In rural Uganda however, public community radio is still a dominant means for news dissemination. Community radio gives audience to the general public especially to individuals living in rural areas, and thus plays an important role in giving a voice to those living in the broadcast area. It is an avenue for participatory communication and a tool relevant in both economic and social development.This is supported by the rise to ubiquity of mobile phones providing access to phone-in or text-in talk shows. In this paper, we describe an approach to analysing the readily available community radio data with machine learning-based speech keyword spotting techniques. We identify the keywords of interest related to agriculture and build models to automatically identify these keywords from audio streams. Our contribution through these techniques is a cost-efficient and effective way to monitor food security concerns particularly in rural areas. Through keyword spotting and radio talk show analysis, issues such as crop diseases, pests, drought and famine can be captured and fed into an early warning system for stakeholders and policy makers.

Overview
This paper presents a very interesting application of speech keyword spotting techniques; the aim is to listen to continuous streams of community radio in Uganda in order to spot keywords of interest related to agriculture to monitor food security concerns in rural areas. The lack of internet infrastructure results in farmers in rural areas using community radio to share concerns related to agriculture. Therefore accurate keyword spotting techniques can potentially help researchers flag up areas of interest. The main engineering challenge that the study deals with is the fact that there isn’t a lot of training data available for languages like Lugandu.
Detailed Comments
 Section 3
1. The word “scrapped” should be replaced with “scraped”.
2. Its not entirely clearly what is meant by “as well as an alternative keyword in form of a stem”. What is meant by “stem” in this context? Maybe a citation or an example would make it more clear.
3. The corpus of keywords supposedly contains 193 distinct keywords however the models in Section 4 are only trained to discriminate between 10 randomly sampled keywords. I don’t understand why this is. Training on the full corpus would allow the network to see more training data and consequently might result in more accurate models.
Section 4
1. The authors refer to Equation 3 in Section 4.1, but I think the reference is to Equation 2.
2. The 1-D CNN has 14 layers but it is not clear to me what the size of the final network is. It would be useful to provide more details of the architecture and a comment about the network size either in terms of number of parameters, number of computations or size of the network weights on disk.
3. The figure for  the Siamese network should be numbered.
4. The authors say that the inputs to the ResNets are of shape 32x100. What do these dimensions refer to? Are their 32 frequency bins? Are their 100 frames as inputs? What are the parameters of the FFT computation? Do 100 frames correspond to a second of audio?
5. How many parameters are their in the Siamese network?
6. Its not clear to me how the Siamese networks are used during inference. The authors say they use the L1 distance to determine if a given test example is similar to a given keyword. How are the examples for the keyword of interest selected? Are all training examples used and the scores average? Are the training examples averaged first to form a centroid vector for  the keyword?
Comments on the methodology
One of the main challenges in this work is the fact that there isn’t a large number of training examples available. However the authors still train relatively large acoustic models with 14 layers for 1-D CNN and a ResNet for the Siamese architecture. There are several studies for keyword spotting on mobile devices that aim to train tiny networks that consume very little power [1] [2]. I think it would be more appropriate to start with architectures similar to these due to the small size of the training dataset. Additionally, it would also be very useful to try and identify languages that are phonetically similar to Lugandu and that have training datasets available for speech recognition. Acoustic models trained on this data can then be adapted to the keyword spotting task using the training set collected for this task. Note that the languages need to have only a small amount of phonetic overlap in order for these acoustic models to be useful starting points for training keyword spotting models.
Comments on the evalation
The keyword spotting models presented here are trained with the intention of applying them to streaming radio data. However, the models are trained and tested on fixed chunks of audio. A big problem with this experimental design is that the models can overfit to confounding audio cues in the training data. For example, if all training data are in the form of 1 second audio chunks and all chunks have at least some silence at the start, then the models learn that a keyword is always preceded by a certain duration of silence. This is not the case when keywords occur in the middle of sentences in streaming audio data. The fact that the evaluation is also performed on individual chunks of audio fails to evaluate how the trained models would behave when presented streaming audio. I am certain that when applied to streaming audio these models would false trigger very often, however this fact isn’t reflected by measuring precision and recall on fixed chunks of test audio.
A more principled evaluation strategy would be to present results in the form of detection-error tradeoff (DET) curves [1], [2]. Here the chunks with examples of the keyword in question can be used to measure the number of false rejects by the system. And long streams of audio data that do not contain the keyword should be used to measure the false alarms. Given that its relatively easy to collect streams of radio data, the false alarms can be measured in terms of the number of false alarms per unit time (minutes, hours or days). This evaluation strategy roughly simulates the streaming conditions in which this model is intended to be deployed. Furthermore, DET curves present the tradeoff between false alarms and false rejects as a function of the operating threshold, which provides much more insight into the performance/accuracy of the trained models.
Summary
I think the area of application of this work is extremely interesting, however the training and evaluation methodologies have to be updated in order to realistically measure the way this system might perform in real-world test conditions.
References
[1] Small-footprint Keyword Spotting Using Deep Neural Networks
[2] Efficient Voice Trigger Detection for Low Resource Hardware


Federated learning improves data privacy and efficiency in machine learning performed over networks of distributed devices, such as mobile phones, IoT and wearable devices, etc. Yet models trained with federated learning can still fail to generalize to new devices due to the problem of domain shift. Domain shift occurs when the labeled data collected by source nodes statistically differs from the target node's unlabeled data. In this work, we present a principled approach to the problem of federated domain adaptation, which aims to align the representations learned among the different nodes with the data distribution of the target node. Our approach extends adversarial adaptation techniques to the constraints of the federated setting. In addition, we devise a dynamic attention mechanism and leverage feature disentanglement to enhance knowledge transfer. Empirically, we perform extensive experiments on several image and text classification tasks and show promising results under unsupervised federated domain adaptation setting.

The authors present a novel algorithm for dealing with domain adaptation in the setting of federated learning (classification, specifically). That is, they tackle the issue of learning a model on a new domain when access to the data points used in training the source models is not possible due to privacy constraints. The approach uses the gradients of the source models, reweighed to account for the differing shifts between the different sources and the target domain, to fit the model on the target domain.
The authors motivate their approach by providing a novel bound on the generalization error of transfer learning when the hypothesis function used on the target domain is a convex combination of hypotheses fitted on multiple source domains. This bound shows that the weighted sum of divergences in the symmetric difference hypothesis space controls the generalization error, so the authors aim at deriving feature representations and using aggregation weights that ensure this weighted sum is small.
The authors use a novel dynamic attention model to get the aggregation weights: they cluster the features in the target domain, and measure how much the intra-cluster variation decreases when information from a given source domain is incorporated. The aggregation weights for the model updates on the target domain are then weighed using a softmax transform of these contribution weights.

The motivation up through and including section 3 is clear, the theoretical results are presented clearly, but the model details in section 4 are unclear:
-  In the dynamic attention mechanism, how does one a priori choose the number of clusters in computing the gap statistics, and what is the impact?
- the notation in the federated adversarial alignment section is unclear: what *exactly* are the model coefficients Theta that are being updated?
- the statement "optimize following objective" is made several times. this is ambiguous, and should be corrected to "miminize" following objective.
- the representation disentanglement process is intricate, and only vaguely addressed. how does one fit the neural net and use (8)? where is the l2 reconstruction loss balanced with the mutual information? the vagueness of this section means Algorithm 1 is not well-specified.
The experiments are reasonable, and compare to baseline domain adaptation methods.
The problem considered is of interest, and the approach is novel and interesting. However, the algorithm is not described in sufficient detail. After reading the paper, and spending considerable time rereading section 4, I still do not understand how Algorithm 1 is implemented in practice. For that reason I lean towards reject. I will update my score if the authors clarify the details of Algorithm 1.
Comments:
- the symmetric difference hypothesis space is incorrectly called the HdeltaH divergence in section 3


This paper tackles a new regression problem, called Dynamic Time-Lag Regression (DTLR), where a cause signal drives an effect signal with an unknown time delay. The motivating application, pertaining to space weather modelling, aims to predict the near-Earth solar wind speed based on estimates of the Sun's coronal magnetic field. DTLR differs from mainstream regression and from sequence-to-sequence learning in two respects: firstly, no ground truth (e.g., pairs of associated subsequences) is available; secondly, the cause signal contains much information irrelevant to the effect signal (the solar magnetic field governs the solar wind propagation in the heliosphere, of which the Earth's magnetosphere is but a minuscule region). A Bayesian approach is presented to tackle the specifics of the DTLR problem, with theoretical justifications based on linear stability analysis. A proof of concept on synthetic problems is presented. Finally, the empirical results on the solar wind modelling task improve on the state of the art in solar wind forecasting.

This paper proposes a bayesian method, Dynamic Time Lag Regression (DTLR), to predict effect-data y from a cause x.
While both, cause and effect, are time series (x = x(t) and y = y(t)), the cause is assumed to have influence on the effect via a time lag, y(x(t), t) = y(phi(t)) = y(t + g(x(t))). The novelity of the problem posed is that the time lag g(x(t)) is allowed to be non-stationary and supposed to be unknown. The authors outlie a a Bayesian approach. The likelihood of the cause-effect data is conidered used to optimize the regressors for a given, finite set of time lags and the probability of these time lags. This discussion is very brief and detailled derivations are given in the appendix. An accompanying stability analysis is performed. In the implementation a neural network-based approach is used, where in turns the parameters of the model are optimised by a typical gradient descent algorithm and afterwards the hyperparemeters are set to values determined by the parameters. The theoretical deviation is accompanied by practical expirements on synthetically generated data and (real) space weather data. The performance of the model is compared to the one of a prediction with a fixed time lag, which it in fact outperforms. On the space weather (solar wind) data the model is shown to outperform several state-of-the-art models. A codebase for an implementation done in the Scala language is given.
The paper presented provides a meaningful contribution to the field of time series analysis and Bayesian methods. The novelty of the problem imposed as well as the accompanying solution is outlined very clearly. The problem is well justified and the motivation is clearly given.
I recommend to accept it, even though a few details could have been described more precisely.
1. The definition of the time lag g(x(t)) could have been made more clearly, providing a few examples at this point could help to clarify these definitions here.
2. For the synthetic data generation a justification would be helpfull in order to motivate the choice of the functions f and v.
3. A chart with sample time series data (y) against the time would be helpful to better visualize the problem.