Meta-learning over a set of distributions can be interpreted as learning different types of parameters corresponding to short-term vs long-term aspects of the mechanisms underlying the generation of data. These are respectively captured by quickly-changing parameters and slowly-changing meta-parameters. We present a new framework for meta-learning causal models where the relationship between each variable and its parents is modeled by a neural network, modulated by structural meta-parameters which capture the overall topology of a directed graphical model. Our approach avoids a discrete search over models in favour of a continuous optimization procedure. We study a setting where interventional distributions are induced as a result of a random intervention on a single unknown variable of an unknown ground truth causal model, and the observations arising after such an intervention constitute one meta-example. To disentangle the slow-changing aspects of each conditional from the fast-changing adaptations to each intervention, we parametrize the neural network into fast parameters and slow meta-parameters. We introduce a meta-learning objective that favours solutions robust to frequent but sparse interventional distribution change, and which generalize well to previously unseen interventions. Optimizing this objective is shown experimentally to recover the structure of the causal graph. Finally, we find that when the learner is unaware of the intervention variable, it is able to infer that information, improving results further and focusing the parameter and meta-parameter updates where needed.
This paper proposes an SCM-model based on masked neural networks to capture arbitrary conditional relationships combined with meta-learning-style adaptation to reflect the effects of various unknown interventions. Overall the paper is well written and easy to follow, but some conceptual issues remain.
- How come there is hardly any discussion of the identifiability issue beyond the few sentences in A.3. This is one of the key issues in learning SCMs and it is strange that the concept of "faithfulness" is not even mentioned in the paper.
In general, there is hardly any discussion of what conditions are required for the proposed estimates to even be valid. The authors seem to be optimistically assuming that their neural network + metalearning model will somehow pick up on the correct structure, without any actual conceptual investigation of this issue.
- The massive downside of neural nets is all the various hyperparameters one has to set (eg. architecture, optimizer, activations, etc). In this setting, how do the authors propose selecting hyperparameter values? How does the reader know the authors did not simply tune their hyperparameters to best match the underlying ground truth (I assume the proposed methodology has many more hyperparameters and thus more degrees of freedom here).
I would like to see the empirical performance of different variants of your model with different hyperparameter values to assess its sensitivity to these choices. 
- Why does one even care about the graph being acylic in this setting?
The mere fact that the authors require a regularizer to ensure acylicity suggests this approach is prone to mis-identifying the ground truth structure (which is always acyclic in the experiments).
- One main reason for SCM modeling in science and policy-making is for analysts to better understand the data generating phenomena.  However your use of neural networks here seems to hamper interpretability, so how do you reconcile this issue? Also is your sparsity regularizer satisfactory to confidently diagnose presence/absence of an edge (in constrast to statistical hypothesis tests, say based on conditional independence). Isn't this heavily influenced by the particular sparsity-regularizer value that happened to be selected?
- Related papers that utilize the same idea of predicting a variable conditioned on subset of other variables via neural network + masking strategy:
Ivanov et al (2019). VARIATIONAL AUTOENCODER WITH ARBITRARY CONDITIONING. 
https://openreview.net/pdf?id=SyxtJh0qYm
Li et al (2019). Flow Models for Arbitrary Conditional Likelihoods.
https://arxiv.org/abs/1909.06319
Yoon et al. GAIN: Missing data imputation using generative adversarial nets. Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, 2018. http://proceedings.mlr.press/v80/yoon18a.html
Douglas et al. A universal marginalizer for amortized inference in generative
models. arXiv preprint arXiv:1711.00695, 2017
For clarity, the authors should highlight the differences of their approach from these works (beyond the causal setting).
- Given the lack of theoretical / conceptual guarantees that the methodology will work, our faith in the proposed methodology rests entirely on the empirical experiments.  However, I find these a bit too basic to be very convincing, and would at least like to see more methods being compared (in particular for the simulated graphs as well).
- The authors should describe what are the underlying interventions in each dataset a bit more.
- The Figures should be better explained (took me a while to figure out what dots/colors represent).
- Why do the authors report cross entropy loss in Table 1? To my knowledge this is not a standard metric for measuring the quality of structure-estimates.
 
- Instead of ICP (which is constrained to be linear which is unrealistically simple), why don't the authors compare against nonlinearICP (which is more flexible like their neural networks):  
Heinze-Deml et al (2018). Invariant Causal Prediction for Nonlinear Models. https://arxiv.org/pdf/1706.08576.pdf