Neural Causal Regularization under the Independence of Mechanisms Assumption
Neural networks provide a powerful framework for learning the association between input and response variables and making accurate predictions. However, in many applications such as healthcare, it is important to identify causal relationships between the inputs and the response variables to be able to change the response variables by intervention on the inputs. In pursuit of models whose predictive power comes maximally from causal variables, we propose a novel causal regularizer based on the independence of mechanisms assumption. We utilize the causal regularizer to steer deep neural network architectures towards causally-interpretable solutions. We perform a large-scale analysis of electronic health records. Employing expert*s judgment as the causal ground-truth, we show that our causally-regularized algorithm outperforms its L1-regularized equivalence both in predictive performance as well as causal relevance. Finally, we show that the proposed causal regularizer can be used together with representation learning algorithms to yield up to 20% improvement in the causality score of the generated hypotheses.
The authors extend their method of causal discovery (Chalupka et al 2016) to include assumptions about sparsity via regularization. They apply this extension to an interesting private dataset from Sutter Health. While an interesting direction, I found the presentation somewhat confused, the methodological novelty smaller than the bulk of ICLR works, and the central results (or perhaps data; see below) inadequate to address questions of causality. First, I found the presentation somewhat unclear. The paper at some points seems to be entirely focused on healthcare data, at other points it uses it as a motivating example, and at other points it is neglected. Also, algorithm 1 seems unreferenced, and I*m not entirely sure why it is needed. Figure 2 is not needed for this community. The key methodological advance in this work appears in section 2.1 (Causal regularizer), but it is introduced amidst toy examples and without clear terminology or standard methodological assumptions/build-up. In Section 3.1 (bottom of first paragraph), key data and results seem to be relegated to the appendices. Thus overall the paper read rather haphazardly. Finally, there seems to be an assumption throughout of fairly intimate familiarity with the Cholupka preprint, which i think should be avoided. This paper should stand alone. Second, while the technical contributions/novelty are not a focus of the paper*s presentation, I am concerned by the lack of methodological advance. Essentially a regularization objective is added to the previous method, which of itself is not a bad idea, but I can*t point to a technical novelty in the paper that the community can not do without. Third, fundamentally i don*t see how the experiments address the central question of causality; they show regularization behaving as expected (or rather, influencing weights as expected), but I don*t think we really have any meaningful quantitative evidence that causality has been learned. This was briefly discussed (see *ground truth causality?* and the response below). I appreciate the technical challenges/impossibility of having such a dataset, but if that*s the case, then I think this work is premature, since there is no way to really validate. Overall it*s clearly a sincere effort, but I found it wanting in terms of a few critical areas.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Neural Causal Regularization under the Independence of Mechanisms Assumption
Neural networks provide a powerful framework for learning the association between input and response variables and making accurate predictions. However, in many applications such as healthcare, it is important to identify causal relationships between the inputs and the response variables to be able to change the response variables by intervention on the inputs. In pursuit of models whose predictive power comes maximally from causal variables, we propose a novel causal regularizer based on the independence of mechanisms assumption. We utilize the causal regularizer to steer deep neural network architectures towards causally-interpretable solutions. We perform a large-scale analysis of electronic health records. Employing expert*s judgment as the causal ground-truth, we show that our causally-regularized algorithm outperforms its L1-regularized equivalence both in predictive performance as well as causal relevance. Finally, we show that the proposed causal regularizer can be used together with representation learning algorithms to yield up to 20% improvement in the causality score of the generated hypotheses.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The authors extend their method of causal discovery (Chalupka et al 2016) to include assumptions about sparsity via regularization. They apply this extension to an interesting private dataset from Sutter Health. While an interesting direction, I found the presentation somewhat confused, the methodological novelty smaller than the bulk of ICLR works, and the central results (or perhaps data; see below) inadequate to address questions of causality. First, I found the presentation somewhat unclear. The paper at some points seems to be entirely focused on healthcare data, at other points it uses it as a motivating example, and at other points it is neglected. Also, algorithm 1 seems unreferenced, and I*m not entirely sure why it is needed. Figure 2 is not needed for this community. The key methodological advance in this work appears in section 2.1 (Causal regularizer), but it is introduced amidst toy examples and without clear terminology or standard methodological assumptions/build-up. In Section 3.1 (bottom of first paragraph), key data and results seem to be relegated to the appendices. Thus overall the paper read rather haphazardly. Finally, there seems to be an assumption throughout of fairly intimate familiarity with the Cholupka preprint, which i think should be avoided. This paper should stand alone. Second, while the technical contributions/novelty are not a focus of the paper*s presentation, I am concerned by the lack of methodological advance. Essentially a regularization objective is added to the previous method, which of itself is not a bad idea, but I can*t point to a technical novelty in the paper that the community can not do without. Third, fundamentally i don*t see how the experiments address the central question of causality; they show regularization behaving as expected (or rather, influencing weights as expected), but I don*t think we really have any meaningful quantitative evidence that causality has been learned. This was briefly discussed (see *ground truth causality?* and the response below). I appreciate the technical challenges/impossibility of having such a dataset, but if that*s the case, then I think this work is premature, since there is no way to really validate. Overall it*s clearly a sincere effort, but I found it wanting in terms of a few critical areas.
assistant
The authors extend their method of causal discovery (Chalupka et al 2016) to include assumptions about sparsity via regularization. They apply this extension to an interesting private dataset from Sutter Health. While an interesting direction, I found the presentation somewhat confused, the methodological novelty smaller than the bulk of ICLR works, and the central results (or perhaps data; see below) inadequate to address questions of causality. First, I found the presentation somewhat unclear. The paper at some points seems to be entirely focused on healthcare data, at other points it uses it as a motivating example, and at other points it is neglected. Also, algorithm 1 seems unreferenced, and I*m not entirely sure why it is needed. Figure 2 is not needed for this community. The key methodological advance in this work appears in section 2.1 (Causal regularizer), but it is introduced amidst toy examples and without clear terminology or standard methodological assumptions/build-up. In Section 3.1 (bottom of first paragraph), key data and results seem to be relegated to the appendices. Thus overall the paper read rather haphazardly. Finally, there seems to be an assumption throughout of fairly intimate familiarity with the Cholupka preprint, which i think should be avoided. This paper should stand alone. Second, while the technical contributions/novelty are not a focus of the paper*s presentation, I am concerned by the lack of methodological advance. Essentially a regularization objective is added to the previous method, which of itself is not a bad idea, but I can*t point to a technical novelty in the paper that the community can not do without. Third, fundamentally i don*t see how the experiments address the central question of causality; they show regularization behaving as expected (or rather, influencing weights as expected), but I don*t think we really have any meaningful quantitative evidence that causality has been learned. This was briefly discussed (see *ground truth causality? * and the response below). I appreciate the technical challenges/impossibility of having such a dataset, but if that*s the case, then I think this work is premature, since there is no way to really validate. Overall it*s clearly a sincere effort, but I found it wanting in terms of a few critical areas.