Causal reasoning has been an indispensable capability for humans and other intelligent animals to interact with the physical world. In this work, we propose to endow an artificial agent with the capability of causal reasoning for completing goal-directed tasks. We develop learning-based approaches to inducing causal knowledge in the form of directed acyclic graphs, which can be used to contextualize a learned goal-conditional policy to perform tasks in novel environments with latent causal structures. We leverage attention mechanisms in our causal induction model and goal-conditional policy, enabling us to incrementally generate the causal graph from the agent's visual observations and to selectively use the induced graph for determining actions. Our experiments show that our method effectively generalizes towards completing new tasks in novel environments with previously unseen causal structures.
[Summary]
This paper proposes an interactive agent that tries to infer the underlying causal structure by interacting with the environment; the authors called it "causal induction."  The inferred graph will later help the agent complete goal-directed tasks referred to as a "causal inference" stage. Notably, the agent directly learns from visual inputs.
Both the induction and inference phases heavily rely upon the attention mechanism, which ensures that the agent only focuses on the relevant components of the causal graph. During the induction phase, the agent incrementally updates the predicted causal graph through each interaction using an attention-based edge decoder. During the inference phase, the attention bottleneck also showed to improve the agent's generalization ability.
 
They have shown that the proposed model outperforms several baselines in a synthetic environment that uses switches to control lights. They have also demonstrated the model's generalization ability by operating on unseen causal graphs and new task goals.
[Major Comments]
My primary concern about this work is the scope of its applicability.
For the causal induction phase, the proposed method makes a strong assumption that it can access the ground truth causal relationship during training. The authors can directly read this information from the synthetic environments used in this paper, yet, in more complex real-world situations, we might not know the underlying causal structure for supervised training the induction model.
For learning the goal-conditioned policies, the authors also assume that they have access to the ground truth causal graph. They use this information to generate the expert demonstrations, which, I presume, are deterministic and unimodal (correct me if I'm wrong). In the real world, a human may be able to infer the underlying causal structure from the observations by interacting with the environment and provide the demonstration data. However, the demonstration may be noisy or form a multi-modal distribution. While I agree that learning from demonstration is an effective way of guiding the learning of the policy, I'm not sure if the method can generalize to more realistic scenarios.
For inferring the causal graph, the authors also assume that they know the "cause" set and the "effect" set, which is already a DAG by construction. Instead of inferring the direction of the edge, they are solving an easier problem of deciding whether a directed edge between a "cause" node and an "effect" node exists or not. The assumption on the graph structure also limits the method's applicability, as, in the real world, the direction of the edge is not always known in advance. Smoking may cause lung cancer, but it is possible that lung cancer may make people smoke more.
I feel this paper makes strong assumptions on both the induction and inference stages, as well as the structure of the causal graph, which greatly limits the applicability of the approach.
[Detailed Comments]
I also have a few questions regarding the details of this paper.
In Section 3.1, the authors said that "N is the number of actions in the environments," which is a bit confusing. Before this point, the authors did not discuss the relationship between the size of the graph and the size of the action set. Only until Section 3.2 did I realize that N is the number of both "cause" set and "effect" set. It would be better to discuss the size of the graph and the action set at an earlier position.
How is the expert planner implemented? Is the expert's policy deterministic and unimodal? I feel this is an important detail to include.
It is related to the previous question. What will happen if we learn the goal-conditional policy from scratch? How much does imitation learning help with policy learning? Again, the assumption that we know the ground truth causal graph may not be feasible in the real world.
In Section 3.3, the authors said that "the expert's action is added to the memory of the policy." However, the authors also noted that "the policy has no memory" in Section 3.2, which seems to contradict each other. Does the "memory" mean replay buffer in Section 3.3?
Does the number of switches fixed across all environments and always the same as the number of lights? Also, are all the lights mounted in the same location? I'm wondering if the model is invariant to the order of the cause nodes and effects nodes in the graph, and can it generalize to larger environments, more lights, and different room configurations.
In Figure 4, TCIN seems to have the best performance in the "Masterswitch" environment when there are 500 seen causal structures. What might be the reason?