title: Invariant Graph Transformer
summary: Rationale discovery is defined as finding a subset of the input data that
  maximally supports the prediction of downstream tasks. In graph machine
  learning context, graph rationale is defined to locate the critical subgraph in
  the given graph topology, which fundamentally determines the prediction
  results. In contrast to the rationale subgraph, the remaining subgraph is named
  the environment subgraph. Graph rationalization can enhance the model
  performance as the mapping between the graph rationale and prediction label is
  viewed as invariant, by assumption. To ensure the discriminative power of the
  extracted rationale subgraphs, a key technique named "intervention" is applied.
  The core idea of intervention is that given any changing environment subgraphs,
  the semantics from the rationale subgraph is invariant, which guarantees the
  correct prediction result. However, most, if not all, of the existing
  rationalization works on graph data develop their intervention strategies on
  the graph level, which is coarse-grained. In this paper, we propose
  well-tailored intervention strategies on graph data. Our idea is driven by the
  development of Transformer models, whose self-attention module provides rich
  interactions between input nodes. Based on the self-attention module, our
  proposed invariant graph Transformer (IGT) can achieve fine-grained, more
  specifically, node-level and virtual node-level intervention. Our comprehensive
  experiments involve 7 real-world datasets, and the proposed IGT shows
  significant performance advantages compared to 13 baseline methods.
paper_authors._id: 657a842e1091657e088c77c4
paper_authors.name: Zhe Xu
paper_authors.status: extracted_infirmed
paper_authors.statusLastChangedAt: 2024-06-11T20:40:01.968Z
paper_authors._id: 657a842e1091657e088c77c5
paper_authors.name: Menghai Pan
paper_authors._id: 657a842e1091657e088c77c6
paper_authors.name: Yuzhong Chen
paper_authors._id: 657a842e1091657e088c77c7
paper_authors.name: Huiyuan Chen
paper_authors._id: 657a842e1091657e088c77c8
paper_authors.name: Yuchen Yan
paper_authors._id: 657a842e1091657e088c77c9
paper_authors.name: Mahashweta Das
paper_authors._id: 657a842e1091657e088c77ca
paper_authors.name: Hanghang Tong
paper_title: Invariant Graph Transformer
paper_summary: Rationale discovery is defined as finding a subset of the input data that
  maximally supports the prediction of downstream tasks. In graph machine
  learning context, graph rationale is defined to locate the critical subgraph in
  the given graph topology, which fundamentally determines the prediction
  results. In contrast to the rationale subgraph, the remaining subgraph is named
  the environment subgraph. Graph rationalization can enhance the model
  performance as the mapping between the graph rationale and prediction label is
  viewed as invariant, by assumption. To ensure the discriminative power of the
  extracted rationale subgraphs, a key technique named "intervention" is applied.
  The core idea of intervention is that given any changing environment subgraphs,
  the semantics from the rationale subgraph is invariant, which guarantees the
  correct prediction result. However, most, if not all, of the existing
  rationalization works on graph data develop their intervention strategies on
  the graph level, which is coarse-grained. In this paper, we propose
  well-tailored intervention strategies on graph data. Our idea is driven by the
  development of Transformer models, whose self-attention module provides rich
  interactions between input nodes. Based on the self-attention module, our
  proposed invariant graph Transformer (IGT) can achieve fine-grained, more
  specifically, node-level and virtual node-level intervention. Our comprehensive
  experiments involve 7 real-world datasets, and the proposed IGT shows
  significant performance advantages compared to 13 baseline methods.
paper_ai_keywords: graph rationale environment subgraph graph rationalization rationale subgraph self-attention module invariant graph Transformer (IGT) node-level intervention virtual node-level intervention