![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-65bb165f-c325-42df-bd65-0776b481da13.jpg)

Neural Network

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-3225e55d-73c4-4e56-a6b8-9e98471b58cb.jpg)

00 (2024) 1-13

## Contrastive Graph Representation Learning with Adversarial Cross-view Reconstruction and Information Bottleneck

Yuntao Shou a c , , Haozhi Lan , Xiangyong Cao 1 a c, , ∗

a

School of Computer Science and Technology, Xi'an Jiaotong University, Xi'an, China b Ministry of Education Key Laboratory for Intelligent Networks and Network Security, Xi'an Jiaotong University, Xi'an, China c School of Artificial Intelligence, Xi'an Jiaotong University

## Abstract

Graph Neural Networks (GNNs) have received extensive research attention due to their powerful information aggregation capabilities. Despite the success of GNNs, most of them suffer from the popularity bias issue in a graph caused by a small number of popular categories. Additionally, real graph datasets always contain incorrect node labels, which hinders GNNs from learning effective node representations. Graph contrastive learning (GCL) has been shown to be effective in solving the above problems for node classification tasks. Most existing GCL methods are implemented by randomly removing edges and nodes to create multiple contrasting views, and then maximizing the mutual information (MI) between these contrasting views to improve the node feature representation. However, maximizing the mutual information between multiple contrasting views may lead the model to learn some redundant information irrelevant to the node classification task. To tackle this issue, we propose an effective Contrastive Graph Representation Learning with Adversarial Cross-view Reconstruction and Information Bottleneck (CGRL) for node classification, which can adaptively learn to mask the nodes and edges in the graph to obtain the optimal graph structure representation. Furthermore, we innovatively introduce the information bottleneck theory into GCLs to remove redundant information in multiple contrasting views while retaining as much information as possible about node classification. Moreover, we add noise perturbations to the original views and reconstruct the augmented views by constructing adversarial views to improve the robustness of node feature representation. Extensive experiments on real-world public datasets demonstrate that our method significantly outperforms existing state-of-the-art algorithms.

© 2024 Published by Elsevier Ltd.

Keywords: Graph Neural Networks, Contrastive Learning, Mutual Information, Information Bottleneck, Adversarial Learning

## 1. Introdution

Graph Neural Networks (GNNs) have attracted extensive attention from researchers due to the increase in large amounts of real-world graph-structured data [1, 2, 3, 4, 5, 6, 7, 8]. Meanwhile, GNNs are widely used in intelligent recommender systems, and social media fields because they provide a practical way to aggregate high-order neighbor information [9, 10, 11, 12, 13].

Although GNNs have achieved reliable performance on node classification tasks, we argue that most of the node classification models based on GNNs suffer from the following two problems. i) Popularity Bias. As shown in

∗ Corresponding author

Email addresses: shouyuntao@stu.xjtu.edu.cn (Yuntao Shou a , ), haozhilan1@gmail.com (Haozhi Lan), caoxiangyong@mail.xjtu.edu.cn (Xiangyong Cao a , )

Fig. 1, different categories of papers have different numbers, and the number of citations of papers also varies, and this unbalanced learning can lead to the popularity bias problem in GNNs learning. In most node classification tasks, the categories and degrees of nodes follow a long-tail distribution, which means that popular categories have many papers, while most papers have few citations. In other words, most of the nodes have less interaction, which hinders the information update of the nodes. The above-mentioned popularity bias problem cause GNNs to tend to learn node representations that are popular and have many interactions, which hinders the representation learning of GNNs. ii) Noise Interference. There may be miscitations in the citation process of papers (i.e., there is a citation relationship between two unrelated papers in different fields), which leads to noise in the information contained in the data. Studies have shown that the fea-

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-122de092-d252-4c98-a7af-799435c9a74c.jpg)

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-f0928157-86b3-4d14-81bd-9c2fcb537486.jpg)

Citeseer

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-48519db1-3c6c-4362-98a3-3161331051b8.jpg)

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-74bf0fc2-2e94-4aba-8ebe-a5625fc81576.jpg)

Citeseer

Figure 1. (a) The proportion of papers in different categories on the cora and citeseer datasets. (b) The degree distribution of different nodes on the cora and citeseer datasets. By analyzing the distribution of categories and degrees, we argue that the constructed graph structure has a long-tail problem.

ture extraction ability of GCN is closely related to the quality of the input graph, which means that the input image with noise may cause the model to learn poor solutions. In response to the above problems, existing graph contrastive learning (GCL) methods [14, 15, 16, 17], [18], [19], [20], [21] propose an effective solution mechanism to alleviate popular bias and improve the robustness of the GCN model.

Nonetheless, the above-mentioned methods suffer from two limitations. 1) Most GCL methods perform data augmentation to optimize the graph structure by randomly masking nodes or perturbing edges. However, the strategy of randomly masking nodes and edge perturbations is too random, which may cause serious damage to the semantic information of the graph structure. For example, in the functional prediction of molecular structural properties, if edges are randomly perturbed, the structural properties of the molecule will change greatly. In addition, the interpretability that can alleviate popular paranoia and improve model robustness through the above methods is relatively poor. 2) The purpose of existing GCL methods [22, 23, 24], [25], [26] to generate multiple views through data augmentation is to maximize the mutual information between views, which may cause the model to capture task-irrelevant feature information. Inspired by the information bottleneck theory [27], we believe that a good GCL method should reduce as much redundant information as possible while retaining as much task-related information as possible.

To tackle the aforementioned issues, we propose a novel method called Contrastive Graph Representation Learning with Adversarial Cross-view Reconstruction and Information Bottleneck (CGRL) for node classification. CGRL consists of two key components: i.e., adaptive automatic generation of graph-augmented views and graph contrastive learning via information bottlenecks.

First, this paper designs an automatic graph augmentation that adaptively learns node masks and edge perturbations to optimize the original graph into relevant views. In addition, SCGCL employs a joint training strategy to train an adaptive learnable view generator and node classifier in an end-to-end manner, thereby generating augmented views with structural heterogeneity but semantic similarity. As a result, the generated augmented view can undersample the popular nodes in the original graph while retaining the majority of isolated nodes to alleviate the model's popularity bias problem. Intuitively, random masking nodes or perturbed edges do not consider the distribution probabilities and neighborhood information of different types of nodes in the original graph, but dropout them randomly. However, GCN based on message passing is difficult to reconstruct the information of isolated nodes and it is easier to optimize the semantic information of popular nodes. Therefore, the model may achieve better classification results on popular nodes and poor classification results on isolated nodes. The method CGRL proposed in this paper takes the augmented views of debias information and inputs them into GCN for node classification, which improves the model's ability to resist popular bias.

Second, we integrate multiple views that are semantically similar and contain complementary information into a shared feature space for compact representation, which can improve the robustness of CGRL. The intuition behind is that when different views contain complementary semantic information, the model can obtain more prior knowledge to improve the performance of node classification tasks [28, 29, 30, 31]. However, we argue that maximizing mutual information between different views forces the model to learn redundant information that is irrelevant to downstream tasks. Inspired by the information bottleneck (IB) theory [32], it obtains optimal solutions by maximizing label information relevant to downstream tasks and minimizing mutual information between different views. Based on the IB strategy, an automatic graph augmenters learns to generate augmented views that remove noise information and contain semantically similar and complementary views. In addition, when calculating the contrastive loss, we not only use the node feature representations of the two augmented views, but also introduce the node feature representations of the original view perturbed in an adversarial manner as a third view. This additional adversarial view introduces perturbations that force the model to not only accurately distinguish the semantic features of the augmented views, but also to maintain an understanding of the semantic integrity of the original graph in the face of perturbations. Through multi-view adversarial reconstruction, we further improve the robustness of the feature representations.

Compared with the previous methods, the contributions of this paper method are summarized as:

Firstly, we propose a Contrastive Graph Representation Learning with Adversarial Cross-view Reconstruction and Information Bottleneck (CGRL) for node classifica-

tion. The CGRL method can alleviate the popularity bias and interaction noises problem of the existing GNNs in aggregating neighbor node information.

Secondly, The CGRL approach provides a learnable approach to adaptively mask nodes and edges for multiple graph contrastive views in an end-to-end manner. In addition, redundant information irrelevant to node classification is discarded by innovatively introducing information bottleneck theory into multi-view graph contrastive learning.

Thirdly, we introduce a cross-view adversarial reconstruction strategy to further improve the robustness of node feature representation.

Finally, extensive experiments also show that the proposed CGRL method outperforms the state-of-the-art methods on seven real-world publicly available datasets.

## 2. Related Work

Graph Representation Learning Early work [33, 34, 35] have made great progress in representation learning tasks (e.g., node classification, entity alignment, and link prediction, etc). Specifically, GNNs on graph representation learning usually follow the information aggregation mechanism to update the feature representation of nodes, i.e., stimulating connected neighbor nodes to have similar semantic information. Inspired by GNNs, several works (e.g., AROPE [36], DeepWalk [37], and GraphSAGE [38], etc) on graph representation learning usually follows the label propagation mechanism to update the feature representation of nodes, i.e., stimulating connected neighbor nodes to have similar labels. In recent years, GNNs (e.g., GraphSMOTE [39], Nodeformer [40], DisGNN [41], and GraphFL [42], etc) have applied GNN algorithms to learn discriminative latent representations on node classification tasks [43, 44].

Contrastive Learning Contrastive learning (CL) [45, 46, 47], which is originally widely used in computer vision to obtain better image feature representations, has received extensive research attention in graph learning. Specifically, graph contrastive learning (GCL) learns discriminative node representations by maximizing mutual information (MI) between multiple graph views. For instance, DGI [14] learns more discriminative node representations by contrasting node embeddings of local and global graph views. GIC [48] maximizes the MI between node clusters with high similarity to make full use of coarsegrained and fine-grained information between nodes. CMC [46] maximizes MI by contrasting feature representations from different views. GMI [25] estimates and maximizes the MI between the input graph and the feature representation from two aspects of node features and network topology. Some recent self-supervised graph learning (SGLs) methods (e.g., MGAE [49], GraphMAE [50], and GAE [51]) generate multiple graph views of nodes and edges and force the consistency between different graph views. On the one hand, all these methods generate multiple contrasting graph views by randomly masking nodes or edges, which may cause some important structural and semantic information to be lost. On the other hand, these methods maximize mutual information between different graph views, which may force the model to learn some taskindependent semantic information. To sum up, existing self-supervised graph learning for node classification suffer from insufficient information utilization, i.e., information associated with the label.

Information-Bottleneck Representation Learning The information bottleneck (IB) theory [27] argues that if the feature representation learned by the model from the input data discards information that is not useful for the given task while retaining as much as possible the information relevant to the given task, it will increase the generalization of downstream tasks. Formally, IB needs to construct multiple views for feature representation learning. Motivated by the effectiveness of the IB, several works have considered transferring information bottleneck theory to graph representation learning tasks. For instance, MIB [52] designs an unsupervised multi-view method and uses the information bottleneck theory to minimize the representation of redundant information. DeepIB [53] reduces redundant information irrelevant to a given task by minimizing mutual information between multiple views and input data. CMIB [28] combines information bottleneck theory to capture the complementarity of intrinsic information between different views and balance the consistency of multi-view latent representations. Nevertheless, almost all the above-mentioned methods aim to obtain a discriminative graph view to replace the original input graph, which may cause some semantic information and topological information of the original graph to be lost. In conclusion, existing information bottleneck methods for node classification suffer from insufficient information utilization, i.e., input graph information and multi-view information.

## 3. Preliminaries

Notations. Suppose G = {V E R W} , , , represents a graph where V = { v , v 1 2 , . . . , v M } is the nodes set, M is the number of nodes, E ⊆ V × V represent the edges set, r ij ( r ij ∈ E ) represents the connection relationship between node i and node j , and ω ij ( ω ij ∈ W , 0 ≤ ω ij ≤ 1) the weight of the edge E ij . The feature matrix and degree matrix of nodes are expressed as X = { x i } M i =1 amd A = { a ij } ∈ { 0 1 , } M M × , respectively, where x i represent the features of the node v i , and if ( v , v i j ) ∈ E then a ij = 1 otherwise a ij = 0. G = G x ( ) is regarded as the process of graph processing.

GCN Processing. For input features X ∈ R M D × , a graph G = F = G X ( ) is constructed based on the features X . A GCN layer is utilized to update features representations between nodes by aggregating information from their

Figure 2. The overview of the proposed CGRL method. Specifically, we combine node dropout and edge dropout views to obtain a more comprehensive representation, where node dropout view can alleviate the problem of popular bias, and edge dropout can alleviate the problem of misconnection between nodes.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-647380e2-3c20-4d24-ac17-a771885ed80f.jpg)

neighbor nodes. Specially, GCN operates as follows:

z . The optimization goals of IB are as follows:

$$\mathcal { G } ^ { \prime } = F ( \mathcal { G }, \mathcal { W } )$$

$$\mathcal { Y } \, & = r \, \left \{ \mathcal { Y }, \nu v \right \} \\ & = U p d a t e \left ( A g g r e g a t e \left ( \mathcal { G }, W _ { a g g } \right ), W _ { u p d a t e } \right ) \quad \text{where} \\ & = W _ { u p d a t e } \left ( \sum _ { r \in \mathcal { R } } \sum _ { j \in N _ { i } ^ { r } } \frac { 1 } { | \mathcal { N } _ { i } ^ { r } | } \left ( \omega _ { i j } W _ { a g g } \mathcal { G } + \omega _ { i i } W _ { a g g } \mathcal { G } \right ) \right ) \quad \text{$C$} \\ & \quad \text{Eq. 3}$$

where N r i is the set of neighbor nodes of node i under the edge relationship r ∈ R , and W agg and W update represent the learnable weights of the nodes in aggregating surrounding neighbor nodes information and updating the aggregated features, respectively.

In addition, we also introduce a multi-head attention mechanism in the GNN layer to capture node feature information and topology information in a more fine-grained manner. The feature vectors x ′′ i after being aggregated and updated is divided into N heads, i.e., h , h 1 2 , . . . , h N , and each head is assigned a learnable parameter. Therefore, the feature vectors x is finally updated as follows:

$$\mathbf x ^ { \prime } _ { i } = \left [ h ^ { 1 } W ^ { 1 } _ { u p d a t e }, h ^ { 2 } W ^ { 2 } _ { u p d a t e }, \dots, h ^ { N } W ^ { N } _ { u p d a t e } \right ], \quad ( 2 )$$

Information Bottleneck (IB). IB is an information theory-based strategy that describes the information in the data that is relevant to downstream tasks. IB argues that if the obtained feature representation excludes semantic information in the original input that is irrelevant to a given downstream task, it improves the robustness of the model. Specifically, for a given input data x , with associated label information is y , using the IB strategy for model optimization can obtain a compact feature representation

$$\max _ { z } I ( \mathbf Y, \mathbf Z, \theta ) - \beta I ( \mathbf X, \mathbf Z, \theta ), \text{ \quad \ \ } ( 3 )$$

where β is an adjustment factor, θ is a learnable parameter. Combining I ( Y, Z ) and mutual information theory in Eq. 3, we can get:

$$\cdot \quad \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { { \lambda \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { \lambda } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { n } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu } \lambda _ { i } ^ { \nu }$$

However, directly calculating p y z ( | ) is quite difficult. Inspired by [54], we use q y z ( | ) as the variational approximation of p y z ( | ).

Since the KL divergence is always greater than or equal to 0, we can get:

$$& \underset { N, \text{ and } } { \text{same} } \quad \sim \sim, \quad \sim \, \cdots \, \cdots \, \circ \cdots \\ & \text{ed and } \quad K L [ p ( y \, | \, z ), q ( y \, | \, z ) ] \geq 0 \Rightarrow \int d y p ( y \, | \, z ) \log \frac { p ( y \, | \, z ) } { q ( y \, | \, z ) } \geq 0 \\ & \quad \Rightarrow \int d y p ( y \, | \, z ) \log p ( y \, | \, z ) \geq \int d y p ( y \, | \, z ) \log q ( y \, | \, z ). \\ &, \quad ( 2 ) \quad \, \quad \, \quad \,.$$

Therefore we can know:

$$\text{Therefore we can know:} \\ I ( \mathbf Y, \mathbf Z ) & \geq \int d y d z p ( \mathbf y, \mathbf z ) \log \frac { q ( \mathbf y \ | \, \mathbf z ) } { p ( \mathbf y ) } \\ & = \int d y d z p ( \mathbf y, \mathbf z ) \log q ( \mathbf y \ | \, \mathbf z ) + H ( \mathbf y ) \\ & \geq \int d y d z p ( \mathbf y, \mathbf z ) \log q ( \mathbf y \ | \, \mathbf z ) \\ & = \int d y p ( \mathbf y ) \int d z p ( \mathbf z \ | \, \mathbf y ) \log q ( \mathbf y \ | \, \mathbf z ).$$

For I ( X,Z ) in Eq. 3, we can get:

$$I \left ( \mathbf Z, \mathbf X \right ) & = \int d \mathbf z d \mathbf x p \left ( \mathbf x, \mathbf z \right ) \log \frac { p \left ( \mathbf z, \mathbf x \right ) } { p \left ( \mathbf z \right ), p \left ( \mathbf x \right ) } \quad \begin{pmatrix} \det \end{pmatrix} \\ & = \int d \mathbf z d \mathbf x p \left ( \mathbf x, \mathbf z \right ) \log \frac { p \left ( \mathbf z \left | \mathbf x \right ) } { p \left ( \mathbf z \right ) }. \quad \begin{pmatrix} \det \end{pmatrix} \\ & = \int d \mathbf z d \mathbf x p \left ( \mathbf x, \mathbf z \right ) \log \frac { p \left ( \mathbf z \left | \mathbf x \right ) } { p \left ( \mathbf z \right ) }. \quad \begin{pmatrix} \det \end{pmatrix}$$

Similarly, it is quite difficult to calculate p z ( ). We use r z ( ) as the variational approximation to estimate p z ( ). Since KL [ p ( z ) , r ( z )] ≥ 0 ⇒ ∫ d p z ( z ) log p ( z ) ≥ ∫ d p z ( z ) log r we can get an upper bound:

$$\begin{array} { c } I \left ( \mathbf Z, \mathbf X \right ) \leq \int d \mathbf x d z p \left ( \mathbf x \right ) p \left ( \mathbf z \, \left | \, \mathbf x \right ) \log \frac { p \left ( \mathbf z \right | \mathbf x \right ) } { r \left ( \mathbf z \right ) } \\ = \int d \mathbf x p \left ( \mathbf x \right ) \int d \mathbf z p \left ( \mathbf z \, \left | \, \mathbf x \right ) \log \frac { p \left ( \mathbf z \right | \mathbf x \right ) } { r \left ( \mathbf z \right ) }. \end{array} \quad \text{(8)} \quad \text{not} \quad \text{for} \quad \end{array}$$

Combining the above analysis and inequalities, we can obtain the lower bound of the information bottleneck theory:

$$\dots \nmid \nmid \nmid \\ & I ( \mathbf Y, \mathbf Z ) - \sum _ { v = 1 } ^ { V } \beta I \left ( \mathbf Z, \mathbf X \right ) \, \begin{array} { c } \text{where} \\ \text{distr} \\ \text{node} \\ \end{array} \\ & \geq \int d \mathbf y p ( \mathbf y ) \int d \mathbf z p ( \mathbf z \, | \, \mathbf y ) \log q ( \mathbf y \, | \, \mathbf y ) \, \end{array} \quad ( 9 ) \quad \text{in th} \\ & - \beta \int d \mathbf x p ( \mathbf x ) \int d \mathbf z p ( \mathbf z \, | \, \mathbf x ) \log \frac { p \left ( \mathbf z \, | \, \mathbf x ) } { r ( \mathbf z ) }. \, \begin{array} { c } \text{where} \\ \text{contract} \\ \text{tory} \\ \text{rem} \\ \text{.,.,.} \end{array}$$

## 4. Methodology

In this section, we introduce our proposed Contrastive Graph Representation Learning with Adversarial Crossview Reconstruction and Information Bottleneck (CGRL) method in detail.

## 4.1. Automatically Generated Multi-view Augmentation

Adversarial View. By adding perturbed adversarial examples to the original image views, the robustness of the model is significantly enhanced. This improvement can be attributed to a possible explanation that there is still non-predictive redundant information in the information shared between the two augmented views. When adversarial examples are introduced, this redundant information is weakened or eliminated, forcing the model to focus on more important and discriminative features. We define adversarial view as follows:

$$\mathcal { G } _ { a d v } ^ { ( l ) } = \mathcal { G } + \delta ^ { * } \quad \quad \quad ( 1 0 ) \quad \alpha$$

$$\delta ^ { * } & = \underset { \| \delta \| _ { \infty } \leqslant \epsilon } { \argmax } \mathcal { L } _ { \text{adv} } \left ( \mathcal { G } _ { E D }, \mathcal { G } _ { N D }, \mathcal { G } + \delta \right ) \underset { \text{adv} } { \text{and} } \underset { \text{in} } { \text{in} } \\ & = \underset { \text{argmax} } { \argmax } \max \left [ \mathcal { L } _ { \text{CL} } \left ( \mathcal { G } _ { N D }, G + \delta ^ { * } \right ) \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{ } \text{  \text{form} }$$

where G ( ) l adv = {V ′ , E ′ , R W } ′ , ′ , V = { v , v 1 2 , . . . , v M } , δ is the randomly initialized Gaussian noise, L CL is the infoNCE loss, ϵ is the radius. Inspired by recent work [55],

(

we add a perturbation δ to the output of the first hidden layer. It has been empirically shown that it can more effectively perturb the intermediate representation of the model than adding perturbations to the initial node features, allowing the model to learn and predict in more complex environments.

z ), Node-Masking View. As shown in Fig. 1, both the category of nodes and the degree of nodes in citation data show data imbalance, which hinders GCN from learning the feature representation of minority class nodes. Therefore, we perform automatic learnable node masking before each information aggregation and feature update of GCN to alleviate the shielding effect of influential nodes on minority class nodes. The node-masking view we created is formulated as follows:

$$\mathcal { G } _ { N D } ^ { ( l ) } = \left \{ \left \{ v _ { i } ^ { \prime } \odot \eta _ { i } ^ { ( l ) } \ | \ v _ { i } ^ { \prime } \in \mathcal { V } ^ { \prime } \right \}, \mathcal { E } ^ { \prime }, \mathcal { R } ^ { \prime }, \mathcal { W } ^ { \prime } \right \} \quad ( 1 2 )$$

where η ( ) l i ∈ { 0 1 , } is sampled from a parameterized Bernoulli distribution Bern ω ( l i ), and η ( ) l i = 0 represents masking node v i , η ( ) l i = 1 represents keeping node v i .

Randomly removing some nodes and their connections in the graph may result in a large loss of minority class node information, thereby affecting the information aggregation of minority class nodes and leading to unsatisfactory classification results. Therefore, instead of directly removing the selected nodes from the graph, we replace the selected nodes by sampling the representation of the local subgraph using a random walk strategy to obtain a local representation of the node.

Edge Perturbation View. The goal of perturbing edges is to generate an optimized graph structure that filters noisy edges and alleviates the problem of popularity bias. The edge perturbation view is formulated as follows:

$$\mathcal { G } _ { E D } ^ { ( l ) } = \left \{ \mathcal { V } ^ { \prime }, \left \{ e _ { i j } ^ { \prime } \odot \eta _ { i j } ^ { ( l ) } \ | \ e _ { i j } ^ { \prime } \in \mathcal { E } ^ { \prime }, \mathcal { R } ^ { \prime }, \mathcal { W } ^ { \prime } \right \} \right \} \quad ( 1 3 )$$

where η ( ) l ij ∈ { 0 1 , } is also sampled from a parameterized Bernoulli distribution Bern ω ( l ij ), and η ( ) l ij = 0 represents perturbating edges e ij , η ( ) l i = 1 represents keeping edge e ij .

To enable the model to automatically learn whether to mask nodes and perturb edges, we formally define the learnable parameter ω l i and ω l ij as follows:

$$( 1 0 ) \quad \omega _ { i } ^ { ( l ) } = \text{Linear} \left ( \mathbf e _ { i } ^ { \prime ( l ) } \right ) ; \ \omega _ { i j } ^ { ( l ) } = \text{Linear} \left ( \left [ \mathbf e _ { i } ^ { \prime ( l ) } ; \mathbf e _ { j } ^ { \prime ( l ) } \right ] \right ) \ ( 1 4 )$$

To ensure that the model can automatically optimize and generate augmented multi-views in an end-to-end learning method, we use reparameterization technology [56] to convert the discretized η into a continuous function. The formula is defined as follows:

$$\delta \text{ is } \frac { \exp \left ( \left ( \log \left ( \pi _ { i } \right ) + g _ { i } \right ) / \tau \right ) } { \sum _ { j = 1 } ^ { m } \exp \left ( \left ( \log \left ( \pi _ { j } \right ) + g _ { j } \right ) / \tau \right ) }, \quad \text{for } i = 1, \dots, m \\ \delta \text{ in } \frac { \left ( 1 5 \right ) } { \sim }$$

where g i = -log ( -log ( ϵ i )) , ϵ i ∼ Uniform (0 1), , τ ∈ R + means annealing temperature, τ represents the class probability, and m represents number of categories.

After obtaining the masked node and edge perturbed views, we input them into GCN for feature representation to obtain optimized multi-views. The formula is defined as follows:

$$\mathbf E _ { N D } ^ { ( l ) } & = G r a p h C o n v \left ( \mathbf E _ { N D } ^ { ( l - 1 ) }, \mathcal { G } _ { N D } ^ { ( l ) } \right ) \\ \mathbf E _ { E D } ^ { ( l ) } & = G r a p h C o n v \left ( \mathbf E _ { E D } ^ { ( l - 1 ) }, \mathcal { G } _ { E D } ^ { ( l ) } \right ) \quad \text{when} \quad.$$

where GraphConv represents the graph convolution operation, and we choose GAT as our graph encoder. E ND and E ED represent the node feature representations of node-masking view and edge perturbation view respectively, G ND and G ED represent node-masking view and edge perturbation view respectively.

## 4.2. Contrastive Learning via IB

Although CGRL combines an automatic learnable view augmentation and a node classification process for model optimization, we argue that relying solely on the classification objective does not well guide the node masking and edge perturbation process to create optimal multiviews. Therefore, we follow the information bottleneck strategy [27] to retain sufficient semantic information relevant to downstream tasks in the augmented node mask and edge perturbation views. Specifically, unlike traditional CL strategies, we encourage maintaining topological heterogeneity between the augmented view and the original graph while maximizing the information relevant to the node classification task. Based on the above strategy, we can obtain topologically heterogeneous but semantically similar enhanced multi-view representations and effectively remove noise information in the graph. Therefore, the objective in Eq. 3 is summarized as:

$$& \min _ { ( E, \tilde { E } ) } \mathcal { L } _ { C L S } + I ( \mathbf E, \tilde { \mathbf E } ) & \mathcal { L } \\ & = - \sum _ { i = 1 } ^ { n } y _ { i } l o g ( \hat { y } ) + I ( \mathbf E, \tilde { \mathbf E } ) & \mathcal { \text{with} } \\ & \underset { \text{$\text{$\text{$will}$} } { \text{$\text{$\text{$will}$} } }$$

where L CLS represents the cross-entropy loss, I ( E E , ˜ ) represents the mutual information between the original view and the augmented view.

Following [57, 58], minimizing the lower bound of mutual information (i.e., Eq. 9) is equivalent to maximizing the InfoNCE loss [59]. Therefore, we adopt negative InfoNCE to optimize the feature representation between the augmented view and the original graph. Specifically, we treat the same node representations in the original graph and the automatically generated view as positive pairs (i.e., { ( e , e ′ i ˜ ) ′ i | v ′ i ∈ V } ′ ), and different node representations

̸

as negative pairs (i.e., { ( e , e ′ i ˜ ) ′ j | v , v ′ i ′ j ∈ V } ′ , i = ). j

$$\overset { \_ \_ } { \shortcirc b } } \quad \underset { \ v _ { j } ^ { \prime } \in \mathcal { V } ^ { \prime } } { \longrightarrow } \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \rightangle \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right | \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right ) \ \left ( \sum \nolimits _ { \nu } \ \right.$$

where s ( ) is used to measure the similarity between posi-· tive and negative sample pairs.

## 4.3. Adversarial Cross-view Reconstruction

To further achieve feature disentanglement, we propose a cross-view reconstruction mechanism. Specifically, we hope that the representation pairs within and across enhanced views can recover the original data. More specifically, we define ( E ND , E ED ), ( E E , ED ), ( E E , ED ) as a cross-view representation pair, and repeat the reconstruction process on it to predict the original view, aiming to ensure that E ND , E ED , E are optimized to approximately disentangle each other. Intuitively, the reconstruction process is able to separate the information of the shared feature set from the information in the unique feature set between the two enhanced views. We formally define the reconstruction process as:

$$\text{$i$-} \\ \text{all} \\ \text{$\cdot$}$$

wherer N represents the number of nodes.

## 4.4. Model Training

We train the model with the goal of optimizing the node mask view, edge perturbation view, and node classification:

$$\mathcal { L } & = \mathcal { L } _ { C L S } + \alpha \mathcal { L } _ { C L S } ^ { N D } \, + \left ( 1 - \alpha \mathcal { L } _ { C L S } ^ { E D } \, \right ) \\ & + \beta \left ( I \left ( \mathbf E, \mathbf E _ { N D } \right ) + I \left ( \mathbf E, \mathbf E _ { E D } \right ) \right ) + \mathcal { L } _ { \text{recon} } + \lambda \| \Theta \| _ { 2 } ^ { 2 }$$

where λ and β represent the learning weights of L2 regularization and information bottleneck contrast learning respectively.

## 5. Theoretic Analysis

Theorem 1. Specifically, we regard the augmented view as ˜ , G the noisy view as G ′ , and the node label information as Y CLS . We assume that ˆ is G not related to the classification information Y CLS . Therefore, the upper bound of I ( ˆ ; G G ˜ ) is defined as follows:

$$I \left ( \hat { \mathcal { G } }, \tilde { \mathcal { G } } \right ) \leq I ( \mathcal { G }, \tilde { \mathcal { G } } ) - I \left ( Y _ { C L S }, \tilde { \mathcal { G } } \right ) \text{ \quad \ \ } ( 2 1 )$$

Proof. We assume that the clean graph G is defined by G ′ and label information Y , and we can get ( G ′ , Y CLS ) →

G → G ˜ according to the Markov chain [60]. Therefore, we can get:

$$\begin{array} { c c c } \cdot \cdot & & I \left ( a \\ I ( \mathcal { G }, \tilde { \mathcal { G } } ) & \geq I \left ( \left ( Y _ { \text{CLS} }, \hat { \mathcal { G } } \right ), \tilde { \mathcal { G } } \right ) & & I \left ( a \\ & = I \left ( \hat { \mathcal { G } }, \tilde { \mathcal { G } } \right ) + I \left ( Y _ { \text{CLS} }, \tilde { \mathcal { G } } \, | \, \hat { \mathcal { G } } \right ) & & I \left ( \hat { \mathcal { G } } \\ & = I \left ( \hat { \mathcal { G } }, \tilde { \mathcal { G } } \right ) + H \left ( Y _ { \text{CLS} } \, | \, \hat { \mathcal { G } } \right ) - H \left ( Y _ { \text{CLS} } \, | \, \hat { \mathcal { G } }, \tilde { \mathcal { G } } \right ) & & I \left ( \hat { \mathcal { G } } \\ & & = \hat { \mathcal { G } } \end{array}$$

Because there is no correlation between ˜ and G Y CLS , H Y ( CLS | G ˜ ) = H Y ( CLS ). Furthermore, H ( Y CLS | G ˆ , ˜ G ) ≤ H Y ( CLS ). Therefore, we simplify the Eq. 22 as follows:

$$I ( \mathcal { G }, \tilde { \mathcal { G } } ) & \geq I \left ( \hat { \mathcal { G } }, \tilde { \mathcal { G } } \right ) + H \left ( Y _ { C L S } \right ) - H \left ( Y _ { C L S } \, | \, \tilde { \mathcal { G } } \right ) \quad \quad \, = \, ; \\ & = I \left ( \hat { \mathcal { G } }, \tilde { \mathcal { G } } \right ) + I \left ( Y _ { C L S }, \tilde { \mathcal { G } } \right ) \quad \, = \, ;$$

Therefore, we prove that Eq. 21 holds. In summary, we provide a theoretical basis to ensure that graph contrastive learning via information bottlenecks can achieve noise invariance by reducing redundant and interfering information in augmented views.

Theorem 2. We optimize the reconstruction by minimizing the entropy H ( E | E ND , E ED ). Ideally, when H ( E | E ND , E ED ) -E E E , ND , E ED [log p ( E | E ND , E ED )] = 0, we achieve the best feature disentanglement. However, in practice, the estimation of conditional probability p ( E | E ND , E ED ) is very tricky and complicated. Therefore, we use an approximate variational distribution q ( E | E ND , E ED ) to simplify the calculation and optimization process. We provide a theoretical upper bound on H ( E | E ND , E ED ) as follows:

$$H ( \mathbf E \, | \, \mathbf E _ { N D }, \mathbf E _ { E D } ) \leqslant \max \{ \| \mathbf E - \mathbf E _ { N D } \| _ { 2 } ^ { 2 }, \| \mathbf E - \mathbf E _ { E D } \| _ { 2 } ^ { 2 } \} \quad P \iota _ { N D },$$

Proof. For a given original view E and two augmented views E ND and E ED , we have:

$$p ( \mathbf E _ { N D }, \mathbf E _ { E D } )$$

Lemma 1. For three given random variables a, b, c , if p b, c ( ) = p b p c ( ) ( ) and p b, c ( | a ) = p b ( | a p c ) ( | a ), then I ( a, b | c ) = I ( a, b ). Based on the definition of mutual information, we deduce:

$$\nu _ { 1 } ( \varpi \varpi + \nu ) &$$

According to Lemma 1, we can derive the theoretical bound of I ( E E ; ND , E ED ) as follows:

$$\text{ive} \quad \text{volume} \quad \text{$\lambda,\$$

Assuming q is a Gaussian distribution, the reconstruction process can be equivalent to minimizing the information entropy and its theoretical upper bound is formally defined as follows:

$$I ( \mathbf E \, | \quad H ( \mathbf E \, |$$

Proof. The estimation of conditional probability p ( E | E ND , E ED ) is very tricky and complicated. Therefore, we use an approximate variational distribution q ( E | E ND , E ED ) to simplify the calculation and optimization process. Therefore, we have,

$$\begin{array} {$$

Assume q ( E | E ND , E ED ) is a Gaussian distribution N ( E | E ND , E ED , σ 2 I ) :

H

(

E

|

E

ND

,

E

ED

)

$$& H ( \mathbf E \, | \, \mathbf E _ { N D }, \mathbf E _ { E D } ) \\ & \leqslant - \mathbb { E } _ { p } ( \mathbf E | \mathbf E _ { N D }, \mathbf E _ { E D } ) \left [ \log q \left ( \mathbf E \, | \, \mathbf E _ { N D }, \mathbf E _ { E D } \right ) \right ] \\ & = - \mathbb { E } _ { p } ( \mathbf E | \mathbf E _ { N D }, \mathbf E _ { E D } ) \left [ \log \left ( \frac { 1 } { \sqrt { 2 \pi I \sigma } } e ^ { - \frac { 1 } { 2 } \left ( \mathbf E - \mathbf E _ { \{ N D, E D \} } \right ) ^ { 2 } } { \left ( \sigma ^ { 2 } \mathbf I \right ) } \right ) \right ] \\ & = - \mathbb { E } _ { p } ( \mathbf E | \mathbf E _ { N D }, \mathbf E _ { E D } ) \left [ \log \left ( \frac { 1 } { \sqrt { 2 \pi I \sigma } } \right ) - \frac { \left ( \mathbf E - \mathbf E _ { \{ N D, E D \} } \right ) ^ { 2 } } { 2 \sigma ^ { 2 } \mathbf I } \right ] \\ & \text{Therefore, we get the upper bound of } H ( \mathbf E \, | \, \mathbf E _ { N D }. \mathbf E _ { E D } ).$$

Therefore, we get the upper bound of H ( E | E ND , E ED ).

## 6. Experiments

## 6.1. Experimental Setup

Datasets Description In our experiments, seven publicly available benchmark datasets are used including two Amazon items datasets [61] (i.e., Computers, and Photo), five citation network datasets [62] (i.e., Citeseer, Pubmed, DBLP, CoraFull, and Cora), three large-scale datasets [63] (i.e., Ogbn-arxiv, Ogbn-mag, and Ogbn-products), page network datasets [64, 65] (i.e., Wiki-CS and Croco).

Evaluation Metrics We use classification accuracy to evaluate the performance of our method CGRL and other comparative methods.

Comparison Methods We compare our method CGRL with twelve state-of-the-art deep learning-based algorithms, including two traditional graph embedding algorithms (i.e., raw features [66], and DeepWalk [37]), three semi-supervised algorithms (i.e., GCN [67], NIGCN[73], and GAT [68]), and nine self-supervised algorithms (i.e., GAE [69], VGAE [69], DGI [14], GCA [18], MVGRL [71], GIC [48], GRACE [70], GMI [25], and CRLC [72]).

Setting-up All the experiments in this paper are implemented on a server with 2 A100 (total 160GB memory). For each experiment, we run the code five times with a random seed to obtain the final mean and corresponding standard deviation to avoid experimental chance. Futhermore, for some parameter settings of the model, we set epochs to 1000/300, batch-size to full-batch/mini-batch, learning rate to 0.005, mask rate to 0.5, α to 0.5, β is 0.5, the activation function to GELU, dropout is 0.2, the weight decay to 1e-4, learning rate scheduling to cosine, warmup epochs to 100, and hidden size to 128. We utilize the Adam optimization algorithm to update parameters.

## 6.2. Results and Analysis

Node classification. Tables 1 and 2 summarize the node classification accuracy of the baselines and the proposed method CGRL on twelve real graph-structured data sets. Specifically, our approach leverages traditional graph embedding algorithms (i.e., raw features and DeepWalk). For example, our method CGRL improves the average accuracy by 20.31% and 10.76% compared to raw features and DeepWalk methods respectively. Compared with selfsupervised methods (e, g., DGI, and GCA, etc.), CGRL also achieves better performance. In addition, CGRL also outperforms semi-supervised algorithms (i.e., GCN , NIGCN, and GAT). The performance improvement may be attributed to the design of the multi-view contrast learning strategy for adaptive masking nodes and perturbed edges, which enables the model to automatically learn whether to mask nodes and perturbed edges. To ensure that our model can reconstruct nodes and edges, we use a random walk strategy to sample the node's subgraph structure to blur its representation. However, most other baselines perform multiview contrastive learning by randomly dropping nodes and edges, which will seriously destroy the semantic information of the graph. In addition, we also introduce the information bottleneck theory to ensure that the augmented views are structurally heterogeneous but semantically similar. The intuition behind this is that maximizing the mutual information between multiple views will lead to a consistent representation of the augmented views learned by the model, which leads to overfitting of the model.

Effect of noise. In order to verify the anti-interference ability of our model CGRL, we perform experiments to demonstrate the performance by varying the noise rate in { 0 0 1 , . , 0 2 . , 0 3 . , 0 4 . , 0 5 . } and the β in { 0 1 . , 0 2 . , 0 3 . , 0 4 . , 0 5 . , 0 6 . , 0 7 . , 0 8 . , 0 9 . } . The experimental results are shown in Fig. 3, We can find that when the parameter β is 0, CGRL has the worst effect on the Cora, Citeseer and PubMed data sets. When β is greater than 0, CGRL's anti-interference ability is significantly reflected, and the best effect is in the range of 0.2 to 0.6. The experimental results show that the introduction of information bottleneck theory can enhance the noise invariance ability of the model.

Hyper-parameter analysis. We set two hyperparameters α and β during the model optimization process. Their settings will have a relatively large impact on the performance of the model. Therefore, we also investigated the impact of hyperparameters on CGRL. We performed the node classification task by varying α and β from 0.1 to 0.9 and visualized the accuracy in Fig. 4. When α and β are set to relatively large values, the node classification effect of the model is better, and when set to smaller values, the performance is poor. The difference in performance can be attributed to the inability of smaller parameter settings to eliminate redundant information in the graph structure and to obtain optimal augmented multi-view.

## 6.3. Ablation Study

We performed three ablation experiments to verify the effectiveness of our proposed node-masking view, edge perturbation view, and information bottleneck theory.

## 6.3.1. Effectiveness of adaptive graph contrastive learning via IB.

We perform ablation studies to analyze the impact of node-masking view, edge perturbation view, and IB criterion on experimental results. The experimental results are shown in Table 3. Firstly, when CGRL includes all modules, the accuracy of node classification on seven data sets is the highest. Secondly, We find that the nodemasking view has a greater impact on the experimental results than the edge perturbation view. Experimental phenomena show that node classification mainly depends on the features of nodes. Thirdly, node-mask view and edge perturbation view combined with IB accuracy can further improve the accuracy of node classification. The performance improvement may be attributed that the IB criterion can promote multi-views to obtain structurally heterogeneous but semantically similar graph structures.

## 6.3.2. Impact of GNN variants.

We explore the impact of different graph neural network variants on experimental results. As shown in Table 4, GAT has the best effect on seven graph structure

Table 1. Experimental results on seven publicly available datasets. Classification accuracy (%) is chosen as our evaluation metric. The best result in each column is in bold.

| Methods          | Cora       | Citeseer   | PubMed     | Photo      | Computers   | Ogbn-arxiv   | Ogbn-products   |
|------------------|------------|------------|------------|------------|-------------|--------------|-----------------|
| Raw Feature [66] | 47.9 ± 0.4 | 49.3 ± 0.3 | 69.1 ± 0.2 | 78.5 ± 0.2 | 73.8 ± 0.1  | 56.3 ± 0.3   | 59.7 ± 0.2      |
| Deep Walk [37]   | 81.5 ± 0.2 | 43.2 ± 0.4 | 65.3 ± 0.5 | 89.4 ± 0.1 | 85.3 ± 0.1  | 63.6 ± 0.4   | 73.2 ± 0.2      |
| GCN [67]         | 81.5 ± 0.2 | 70.3 ± 0.4 | 79.0 ± 0.5 | 91.8 ± 0.1 | 84.5 ± 0.1  | 70.4 ± 0.3   | 81.6 ± 0.4      |
| GAT [68]         | 83.0 ± 0.2 | 72.5 ± 0.3 | 79.0 ± 0.5 | 91.8 ± 0.1 | 85.7 ± 0.1  | 70.6 ± 0.3   | 82.4 ± 0.4      |
| GAE [69]         | 74.9 ± 0.4 | 65.6 ± 0.5 | 74.2 ± 0.3 | 91.0 ± 0.1 | 85.1 ± 0.4  | 63.6 ± 0.5   | 72.1 ± 0.1      |
| VGAE [69]        | 76.2 ± 0.4 | 66.7 ± 0.5 | 75.7 ± 0.3 | 91.4 ± 0.1 | 85.7 ± 0.3  | 64.8 ± 0.2   | 72.9 ± 0.2      |
| DGI [14]         | 82.3 ± 0.5 | 71.5 ± 0.4 | 79.4 ± 0.3 | 91.3 ± 0.1 | 87.8 ± 0.2  | 65.1 ± 0.4   | 77.9 ± 0.2      |
| GMI [25]         | 83.0 ± 0.2 | 71.5 ± 0.4 | 79.9 ± 0.4 | 90.6 ± 0.2 | 82.2 ± 0.4  | 68.2 ± 0.2   | 76.8 ± 0.3      |
| GRACE [70]       | 83.1 ± 0.2 | 72.1 ± 0.1 | 79.6 ± 0.5 | 91.9 ± 0.3 | 86.8 ± 0.2  | 68.7 ± 0.4   | 77.4 ± 0.4      |
| MVGRL [71]       | 82.9 ± 0.3 | 72.6 ± 0.4 | 80.1 ± 0.7 | 91.7 ± 0.1 | 86.9 ± 0.1  | 68.1 ± 0.1   | 78.1 ± 0.1      |
| GCA [18]         | 81.8 ± 0.2 | 71.9 ± 0.4 | 81.0 ± 0.3 | 92.4 ± 0.4 | 87.7 ± 0.1  | 68.2 ± 0.2   | 78.4 ± 0.3      |
| GIC [48]         | 81.7 ± 0.5 | 71.9 ± 0.9 | 77.4 ± 0.5 | 91.6 ± 0.1 | 84.9 ± 0.2  | 68.4 ± 0.4   | 75.8 ± 0.2      |
| CRLC [72]        | 83.5 ± 0.2 | 72.4 ± 0.5 | 82.0 ± 0.1 | 92.2 ± 0.2 | 87.2 ± 0.4  | 68.8 ± 0.3   | 82.6 ± 0.3      |
| NIGCN [73]       | 83.4 ± 0.3 | 71.6 ± 0.3 | 81.6 ± 0.3 | 91.6 ± 0.2 | 85.7 ± 0.3  | 60.5 ± 0.5   | 74.5 ± 0.5      |
| CGRL (Ours)      | 86.3 ± 0.2 | 75.4 ± 0.2 | 84.5 ± 0.6 | 93.7 ± 0.3 | 89.7 ± 0.5  | 74.7 ± 0.4   | 85.1 ± 0.3      |

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-9b8c3cfe-66e8-405a-a1ae-8b2567fdf3bc.jpg)

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-69adeef4-f700-4255-b5dd-f2476549371c.jpg)

Figure 3. The impact of different noise rates on experimental results on Cora, Citeseer and PubMed datasets.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-7b239925-6120-4d3a-85cc-d5392c01727f.jpg)

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-923a2420-e35a-42b8-bbeb-47bf9fbc0ff3.jpg)

## 6.3.3. Effectiveness of multi-view optimization.

Figure 4. The impact of hyperparameter settings (i.e., α and β ) on experimental results in four data sets (i.e., Cora, Citeer, and PubMed).

data sets, with node classification accuracy rates of 86.3%, 75.4% and 84.5% on the Cora, Citeseer, and PubMed data sets, respectively, followed by GraphSAGE, with node classification accuracy rates of 83.6%, 72.8% and 82.0% , respectively, and GCN has the worst effect, with node classification accuracy rates of 82.7%, 70.7% and 81.4%, respectively. Therefore, we choose GAT as our encoder in our experiments.

We perform ablation experiments to verify the impact of multi-view augmentation on the optimization process of model training. The experimental results are shown in Fig. 5. On Cora, Citeseer, and PubMed datasets, we found that in the absence of node-masking view or edge perturbation view, the loss value of the model cannot converge to the optimal value, and the edge perturbation view has the worst convergence effect. When CGRL combines the node-masking view, edge perturbation view and information bottleneck criterion, the loss value of the model can converge to close to 0. The experimental results show that the performance of the augmented node-masking view is better than the augmented edge perturbation view, but worse than the augmented multi-view that combines information bottlenecks, which shows that the information bottleneck theory can further enhance the utilization of information.

## 6.3.4. Visualization.

Weuse the T-SNE method to project the high-dimensional node features into a 2-dimensional space and visualize them. The visualization results are shown in Figs. 6 and 7. In the

Table 2. Experimental results on five publicly available datasets. Classification accuracy (%) is chosen as our evaluation metric. The best result in each column is in bold.

| Methods           | Wiki-CS    | DBLP       | Croco      | CoraFull   | Ogbn-mag   |
|-------------------|------------|------------|------------|------------|------------|
| Raw Feature [66]  | 72.0 ± 0.9 | 71.6 ± 0.6 | 41.7+0.4   | 43.6 ± 0.7 | 22.1 ± 0.3 |
| DeepWalk [37]     | 74.4 ± 0.8 | 76.0 ± 0.7 | 42.5 ± 0.7 | 53.2 ± 0.5 | 25.6 ± 0.3 |
| GCN [67]          | 74.0 ± 0.7 | 77.8 ± 0.5 | 52.6 ± 0.8 | 59.4+0.6   | 30.1 ± 0.3 |
| GAT [68]          | 77.6 ± 0.6 | 78.2 ± 1.5 | 53.3+1.0   | 58.6 ± 0.5 | 30.5 ± 0.3 |
| DGI [14]          | 74.8 ± 0.7 | 83.1 ± 0.5 | 53.1 ± 0.7 | 55.1 ± 0.6 | 30.6 ± 0.3 |
| GIC [48]          | 75.9 ± 0.6 | 81.9 ± 0.8 | 56.8+0.6   | 58.2 ± 0.7 | 29.8 ± 0.2 |
| GRACE [70]        | 75.3 ± 0.7 | 84.2 ± 0.6 | 58.3 ± 0.4 | 54.0 ± 0.6 | 31.1 ± 0.3 |
| GMI [25]          | 74.8 ± 0.7 | 83.9 ± 0.8 | 54.3 ± 0.9 | 54.6+0.8   | 27.2 ± 0.1 |
| MVRLG [71]        | 76.3 ± 1.1 | 79.5 ± 0.8 | 57.9 ± 0.6 | 58.8 ± 0.7 | 30.4 ± 0.4 |
| Contrast-Reg [74] | 77.0 ± 0.6 | 83.6 ± 0.8 | 58.4 ± 0.7 | 58.9 ± 0.6 | 30.9 ± 0.4 |
| GRLC              | 77.9 ± 0.5 | 84.2 ± 0.6 | 59.5 ± 0.7 | 59.4 ± 0.6 | 31.6 ± 0.2 |
| CGRL (Ours)       | 80.4 ± 0.3 | 87.2 ± 0.7 | 63.6 ± 0.4 | 62.9 ± 0.5 | 35.6 ± 0.4 |

Table 3. Ablation studies are performed on seven datasets to verify the effectiveness of node-masking view, edge perturbation view, and information bottleneck strategy. Classification accuracy (%) is chosen as our evaluation metric. The best result in each column is in bold.

Table 4. Experiments were conducted using different graph convolutional neural networks on seven data sets to verify their impact on experimental results. Classification accuracy (%) is chosen as our evaluation metric. The best result in each column is in bold.

| L ND CLS   | L ED CLS   | I ( E,E ˜ )   | Cora   | Citeseer   | PubMed   |   Photo |   Computers |   Ogbn-arxiv |   Ogbn-products |
|------------|------------|---------------|--------|------------|----------|---------|-------------|--------------|-----------------|
| %          | %          | % 83.0        | 72.5   | 79.0       |          |    91.8 |        84.5 |         70.4 |            81.6 |
| "          | %          | % 83.7        | 72.7   |            | 79.5     |    92.1 |        85.3 |         70.8 |            82   |
| %          | "          | % 83.2        |        | 72.5       | 79.3     |    92   |        84.7 |         70.5 |            81.6 |
| "          | "          | %             | 84.0   | 72.9       | 81.8     |    92.4 |        86.6 |         71.5 |            82.4 |
| "          | %          | "             | 84.0   | 73.1       | 82.3     |    92.6 |        88.1 |         71.9 |            83   |
| %          | "          | "             | 83.8   | 72.8       | 80.0     |    92.2 |        85.6 |         70.8 |            82.1 |
| "          | "          | "             | 86.3   | 75.4       | 84.5     |    93.7 |        89.7 |         74.7 |            85.1 |

| GraphConv      |   Cora |   Citeseer |   PubMed |   Photo |   Computers |   Ogbn-arxiv |   Ogbn-products |
|----------------|--------|------------|----------|---------|-------------|--------------|-----------------|
| GCN [67]       |   82.7 |       70.7 |     81.4 |    88.6 |        86.1 |         70.9 |            82.3 |
| GAE [69]       |   79.3 |       68.4 |     76.8 |    90.6 |        85.9 |         67.2 |            74.6 |
| VGAE [69]      |   79.8 |       71.2 |     77.4 |    91.2 |        86.6 |         68.1 |            77.4 |
| GIN [75]       |   83.3 |       71.8 |     81.2 |    90.5 |        86.9 |         71.3 |            82.6 |
| GraphSAGE [38] |   83.6 |       72.8 |     82   |    92.4 |        87.6 |         71.9 |            82.8 |
| GAT [68]       |   86.3 |       75.4 |     84.5 |    93.7 |        89.7 |         74.7 |            85.1 |

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-d80cf65a-b435-4c59-948d-5fe47bab9209.jpg)

Figure 5. Effect of node-masking view and edge perturbation view on training loss.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-77bfe101-18f1-48b3-b972-f5856332f242.jpg)

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-1f6585f1-e0ad-4b37-a5ee-e559d982f23a.jpg)

Figure 6. Visualization of feature embeddings in 2-dimensional space for different comparison algorithms on the Cora dataset.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-33e9aaaf-9e86-43eb-bf9d-1a6feeb0f2e0.jpg)

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-5857a538-6794-487a-9a94-7030a7a75d8a.jpg)

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-6027977c-074f-42a0-8963-63488f572bb6.jpg)

Figure 7. Visualization of feature embeddings in 2-dimensional space for different comparison algorithms on the Citeer dataset.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-58dd00ba-3418-4d8e-a7bf-6aa336397f04.jpg)

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-9948b3a9-a7a1-4400-ba8f-f9e96cf0ee86.jpg)

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00295v1-figure-45c6ae2a-81fe-4285-9182-ba0db7323f9f.jpg)

Cora data set, GCN and GAT have more overlaps between different node categories and the class boundaries are not clear enough, while CGRL has clearer class boundaries between different node categories. In the Citeseer data set, the feature embeddings of GCN and GAT are more scattered among the same node categories, and there is excessive overlap between different categories, while CGRL is more densely distributed for the same category of nodes and there is less overlap between different categories.

## 7. Conclusions

In this paper, we propose a Contrastive Graph Representation Learning with Adversarial Cross-view Reconstruction and Information Bottleneck for node classification to automatically generate structurally heterogeneous but semantically similar multi-views. Specifically, CGRL can adaptively learn to mask nodes and perturb edges in the graph to obtain optimal graph structure representation. Furthermore, we innovatively introduce the information bottleneck theory into GCL to eliminate redundant information in multiple contrasting views while retaining as much information about node classification as possible. Moreover, we add noise perturbations to the original views and reconstruct the augmented views by constructing adversarial views to improve the robustness of node feature representation. Extensive experiments on real-world public datasets show that our approach significantly outperforms existing the SOTA algorithms.

## References

- [1] X. Xu, T. Wang, Y. Yang, A. Hanjalic, H. T. Shen, Radial graph convolutional network for visual question generation, IEEE Transactions on Neural Networks and Learning Systems 32 (4) (2020) 1654-1667.
- [2] X. Zhou, F. Shen, L. Liu, W. Liu, L. Nie, Y. Yang, H. T. Shen, Graph convolutional network hashing, IEEE Transactions on Cybernetics 50 (4) (2018) 1460-1472.
- [3] Y. Shou, T. Meng, W. Ai, S. Yang, K. Li, Conversational emotion recognition studies based on graph convolutional neural networks and a dependent syntactic analysis, Neurocomputing 501 (2022) 629-639.
- [4] Y. Shou, T. Meng, W. Ai, N. Yin, K. Li, A comprehensive survey on multi-modal conversational emotion recognition with deep learning, arXiv preprint arXiv:2312.05735 (2023).
- [5] T. Meng, Y. Shou, W. Ai, N. Yin, K. Li, Deep imbalanced learning for multimodal emotion recognition in conversations, arXiv preprint arXiv:2312.06337 (2023).
- [6] T. Meng, Y. Shou, W. Ai, J. Du, H. Liu, K. Li, A multi-message passing framework based on heterogeneous graphs in conversational emotion recognition, Neurocomputing 569 (2024) 127109.
- [7] Y. Shou, T. Meng, W. Ai, K. Li, Adversarial representation with intra-modal and inter-modal graph contrastive learning for multimodal emotion recognition, arXiv preprint arXiv:2312.16778 (2023).
- [8] W. Ai, Y. Shou, T. Meng, K. Li, Der-gcn: Dialog and event relation-aware graph convolutional neural network for multimodal dialog emotion recognition, IEEE Transactions on Neural Networks and Learning Systems (2024).
- [9] X. He, K. Deng, X. Wang, Y. Li, Y. Zhang, M. Wang, Lightgcn: Simplifying and powering graph convolution network for recommendation, in: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2020, pp. 639-648.
- [10] X. Wang, X. He, M. Wang, F. Feng, T.-S. Chua, Neural graph collaborative filtering, in: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2019, pp. 165-174.
- [11] W. Ai, F. Zhang, T. Meng, Y. Shou, H. Shao, K. Li, A twostage multimodal emotion recognition model based on graph contrastive learning, in: 2023 IEEE 29th International Conference on Parallel and Distributed Systems (ICPADS), IEEE, 2023, pp. 397-404.
- [12] Y. Shou, X. Cao, D. Meng, Masked contrastive graph representation learning for age estimation, arXiv preprint arXiv:2306.17798 (2023).
- [13] Y. Shou, T. Meng, W. Ai, C. Xie, H. Liu, Y. Wang, Object detection in medical images based on hierarchical transformer and mask mechanism, Computational Intelligence and Neuroscience 2022 (1) (2022) 5863782.
- [14] P. Veliˇkovi´, c c W. Fedus, W. L. Hamilton, P. Li`, Y. Bengio, o R. D. Hjelm, Deep graph infomax, in: International Conference on Learning Representations.
- [15] Y. Shou, X. Cao, D. Meng, B. Dong, Q. Zheng, A lowrank matching attention based cross-modal feature fusion method for conversational emotion recognition, arXiv preprint arXiv:2306.17799 (2023).
- [16] W. Ai, Y. Shou, T. Meng, K. Li, Der-gcn: Dialogue and event relation-aware graph convolutional neural network for multimodal dialogue emotion recognition, arXiv preprint arXiv:2312.10579 (2023).
- [17] Y. Shou, T. Meng, W. Ai, F. Zhang, N. Yin, K. Li, Adversarial alignment and graph fusion via information bottleneck for multimodal emotion recognition in conversations, Information Fusion (2024) 102590.
- [18] Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, L. Wang, Graph contrastive learning with adaptive augmentation, in: Proceedings of the Web Conference 2021, 2021, pp. 2069-2080.
- [19] J. Wu, X. Wang, F. Feng, X. He, L. Chen, J. Lian, X. Xie, Selfsupervised graph learning for recommendation, in: Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, 2021, pp. 726-735.
- [20] J. Yu, H. Yin, J. Li, Q. Wang, N. Q. V. Hung, X. Zhang, Selfsupervised multi-channel hypergraph convolutional network for social recommendation, in: Proceedings of the web conference 2021, 2021, pp. 413-424.
- [21] X. Xia, H. Yin, J. Yu, Y. Shao, L. Cui, Self-supervised graph co-training for session-based recommendation, in: Proceedings of the 30th ACM international conference on information &amp; knowledge management, 2021, pp. 2180-2190.
- [22] P. Bachman, R. D. Hjelm, W. Buchwalter, Learning representations by maximizing mutual information across views, Advances in neural information processing systems 32 (2019).

- [23] Y. Shou, W. Ai, T. Meng, Graph information bottleneck for remote sensing segmentation, arXiv preprint arXiv:2312.02545 (2023).
- [24] Y. Shou, W. Ai, T. Meng, K. Li, Czl-ciae: Clip-driven zero-shot learning for correcting inverse age estimation, arXiv preprint arXiv:2312.01758 (2023).
- [25] Z. Peng, W. Huang, M. Luo, Q. Zheng, Y. Rong, T. Xu, J. Huang, Graph representation learning via graphical mutual information maximization, in: Proceedings of The Web Conference 2020, 2020, pp. 259-270.
- [26] M. Tschannen, J. Djolonga, P. K. Rubenstein, S. Gelly, M. Lucic, On mutual information maximization for representation learning, in: International Conference on Learning Representations, 2019.
- [27] N. TISHBY, The information bottleneck method, in: Proc. of the 37th Allerton Conference on Communication and Computation, 1999, 1999.
- [28] Z. Wan, C. Zhang, P. Zhu, Q. Hu, Multi-view informationbottleneck representation learning, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, 2021, pp. 1008510092.
- [29] Y. Shou, W. Ai, J. Du, T. Meng, H. Liu, Efficient longdistance latent relation-aware graph neural network for multimodal emotion recognition in conversations, arXiv preprint arXiv:2407.00119 (2024).
- [30] Y. Shou, T. Meng, F. Zhang, N. Yin, K. Li, Revisiting multimodal emotion learning with broad state space models and probability-guidance fusion, arXiv preprint arXiv:2404.17858 (2024).
- [31] T. Meng, F. Zhang, Y. Shou, H. Shao, W. Ai, K. Li, Masked graph learning with recurrent alignment for multimodal emotion recognition in conversation, IEEE/ACM Transactions on Audio, Speech, and Language Processing (2024).
- [32] J. Yu, J. Cao, R. He, Improving subgraph recognition with variational graph information bottleneck, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 19396-19405.
- [33] M. Ou, P. Cui, J. Pei, Z. Zhang, W. Zhu, Asymmetric transitivity preserving graph embedding, in: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 1105-1114.
- [34] G. Ji, S. He, L. Xu, K. Liu, J. Zhao, Knowledge graph embedding via dynamic mapping matrix, in: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (volume 1: Long papers), 2015, pp. 687696.
- [35] F. Nie, W. Zhu, X. Li, Unsupervised large graph embedding, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31, 2017.
- [36] Z. Zhang, P. Cui, X. Wang, J. Pei, X. Yao, W. Zhu, Arbitraryorder proximity preserved network embedding, in: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery &amp; Data Mining, 2018, pp. 2778-2786.
- [37] B. Perozzi, R. Al-Rfou, S. Skiena, Deepwalk: Online learning of social representations, in: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2014, pp. 701-710.
- [38] W. Hamilton, Z. Ying, J. Leskovec, Inductive representation learning on large graphs, Advances in Neural Information Processing Systems 30 (2017).
- [39] T. Zhao, X. Zhang, S. Wang, Graphsmote: Imbalanced node classification on graphs with graph neural networks, in: Proceedings of the 14th ACM International Conference on Web Search and Data Mining, 2021, pp. 833-841.
- [40] Q. Wu, W. Zhao, Z. Li, D. P. Wipf, J. Yan, Nodeformer: A scalable graph structure learning transformer for node classification, Advances in Neural Information Processing Systems 35 (2022) 27387-27401.
- [41] T. Zhao, X. Zhang, S. Wang, Exploring edge disentanglement for node classification, in: Proceedings of the ACM Web Conference 2022, 2022, pp. 1028-1036.
- [42] B. Wang, A. Li, M. Pang, H. Li, Y. Chen, Graphfl: A federated learning framework for semi-supervised node classification on graphs, in: 2022 IEEE International Conference on Data Mining

(ICDM), IEEE, 2022, pp. 498-507.

- [43] Y. Shou, W. Ai, T. Meng, F. Zhang, K. Li, Graphunet: Graph make strong encoders for remote sensing segmentation, in: 2023 IEEE 29th International Conference on Parallel and Distributed Systems (ICPADS), IEEE, 2023, pp. 2734-2737.
- [44] T. Meng, F. Zhang, Y. Shou, W. Ai, N. Yin, K. Li, Revisiting multimodal emotion recognition in conversation from the perspective of graph spectrum, arXiv preprint arXiv:2404.17862 (2024).
- [45] X. Xu, H. Lu, J. Song, Y. Yang, H. T. Shen, X. Li, Ternary adversarial networks with self-supervision for zero-shot crossmodal retrieval, IEEE Transactions on Cybernetics 50 (6) (2019) 2400-2413.
- [46] Y. Tian, D. Krishnan, P. Isola, Contrastive multiview coding, in: Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XI 16, Springer, 2020, pp. 776-794.
- [47] R. D. Hjelm, A. Fedorov, S. Lavoie-Marchildon, K. Grewal, P. Bachman, A. Trischler, Y. Bengio, Learning deep representations by mutual information estimation and maximization, in: International Conference on Learning Representations.
- [48] C. Mavromatis, G. Karypis, Graph infoclust: Maximizing coarse-grain mutual information in graphs, in: Advances in Knowledge Discovery and Data Mining: 25th Pacific-Asia Conference, PAKDD 2021, Virtual Event, May 11-14, 2021, Proceedings, Part I, Springer, 2021, pp. 541-553.
- [49] Q. Tan, N. Liu, X. Huang, R. Chen, S.-H. Choi, X. Hu, Mgae: Masked autoencoders for self-supervised learning on graphs, arXiv preprint arXiv:2201.02534 (2022).
- [50] Z. Hou, X. Liu, Y. Cen, Y. Dong, H. Yang, C. Wang, J. Tang, Graphmae: Self-supervised masked graph autoencoders, in: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022, pp. 594-604.
- [51] J. Li, R. Wu, W. Sun, L. Chen, S. Tian, L. Zhu, C. Meng, Z. Zheng, W. Wang, Maskgae: masked graph modeling meets graph autoencoders, arXiv preprint arXiv:2205.10053 (2022).
- [52] M. Federici, A. Dutta, P. Forr´, N. Kushman, Z. Akata, Learne ing robust representations via multi-view information bottleneck, in: 8th International Conference on Learning Representations, OpenReview. net, 2020.
- [53] Q. Wang, C. Boudreau, Q. Luo, P.-N. Tan, J. Zhou, Deep multiview information bottleneck, in: Proceedings of the 2019 SIAM International Conference on Data Mining, SIAM, 2019, pp. 3745.
- [54] A. A. Alemi, I. Fischer, J. V. Dillon, K. Murphy, Deep variational information bottleneck, in: International Conference on Learning Representations, 2016.
- [55] L. Yang, L. Zhang, W. Yang, Graph adversarial self-supervised learning, Advances in Neural Information Processing Systems 34 (2021) 14887-14899.
- [56] E. Jang, S. Gu, B. Poole, Categorical reparameterization with gumbel-softmax, in: International Conference on Learning Representations, 2016.
- [57] A. v. d. Oord, Y. Li, O. Vinyals, Representation learning with contrastive predictive coding, arXiv preprint arXiv:1807.03748 (2018).
- [58] F.-Y. Sun, J. Hoffman, V. Verma, J. Tang, Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization, in: International Conference on Learning Representations, 2019.
- [59] M. Gutmann, A. Hyv¨ arinen, Noise-contrastive estimation: A new estimation principle for unnormalized statistical models, in: Proceedings of the thirteenth international conference on artificial intelligence and statistics, JMLR Workshop and Conference Proceedings, 2010, pp. 297-304.
- [60] A. Achille, S. Soatto, Emergence of invariance and disentanglement in deep representations, The Journal of Machine Learning Research 19 (1) (2018) 1947-1980.
- [61] O. Shchur, M. Mumme, A. Bojchevski, S. G¨ unnemann, Pitfalls of graph neural network evaluation, arXiv preprint arXiv:1811.05868 (2018).
- [62] Z. Yang, W. Cohen, R. Salakhudinov, Revisiting semisupervised learning with graph embeddings, in: International Conference on Machine Learning, PMLR, 2016, pp. 40-48.
- [63] W. Hu, M. Fey, M. Zitnik, Y. Dong, H. Ren, B. Liu, M. Catasta,

- J. Leskovec, Open graph benchmark: Datasets for machine learning on graphs, Advances in Neural Information Processing Systems 33 (2020) 22118-22133.
- [64] P. Mernyei, C. Cangea, Wiki-cs: A wikipedia-based benchmark for graph neural networks, arXiv preprint arXiv:2007.02901 (2020).
- [65] B. Rozemberczki, C. Allen, R. Sarkar, Multi-scale attributed node embedding, Journal of Complex Networks 9 (2) (2021) cnab014.
- [66] Y. Mo, L. Peng, J. Xu, X. Shi, X. Zhu, Simple unsupervised graph representation learning, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36, 2022, pp. 77977805.
- [67] T. N. Kipf, M. Welling, Semi-supervised classification with graph convolutional networks, in: International Conference on Learning Representations.
- [68] P. Veliˇkovi´, c c G. Cucurull, A. Casanova, A. Romero, P. Li`, o Y. Bengio, Graph attention networks, in: International Conference on Learning Representations.
- [69] T. N. Kipf, M. Welling, Variational graph auto-encoders, arXiv preprint arXiv:1611.07308 (2016).
- [70] Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, L. Wang, Deep graph contrastive representation learning, arXiv preprint arXiv:2006.04131 (2020).
- [71] K. Hassani, A. H. Khasahmadi, Contrastive multi-view representation learning on graphs, in: International Conference on Machine Learning, PMLR, 2020, pp. 4116-4126.
- [72] L. Peng, Y. Mo, J. Xu, J. Shen, X. Shi, X. Li, H. T. Shen, X. Zhu, Grlc: Graph representation learning with constraints, IEEE Transactions on Neural Networks and Learning Systems (2023) 1-14.
- [73] K. Huang, J. Tang, J. Liu, R. Yang, X. Xiao, Node-wise diffusion for scalable graph learning, in: Proceedings of the ACM Web Conference 2023, 2023, pp. 1723-1733.
- [74] K. Ma, H. Yang, H. Yang, T. Jin, P. Chen, Y. Chen, B. F. Kamhoua, J. Cheng, Improving graph representation learning by contrastive regularization, arXiv preprint arXiv:2101.11525 (2021) 20.
- [75] K. Xu, W. Hu, J. Leskovec, S. Jegelka, How powerful are graph neural networks?, in: International Conference on Learning Representations, 2018.
Yuntao Shou Haozhi Lan Xiangyong Cao
Contrastive Graph Representation Learning with Adversarial Cross-view Reconstruction and Information Bottleneck
Graph Neural Networks (GNNs) have received extensive research attention due
to their powerful information aggregation capabilities. Despite the success of
GNNs, most of them suffer from the popularity bias issue in a graph caused by a
small number of popular categories. Additionally, real graph datasets always
contain incorrect node labels, which hinders GNNs from learning effective node
representations. Graph contrastive learning (GCL) has been shown to be
effective in solving the above problems for node classification tasks. Most
existing GCL methods are implemented by randomly removing edges and nodes to
create multiple contrasting views, and then maximizing the mutual information
(MI) between these contrasting views to improve the node feature
representation. However, maximizing the mutual information between multiple
contrasting views may lead the model to learn some redundant information
irrelevant to the node classification task. To tackle this issue, we propose an
effective Contrastive Graph Representation Learning with Adversarial Cross-view
Reconstruction and Information Bottleneck (CGRL) for node classification, which
can adaptively learn to mask the nodes and edges in the graph to obtain the
optimal graph structure representation. Furthermore, we innovatively introduce
the information bottleneck theory into GCLs to remove redundant information in
multiple contrasting views while retaining as much information as possible
about node classification. Moreover, we add noise perturbations to the original
views and reconstruct the augmented views by constructing adversarial views to
improve the robustness of node feature representation. Extensive experiments on
real-world public datasets demonstrate that our method significantly
outperforms existing state-of-the-art algorithms.