Online GNN Evaluation Under Test-time Graph Distribution Shifts
<|reference_start|>Online GNN Evaluation Under Test-time Graph Distribution Shifts: Evaluating the performance of a well-trained GNN model on real-world graphs is a pivotal step for reliable GNN online deployment and serving. Due to a lack of test node labels and unknown potential training-test graph data distribution shifts, conventional model evaluation encounters limitations in calculating performance metrics (e.g., test error) and measuring graph data-level discrepancies, particularly when the training graph used for developing GNNs remains unobserved during test time. In this paper, we study a new research problem, online GNN evaluation, which aims to provide valuable insights into the well-trained GNNs's ability to effectively generalize to real-world unlabeled graphs under the test-time graph distribution shifts. Concretely, we develop an effective learning behavior discrepancy score, dubbed LeBeD, to estimate the test-time generalization errors of well-trained GNN models. Through a novel GNN re-training strategy with a parameter-free optimality criterion, the proposed LeBeD comprehensively integrates learning behavior discrepancies from both node prediction and structure reconstruction perspectives. This enables the effective evaluation of the well-trained GNNs' ability to capture test node semantics and structural representations, making it an expressive metric for estimating the generalization error in online GNN evaluation. Extensive experiments on real-world test graphs under diverse graph distribution shifts could verify the effectiveness of the proposed method, revealing its strong correlation with ground-truth test errors on various well-trained GNN models.<|reference_end|>
@article{zheng2024online,
    title={Online GNN Evaluation Under Test-time Graph Distribution Shifts},
    author={Xin Zheng, Dongjin Song, Qingsong Wen, Bo Du, Shirui Pan},
    journal={arXiv preprint arXiv:2403.09953},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.09953},
    primaryClass={cs.LG}
}