Enhancing Graph Neural Networks with Limited Labeled Data by Actively Distilling Knowledge from Large Language Models
<|reference_start|>Enhancing Graph Neural Networks with Limited Labeled Data by Actively Distilling Knowledge from Large Language Models: Graphs are pervasive in the real-world, such as social network analysis, bioinformatics, and knowledge graphs. Graph neural networks (GNNs) have great ability in node classification, a fundamental task on graphs. Unfortunately, conventional GNNs still face challenges in scenarios with few labeled nodes, despite the prevalence of few-shot node classification tasks in real-world applications. To address this challenge, various approaches have been proposed, including graph meta-learning, transfer learning, and methods based on Large Language Models (LLMs). However, traditional meta-learning and transfer learning methods often require prior knowledge from base classes or fail to exploit the potential advantages of unlabeled nodes. Meanwhile, LLM-based methods may overlook the zero-shot capabilities of LLMs and rely heavily on the quality of generated contexts. In this paper, we propose a novel approach that integrates LLMs and GNNs, leveraging the zero-shot inference and reasoning capabilities of LLMs and employing a Graph-LLM-based active learning paradigm to enhance GNNs' performance. Extensive experiments demonstrate the effectiveness of our model in improving node classification accuracy with considerably limited labeled data, surpassing state-of-the-art baselines by significant margins.<|reference_end|>
@article{li2024enhancing,
    title={Enhancing Graph Neural Networks with Limited Labeled Data by Actively
  Distilling Knowledge from Large Language Models},
    author={Quan Li, Tianxiang Zhao, Lingwei Chen, Junjie Xu, Suhang Wang},
    journal={arXiv preprint arXiv:2407.13989},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.13989},
    primaryClass={cs.LG cs.AI}
}