Pre-Training and Prompting for Few-Shot Node Classification on Text-Attributed Graphs
<|reference_start|>Pre-Training and Prompting for Few-Shot Node Classification on Text-Attributed Graphs: The text-attributed graph (TAG) is one kind of important real-world graph-structured data with each node associated with raw texts. For TAGs, traditional few-shot node classification methods directly conduct training on the pre-processed node features and do not consider the raw texts. The performance is highly dependent on the choice of the feature pre-processing method. In this paper, we propose P2TAG, a framework designed for few-shot node classification on TAGs with graph pre-training and prompting. P2TAG first pre-trains the language model (LM) and graph neural network (GNN) on TAGs with self-supervised loss. To fully utilize the ability of language models, we adapt the masked language modeling objective for our framework. The pre-trained model is then used for the few-shot node classification with a mixed prompt method, which simultaneously considers both text and graph information. We conduct experiments on six real-world TAGs, including paper citation networks and product co-purchasing networks. Experimental results demonstrate that our proposed framework outperforms existing graph few-shot learning methods on these datasets with +18.98% ~ +35.98% improvements.<|reference_end|>
@article{zhao2024pre-training,
    title={Pre-Training and Prompting for Few-Shot Node Classification on
  Text-Attributed Graphs},
    author={Huanjing Zhao, Beining Yang, Yukuo Cen, Junyu Ren, Chenhui Zhang,
  Yuxiao Dong, Evgeny Kharlamov, Shu Zhao, Jie Tang},
    journal={arXiv preprint arXiv:2407.15431},
    year={2024},
    doi={10.1145/3637528.3671952},
    archivePrefix={arXiv},
    eprint={2407.15431},
    primaryClass={cs.SI cs.AI cs.LG}
}