Text-Free Multi-domain Graph Pre-training: Toward Graph Foundation Models
<|reference_start|>Text-Free Multi-domain Graph Pre-training: Toward Graph Foundation Models: Given the ubiquity of graph data, it is intriguing to ask: Is it possible to train a graph foundation model on a broad range of graph data across diverse domains? A major hurdle toward this goal lies in the fact that graphs from different domains often exhibit profoundly divergent characteristics. Although there have been some initial efforts in integrating multi-domain graphs for pre-training, they primarily rely on textual descriptions to align the graphs, limiting their application to text-attributed graphs. Moreover, different source domains may conflict or interfere with each other, and their relevance to the target domain can vary significantly. To address these issues, we propose MDGPT, a text free Multi-Domain Graph Pre-Training and adaptation framework designed to exploit multi-domain knowledge for graph learning. First, we propose a set of domain tokens to to align features across source domains for synergistic pre-training. Second, we propose a dual prompts, consisting of a unifying prompt and a mixing prompt, to further adapt the target domain with unified multi-domain knowledge and a tailored mixture of domain-specific knowledge. Finally, we conduct extensive experiments involving six public datasets to evaluate and analyze MDGPT, which outperforms prior art by up to 37.9%.<|reference_end|>
@article{yu2024text-free,
    title={Text-Free Multi-domain Graph Pre-training: Toward Graph Foundation
  Models},
    author={Xingtong Yu, Chang Zhou, Yuan Fang, Xinming Zhang},
    journal={arXiv preprint arXiv:2405.13934},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.13934},
    primaryClass={cs.LG}
}