The Power of Noise: Toward a Unified Multi-modal Knowledge Graph Representation Framework
<|reference_start|>The Power of Noise: Toward a Unified Multi-modal Knowledge Graph Representation Framework: The advancement of Multi-modal Pre-training highlights the necessity for a robust Multi-Modal Knowledge Graph (MMKG) representation learning framework. This framework is crucial for integrating structured knowledge into multi-modal Large Language Models (LLMs) at scale, aiming to alleviate issues like knowledge misconceptions and multi-modal hallucinations. In this work, to evaluate models' ability to accurately embed entities within MMKGs, we focus on two widely researched tasks: Multi-modal Knowledge Graph Completion (MKGC) and Multi-modal Entity Alignment (MMEA). Building on this foundation, we propose a novel SNAG method that utilizes a Transformer-based architecture equipped with modality-level noise masking for the robust integration of multi-modal entity features in KGs. By incorporating specific training objectives for both MKGC and MMEA, our approach achieves SOTA performance across a total of ten datasets (three for MKGC and seven for MEMA), demonstrating its robustness and versatility. Besides, SNAG can not only function as a standalone model but also enhance other existing methods, providing stable performance improvements. Our code and data are available at: https://github.com/zjukg/SNAG.<|reference_end|>
@article{chen2024the,
    title={The Power of Noise: Toward a Unified Multi-modal Knowledge Graph
  Representation Framework},
    author={Zhuo Chen, Yin Fang, Yichi Zhang, Lingbing Guo, Jiaoyan Chen, Huajun
  Chen, Wen Zhang},
    journal={arXiv preprint arXiv:2403.06832},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.06832},
    primaryClass={cs.CL cs.AI}
}