MAVEN-Fact: A Large-scale Event Factuality Detection Dataset
<|reference_start|>MAVEN-Fact: A Large-scale Event Factuality Detection Dataset: Event Factuality Detection (EFD) task determines the factuality of textual events, i.e., classifying whether an event is a fact, possibility, or impossibility, which is essential for faithfully understanding and utilizing event knowledge. However, due to the lack of high-quality large-scale data, event factuality detection is under-explored in event understanding research, which limits the development of EFD community. To address these issues and provide faithful event understanding, we introduce MAVEN-Fact, a large-scale and high-quality EFD dataset based on the MAVEN dataset. MAVEN-Fact includes factuality annotations of 112,276 events, making it the largest EFD dataset. Extensive experiments demonstrate that MAVEN-Fact is challenging for both conventional fine-tuned models and large language models (LLMs). Thanks to the comprehensive annotations of event arguments and relations in MAVEN, MAVEN-Fact also supports some further analyses and we find that adopting event arguments and relations helps in event factuality detection for fine-tuned models but does not benefit LLMs. Furthermore, we preliminarily study an application case of event factuality detection and find it helps in mitigating event-related hallucination in LLMs. Our dataset and codes can be obtained from \url{https://github.com/lcy2723/MAVEN-FACT}<|reference_end|>
@article{li2024maven-fact:,
    title={MAVEN-Fact: A Large-scale Event Factuality Detection Dataset},
    author={Chunyang Li, Hao Peng, Xiaozhi Wang, Yunjia Qi, Lei Hou, Bin Xu,
  Juanzi Li},
    journal={arXiv preprint arXiv:2407.15352},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.15352},
    primaryClass={cs.CL}
}