ANHALTEN: Cross-Lingual Transfer for German Token-Level Reference-Free Hallucination Detection
<|reference_start|>ANHALTEN: Cross-Lingual Transfer for German Token-Level Reference-Free Hallucination Detection: Research on token-level reference-free hallucination detection has predominantly focused on English, primarily due to the scarcity of robust datasets in other languages. This has hindered systematic investigations into the effectiveness of cross-lingual transfer for this important NLP application. To address this gap, we introduce ANHALTEN, a new evaluation dataset that extends the English hallucination detection dataset to German. To the best of our knowledge, this is the first work that explores cross-lingual transfer for token-level reference-free hallucination detection. ANHALTEN contains gold annotations in German that are parallel (i.e., directly comparable to the original English instances). We benchmark several prominent cross-lingual transfer approaches, demonstrating that larger context length leads to better hallucination detection in German, even without succeeding context. Importantly, we show that the sample-efficient few-shot transfer is the most effective approach in most setups. This highlights the practical benefits of minimal annotation effort in the target language for reference-free hallucination detection. Aiming to catalyze future research on cross-lingual token-level reference-free hallucination detection, we make ANHALTEN publicly available: https://github.com/janekh24/anhalten<|reference_end|>
@article{herrlein2024anhalten:,
    title={ANHALTEN: Cross-Lingual Transfer for German Token-Level Reference-Free
  Hallucination Detection},
    author={Janek Herrlein, Chia-Chien Hung, Goran Glavav{s}},
    journal={arXiv preprint arXiv:2407.13702},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.13702},
    primaryClass={cs.CL}
}