Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic
<|reference_start|>Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic: Recent language models enable new opportunities for structured reasoning with text, such as the construction of intuitive, proof-like textual entailment trees without relying on brittle formal logic. However, progress in this direction has been hampered by a long-standing lack of a clear protocol for determining what valid compositional entailment is. This absence causes noisy datasets and limited performance gains by modern neuro-symbolic engines. To address these problems, we formulate a consistent and theoretically grounded approach to annotating decompositional entailment and evaluate its impact on LLM-based textual inference. We find that our new dataset, RDTE (Recognizing Decompositional Textual Entailment), has a substantially higher internal consistency (+9%) than prior decompositional entailment datasets. We also find that training an RDTE-oriented entailment classifier via knowledge distillation and employing it in an entailment tree reasoning engine significantly improves both accuracy and proof quality, illustrating the practical benefit of this advance for textual inference.<|reference_end|>
@article{weir2024enhancing,
    title={Enhancing Systematic Decompositional Natural Language Inference Using
  Informal Logic},
    author={Nathaniel Weir, Kate Sanders, Orion Weller, Shreya Sharma, Dongwei
  Jiang, Zhengping Jiang, Bhavana Dalvi Mishra, Oyvind Tafjord, Peter Jansen,
  Peter Clark, Benjamin Van Durme},
    journal={arXiv preprint arXiv:2402.14798},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.14798},
    primaryClass={cs.CL cs.AI}
}