To Aggregate or Not to Aggregate That is the Question: A Case Study on Annotation Subjectivity in Span Prediction
<|reference_start|>To Aggregate or Not to Aggregate That is the Question: A Case Study on Annotation Subjectivity in Span Prediction: This paper explores the task of automatic prediction of text spans in a legal problem description that support a legal area label. We use a corpus of problem descriptions written by laypeople in English that is annotated by practising lawyers. Inherent subjectivity exists in our task because legal area categorisation is a complex task, and lawyers often have different views on a problem, especially in the face of legally-imprecise descriptions of issues. Experiments show that training on majority-voted spans outperforms training on disaggregated ones.<|reference_end|>
@article{kurniawan2024to,
    title={To Aggregate or Not to Aggregate. That is the Question: A Case Study on
  Annotation Subjectivity in Span Prediction},
    author={Kemal Kurniawan, Meladel Mistica, Timothy Baldwin, and Jey Han Lau},
    journal={arXiv preprint arXiv:2408.02257},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.02257},
    primaryClass={cs.CL}
}