FRACTAL: Fine-Grained Scoring from Aggregate Text Labels
<|reference_start|>FRACTAL: Fine-Grained Scoring from Aggregate Text Labels: Large language models (LLMs) are being increasingly tuned to power complex generation tasks such as writing, fact-seeking, querying and reasoning. Traditionally, human or model feedback for evaluating and further tuning LLM performance has been provided at the response level, enabling faster and more cost-effective assessments. However, recent works (Amplayo et al. [2022], Wu et al. [2023]) indicate that sentence-level labels may provide more accurate and interpretable feedback for LLM optimization. In this work, we introduce methods to disaggregate response-level labels into sentence-level (pseudo-)labels. Our approach leverages multiple instance learning (MIL) and learning from label proportions (LLP) techniques in conjunction with prior information (e.g., document-sentence cosine similarity) to train a specialized model for sentence-level scoring. We also employ techniques which use model predictions to pseudo-label the train-set at the sentence-level for model training to further improve performance.   We conduct extensive evaluations of our methods across six datasets and four tasks: retrieval, question answering, summarization, and math reasoning. Our results demonstrate improved performance compared to multiple baselines across most of these tasks. Our work is the first to develop response-level feedback to sentence-level scoring techniques, leveraging sentence-level prior information, along with comprehensive evaluations on multiple tasks as well as end-to-end finetuning evaluation showing performance comparable to a model trained on fine-grained human annotated labels.<|reference_end|>
@article{makhija2024fractal:,
    title={FRACTAL: Fine-Grained Scoring from Aggregate Text Labels},
    author={Yukti Makhija and Priyanka Agrawal and Rishi Saket and Aravindan
  Raghuveer},
    journal={arXiv preprint arXiv:2404.04817},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.04817},
    primaryClass={cs.CL}
}