Metric-aware LLM inference for regression and scoring
<|reference_start|>Metric-aware LLM inference for regression and scoring: Large language models (LLMs) have demonstrated strong results on a range of NLP tasks. Typically, outputs are obtained via autoregressive sampling from the LLM's underlying distribution. Building on prior work on Minimum Bayes Risk Decoding, we show that this inference strategy can be suboptimal for a range of regression and scoring tasks, and associated evaluation metrics. As a remedy, we propose metric aware LLM inference: a decision theoretic approach optimizing for custom regression and scoring metrics at inference time. We report improvements over baselines on academic benchmarks and publicly available models.<|reference_end|>
@article{lukasik2024regression-aware,
    title={Regression-aware Inference with LLMs},
    author={Michal Lukasik, Harikrishna Narasimhan, Aditya Krishna Menon, Felix
  Yu, Sanjiv Kumar},
    journal={EMNLP Findings 2024},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.04182},
    primaryClass={cs.CL cs.AI}
}