Batch Universal Prediction
<|reference_start|>Batch Universal Prediction: Large language models (LLMs) have recently gained much popularity due to their surprising ability at generating human-like English sentences. LLMs are essentially predictors, estimating the probability of a sequence of words given the past. Therefore, it is natural to evaluate their performance from a universal prediction perspective. In order to do that fairly, we introduce the notion of batch regret as a modification of the classical average regret, and we study its asymptotical value for add-constant predictors, in the case of memoryless sources and first-order Markov sources.<|reference_end|>
@article{bondaschi2024batch,
    title={Batch Universal Prediction},
    author={Marco Bondaschi, Michael Gastpar},
    journal={arXiv preprint arXiv:2402.03901},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.03901},
    primaryClass={cs.IT cs.LG math.IT stat.ML}
}