Asymptotic Log-loss of Prequential Maximum Likelihood Codes
<|reference_start|>Asymptotic Log-loss of Prequential Maximum Likelihood Codes: We analyze the Dawid-Rissanen prequential maximum likelihood codes relative to one-parameter exponential family models M. If data are i.i.d. according to an (essentially) arbitrary P, then the redundancy grows at rate c/2 ln n. We show that c=v1/v2, where v1 is the variance of P, and v2 is the variance of the distribution m* in M that is closest to P in KL divergence. This shows that prequential codes behave quite differently from other important universal codes such as the 2-part MDL, Shtarkov and Bayes codes, for which c=1. This behavior is undesirable in an MDL model selection setting.<|reference_end|>
@article{grunwald2005asymptotic,
    title={Asymptotic Log-loss of Prequential Maximum Likelihood Codes},
    author={Peter Grunwald and Steven de Rooij},
    journal={arXiv preprint arXiv:cs/0502004},
    year={2005},
    archivePrefix={arXiv},
    eprint={cs/0502004},
    primaryClass={cs.LG cs.IT math.IT}
}