Strong Asymptotic Assertions for Discrete MDL in Regression and Classification
<|reference_start|>Strong Asymptotic Assertions for Discrete MDL in Regression and Classification: We study the properties of the MDL (or maximum penalized complexity) estimator for Regression and Classification, where the underlying model class is countable. We show in particular a finite bound on the Hellinger losses under the only assumption that there is a "true" model contained in the class. This implies almost sure convergence of the predictive distribution to the true one at a fast rate. It corresponds to Solomonoff's central theorem of universal induction, however with a bound that is exponentially larger.<|reference_end|>
@article{poland2005strong,
    title={Strong Asymptotic Assertions for Discrete MDL in Regression and
  Classification},
    author={Jan Poland and Marcus Hutter},
    journal={Proc. 14th Dutch-Belgium Conf. on Machine Learning (Benelearn
  2005) 67-72},
    year={2005},
    number={IDSIA-02-05},
    archivePrefix={arXiv},
    eprint={math/0502315},
    primaryClass={math.ST cs.AI cs.IT cs.LG math.IT math.PR stat.TH}
}