Using the Distribution of Performance for Studying Statistical NLP Systems and Corpora
<|reference_start|>Using the Distribution of Performance for Studying Statistical NLP Systems and Corpora: Statistical NLP systems are frequently evaluated and compared on the basis of their performances on a single split of training and test data. Results obtained using a single split are, however, subject to sampling noise. In this paper we argue in favour of reporting a distribution of performance figures, obtained by resampling the training data, rather than a single number. The additional information from distributions can be used to make statistically quantified statements about differences across parameter settings, systems, and corpora.<|reference_end|>
@article{krymolowski2001using,
    title={Using the Distribution of Performance for Studying Statistical NLP
  Systems and Corpora},
    author={Yuval Krymolowski},
    journal={arXiv preprint arXiv:cs/0106043},
    year={2001},
    archivePrefix={arXiv},
    eprint={cs/0106043},
    primaryClass={cs.CL}
}