Aggregate and mixed-order Markov models for statistical language processing
<|reference_start|>Aggregate and mixed-order Markov models for statistical language processing: We consider the use of language models whose size and accuracy are intermediate between different order n-gram models. Two types of models are studied in particular. Aggregate Markov models are class-based bigram models in which the mapping from words to classes is probabilistic. Mixed-order Markov models combine bigram models whose predictions are conditioned on different words. Both types of models are trained by Expectation-Maximization (EM) algorithms for maximum likelihood estimation. We examine smoothing procedures in which these models are interposed between different order n-grams. This is found to significantly reduce the perplexity of unseen word combinations.<|reference_end|>
@article{saul1997aggregate,
    title={Aggregate and mixed-order Markov models for statistical language
  processing},
    author={Lawrence Saul and Fernando Pereira (AT&T Labs -- Research)},
    journal={arXiv preprint arXiv:cmp-lg/9706007},
    year={1997},
    archivePrefix={arXiv},
    eprint={cmp-lg/9706007},
    primaryClass={cmp-lg cs.CL}
}