Inducing Probabilistic Grammars by Bayesian Model Merging
<|reference_start|>Inducing Probabilistic Grammars by Bayesian Model Merging: We describe a framework for inducing probabilistic grammars from corpora of positive samples. First, samples are {\em incorporated} by adding ad-hoc rules to a working grammar; subsequently, elements of the model (such as states or nonterminals) are {\em merged} to achieve generalization and a more compact representation. The choice of what to merge and when to stop is governed by the Bayesian posterior probability of the grammar given the data, which formalizes a trade-off between a close fit to the data and a default preference for simpler models (`Occam's Razor'). The general scheme is illustrated using three types of probabilistic grammars: Hidden Markov models, class-based $n$-grams, and stochastic context-free grammars.<|reference_end|>
@article{stolcke1994inducing,
    title={Inducing Probabilistic Grammars by Bayesian Model Merging},
    author={Andreas Stolcke (ICSI, Berkeley, CA), Stephen M. Omohundro (ICSI,
  Berkeley, CA)},
    journal={Grammatical Inference and Applications. ICGI 1994, pp. 106-118,
  Sept. 1994},
    year={1994},
    doi={10.1007/3-540-58473-0_141},
    archivePrefix={arXiv},
    eprint={cmp-lg/9409010},
    primaryClass={cmp-lg cs.CL}
}