The invention comprises a core algorithm to use language regularity in large collections of human created textual documents, as well as optimization techniques to make the algorithm tractable. The core algorithm includes receiving tuples of text units that may be grammatically linked and processing a stream of such tuples to discover language regularities. After this learning is completed, the algorithm's output is used to evaluate the perceived likelihood that different interpretations of novel sentences would have been intended by a speaker of the language.
Method for unsupervised learning of grammatical parsers