The most effective and general methods for automatic translation are based on (Manning & Schütze, 1999). When translating statistical machine translation models a document or a web page, in contrast to a query, not only is ambiguity a problem, but the translated sentences should also be grammatically correct. Words can change order, disappear, or become multiple words when a sentence is translated. Statistical translation models represent each of these changes with a probability. This means that the model describes the probability that a word is translated into another word, the probability that words change order, and the probability that words disappear or become multiple words. These probabilities are used to calculate the most likely translation for a sentence Although a model that is based on word-to-word translation probabilities has some similarities to a dictionary-based approach, if the translation probabilities are accurate, they can make a large difference to the quality of the translation. Unusual translations for an ambiguous word can then be easily distinguished from more typical translations. More recent versions of these models, called phrase based further improve the use of context in translation models, the translation by calculating the probabilities of translating sequences of words, rather than just individual words. The probabilities in statistical machine translation models are estimated primarily by using parallel corpora. These are collections of documents in one language together with the translations into one or more other languages. The corpora are obtained primarily from government organizations (such as the Govt of India), news organizations, and by mining the Web, since there are hundreds of thousands of translated pages. The sentences in the parallel corpora are aligned either manually or automatically, which means that sentences are paired with their translations. The aligned sentences are then used for training the translation model. Special attention has to be paid to the translation of unusual especially proper nouns such as people’s names. For these words in particular, the Web is a rich resource. Automatic transliteration techniques are also used to address the problem of people’s names. Proper names are not usually translated into another
The most effective and general methods for automatic translation are based on statistical machine translation models (Manning & Schütze, 1999). When translating a document or a web page, in contrast to a query, not only is ambiguity a problem, but the translated sentences should also be grammatically correct. Words can change order, disappear, or become multiple words when a sentence is translated. Statistical translation models represent each of these changes with a probability. This means that the model describes the probability that a word is translated into another word, the probability that words change order, and the probability that words disappear or become multiple words. These probabilities are used to calculate the most likely translation for a sentence Although a model that is based on word-to-word translation probabilities has some similarities to a dictionary-based approach, if the translation probabilities are accurate, they can make a large difference to the quality of the translation. Unusual translations for an ambiguous word can then be easily distinguished from more typical translations. More recent versions of these models, called phrase based translation models, further improve the use of context in the translation by calculating the probabilities of translating sequences of words, rather than just individual words. The probabilities in statistical machine translation models are estimated primarily by using parallel corpora. These are collections of documents in one language together with the translations into one or more other languages. The corpora are obtained primarily from government organizations (such as the Govt of India), news organizations, and by mining the Web, since there are hundreds of thousands of translated pages. The sentences in the parallel corpora are aligned either manually or automatically, which means that sentences are paired with their translations. The aligned sentences are then used for training the translation model. Special attention has to be paid to the translation of unusual especially proper nouns such as people’s names. For these words in particular, the Web is a rich resource. Automatic transliteration techniques are also used to address the problem of people’s names. Proper names are not usually translated into another
unit-iv-web-search.docx