Answers.Body: <p>You are basically describing the way <a href="https://en.wikipedia.org/wiki/Google_Translate#Method_of_translation" rel="nofollow noreferrer">Google Translate</a> works.</p>
  
  <p>There has been a lot of research in text alignment in the area of multi-lingual corpus linguistics. An early paper (with sourcode) is Gale and Church's <em>A Program for Aligning Sentences in Bilingual Corpora</em> (<a href="https://s3.amazonaws.com/tm-town-nlp-resources/Gale-Church+alignment.pdf" rel="nofollow noreferrer">PDF</a>). </p>
  
  <p>In linguistics these are called <a href="https://en.wikipedia.org/wiki/Parallel_text" rel="nofollow noreferrer">parallel texts</a>. On the wikipedia page you will find links to a number of alignment programs which attempt to solve that issue.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-10T22:51:58.753Z
Answers.Text: Awesome. Although your answer doesn't spell out a way to implement this, the terms you gave are the nudge I needed to continue my research.
Answers.UserDisplayName: Evgeniy Berezovsky
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-11T09:16:04.217Z
Answers.Text: There is C source code in the Gale and Church PDF.
Answers.UserDisplayName: Oliver Mason
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-10T16:08:33.820Z
Answers.LastActivityDate: 2019-03-10T16:08:33.820Z
Answers.UserDisplayName: Oliver Mason
Body: <p>There are many, many literary works in the public domain, along with human translations, many of which have entered the public domain as well. (Public domain = easily available)</p>
  
  <p>In order for me to advance my knowledge of e.g. Japanese, I'd like to read texts in Japanese using a yet to be written tool, and when I encounter an unknown word/phrase/sentence, I'd like to just click on a position in the text and be transferred to the corresponding position in the e.g. English translation (or original) of the text in question.</p>
  
  <p>Let's also assume we have access to a dictionary that translates a fair amount of words between those two languages (and there are of course free dictionaries for many language pairs).</p>
  
  <p><strong>What ways are there to use AI toolkits plus some wiring and perhaps scripting/programming, to auto-correlate the positions of two versions of the same text, in two languages?</strong> The results do not - and in fact in many cases cannot - be perfect, but they should be roughly correct.</p>
  
  <p>I'm aware that this is still not a straightforwards task, as there are complicating factors like inflection of verbs and other grammatical properties that make the use of dictionary tools much harder. Also, translators will often translate to words that don't have that mapping in any dictionary. Then there is the fact that words aren't delimited by spaces in languages like my example language Japanese - (but if it is easier to work with only space-separated languages like, say, Spanish, or Russian, I'd like to hear answers to this simpler problem as well). Also the order of words and even hole sub-clauses differs from language to language.</p>
  
  <p>A simple, non-AI approximation would be to</p>
  
  <ol>
  <li>figure out at what relative position in the source language text the user clicked (e.g. at the character on position 50.234%)</li>
  <li>then go to that same relative position 50.234% in the target language text</li>
  </ol>
  
  <p>This approximation could perhaps be used as the starting point for the AI, which would then use words and dictionaries to make the results more accurate.</p>
  
Tags: <algorithm><learning-algorithms>
Title: Use AI to auto-correlate the words of human-translated texts?
UserDisplayName: Evgeniy Berezovsky
ThreadText: # Use AI to auto-correlate the words of human-translated texts?
  
  (Asked by: username_0 on 2019-03-09)
  
  There are many, many literary works in the public domain, along with human translations, many of which have entered the public domain as well. (Public domain = easily available)
  
  In order for me to advance my knowledge of e.g. Japanese, I'd like to read texts in Japanese using a yet to be written tool, and when I encounter an unknown word/phrase/sentence, I'd like to just click on a position in the text and be transferred to the corresponding position in the e.g. English translation (or original) of the text in question.
  
  Let's also assume we have access to a dictionary that translates a fair amount of words between those two languages (and there are of course free dictionaries for many language pairs).
  
  **What ways are there to use AI toolkits plus some wiring and perhaps scripting/programming, to auto-correlate the positions of two versions of the same text, in two languages?** The results do not - and in fact in many cases cannot - be perfect, but they should be roughly correct.
  
  I'm aware that this is still not a straightforwards task, as there are complicating factors like inflection of verbs and other grammatical properties that make the use of dictionary tools much harder. Also, translators will often translate to words that don't have that mapping in any dictionary. Then there is the fact that words aren't delimited by spaces in languages like my example language Japanese - (but if it is easier to work with only space-separated languages like, say, Spanish, or Russian, I'd like to hear answers to this simpler problem as well). Also the order of words and even hole sub-clauses differs from language to language.
  
  A simple, non-AI approximation would be to
  
  1. figure out at what relative position in the source language text the user clicked (e.g. at the character on position 50.234%)
  2. then go to that same relative position 50.234% in the target language text
  
  This approximation could perhaps be used as the starting point for the AI, which would then use words and dictionaries to make the results more accurate.
  
  **Tags:** algorithm, learning-algorithms
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-03-10. Score: 1
  
  You are basically describing the way [Google Translate](https://en.wikipedia.org/wiki/Google_Translate#Method_of_translation) works.
  
  There has been a lot of research in text alignment in the area of multi-lingual corpus linguistics. An early paper (with sourcode) is Gale and Church's *A Program for Aligning Sentences in Bilingual Corpora* ([PDF](https://s3.amazonaws.com/tm-town-nlp-resources/Gale-Church+alignment.pdf)).
  
  In linguistics these are called [parallel texts](https://en.wikipedia.org/wiki/Parallel_text). On the wikipedia page you will find links to a number of alignment programs which attempt to solve that issue.
  
  > **Comments:**
  > **username_0**: Awesome. Although your answer doesn't spell out a way to implement this, the terms you gave are the nudge I needed to continue my research.  
  > **username_1**: There is C source code in the Gale and Church PDF.  
  
  ---
  
  