Answers.Body: <p>For the first answer, the general way is using the seq2seq model:</p>
  
  <p><a href="https://www.tensorflow.org/tutorials/seq2seq" rel="nofollow noreferrer">https://www.tensorflow.org/tutorials/seq2seq</a></p>
  
  <p>(a more specific example of the above category is this paper:</p>
  
  <p><a href="http://emnlp2014.org/papers/pdf/EMNLP2014179.pdf" rel="nofollow noreferrer">http://emnlp2014.org/papers/pdf/EMNLP2014179.pdf</a>)</p>
  
  <p>For the 2nd answer, one way is to use word2vec and apply supervised learning on the groups of similar words:</p>
  
  <p><a href="https://towardsdatascience.com/word-to-vectors-natural-language-processing-b253dd0b0817" rel="nofollow noreferrer">https://towardsdatascience.com/word-to-vectors-natural-language-processing-b253dd0b0817</a>   </p>
  
  <p><a href="https://spacy.io/usage/vectors-similarity" rel="nofollow noreferrer">https://spacy.io/usage/vectors-similarity</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-11T02:47:14.710Z
Answers.LastActivityDate: 2018-02-11T02:47:14.710Z
Answers.UserDisplayName: Peter Teoh
Answers.Body: <p>Let us simplify case 1 of the question: assume two files, first one with numbers written in indo-arabic numeral system (i.e. 123, 9, 186754, ...), second one in roman numeral system (XX, LCVI, IX, ...). How do you match pairs of symbols with same meaning ?</p>
  
  <p>Without external information or assumptions, you can not. You could made the hypothesis that probability of one specific number is the same in both examples, and base your pairing on it. But then you need to find two input files that fulfills this condition.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-10T11:03:40.143Z
Answers.Text: It is possible to get this information. Because same words in both languages are used in sentences that have same structure. For example lion and Africa are often used in same sentence, or stars and universe and telescope. But on the other hand stars and walking are less related, you fly to stars not walk, and aeroplane also flies not walks, and people fly on aeroplanes. So there you get these specific relationship between these words that are true in both languages and specific structure of relationships is true only for these specific words. Thats how you can know they must mean the same.
Answers.UserDisplayName: Tone Škoda
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-10T11:05:32.057Z
Answers.Text: How to compare these structures is not yet entirely clear to me. It is complicated. But you have to use analogies, statistics and logic. Neural networks only use analogies or simple similarity but ignore logic, that is their problem I think.
Answers.UserDisplayName: Tone Škoda
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-10T11:24:16.850Z
Answers.Text: About the examples you provide: in the pure case, you can know from text1 that there are a statistic relation between "lion" and "Africa", from text2 a relation between "estrellas" and "universo". Is "estrellas" synonym of "lyon" ? You can improve the algorithm starting by some well known fixed matches (the external information I refer in m answer),  i.e 5=V, and, from this one and using statistics, infers new matches like 6=VI.  However, in natural language, all these approach will have a lot of troubles. Nice as theoretical exercise, but easier to use an existing dictionary.
Answers.UserDisplayName: pasaba por aqui
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-10T11:32:21.663Z
Answers.Text: Well the ultimate goals is to write a program that will be able to learn itself new languages just from reading large amounts of texts. That this is possible we know from human babies, who are capable of this.
Answers.UserDisplayName: Tone Škoda
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-10T11:37:14.760Z
Answers.Text: Remember the case of Egyptian scripts: no translation until rosetta stone (external information) was found.
Answers.UserDisplayName: pasaba por aqui
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-10T12:27:24.957Z
Answers.Text: So you are saying this is very hard program to write? Good to know so I wont feel bad when I fail. I will read about Egyptian scripts, sounds interesting. Maybe my program will be able to decrypt them, it should
Answers.UserDisplayName: Tone Škoda
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-10T12:45:39.583Z
Answers.Text: You have my best wishes in your target. Yes, when you say "I'm planning on writing these two programs", be prepared to say "I'm planning to research on this issue and implement some test programs". You will be famous if you decrypt Voynich manuscript.
Answers.UserDisplayName: pasaba por aqui
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-10T10:04:31.650Z
Answers.LastActivityDate: 2018-02-10T10:04:31.650Z
Answers.UserDisplayName: pasaba por aqui
Body: <p>I'm wondering if these 2 specific programs already exist and if not how hard would it be to write them:</p>
  
  <ol>
  <li><p>A program that would figure out (by only "reading" large amounts of texts in human language 1 and 2) which words in second language have the same meaning as a word in first language.  You would give for input texts in both languages and for output you would get for every word in first language a list of words in second language that are most similar to it with a probability that they mean the same thing. </p></li>
  <li><p>A program that would figure out which words have the most similar meaning by analyzing large amounts of texts in one human language.</p></li>
  </ol>
  
  <p>I'm planning on writing these two programs and it would be nice if I could get existing programs that do this so that I could compare results of my program to those of existing programs.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-02-10T16:11:45.560Z
Comments.Text: A point of consideration: words are only one of many forms of structure in a language. That is to say, different languages will construct the context around synonymous words in slightly different ways. Any good model outputting a translation of sorts will almost certainly have to account for this heterogenous word context to some degree. For a great book that covers some interesting ideas around differences in mechanics between language, check out Through The Language Glass by Guy Deutscher.
Comments.UserDisplayName: Greenstick
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-02-10T17:09:49.150Z
Comments.Text: Can you give an example of what you mean? Yes for example order of words can be different in different languages. But the way one word is related in meaning to other words is same in all languages, because languages live in the same worlds where logic is the same.
Comments.UserDisplayName: Tone Škoda
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-02-11T02:36:57.287Z
Comments.Text: That’s a peerless sentence right there! @Bobs Yes, the short of it is that the variation in grammatical constructs and the structure of relationships between words (among other things) need to be understood by your program (this is not covered in the book I mentioned). Given the complexity of what your asking, this program almost certainly has at least one statistical learning model at its core. Also, thinking of synonyms we can see how context is critical: a given synonym in language A undoubtedly does not share the same set of synonyms for its synonym in language B.
Comments.UserDisplayName: Greenstick
Tags: <machine-learning><natural-language-processing>
Title: Existing programs that find out words with same meanings
UserDisplayName: Tone Škoda
ThreadText: # Existing programs that find out words with same meanings
  
  (Asked by: username_0 on 2018-02-10)
  
  I'm wondering if these 2 specific programs already exist and if not how hard would it be to write them:
  
  1. A program that would figure out (by only "reading" large amounts of texts in human language 1 and 2) which words in second language have the same meaning as a word in first language. You would give for input texts in both languages and for output you would get for every word in first language a list of words in second language that are most similar to it with a probability that they mean the same thing.
  2. A program that would figure out which words have the most similar meaning by analyzing large amounts of texts in one human language.
  
  I'm planning on writing these two programs and it would be nice if I could get existing programs that do this so that I could compare results of my program to those of existing programs.
  
  **Tags:** machine-learning, natural-language-processing
  > **Comments:**
  > **username_1**: A point of consideration: words are only one of many forms of structure in a language. That is to say, different languages will construct the context around synonymous words in slightly different ways. Any good model outputting a translation of sorts will almost certainly have to account for this heterogenous word context to some degree. For a great book that covers some interesting ideas around differences in mechanics between language, check out Through The Language Glass by Guy Deutscher.  
  > **username_0**: Can you give an example of what you mean? Yes for example order of words can be different in different languages. But the way one word is related in meaning to other words is same in all languages, because languages live in the same worlds where logic is the same.  
  > **username_1**: That’s a peerless sentence right there! @Bobs Yes, the short of it is that the variation in grammatical constructs and the structure of relationships between words (among other things) need to be understood by your program (this is not covered in the book I mentioned). Given the complexity of what your asking, this program almost certainly has at least one statistical learning model at its core. Also, thinking of synonyms we can see how context is critical: a given synonym in language A undoubtedly does not share the same set of synonyms for its synonym in language B.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2018-02-11. Score: 1 | Accepted answer
  
  For the first answer, the general way is using the seq2seq model:
  
  <https://www.tensorflow.org/tutorials/seq2seq>
  
  (a more specific example of the above category is this paper:
  
  <http://emnlp2014.org/papers/pdf/EMNLP2014179.pdf>)
  
  For the 2nd answer, one way is to use word2vec and apply supervised learning on the groups of similar words:
  
  <https://towardsdatascience.com/word-to-vectors-natural-language-processing-b253dd0b0817>
  
  <https://spacy.io/usage/vectors-similarity>
  
  
  ---
  
  ### Answer by username_3 on 2018-02-10. Score: 1
  
  Let us simplify case 1 of the question: assume two files, first one with numbers written in indo-arabic numeral system (i.e. 123, 9, 186754, ...), second one in roman numeral system (XX, LCVI, IX, ...). How do you match pairs of symbols with same meaning ?
  
  Without external information or assumptions, you can not. You could made the hypothesis that probability of one specific number is the same in both examples, and base your pairing on it. But then you need to find two input files that fulfills this condition.
  
  > **Comments:**
  > **username_0**: It is possible to get this information. Because same words in both languages are used in sentences that have same structure. For example lion and Africa are often used in same sentence, or stars and universe and telescope. But on the other hand stars and walking are less related, you fly to stars not walk, and aeroplane also flies not walks, and people fly on aeroplanes. So there you get these specific relationship between these words that are true in both languages and specific structure of relationships is true only for these specific words. Thats how you can know they must mean the same.  
  > **username_0**: How to compare these structures is not yet entirely clear to me. It is complicated. But you have to use analogies, statistics and logic. Neural networks only use analogies or simple similarity but ignore logic, that is their problem I think.  
  > **username_3**: About the examples you provide: in the pure case, you can know from text1 that there are a statistic relation between "lion" and "Africa", from text2 a relation between "estrellas" and "universo". Is "estrellas" synonym of "lyon" ? You can improve the algorithm starting by some well known fixed matches (the external information I refer in m answer),  i.e 5=V, and, from this one and using statistics, infers new matches like 6=VI.  However, in natural language, all these approach will have a lot of troubles. Nice as theoretical exercise, but easier to use an existing dictionary.  
  > **username_0**: Well the ultimate goals is to write a program that will be able to learn itself new languages just from reading large amounts of texts. That this is possible we know from human babies, who are capable of this.  
  > **username_3**: Remember the case of Egyptian scripts: no translation until rosetta stone (external information) was found.  
  > **username_0**: So you are saying this is very hard program to write? Good to know so I wont feel bad when I fail. I will read about Egyptian scripts, sounds interesting. Maybe my program will be able to decrypt them, it should  
  > **username_3**: You have my best wishes in your target. Yes, when you say "I'm planning on writing these two programs", be prepared to say "I'm planning to research on this issue and implement some test programs". You will be famous if you decrypt Voynich manuscript.  
  
  ---
  
  