Answers.Body: <p>For this example the function below will do:
  <code>
  TSAI.Analogies.FindAnalogy(List ex1, List ex2, List ex3, out List ex4)
  </code>
  ex1 is to ex2 as ex3 is to ex4.
  Figure out ex4.</p>
  
  <p>Fill ex4 with values from ex2.
  For every value in ex3: find out to which positions in ex4 we have to copy this value, based on value in ex1 at the same position that was repeated in ex2.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-28T15:05:57.140Z
Answers.LastActivityDate: 2018-01-28T15:05:57.140Z
Answers.UserDisplayName: Tone Škoda
Answers.Body: <p>You are implying that such ideas are novel, and that such tools do not exist. But the idea is very popular, and there are numerous tools.</p>
  
  <blockquote>
    <p>We need to write a program that would recognize that a word is connected to other words in the same way in both language. Then it would know those two words must have the same meaning.</p>
  </blockquote>
  
  <p>You are describing the essence of known natural language processing (NLP) tasks such as word alignment (link words in different languages that have the same meaning) and, of course, machine translation.</p>
  
  <p>While learning a machine translation model, we actually do discover which words (or parts of words, or sequences of words) in different languages have the same meaning.</p>
  
  <p>Here are some concepts I would recommend for further study of this subject:</p>
  
  <ul>
  <li>Word alignment, an example for a well-known and popular tool would be <code>fast_align</code></li>
  <li>Word embeddings, <code>word2vec</code> is a widely used tool</li>
  <li>Modern machine translation with sequence-to-sequence models, well-known tools are <code>fairseq</code>, or <code>Sockeye</code></li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-28T19:22:30.837Z
Answers.LastActivityDate: 2020-01-28T19:22:30.837Z
Answers.UserDisplayName: Mathias Müller
Answers.Body: <p>Isn't this what already <code>Word2Vec</code> and other <code>word-embedding</code> techniques already use. <code>You know your word by the company it keeps</code> is an idea that has been around for some time now.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-27T22:07:16.943Z
Answers.Text: Never heard of it. I'll read about it.
Answers.UserDisplayName: Tone Škoda
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-25T11:07:27.393Z
Answers.LastActivityDate: 2018-01-25T11:07:27.393Z
Answers.UserDisplayName: Krishna Sangeeth K S
Answers.Body: <p>It is wrong to assume that just the connections to other words define their meaning.</p>
  
  <p>Give an AI a hundred novels and it would still not know what the word "cat" means.</p>
  
  <p>Show the AI a picture of a cat with the word "cat" underneath it and it would know straight away.</p>
  
  <p>In this way an AI needs to know a minimum number of words through experience other than combinations of other words. From then it may be able to deduce meanings of new words.</p>
  
  <p>Just like, if I gave you a hundred novels in Chinese you would never be able to understand Chinese. I show you a picture book in Chinese and maybe you have a chance.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-28T15:21:42.583Z
Answers.Text: If your theory was true then blind people wouldn't understand words.
Answers.UserDisplayName: Tone Škoda
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-28T19:14:37.760Z
Answers.Text: My apologies, but this answer is completely inaccurate. It is absolutely true that words can be defined by their context, most modern language technology tools are based on this idea (word embeddings). It is not true that in order to learn the meaning of words they need to be presented together with images.
Answers.UserDisplayName: Mathias Müller
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-28T22:33:20.633Z
Answers.Text: @tone nonsense blind people use their senses like touch and sound to associate the word cat with cats
Answers.UserDisplayName: zooby
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-28T22:35:21.673Z
Answers.Text: @mathias. It is true that after you have learnt a minimum number of words you can use these to define other words... But first these minimum words beed to be learnt from association with the real world... Unless u are learning a second language. Also word emedding ai have no concept of what a cat is.
Answers.UserDisplayName: zooby
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-29T07:44:12.240Z
Answers.Text: U haven't heard of Helen Mirren? She had no hearing or sight, she had touch but she used it to communicate, not to touch things to understand them. You never saw atom and still know what it is.
Answers.UserDisplayName: Tone Škoda
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-29T08:12:19.147Z
Answers.Text: @zooby "It is true that after you have learnt a minimum number of words you can use these to define other words..." - I can only repeat myself: It is not true for modern NLP tools that in order to learn the meaning of words they need to be presented together with images or other kinds of sensory input.
Answers.UserDisplayName: Mathias Müller
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-29T08:18:16.587Z
Answers.Text: @zooby "Also word emedding ai have no concept of what a cat is." - Which facts are you basing this observation on? Please back up with some sources.
Answers.UserDisplayName: Mathias Müller
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-29T11:35:24.390Z
Answers.Text: I think that this answer can highly be improved by emphasizing the term "meaning", because some people will claim that AI really understands the meaning and some people will say that they perform only pattern recognition (or whatever). In other words, just say that this is also a philosophical issue.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-29T12:28:58.713Z
Answers.Text: @ToneŠkoda I assume you mean Helen Keller and not the actress XD
Answers.UserDisplayName: Fnguyen
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-29T13:05:56.663Z
Answers.Text: yes you are correct :)
Answers.UserDisplayName: Tone Škoda
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-28T05:49:16.843Z
Answers.LastActivityDate: 2020-01-28T05:49:16.843Z
Answers.UserDisplayName: zooby
Body: <p>Imagine two languages that have only these words:</p>
  
  <pre><code>Man = 1,
  deer = 2, 
  eat = 3,
  grass = 4 
  </code></pre>
  
  <p>And you would form all sentences possible from these words:</p>
  
  <pre><code>Man eats deer.
  Deer eats grass.
  Man eats.
  Deer eats.
  </code></pre>
  
  <p>German:</p>
  
  <pre><code>Mensch = 5,
  Gras = 6, 
  isst = 7, 
  Hirsch = 8
  </code></pre>
  
  <p>Possible german sentences:</p>
  
  <pre><code>Mensch isst Hirsch.
  Hirsch isst Gras.
  Mensch isst.
  Hirsch isst.
  </code></pre>
  
  <p>How would you write a program that would figure out which words have the same meaning in English and German?</p>
  
  <p>It is possible.</p>
  
  <p>All words get their meaning from the information in which sentences they can be used. The connection with other words defines their meaning.</p>
  
  <p>We need to write a program that would recognize that a word is connected to other words in the same way in both languages. Then it would know those two words must have the same meaning.</p>
  
  <p>If we take the word "deer" (2) it has this structure in English</p>
  
  <pre><code>1-3-2
  2-3-4
  </code></pre>
  
  <p>In german (8):</p>
  
  <pre><code>5-6-8
  8-6-7
  </code></pre>
  
  <p>We get the same structure (pattern) in both languages: both 8 and 2 lie in first and last position, and the middle word is the same in both languages, the other word is different in both languages. So we can conclude that 8=2 because both elements are connected with other elements the same way.</p>
  
  <p>Maybe we just need to write a very good program for recognizing analogies and we will be on the right track to creating AI?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-25T08:23:10.637Z
Comments.Text: This is similar to the way that [example-based machine translation](https://en.wikipedia.org/wiki/Example-based_machine_translation) systems are designed.
Comments.UserDisplayName: Anderson Green
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-25T18:33:26.993Z
Comments.Text: Welcome to AI!  My sense is that language development is partly creative, so it may require algorithms that can think abstractly in a general semantic context.   Combinatorially speaking, I think you'd also need to include ["grass eats", "grass eats deer", "deer eats man"] and give a distinct value to "eats" as a verb, to distinguish from nouns (possibly this could be approached logarithmically.)   You'll also have to contend with poetic constructions "Deer, man eats", etc., nonsense such as "deer eats man", and abstractions "grass eats man" (in the sense of "dust to dust", as fertilizer.)
Comments.UserDisplayName: DukeZhou
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-27T21:56:26.183Z
Comments.Text: Yes, we can add all those metaphorical sentences. Basically every sentence that is syntatically correct and not total nonsense. But for the sake of simplicity I left them out. Program would work that way anyway that it would count frequency of sentences used in large amounts of texts. Your examples would have very small frequency and so connections between those words wouldn't have that much meaning for the definition of word as would frequent sentences.
Comments.UserDisplayName: Tone Škoda
Tags: <natural-language-processing><statistical-ai>
Title: How to figure out which words have the same meaning in two different languages?
UserDisplayName: Tone Škoda
ThreadText: # How to figure out which words have the same meaning in two different languages?
  
  (Asked by: username_0 on 2018-01-24)
  
  Imagine two languages that have only these words:
  Man = 1,
  deer = 2, 
  eat = 3,
  grass = 4 
  
  And you would form all sentences possible from these words:
  Man eats deer.
  Deer eats grass.
  Man eats.
  Deer eats.
  
  German:
  Mensch = 5,
  Gras = 6, 
  isst = 7, 
  Hirsch = 8
  
  Possible german sentences:
  Mensch isst Hirsch.
  Hirsch isst Gras.
  Mensch isst.
  Hirsch isst.
  
  How would you write a program that would figure out which words have the same meaning in English and German?
  It is possible.
  All words get their meaning from the information in which sentences they can be used. The connection with other words defines their meaning.
  We need to write a program that would recognize that a word is connected to other words in the same way in both languages. Then it would know those two words must have the same meaning.
  If we take the word "deer" (2) it has this structure in English
  1-3-2
  2-3-4
  
  In german (8):
  5-6-8
  8-6-7
  
  We get the same structure (pattern) in both languages: both 8 and 2 lie in first and last position, and the middle word is the same in both languages, the other word is different in both languages. So we can conclude that 8=2 because both elements are connected with other elements the same way.
  Maybe we just need to write a very good program for recognizing analogies and we will be on the right track to creating AI?
  
  **Tags:** natural-language-processing, statistical-ai
  > **Comments:**
  > **username_1**: This is similar to the way that [example-based machine translation](https://en.wikipedia.org/wiki/Example-based_machine_translation) systems are designed.  
  > **username_2**: Welcome to AI!  My sense is that language development is partly creative, so it may require algorithms that can think abstractly in a general semantic context.   Combinatorially speaking, I think you'd also need to include ["grass eats", "grass eats deer", "deer eats man"] and give a distinct value to "eats" as a verb, to distinguish from nouns (possibly this could be approached logarithmically.)   You'll also have to contend with poetic constructions "Deer, man eats", etc., nonsense such as "deer eats man", and abstractions "grass eats man" (in the sense of "dust to dust", as fertilizer.)  
  > **username_0**: Yes, we can add all those metaphorical sentences. Basically every sentence that is syntatically correct and not total nonsense. But for the sake of simplicity I left them out. Program would work that way anyway that it would count frequency of sentences used in large amounts of texts. Your examples would have very small frequency and so connections between those words wouldn't have that much meaning for the definition of word as would frequent sentences.  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_0 on 2018-01-28. Score: -1 | Accepted answer
  
  For this example the function below will do:
  `TSAI.Analogies.FindAnalogy(List ex1, List ex2, List ex3, out List ex4)`
  ex1 is to ex2 as ex3 is to ex4.
  Figure out ex4.
  
  Fill ex4 with values from ex2.
  For every value in ex3: find out to which positions in ex4 we have to copy this value, based on value in ex1 at the same position that was repeated in ex2.
  
  
  ---
  
  ### Answer by username_3 on 2020-01-28. Score: 2
  
  You are implying that such ideas are novel, and that such tools do not exist. But the idea is very popular, and there are numerous tools.
  
  > We need to write a program that would recognize that a word is connected to other words in the same way in both language. Then it would know those two words must have the same meaning.
  
  You are describing the essence of known natural language processing (NLP) tasks such as word alignment (link words in different languages that have the same meaning) and, of course, machine translation.
  
  While learning a machine translation model, we actually do discover which words (or parts of words, or sequences of words) in different languages have the same meaning.
  
  Here are some concepts I would recommend for further study of this subject:
  
  * Word alignment, an example for a well-known and popular tool would be `fast_align`
  * Word embeddings, `word2vec` is a widely used tool
  * Modern machine translation with sequence-to-sequence models, well-known tools are `fairseq`, or `Sockeye`
  
  
  ---
  
  ### Answer by username_4 on 2018-01-25. Score: 1
  
  Isn't this what already `Word2Vec` and other `word-embedding` techniques already use. `You know your word by the company it keeps` is an idea that has been around for some time now.
  
  > **Comments:**
  > **username_0**: Never heard of it. I'll read about it.  
  
  ---
  
  ### Answer by username_5 on 2020-01-28. Score: -1
  
  It is wrong to assume that just the connections to other words define their meaning.
  
  Give an AI a hundred novels and it would still not know what the word "cat" means.
  
  Show the AI a picture of a cat with the word "cat" underneath it and it would know straight away.
  
  In this way an AI needs to know a minimum number of words through experience other than combinations of other words. From then it may be able to deduce meanings of new words.
  
  Just like, if I gave you a hundred novels in Chinese you would never be able to understand Chinese. I show you a picture book in Chinese and maybe you have a chance.
  
  > **Comments:**
  > **username_0**: If your theory was true then blind people wouldn't understand words.  
  > **username_3**: My apologies, but this answer is completely inaccurate. It is absolutely true that words can be defined by their context, most modern language technology tools are based on this idea (word embeddings). It is not true that in order to learn the meaning of words they need to be presented together with images.  
  > **username_5**: @tone nonsense blind people use their senses like touch and sound to associate the word cat with cats  
  > **username_5**: @mathias. It is true that after you have learnt a minimum number of words you can use these to define other words... But first these minimum words beed to be learnt from association with the real world... Unless u are learning a second language. Also word emedding ai have no concept of what a cat is.  
  > **username_0**: U haven't heard of Helen Mirren? She had no hearing or sight, she had touch but she used it to communicate, not to touch things to understand them. You never saw atom and still know what it is.  
  > **username_3**: @zooby "It is true that after you have learnt a minimum number of words you can use these to define other words..." - I can only repeat myself: It is not true for modern NLP tools that in order to learn the meaning of words they need to be presented together with images or other kinds of sensory input.  
  > **username_3**: @zooby "Also word emedding ai have no concept of what a cat is." - Which facts are you basing this observation on? Please back up with some sources.  
  > **username_6**: I think that this answer can highly be improved by emphasizing the term "meaning", because some people will claim that AI really understands the meaning and some people will say that they perform only pattern recognition (or whatever). In other words, just say that this is also a philosophical issue.  
  > **username_7**: @ToneŠkoda I assume you mean Helen Keller and not the actress XD  
  > **username_0**: yes you are correct :)  
  
  ---
  
  