Answers.Body: <p>I think that what you're really asking about is the question of <a href="https://en.wikipedia.org/wiki/Knowledge_representation_and_reasoning" rel="nofollow noreferrer">knowledge representation</a>.  Regardless of how you train your AI, one of the most fundamental questions is how do you represent "knowledge" and especially when it exists at different levels of abstraction, may be mutually recursive, etc.  Along with that goes the question of <a href="https://en.wikipedia.org/wiki/Belief_revision" rel="nofollow noreferrer">belief revision</a> which deals with how you update existing beliefs/knowledge in the light of new information.</p>
  
  <p>Both of these areas are still subject to plenty of active research and neither has entirely settled answers to the core questions.  But progress has been made in both areas.  </p>
  
  <p>Personally I suspect that something like <a href="https://en.wikipedia.org/wiki/Semantic_network" rel="nofollow noreferrer">semantic networks</a> or <a href="https://en.wikipedia.org/wiki/Conceptual_graph" rel="nofollow noreferrer">conceptual graphs</a> will be the best answer to the KR problem.  Dealing with belief revision seems even fuzzier to me, although there are known strategies (like the <a href="https://en.wikipedia.org/wiki/Belief_revision#The_AGM_postulates" rel="nofollow noreferrer">AGM postulates</a>) that work to a point. Something like <a href="https://en.wikipedia.org/wiki/Bayesian_network" rel="nofollow noreferrer">Bayesian Belief Networks</a> may also prove useful.  </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-08-20T21:01:02.073Z
Answers.LastActivityDate: 2018-08-20T21:01:02.073Z
Answers.UserDisplayName: mindcrime
Answers.Body: <p>I have recently watched <a href="https://www.youtube.com/watch?v=STFcvzoxVw4" rel="nofollow noreferrer">a podcast by Lex Fridman where he interviews Vladimir Vapnik</a>, who talks about this way of teaching with &quot;predicates&quot; and &quot;invariants&quot;.</p>
  <p>If you were really able to teach a model with sentences like &quot;a bird is an animal that is able to fly&quot; but &quot;not all birds fly, such as penguins&quot;, that would probably represent one of the biggest milestones in machine learning, especially, if the machine was able to learn and apply this learned knowledge as <em>efficiently</em> as humans do. However, we are not quite there yet!</p>
  <p>To know more about this new learning paradigm <strong>Learning Using Statistical Invariants (LUSI)</strong>, you probably should read the paper <a href="https://link.springer.com/article/10.1007/s10994-018-5742-0" rel="nofollow noreferrer">Rethinking statistical learning theory: learning using statistical invariants</a> (2019), by V. Vapnik and R. Izmailov, which will probably be difficult to follow if you have no knowledge of learning theory and your mathematical background is poor. However, there are some sections of this paper that are accessible to everyone. For example, section 6.6</p>
  <blockquote>
  <p>Suppose that the Teacher teaches Student to recognize digits by providing a number of examples and also suggesting the following heuristics: &quot;In order to recognize the digit zero, look at the center of the picture — it is usually light; in order to recognize the digit 2, look at the bottom of the picture - it usually has a dark tail&quot; and so on.</p>
  <p>From the theory above, the Teacher wants the Student to construct specific predicates <span class="math-container">$\psi(x)$</span> to use them for invariants. However, the Student does not necessarily construct exactly the same predicate that the Teacher had in mind (the Student's understanding of concepts &quot;center of the picture&quot; or &quot;bottom of the picture&quot;:  can be different). Instead of <span class="math-container">$\psi(x)$</span>, the Student constructs function <span class="math-container">$\hat{\psi}(x)$</span>. However, this is acceptable, since any function from <span class="math-container">$L_2$</span> can serve as a predicate for an invariant.</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-30T00:34:20.203Z
Answers.LastActivityDate: 2020-04-30T00:34:20.203Z
Answers.LastEditDate: 2020-06-17T09:57:20.710Z
Answers.UserDisplayName: nbro
Body: <p>Could we teach an AI with sentences such as "ants are small" and "the sky is blue"? Is there any research work that attempts to do this?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-08-16T21:29:53.387Z
Comments.Text: You may want to look at the [symbol grounding problem](https://en.wikipedia.org/wiki/Symbol_grounding_problem).  Significant thinkers don't think automata will need to understand to imitate, but are you asking about an NN that actually *understands* the meaning of the symbols (words)?  I was reading something recently suggesting that mathematics itself is form of "language game". If you're talking [syntactics](https://en.wikipedia.org/wiki/Syntax), it doesn't seem to be out of reach, but if you're talking [semantics](https://en.wikipedia.org/wiki/Semantics), it's a whole different ballgame.
Comments.UserDisplayName: DukeZhou
Tags: <neural-networks><machine-learning><natural-language-processing><computational-learning-theory>
Title: Can we teach an artificial intelligence through sentences?
UserDisplayName: zooby
ThreadText: # Can we teach an artificial intelligence through sentences?
  
  (Asked by: username_0 on 2018-08-16)
  
  Could we teach an AI with sentences such as "ants are small" and "the sky is blue"? Is there any research work that attempts to do this?
  
  **Tags:** neural-networks, machine-learning, natural-language-processing, computational-learning-theory
  
  > **Comments:**
  > **username_1**: You may want to look at the [symbol grounding problem](https://en.wikipedia.org/wiki/Symbol_grounding_problem).  Significant thinkers don't think automata will need to understand to imitate, but are you asking about an NN that actually *understands* the meaning of the symbols (words)?  I was reading something recently suggesting that mathematics itself is form of "language game". If you're talking [syntactics](https://en.wikipedia.org/wiki/Syntax), it doesn't seem to be out of reach, but if you're talking [semantics](https://en.wikipedia.org/wiki/Semantics), it's a whole different ballgame.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2018-08-20. Score: 1
  
  I think that what you're really asking about is the question of [knowledge representation](https://en.wikipedia.org/wiki/Knowledge_representation_and_reasoning). Regardless of how you train your AI, one of the most fundamental questions is how do you represent "knowledge" and especially when it exists at different levels of abstraction, may be mutually recursive, etc. Along with that goes the question of [belief revision](https://en.wikipedia.org/wiki/Belief_revision) which deals with how you update existing beliefs/knowledge in the light of new information.
  
  Both of these areas are still subject to plenty of active research and neither has entirely settled answers to the core questions. But progress has been made in both areas.
  
  Personally I suspect that something like [semantic networks](https://en.wikipedia.org/wiki/Semantic_network) or [conceptual graphs](https://en.wikipedia.org/wiki/Conceptual_graph) will be the best answer to the KR problem. Dealing with belief revision seems even fuzzier to me, although there are known strategies (like the [AGM postulates](https://en.wikipedia.org/wiki/Belief_revision#The_AGM_postulates)) that work to a point. Something like [Bayesian Belief Networks](https://en.wikipedia.org/wiki/Bayesian_network) may also prove useful.
  
  
  ---
  
  ### Answer by username_3 on 2020-04-30. Score: 0
  
  I have recently watched [a podcast by Lex Fridman where he interviews Vladimir Vapnik](https://www.youtube.com/watch?v=STFcvzoxVw4), who talks about this way of teaching with "predicates" and "invariants".
  
  If you were really able to teach a model with sentences like "a bird is an animal that is able to fly" but "not all birds fly, such as penguins", that would probably represent one of the biggest milestones in machine learning, especially, if the machine was able to learn and apply this learned knowledge as *efficiently* as humans do. However, we are not quite there yet!
  
  To know more about this new learning paradigm **Learning Using Statistical Invariants (LUSI)**, you probably should read the paper [Rethinking statistical learning theory: learning using statistical invariants](https://link.springer.com/article/10.1007/s10994-018-5742-0) (2019), by V. Vapnik and R. Izmailov, which will probably be difficult to follow if you have no knowledge of learning theory and your mathematical background is poor. However, there are some sections of this paper that are accessible to everyone. For example, section 6.6
  
  > Suppose that the Teacher teaches Student to recognize digits by providing a number of examples and also suggesting the following heuristics: "In order to recognize the digit zero, look at the center of the picture — it is usually light; in order to recognize the digit 2, look at the bottom of the picture - it usually has a dark tail" and so on.
  >
  > From the theory above, the Teacher wants the Student to construct specific predicates $\psi(x)$ to use them for invariants. However, the Student does not necessarily construct exactly the same predicate that the Teacher had in mind (the Student's understanding of concepts "center of the picture" or "bottom of the picture": can be different). Instead of $\psi(x)$, the Student constructs function $\hat{\psi}(x)$. However, this is acceptable, since any function from $L\_2$ can serve as a predicate for an invariant.
  
  
  ---
  
  