Answers.Body: <p>Note that, in <strong><a href="https://plato.stanford.edu/entries/logic-ai/" rel="nofollow noreferrer">knowledge representation</a></strong> and reasoning, <a href="https://plato.stanford.edu/entries/common-knowledge/" rel="nofollow noreferrer"><strong>common-sense knowledge</strong></a> is traditionally represented as sentences (in logic). For example, one possible sentence that you could store in a <a href="https://ai.stackexchange.com/a/32666/2444">knowledge base</a> is</p>
  <blockquote>
  <p>The earth revolves around the sun.</p>
  </blockquote>
  <p>This is common-sense knowledge (ignoring the fact that this was not the case in the past until Copernicus and Galileo came along in the 1500s). The programming language PROLOG is based on this type of knowledge, facts, and deduction.</p>
  <p><a href="https://ai.stackexchange.com/q/10623/2444">Self-supervised learning (SSL)</a> has been used to <strong>learn representations of data</strong> (which is often done in the context of natural language processing), but these representations may not be <strong>knowledge</strong>, in the sense that we may not understand what is encoded in these representations or whether they are related to our common-sense knowledge.</p>
  <p>So, whether or not SSL can be used for acquiring/approximating knowledge depends on the definition of knowledge that you use.</p>
  <p>If you use the <a href="https://plato.stanford.edu/entries/common-knowledge/" rel="nofollow noreferrer">traditional definition of common-sense knowledge</a>, SSL is not usually used for knowledge representation, but it has been used in the context of knowledge graphs (see <a href="https://keg.cs.tsinghua.edu.cn/jietang/publications/TKDE21-Liu-et-al-OAG-know.pdf" rel="nofollow noreferrer">the SelfLinKG approach</a>), which can be viewed as a graphical representation of a knowledge base. So, it's possible that SSL can also be useful for approximating common-sense knowledge.</p>
  <p>To answer your question more directly, my impression is that <a href="https://ai.stackexchange.com/a/32666/2444"><strong>knowledge graphs</strong></a> are promising and they are useful in practice. <a href="https://blog.google/products/search/introducing-knowledge-graph-things-not/" rel="nofollow noreferrer">Google uses them in its search engine</a> (and probably also in the Google Assistant). Whenever you search, for example, for a famous person (for example, Gandhi), on the right side, you should see a window describing certain details of that famous person. That's done using Google's knowledge graph.</p>
  <p>Right now, people are trying to develop techniques to learn embeddings of the nodes or relations in a knowledge graph, with the goal of using these embeddings for discovering new knowledge. This area is called <strong>knowledge graph embedding (KGE)</strong>. <a href="https://ai.stackexchange.com/a/32666/2444">My other answer</a> provides more details (but there are many surveys on KGEs that you can find, for instance, on Google Scholar).</p>
  <p>Having said this, knowledge representation has been a big problem since the early days of AI (for example, see the <a href="https://plato.stanford.edu/entries/frame-problem/" rel="nofollow noreferrer">frame problem</a>) and there hasn't been much progress in this area (AFAIK). Most people are focusing on neural networks right now, but <a href="https://ai.stackexchange.com/q/12870/2444">explainable AI techniques</a> for neural networks could potentially play a role in knowledge representation.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-01T11:58:34.663Z
Answers.LastActivityDate: 2021-12-17T15:53:19.773Z
Answers.LastEditDate: 2021-12-17T15:53:19.773Z
Answers.UserDisplayName: nbro
Body: <p>I learned from this blog post <a href="https://ai.facebook.com/blog/self-supervised-learning-the-dark-matter-of-intelligence/" rel="nofollow noreferrer">Self-Supervised Learning: The Dark Matter of Intelligence</a> that</p>
  <blockquote>
  <p>We believe that self-supervised learning (SSL) is one of the most promising ways to build such background knowledge and approximate a form of common sense in AI systems.</p>
  </blockquote>
  <p>What are other most promising ways which would be competitive to self-supervised learning?</p>
  <p>I only know the <em>knowledge base</em>, but I don't think it would be that promising due to <a href="https://ai.stackexchange.com/a/25788/5351">the curation problem of the large scale Automated Knowledge Base Construction</a>.</p>
  
Tags: <neural-networks><pretrained-models><self-supervised-learning><commonsense-knowledge>
Title: What are some most promising ways to approximate common sense and background knowledge?
UserDisplayName: Lerner Zhang
ThreadText: # What are some most promising ways to approximate common sense and background knowledge?
  
  (Asked by: username_0 on 2021-03-20)
  
  I learned from this blog post [Self-Supervised Learning: The Dark Matter of Intelligence](https://ai.facebook.com/blog/self-supervised-learning-the-dark-matter-of-intelligence/) that
  
  > We believe that self-supervised learning (SSL) is one of the most promising ways to build such background knowledge and approximate a form of common sense in AI systems.
  
  What are other most promising ways which would be competitive to self-supervised learning?
  
  I only know the *knowledge base*, but I don't think it would be that promising due to [the curation problem of the large scale Automated Knowledge Base Construction](https://ai.stackexchange.com/a/25788/5351).
  
  **Tags:** neural-networks, pretrained-models, self-supervised-learning, commonsense-knowledge
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-12-01. Score: 1
  
  Note that, in **[knowledge representation](https://plato.stanford.edu/entries/logic-ai/)** and reasoning, [**common-sense knowledge**](https://plato.stanford.edu/entries/common-knowledge/) is traditionally represented as sentences (in logic). For example, one possible sentence that you could store in a [knowledge base](https://ai.stackexchange.com/a/32666/2444) is
  
  > The earth revolves around the sun.
  
  This is common-sense knowledge (ignoring the fact that this was not the case in the past until Copernicus and Galileo came along in the 1500s). The programming language PROLOG is based on this type of knowledge, facts, and deduction.
  
  [Self-supervised learning (SSL)](https://ai.stackexchange.com/q/10623/2444) has been used to **learn representations of data** (which is often done in the context of natural language processing), but these representations may not be **knowledge**, in the sense that we may not understand what is encoded in these representations or whether they are related to our common-sense knowledge.
  
  So, whether or not SSL can be used for acquiring/approximating knowledge depends on the definition of knowledge that you use.
  
  If you use the [traditional definition of common-sense knowledge](https://plato.stanford.edu/entries/common-knowledge/), SSL is not usually used for knowledge representation, but it has been used in the context of knowledge graphs (see [the SelfLinKG approach](https://keg.cs.tsinghua.edu.cn/jietang/publications/TKDE21-Liu-et-al-OAG-know.pdf)), which can be viewed as a graphical representation of a knowledge base. So, it's possible that SSL can also be useful for approximating common-sense knowledge.
  
  To answer your question more directly, my impression is that [**knowledge graphs**](https://ai.stackexchange.com/a/32666/2444) are promising and they are useful in practice. [Google uses them in its search engine](https://blog.google/products/search/introducing-knowledge-graph-things-not/) (and probably also in the Google Assistant). Whenever you search, for example, for a famous person (for example, Gandhi), on the right side, you should see a window describing certain details of that famous person. That's done using Google's knowledge graph.
  
  Right now, people are trying to develop techniques to learn embeddings of the nodes or relations in a knowledge graph, with the goal of using these embeddings for discovering new knowledge. This area is called **knowledge graph embedding (KGE)**. [My other answer](https://ai.stackexchange.com/a/32666/2444) provides more details (but there are many surveys on KGEs that you can find, for instance, on Google Scholar).
  
  Having said this, knowledge representation has been a big problem since the early days of AI (for example, see the [frame problem](https://plato.stanford.edu/entries/frame-problem/)) and there hasn't been much progress in this area (AFAIK). Most people are focusing on neural networks right now, but [explainable AI techniques](https://ai.stackexchange.com/q/12870/2444) for neural networks could potentially play a role in knowledge representation.
  
  
  ---
  
  