Answers.Body: <p>I think you would be interested in <a href="https://arxiv.org/pdf/1711.03902.pdf" rel="nofollow noreferrer">Neural-Symbolic Learning and Reasoning</a>, a recent survey on the intersection between connectionist models (e.g. neural nets) and symbolic reasoning. It's a long paper, but it has a lot in it that is relevant to your question, if you're willing to dig. For example, it talks about learning networks that implement boolean logic.</p>
  
  <p><a href="https://i.sstatic.net/yzaeK.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/yzaeK.png" alt="enter image description here"></a></p>
  
  <blockquote>
    <p>This
    network searches for satisfying solutions for the weighted conjunctive normal form (CNF):<span class="math-container">$\text{(¬X ∨ ¬Y ∨ Z) ∧ (X ∨ Y )}$</span>.</p>
  </blockquote>
  
  <p>The closest thing I could find to the "functionals" you mentioned was a discussion on learning "relations" with neural networks. For this, it cited <a href="https://epdf.tips/neural-symbolic-cognitive-reasoning3f30592b99b4240b1790fcf45c13e1e260069.html" rel="nofollow noreferrer">Neural-Symbolic Cognitive Reasoning</a>, a book also about the intersection between neural networks and symbolic reasoning. In Chapter 10, it gives the example of teaching a neural network to learn the relation of "grandparent."</p>
  
  <blockquote>
    <p>Our goal is to learn a description of grandparent from examples of the father, mother, and grandparent relations such as <em>father</em>(charles, william), <em>mother</em>(elisabeth, charles), and <em>grandparent</em>(elisabeth, william).</p>
  </blockquote>
  
  <p>I could imagine something like this being done with the example inputs being image features from a CNN. Perhaps then you could start learning functionals like "on the shirt."</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-09T18:56:14.850Z
Answers.Text: Similarly, there have been several attempts to develop [theorem provers using neural networks](https://www.google.com/search?q=%22theorem+prover%22+neural).
Answers.UserDisplayName: Anderson Green
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-17T22:31:40.493Z
Answers.LastActivityDate: 2019-03-17T22:31:40.493Z
Answers.UserDisplayName: Philip Raeisghasem
Body: <p>Does anyone work out ways of relating trained neural networks by symbolic AI?</p>
  <p>For example, if I train a network on pictures of dogs, and I train a network on pictures of shirts. You could imagine that the simplest way (without going through the process from scratch) of identifying &quot;dog AND shirt&quot; would be to perform an AND operation on the last output of the individual cat &amp; dog neural nets. So, &quot;dog AND shirt&quot; would amount to AND'ing the output of two nets.</p>
  <p>But this operation AND could be replaced with a more complicated operation. And, in principle, I could &quot;train&quot; a neural network to act as one of these operations.  For instance, maybe I could figure out the net that describes some changeable output &quot;X&quot; being &quot;on the shirt.&quot; This would be sort of like a &quot;functional&quot; in mathematics (in which we are operating are considering the behavior of a network whos input could be any network). If I can figure out this &quot;functional&quot;, then I would be able to use it symbolically and determine queries like &quot;dog on the shirt&quot;? - &quot;cat on the shirt&quot;?</p>
  <p>It seems like to me there's a lot of sense to turn specific neural networks into more &quot;abstract&quot; objects - and that there would be a lot of power in doing so.</p>
  
Tags: <neural-networks><reference-request><logic><symbolic-ai><neurosymbolic-ai>
Title: Can (trained) neural networks be combined with symbolic AI to perform operations like AND?
UserDisplayName: Steven Sagona
ThreadText: # Can (trained) neural networks be combined with symbolic AI to perform operations like AND?
  
  (Asked by: username_0 on 2019-02-20)
  
  Does anyone work out ways of relating trained neural networks by symbolic AI?
  
  For example, if I train a network on pictures of dogs, and I train a network on pictures of shirts. You could imagine that the simplest way (without going through the process from scratch) of identifying "dog AND shirt" would be to perform an AND operation on the last output of the individual cat & dog neural nets. So, "dog AND shirt" would amount to AND'ing the output of two nets.
  
  But this operation AND could be replaced with a more complicated operation. And, in principle, I could "train" a neural network to act as one of these operations. For instance, maybe I could figure out the net that describes some changeable output "X" being "on the shirt." This would be sort of like a "functional" in mathematics (in which we are operating are considering the behavior of a network whos input could be any network). If I can figure out this "functional", then I would be able to use it symbolically and determine queries like "dog on the shirt"? - "cat on the shirt"?
  
  It seems like to me there's a lot of sense to turn specific neural networks into more "abstract" objects - and that there would be a lot of power in doing so.
  
  **Tags:** neural-networks, reference-request, logic, symbolic-ai, neurosymbolic-ai
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-03-17. Score: 3
  
  I think you would be interested in [Neural-Symbolic Learning and Reasoning](https://arxiv.org/pdf/1711.03902.pdf), a recent survey on the intersection between connectionist models (e.g. neural nets) and symbolic reasoning. It's a long paper, but it has a lot in it that is relevant to your question, if you're willing to dig. For example, it talks about learning networks that implement boolean logic.
  
  [![enter image description here](https://i.sstatic.net/yzaeK.png)](https://i.sstatic.net/yzaeK.png)
  
  > This
  > network searches for satisfying solutions for the weighted conjunctive normal form (CNF):$\text{(¬X ∨ ¬Y ∨ Z) ∧ (X ∨ Y )}$.
  
  The closest thing I could find to the "functionals" you mentioned was a discussion on learning "relations" with neural networks. For this, it cited [Neural-Symbolic Cognitive Reasoning](https://epdf.tips/neural-symbolic-cognitive-reasoning3f30592b99b4240b1790fcf45c13e1e260069.html), a book also about the intersection between neural networks and symbolic reasoning. In Chapter 10, it gives the example of teaching a neural network to learn the relation of "grandparent."
  
  > Our goal is to learn a description of grandparent from examples of the father, mother, and grandparent relations such as *father*(charles, william), *mother*(elisabeth, charles), and *grandparent*(elisabeth, william).
  
  I could imagine something like this being done with the example inputs being image features from a CNN. Perhaps then you could start learning functionals like "on the shirt."
  
  > **Comments:**
  > **username_2**: Similarly, there have been several attempts to develop [theorem provers using neural networks](https://www.google.com/search?q=%22theorem+prover%22+neural).  
  
  ---
  
  