Answers.Body: <p>I think the colloquial understanding of Gödel's incompleteness theorems allows them to be too broadly applied. Gödel's second incompleteness regards the <strong>consistency</strong> of a formal system, which is a <a href="https://en.wikipedia.org/wiki/Consistency" rel="nofollow noreferrer">technical concept</a> of formal systems that means the system cannot prove every formula. It is commonly framed as a system not being able to prove both a formula and its negation (e.g. <span class="math-container">$2+2=4$</span> and <span class="math-container">$2+2 \neq 4$</span>), since many logical systems allow <a href="https://en.wikipedia.org/wiki/Principle_of_explosion" rel="nofollow noreferrer">anything to be proven from a contradiction</a>.</p>
  <p>The second incompleteness theorem states that if a consistent formal system is expressive enough to encode basic arithmetic (<a href="https://en.wikipedia.org/wiki/Peano_axioms" rel="nofollow noreferrer">Peano arithmetic</a>), then that system cannot prove its own consistency. This implies that we must use a stronger system B to prove the consistency of A. The system needs to be able to represent arithmetic because that is what is used to define the representability conditions of Gödel's proof that allowed him to formally construct the self-referential formulae central to the incompleteness theorems.</p>
  <p>Here I diverge with my own opinion on this, I feel the concept of consistency in formal systems has no obvious bearing on the limits of artificial intelligence. An intelligent agent need not know anything about formal consistency to reach its level of intelligence -- the vast majority of humans have never encountered this concept, and yet they are still intelligent. Even many mathematicians don't give it a second thought unless they are in the trenches of mathematical logic. One would have to take an overly narrow view of artificial intelligence to allow Gödel's second incompleteness to serve as a limitation to it.</p>
  <p>I caution against the popular informal restatements of Gödel's incompleteness theorems. These theorems were undoubtedly earth-shattering in the study of foundational mathematics and still have grand implications today, but projecting those results too far away from their rigorous origins is going to lead to many stray conclusions.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-11T06:32:56.127Z
Answers.LastActivityDate: 2020-12-11T18:12:18.183Z
Answers.LastEditDate: 2020-12-11T18:12:18.183Z
Answers.UserDisplayName: adamconkey
Body: <p>According to Brian Cantwell Smith</p>
  <blockquote>
  <p>no calculation without representation</p>
  </blockquote>
  <p>Therefore, computers depend on models. So, we can say that AI is limited internally by the model and externally by the environment. This problem is discussed <a href="https://ai.stackexchange.com/q/17999/2444">here</a> in a previous question I have asked.</p>
  <p>Now, consider <em>Gödel's second incompleteness theorem</em></p>
  <blockquote>
  <p>a coherent theory does not demonstrate its own coherence</p>
  </blockquote>
  <p>Can we say that Gödel's second incompleteness theorem puts a limitation on artificial intelligence? How could AI bypass Gödel's second incompleteness theorem?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-13T10:18:10.043Z
Comments.Text: [This](https://ai.stackexchange.com/q/11517/2444) and [this](https://ai.stackexchange.com/q/3209/2444) are two very related questions.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-11T20:27:12.247Z
Comments.Text: Why should AI systems be consistent? Intelligent humans are not!
Comments.UserDisplayName: Basile Starynkevitch
Tags: <philosophy><math><agi><incompleteness-theorems>
Title: Does Gödel's second incompleteness theorem put a limitation on artificial intelligence systems?
UserDisplayName: alamata
ThreadText: # Does Gödel's second incompleteness theorem put a limitation on artificial intelligence systems?
  
  (Asked by: username_0 on 2020-05-13)
  
  According to Brian Cantwell Smith
  
  > no calculation without representation
  
  Therefore, computers depend on models. So, we can say that AI is limited internally by the model and externally by the environment. This problem is discussed [here](https://ai.stackexchange.com/q/17999/2444) in a previous question I have asked.
  
  Now, consider *Gödel's second incompleteness theorem*
  
  > a coherent theory does not demonstrate its own coherence
  
  Can we say that Gödel's second incompleteness theorem puts a limitation on artificial intelligence? How could AI bypass Gödel's second incompleteness theorem?
  
  **Tags:** philosophy, math, agi, incompleteness-theorems
  
  > **Comments:**
  > **username_1**: [This](https://ai.stackexchange.com/q/11517/2444) and [this](https://ai.stackexchange.com/q/3209/2444) are two very related questions.  
  > **username_2**: Why should AI systems be consistent? Intelligent humans are not!  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2020-12-11. Score: 4
  
  I think the colloquial understanding of Gödel's incompleteness theorems allows them to be too broadly applied. Gödel's second incompleteness regards the **consistency** of a formal system, which is a [technical concept](https://en.wikipedia.org/wiki/Consistency) of formal systems that means the system cannot prove every formula. It is commonly framed as a system not being able to prove both a formula and its negation (e.g. $2+2=4$ and $2+2 \neq 4$), since many logical systems allow [anything to be proven from a contradiction](https://en.wikipedia.org/wiki/Principle_of_explosion).
  
  The second incompleteness theorem states that if a consistent formal system is expressive enough to encode basic arithmetic ([Peano arithmetic](https://en.wikipedia.org/wiki/Peano_axioms)), then that system cannot prove its own consistency. This implies that we must use a stronger system B to prove the consistency of A. The system needs to be able to represent arithmetic because that is what is used to define the representability conditions of Gödel's proof that allowed him to formally construct the self-referential formulae central to the incompleteness theorems.
  
  Here I diverge with my own opinion on this, I feel the concept of consistency in formal systems has no obvious bearing on the limits of artificial intelligence. An intelligent agent need not know anything about formal consistency to reach its level of intelligence -- the vast majority of humans have never encountered this concept, and yet they are still intelligent. Even many mathematicians don't give it a second thought unless they are in the trenches of mathematical logic. One would have to take an overly narrow view of artificial intelligence to allow Gödel's second incompleteness to serve as a limitation to it.
  
  I caution against the popular informal restatements of Gödel's incompleteness theorems. These theorems were undoubtedly earth-shattering in the study of foundational mathematics and still have grand implications today, but projecting those results too far away from their rigorous origins is going to lead to many stray conclusions.
  
  
  ---
  
  