Answers.Body: <p><sup>Your question is mostly philosophical, not technical or scientific. So I am giving opinions and references here.</sup></p>
  
  <blockquote>
    <p>the core of AI boils down to design algorithms</p>
  </blockquote>
  
  <p>I am noticing that you are not even try to define <a href="https://en.wikipedia.org/wiki/Artificial_intelligence" rel="nofollow noreferrer">AI</a> (whose definition changed since the previous century). You could look at the table of contents of the <a href="https://www.journals.elsevier.com/artificial-intelligence" rel="nofollow noreferrer"><em>Artificial Intelligence</em> journal</a> and notice how topics covered there changed drastically in a few decades (even experimental approaches have declined).</p>
  
  <p>You might be interested in reading more about <a href="https://en.wikipedia.org/wiki/Artificial_general_intelligence" rel="nofollow noreferrer">AGI</a> and follow the few conferences about it. Beware, there is a lot of too much simplified approaches, and even a lot of bullshit (e.g. on <a href="https://agi.topicbox.com/groups/agi" rel="nofollow noreferrer">this</a> AGI mailing list, but some messages there are gems)</p>
  
  <p>I assume you accept the <a href="https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis" rel="nofollow noreferrer">Church-Turing</a> philosophical thesis: every intelligent cognition (either natural i.e. biological or artificial) is some symbolic computation. In particular, the work of a mathematician can be abstracted as a Turing machine (that was the major insight of Turing and in the <a href="https://en.wikipedia.org/wiki/Halting_problem" rel="nofollow noreferrer">halting problem</a>). Be also aware of the related <a href="https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspondence" rel="nofollow noreferrer">Curry-Howard</a> correspondence and <a href="https://en.wikipedia.org/wiki/Rice%27s_theorem" rel="nofollow noreferrer">Rice's theorem</a>. Read <a href="https://en.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach" rel="nofollow noreferrer"><em>Gödel, Escher, Bach</em></a> !</p>
  
  <p>We don't know yet how to make AGI. You could read Bostrom's <a href="https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies" rel="nofollow noreferrer"><em>SuperIntelligence</em></a> book about potential dangers. You could also read J.Pitrat's book <a href="https://dl.acm.org/citation.cfm?id=1717130" rel="nofollow noreferrer"><em>Artificial Beings</em></a> (which gives much more positive and constructive insights about eventually making some AGI) and <a href="http://bootstrappingartificialintelligence.fr/WordPress3/" rel="nofollow noreferrer">blog</a>.</p>
  
  <p>My personal belief (just an opinion) is that AGI could be perhaps achieved (in many dozens of years), should be definitely get much more funding -and more time- as a research topic (e.g. as much as the <a href="https://en.wikipedia.org/wiki/ITER" rel="nofollow noreferrer">ITER</a> reactor; see also <a href="https://www.softwareheritage.org/" rel="nofollow noreferrer">softwareheritage.org</a> and the motivations there), but won't be achieved by any single technique, but by a clever combination of many AI techniques (both <a href="https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence" rel="nofollow noreferrer">symbolic AI</a> -e.g. for <a href="https://en.wikipedia.org/wiki/Planning" rel="nofollow noreferrer">planning</a>- and <a href="https://en.wikipedia.org/wiki/Machine_learning" rel="nofollow noreferrer">machine learning</a> or <a href="https://en.wikipedia.org/wiki/Connectionism" rel="nofollow noreferrer">connectionnist</a> approaches, with inspiration from cognitive psychology).</p>
  
  <blockquote>
    <p>The conclusion is, in my understanding, that to create AI systems we must accept incompleteness or inconsistency. </p>
  </blockquote>
  
  <p>We, members of the <a href="https://en.wikipedia.org/wiki/Human_taxonomy#Subspecies" rel="nofollow noreferrer">Homo Sapiens Sapiens</a> species (in latin, the humans who know that they know, so capable of <a href="https://en.wikipedia.org/wiki/Metaknowledge" rel="nofollow noreferrer">metaknowledge</a>), claim to be intelligent. But all of us have a globally incomplete and inconsistent behavior, because each of us have contradictions (e.g. in our personal lives or <a href="https://en.wikipedia.org/wiki/Ethical_dilemma" rel="nofollow noreferrer">ethical</a> beliefs). So, logically speaking, incompleteness or inconsistency is not opposed to intelligence. Read also more about <a href="https://en.wikipedia.org/wiki/Artificial_intelligence,_situated_approach" rel="nofollow noreferrer">situated AI</a> and <a href="https://en.wikipedia.org/wiki/Machine_ethics" rel="nofollow noreferrer">machine ethics</a>. BTW, I believe (since educated by J.Pitrat about this) that explicit and <a href="https://en.wikipedia.org/wiki/Descriptive_knowledge" rel="nofollow noreferrer">declarative</a> <a href="https://www.coursehero.com/file/ptcioic/CS-484-Artificial-Intelligence-8-Meta-Rules-The-rules-that-determine-the/" rel="nofollow noreferrer">meta</a>knowledge is required in any AGI system.</p>
  
  <blockquote>
    <p>Does that mean that AI will never be employed for real-time critical applications? </p>
  </blockquote>
  
  <p>Notice that <a href="https://en.wikipedia.org/wiki/Lethal_autonomous_weapon" rel="nofollow noreferrer">autonomous killing robots</a> are already a controversial research topic today. <a href="https://en.wikipedia.org/wiki/Autonomous_robot" rel="nofollow noreferrer">Autonomous robots</a> already exist (e.g. <a href="https://en.wikipedia.org/wiki/Mars_rover" rel="nofollow noreferrer">Mars rovers</a> cannot be teleoperated -for every elementary movement- from Earth, because any radio signal takes minutes to reach Mars). And autonomous vehicles (à la <a href="https://en.wikipedia.org/wiki/Waymo" rel="nofollow noreferrer">Google car</a>) claim today to use AI techniques and are <a href="https://en.wikipedia.org/wiki/Real-time_computing" rel="nofollow noreferrer">real-time</a> <a href="https://en.wikipedia.org/wiki/Safety-critical_system" rel="nofollow noreferrer">safety-critical systems</a>. Today's Airbus or Boeing (cf <a href="https://en.wikipedia.org/wiki/DO-178C" rel="nofollow noreferrer">DO-178C</a>) are flying automatically most of the time. <a href="https://en.wikipedia.org/wiki/Cruise_missile" rel="nofollow noreferrer">Cruise missiles</a> and <a href="https://en.wikipedia.org/wiki/Intercontinental_ballistic_missile" rel="nofollow noreferrer">ICBM</a>s are <a href="https://en.wikipedia.org/wiki/Fire-and-forget" rel="nofollow noreferrer">fire-and-forget</a> devices. Many <a href="https://en.wikipedia.org/wiki/High-frequency_trading" rel="nofollow noreferrer">high-frequency trading</a> systems claim to use AI techniques and are real-time.</p>
  
  <p>PS. Notice that what was called AI in the previous century is today called AGI. My PhD in AI was defended in 1990 (and was about explicit metaknowledge for metaprogramming goals, see e.g. <a href="https://inis.iaea.org/collection/NCLCollectionStore/_Public/19/059/19059867.pdf?r=1&amp;r=1" rel="nofollow noreferrer">this</a> old 1987 paper)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-16T13:18:05.733Z
Answers.Text: Thanks for the references! I refer to AI  as "machines (or computers) that mimic "cognitive" functions that humans associate with the human mind, such as "learning" and "problem solving"" (Wikipedia). AGI is also known as Strong AI, but I don't mind weak or strong AI for my argument. Yes, I assume the Chruch-Turing thesis. I further agree that intelligence is not opposed to incompleteness or inconsistency. But then AI is also incomplete and inconsistent. This means that it can make evaluation mistakes and that may not be socially accepted for some apps. Think e.g. to FULLY autonomous drive.
Answers.UserDisplayName: Barzi2001
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-16T13:28:44.373Z
Answers.Text: I just read about a crush of a Google self-driving car that "The company also stated "This type of misunderstanding happens between human drivers on the road every day"" (Wikipedia). This episode (along with Google answer) would strengthen the hypothesis that an A(G)I system could make evaluation errors and could potentially harm human, or?
Answers.UserDisplayName: Barzi2001
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-17T19:23:13.207Z
Answers.Text: If you feell my answer relevant to your question, consider upvoting it. I don't read well enough your English (since my native language is French, and my parents spoke to me in Russian) to understand what the ending "or?" means in your comment above
Answers.UserDisplayName: Basile Starynkevitch
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-18T13:20:53.637Z
Answers.Text: Your answer was actually  very useful. You can replace the ”or?” at the end of my comment  with ”is that correct?”  It is an expression often used in colloquial speaking.
Answers.UserDisplayName: Barzi2001
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-16T05:40:48.770Z
Answers.LastActivityDate: 2019-08-18T13:56:54.867Z
Answers.LastEditDate: 2019-08-18T13:56:54.867Z
Answers.UserDisplayName: Basile Starynkevitch
Answers.Body: <p>Your initial statement on the core of AI is rather limited. In general, AI is concerned with modeling human behaviour either by imitation (<em>soft AI</em>) or by replicating the way human cognition works (<em>hard AI</em>). So far there have been some successes with soft AI, as computers can perform tasks that required some "intelligence", though the degree of this intelligence is questionable. This is partly due to the fact that even we as humans don't really have a clear idea what it means for a computer to "understand" something.</p>
  
  <p>But your conclusion is correct: if we build an AI system with human characteristics, then it will make mistakes, just as humans make mistakes. And any system designed by humans (or machines!) will make mistakes. However, not being able to deal with an imperfect world is not really relevant to AI alone: even systems that do not use AI methods will have to face that, and whether a system is suitable for real-time critical applications has got nothing to do with whether it is based on AI or not.</p>
  
  <p>UPDATE: There seem to be two distinct issues at play here: decidability and real-time processing.</p>
  
  <ol>
  <li><p>Real-time computing (RTC): This is not really related to AI. Even ordinary programmes written in Java are not really safe for RTC, as they could start a garbage collection cycle at any time which pauses execution of the program. Just imagine a reactor core starts overheating just as your controller runs out of memory and garbage collection kicks in, halting the program for a few minutes. If you implement AI methods in RTC-safe systems, that should not be an issue.</p></li>
  <li><p>Decidability: Your reasoning is that AI systems attempt to mirror human cognition, thus incorporating the ability to make mistakes. This is a more philosophical issue &mdash; if a human can control a system, then an AI system with the same capabilities should be able to do it too. This assumes that we are able to replicate human behaviour (which we are not). There are AI methods which are deterministic, so would come to the same conclusions given identical environments. So I would say that they would not perform worse than non-AI methods. It partly depends what you want to call AI; the distinction between traditional AI and statistical methods keeps getting blurred at present.</p></li>
  </ol>
  
  <p>To conclude: No, AI methods should be suitable, as they can also be deterministic. It depends on the actual application and method if they are. And, of course, on what you count as AI.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-08T16:38:39.217Z
Answers.LastActivityDate: 2019-08-13T14:28:34.447Z
Answers.LastEditDate: 2019-08-13T14:28:34.447Z
Answers.UserDisplayName: Oliver Mason
Answers.Body: <p>Artificial intelligence cannot be boiled down to designing algorithms, binary or otherwise, simply because the exhibition of intelligence in biological systems predated the invention of algorithmic computing.  From this, we can further draw the conclusion that algorithms are not a necessary component of systems that exhibit behavior we deem intelligent.</p>
  
  <p>A decision was made, per the recommendation of John von Neumann, to increase reliability of computing machinery by delegating to a single binary central processing unit all computation. This choice and the prior work upon which it was based (Shannon, Church, and Turing) led to the preeminence of algorithm specification in computer languages. The foundation of expressing functional design in algorithmic terms was laid and the software industry was born.</p>
  
  <p>Since that time, there has existed a parallel trend in research back toward the biological inspiration of computing machinery and, more specifically, parallel processing.  We see this at several levels.</p>
  
  <ul>
  <li>Movement of floating point arithmetic, video rendering, and machine learning bottlenecks to dedicated VLSI hardware acceleration</li>
  <li>Multiple core VLSI processors</li>
  <li>Computing clusters and processing frameworks, containers, and environments that expose interfaces through which compiler and kernel programmers can control parallel machinery explicitly or implicitly</li>
  <li>Multiple thread and processes delegated to multiple cores, agents, or hosts in computing clusters</li>
  <li>Sophisticated VLSI level caching to maximize the efficiency of parallel operations</li>
  <li>Language and compiler features to support the trends toward deployment to multiprocessing environments, such as declarative languages for Big Data platforms (ECL for example)</li>
  <li>Development of AI chip designs that completely or partially shift the computing paradigm to prior to the emergence of the CPU in some ways, returning to considerable parallelism and departing from centralized processing (yet capitalizing on lessons learned in computer vision, cognitive science, reverse engineering of brain genetics, mental signal tracing, the use of gradient descent with back propagation, reinforcement designs, and applied robotics) &mdash; This is likely a major research direction for the 2020s.</li>
  </ul>
  
  <p>Some believe that an implication of Gödel's two incompleteness theorems is that the human mind does not meet the criteria of a computing machine as Turing defined one, but these are largely tangential issues.</p>
  
  <p>It is true that The working out of a proof that RNNs of sufficient resolution, depth, and width can be trained to be equivalent to any Turing Machine by Hava Siegelmann. It is true that her work is considered support for Marvin Minski's bold assertion that the human brain is a meat machine. However, the work on determinism by John Lucas and Roger Penrose's <em>The Emperor's New Mind</em> are not refutations of either of Gödel's theorems.  They are refutations of what some thought were consequences Gödel's theorems and some of the implications of Minski's declaration.</p>
  
  <p>Gödel clearly explains his intentions in the early portion of the paper presenting the theorems, and they had nothing to do with computing.  He intended to and succeeded in proving that theorems within a concrete mathematical system not always be proven even if they are true.  Gödel's work placed unwanted doubt on the initiative to prove all remaining unproven mathematical theorems.  Mathematicians naturally tended to think of mathematics as the perfect human endeavor, and a legitimate proof of incongruity between what is true and what is provable seemed an imperfect irritation.</p>
  
  <p>Perhaps the most profound response to Gödel's incompleteness theorems came from Alan Turing, who likely deliberately placed the word Completeness in the name of his theorem.  But this was not a refutation either.  He worked around incompleteness by defining a class of mathematical operations and finite data structures upon which they can operate that he could prove could be complete. Upon doing so, he put into place an important portion of the basis for algorithm development.</p>
  
  <p>Nonetheless, it is probably wise for present day AI researchers to accept both incompleteness and inconsistency and realize that intelligence, artificial or not, is likely fallible after any finite degree of learning.  This is likely because one cannot provide an infinite range of problem types to a learning system in a finite amount of time.  There may always be at least one problem that the current state of learning cannot address.  The practical colloquialism for this condition of partial knowledge is, "We don't know what we don't know."</p>
  
  <p>Furthermore, a clear implication of the work of Gödel is that no proof may be found for some things that are true, ever, by any type of intelligence.  Similarly, we cannot be sure that the most intelligent searching for a counter example to dispute a false assertion may end in finding one, ever.  The PAC Learning framework addresses categories of problems that are solvable or not from a mathematical perspective and is worthy of study.</p>
  
  <p>Lastly, but perhaps most profoundly, it is not clear that a type of intelligence exists that can learn anything, as opposed to be programmed to accomplish anything.  Said another way, general intelligence may be an ideal conception never achieved but possibly approached. What may seem like super intelligence in one environment and during one specific time period may be entirely ineffective or even counter-intelligent and problematic in another environment or during a different time period.</p>
  
  <p>This cannot be stressed too much, with so many statements about AI being made in the guise of science that have no origin in scientific rigor.</p>
  
  <p>Nonetheless, even with these likely limitations on both AI and human intelligence, one cannot conclude that AI will be ineffective in real time critical applications.  One cannot conclude that AI will be less effective than human intelligence in any particular domain either.</p>
  
  <p>It is actually difficult to conclude anything about intelligence at all, without defining it formally and reaching a consensus in that definition, which continues to escape us.  We can see that in the absence of this formality, mail industry from continuing to sort mail automatically.  The automotive industry continues to pursue the invention of better artificial drivers than the average human driver.  The game industry implements artificial opponents that have to deliberately make mistakes to let people win in an otherwise fair, real time game.</p>
  
  <p>Clearly AI is evolving faster than the DNA components that affect the human brain.</p>
  
  <p>People are less startled today than they would have been ten years ago by the proposition that, some time during this century, driving a car will be illegal in some jurisdictions, when the human and property loss statistics prove automated drivers to be substantially safer than nearly all manual ones.  The bar for driving safety set by humans is not very high, with day dreaming, texting, occasional tiredness or inebriation slowing an already insufficient reaction time for many street events.</p>
  
  <p>If the driving computing agent panics because there is determines the trajectory of a dog, a child, and an elderly person is intersecting with the car's trajectory, it may resolve the panic and plot a safe course in a millisecond (perhaps avoiding all three or perhaps sacrificing the dog to save the two people), whereas the human may resolve the panic only after hitting some one.</p>
  
  <p>In summary, it is not infallibility that determines the proper balance or volume of AI deployment but the comparison of the distribution of human performances compared with the distribution found with the machine replacements under similar conditions.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-17T18:10:40.850Z
Answers.LastActivityDate: 2019-08-18T11:25:35.303Z
Answers.LastEditDate: 2019-08-18T11:25:35.303Z
Answers.UserDisplayName: Douglas Daseeco
Body: <p>So far I understand - I know very little on the topic - the core of AI boils down to design algorithms that shall provide a TRUE/FALSE answer to a given statement. Nevertheless, I am aware of the limitations provided by the Gödel's incomplete theorems but I am also aware that there have been long debates such as the Lucas and Penrose arguments with all the consequent objections during the past 60 years. </p>
  
  <p>The conclusion is, in my understanding, that to create AI systems we must accept incompleteness or inconsistency. </p>
  
  <p>Does that mean that intelligence systems (including artificial ones), like humans, may end up in some undecidable situation that may lead to take a wrong decision? </p>
  
  <p>If this may be acceptable in some application (for example, if every once in a while a spam email ends up in the inbox folder - or vice versa - despite an AI-based anti-spam filter) in some other application it may not. I am referring to real-time critical applications when a "wrong" action from a machine may harm people. </p>
  
  <p>Does that mean that AI will never be employed for real-time critical applications? </p>
  
  <p>Would in that case more safe to use deterministic methods that do not leave room for any kind of undecidability?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-08-15T19:07:25.197Z
Comments.Text: An example of an undecidable problem, from my own experience, is selecting a meal off the menu at a restaurant.
Comments.UserDisplayName: Somatic Custard
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-13T20:07:30.987Z
Comments.Text: Please, next time, **focus on one single question/problem per post**.
Comments.UserDisplayName: nbro
Tags: <philosophy><incompleteness-theorems>
Title: Do Gödel's theorems imply that intelligence systems may end up in some undecidable situation (that may make them take a wrong decision)?
UserDisplayName: Barzi2001
ThreadText: # Do Gödel's theorems imply that intelligence systems may end up in some undecidable situation (that may make them take a wrong decision)?
  
  (Asked by: username_0 on 2019-08-08)
  
  So far I understand - I know very little on the topic - the core of AI boils down to design algorithms that shall provide a TRUE/FALSE answer to a given statement. Nevertheless, I am aware of the limitations provided by the Gödel's incomplete theorems but I am also aware that there have been long debates such as the Lucas and Penrose arguments with all the consequent objections during the past 60 years.
  
  The conclusion is, in my understanding, that to create AI systems we must accept incompleteness or inconsistency.
  
  Does that mean that intelligence systems (including artificial ones), like humans, may end up in some undecidable situation that may lead to take a wrong decision?
  
  If this may be acceptable in some application (for example, if every once in a while a spam email ends up in the inbox folder - or vice versa - despite an AI-based anti-spam filter) in some other application it may not. I am referring to real-time critical applications when a "wrong" action from a machine may harm people.
  
  Does that mean that AI will never be employed for real-time critical applications?
  
  Would in that case more safe to use deterministic methods that do not leave room for any kind of undecidability?
  
  **Tags:** philosophy, incompleteness-theorems
  
  > **Comments:**
  > **username_1**: An example of an undecidable problem, from my own experience, is selecting a meal off the menu at a restaurant.  
  > **username_2**: Please, next time, **focus on one single question/problem per post**.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_3 on 2019-08-16. Score: 1 | Accepted answer
  
  Your question is mostly philosophical, not technical or scientific. So I am giving opinions and references here.
  
  > the core of AI boils down to design algorithms
  
  I am noticing that you are not even try to define [AI](https://en.wikipedia.org/wiki/Artificial_intelligence) (whose definition changed since the previous century). You could look at the table of contents of the [*Artificial Intelligence* journal](https://www.journals.elsevier.com/artificial-intelligence) and notice how topics covered there changed drastically in a few decades (even experimental approaches have declined).
  
  You might be interested in reading more about [AGI](https://en.wikipedia.org/wiki/Artificial_general_intelligence) and follow the few conferences about it. Beware, there is a lot of too much simplified approaches, and even a lot of bullshit (e.g. on [this](https://agi.topicbox.com/groups/agi) AGI mailing list, but some messages there are gems)
  
  I assume you accept the [Church-Turing](https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis) philosophical thesis: every intelligent cognition (either natural i.e. biological or artificial) is some symbolic computation. In particular, the work of a mathematician can be abstracted as a Turing machine (that was the major insight of Turing and in the [halting problem](https://en.wikipedia.org/wiki/Halting_problem)). Be also aware of the related [Curry-Howard](https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspondence) correspondence and [Rice's theorem](https://en.wikipedia.org/wiki/Rice%27s_theorem). Read [*Gödel, Escher, Bach*](https://en.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach) !
  
  We don't know yet how to make AGI. You could read Bostrom's [*SuperIntelligence*](https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies) book about potential dangers. You could also read J.Pitrat's book [*Artificial Beings*](https://dl.acm.org/citation.cfm?id=1717130) (which gives much more positive and constructive insights about eventually making some AGI) and [blog](http://bootstrappingartificialintelligence.fr/WordPress3/).
  
  My personal belief (just an opinion) is that AGI could be perhaps achieved (in many dozens of years), should be definitely get much more funding -and more time- as a research topic (e.g. as much as the [ITER](https://en.wikipedia.org/wiki/ITER) reactor; see also [softwareheritage.org](https://www.softwareheritage.org/) and the motivations there), but won't be achieved by any single technique, but by a clever combination of many AI techniques (both [symbolic AI](https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence) -e.g. for [planning](https://en.wikipedia.org/wiki/Planning)- and [machine learning](https://en.wikipedia.org/wiki/Machine_learning) or [connectionnist](https://en.wikipedia.org/wiki/Connectionism) approaches, with inspiration from cognitive psychology).
  
  > The conclusion is, in my understanding, that to create AI systems we must accept incompleteness or inconsistency.
  
  We, members of the [Homo Sapiens Sapiens](https://en.wikipedia.org/wiki/Human_taxonomy#Subspecies) species (in latin, the humans who know that they know, so capable of [metaknowledge](https://en.wikipedia.org/wiki/Metaknowledge)), claim to be intelligent. But all of us have a globally incomplete and inconsistent behavior, because each of us have contradictions (e.g. in our personal lives or [ethical](https://en.wikipedia.org/wiki/Ethical_dilemma) beliefs). So, logically speaking, incompleteness or inconsistency is not opposed to intelligence. Read also more about [situated AI](https://en.wikipedia.org/wiki/Artificial_intelligence,_situated_approach) and [machine ethics](https://en.wikipedia.org/wiki/Machine_ethics). BTW, I believe (since educated by J.Pitrat about this) that explicit and [declarative](https://en.wikipedia.org/wiki/Descriptive_knowledge) [meta](https://www.coursehero.com/file/ptcioic/CS-484-Artificial-Intelligence-8-Meta-Rules-The-rules-that-determine-the/)knowledge is required in any AGI system.
  
  > Does that mean that AI will never be employed for real-time critical applications?
  
  Notice that [autonomous killing robots](https://en.wikipedia.org/wiki/Lethal_autonomous_weapon) are already a controversial research topic today. [Autonomous robots](https://en.wikipedia.org/wiki/Autonomous_robot) already exist (e.g. [Mars rovers](https://en.wikipedia.org/wiki/Mars_rover) cannot be teleoperated -for every elementary movement- from Earth, because any radio signal takes minutes to reach Mars). And autonomous vehicles (à la [Google car](https://en.wikipedia.org/wiki/Waymo)) claim today to use AI techniques and are [real-time](https://en.wikipedia.org/wiki/Real-time_computing) [safety-critical systems](https://en.wikipedia.org/wiki/Safety-critical_system). Today's Airbus or Boeing (cf [DO-178C](https://en.wikipedia.org/wiki/DO-178C)) are flying automatically most of the time. [Cruise missiles](https://en.wikipedia.org/wiki/Cruise_missile) and [ICBM](https://en.wikipedia.org/wiki/Intercontinental_ballistic_missile)s are [fire-and-forget](https://en.wikipedia.org/wiki/Fire-and-forget) devices. Many [high-frequency trading](https://en.wikipedia.org/wiki/High-frequency_trading) systems claim to use AI techniques and are real-time.
  
  PS. Notice that what was called AI in the previous century is today called AGI. My PhD in AI was defended in 1990 (and was about explicit metaknowledge for metaprogramming goals, see e.g. [this](https://inis.iaea.org/collection/NCLCollectionStore/_Public/19/059/19059867.pdf?r=1&r=1) old 1987 paper)
  
  > **Comments:**
  > **username_0**: Thanks for the references! I refer to AI  as "machines (or computers) that mimic "cognitive" functions that humans associate with the human mind, such as "learning" and "problem solving"" (Wikipedia). AGI is also known as Strong AI, but I don't mind weak or strong AI for my argument. Yes, I assume the Chruch-Turing thesis. I further agree that intelligence is not opposed to incompleteness or inconsistency. But then AI is also incomplete and inconsistent. This means that it can make evaluation mistakes and that may not be socially accepted for some apps. Think e.g. to FULLY autonomous drive.  
  > **username_0**: I just read about a crush of a Google self-driving car that "The company also stated "This type of misunderstanding happens between human drivers on the road every day"" (Wikipedia). This episode (along with Google answer) would strengthen the hypothesis that an A(G)I system could make evaluation errors and could potentially harm human, or?  
  > **username_3**: If you feell my answer relevant to your question, consider upvoting it. I don't read well enough your English (since my native language is French, and my parents spoke to me in Russian) to understand what the ending "or?" means in your comment above  
  > **username_0**: Your answer was actually  very useful. You can replace the ”or?” at the end of my comment  with ”is that correct?”  It is an expression often used in colloquial speaking.  
  
  ---
  
  ### Answer by username_4 on 2019-08-08. Score: 4
  
  Your initial statement on the core of AI is rather limited. In general, AI is concerned with modeling human behaviour either by imitation (*soft AI*) or by replicating the way human cognition works (*hard AI*). So far there have been some successes with soft AI, as computers can perform tasks that required some "intelligence", though the degree of this intelligence is questionable. This is partly due to the fact that even we as humans don't really have a clear idea what it means for a computer to "understand" something.
  
  But your conclusion is correct: if we build an AI system with human characteristics, then it will make mistakes, just as humans make mistakes. And any system designed by humans (or machines!) will make mistakes. However, not being able to deal with an imperfect world is not really relevant to AI alone: even systems that do not use AI methods will have to face that, and whether a system is suitable for real-time critical applications has got nothing to do with whether it is based on AI or not.
  
  UPDATE: There seem to be two distinct issues at play here: decidability and real-time processing.
  
  1. Real-time computing (RTC): This is not really related to AI. Even ordinary programmes written in Java are not really safe for RTC, as they could start a garbage collection cycle at any time which pauses execution of the program. Just imagine a reactor core starts overheating just as your controller runs out of memory and garbage collection kicks in, halting the program for a few minutes. If you implement AI methods in RTC-safe systems, that should not be an issue.
  2. Decidability: Your reasoning is that AI systems attempt to mirror human cognition, thus incorporating the ability to make mistakes. This is a more philosophical issue — if a human can control a system, then an AI system with the same capabilities should be able to do it too. This assumes that we are able to replicate human behaviour (which we are not). There are AI methods which are deterministic, so would come to the same conclusions given identical environments. So I would say that they would not perform worse than non-AI methods. It partly depends what you want to call AI; the distinction between traditional AI and statistical methods keeps getting blurred at present.
  
  To conclude: No, AI methods should be suitable, as they can also be deterministic. It depends on the actual application and method if they are. And, of course, on what you count as AI.
  
  
  ---
  
  ### Answer by username_5 on 2019-08-17. Score: 2
  
  Artificial intelligence cannot be boiled down to designing algorithms, binary or otherwise, simply because the exhibition of intelligence in biological systems predated the invention of algorithmic computing. From this, we can further draw the conclusion that algorithms are not a necessary component of systems that exhibit behavior we deem intelligent.
  
  A decision was made, per the recommendation of John von Neumann, to increase reliability of computing machinery by delegating to a single binary central processing unit all computation. This choice and the prior work upon which it was based (Shannon, Church, and Turing) led to the preeminence of algorithm specification in computer languages. The foundation of expressing functional design in algorithmic terms was laid and the software industry was born.
  
  Since that time, there has existed a parallel trend in research back toward the biological inspiration of computing machinery and, more specifically, parallel processing. We see this at several levels.
  
  * Movement of floating point arithmetic, video rendering, and machine learning bottlenecks to dedicated VLSI hardware acceleration
  * Multiple core VLSI processors
  * Computing clusters and processing frameworks, containers, and environments that expose interfaces through which compiler and kernel programmers can control parallel machinery explicitly or implicitly
  * Multiple thread and processes delegated to multiple cores, agents, or hosts in computing clusters
  * Sophisticated VLSI level caching to maximize the efficiency of parallel operations
  * Language and compiler features to support the trends toward deployment to multiprocessing environments, such as declarative languages for Big Data platforms (ECL for example)
  * Development of AI chip designs that completely or partially shift the computing paradigm to prior to the emergence of the CPU in some ways, returning to considerable parallelism and departing from centralized processing (yet capitalizing on lessons learned in computer vision, cognitive science, reverse engineering of brain genetics, mental signal tracing, the use of gradient descent with back propagation, reinforcement designs, and applied robotics) — This is likely a major research direction for the 2020s.
  
  Some believe that an implication of Gödel's two incompleteness theorems is that the human mind does not meet the criteria of a computing machine as Turing defined one, but these are largely tangential issues.
  
  It is true that The working out of a proof that RNNs of sufficient resolution, depth, and width can be trained to be equivalent to any Turing Machine by Hava Siegelmann. It is true that her work is considered support for Marvin Minski's bold assertion that the human brain is a meat machine. However, the work on determinism by John Lucas and Roger Penrose's *The Emperor's New Mind* are not refutations of either of Gödel's theorems. They are refutations of what some thought were consequences Gödel's theorems and some of the implications of Minski's declaration.
  
  Gödel clearly explains his intentions in the early portion of the paper presenting the theorems, and they had nothing to do with computing. He intended to and succeeded in proving that theorems within a concrete mathematical system not always be proven even if they are true. Gödel's work placed unwanted doubt on the initiative to prove all remaining unproven mathematical theorems. Mathematicians naturally tended to think of mathematics as the perfect human endeavor, and a legitimate proof of incongruity between what is true and what is provable seemed an imperfect irritation.
  
  Perhaps the most profound response to Gödel's incompleteness theorems came from Alan Turing, who likely deliberately placed the word Completeness in the name of his theorem. But this was not a refutation either. He worked around incompleteness by defining a class of mathematical operations and finite data structures upon which they can operate that he could prove could be complete. Upon doing so, he put into place an important portion of the basis for algorithm development.
  
  Nonetheless, it is probably wise for present day AI researchers to accept both incompleteness and inconsistency and realize that intelligence, artificial or not, is likely fallible after any finite degree of learning. This is likely because one cannot provide an infinite range of problem types to a learning system in a finite amount of time. There may always be at least one problem that the current state of learning cannot address. The practical colloquialism for this condition of partial knowledge is, "We don't know what we don't know."
  
  Furthermore, a clear implication of the work of Gödel is that no proof may be found for some things that are true, ever, by any type of intelligence. Similarly, we cannot be sure that the most intelligent searching for a counter example to dispute a false assertion may end in finding one, ever. The PAC Learning framework addresses categories of problems that are solvable or not from a mathematical perspective and is worthy of study.
  
  Lastly, but perhaps most profoundly, it is not clear that a type of intelligence exists that can learn anything, as opposed to be programmed to accomplish anything. Said another way, general intelligence may be an ideal conception never achieved but possibly approached. What may seem like super intelligence in one environment and during one specific time period may be entirely ineffective or even counter-intelligent and problematic in another environment or during a different time period.
  
  This cannot be stressed too much, with so many statements about AI being made in the guise of science that have no origin in scientific rigor.
  
  Nonetheless, even with these likely limitations on both AI and human intelligence, one cannot conclude that AI will be ineffective in real time critical applications. One cannot conclude that AI will be less effective than human intelligence in any particular domain either.
  
  It is actually difficult to conclude anything about intelligence at all, without defining it formally and reaching a consensus in that definition, which continues to escape us. We can see that in the absence of this formality, mail industry from continuing to sort mail automatically. The automotive industry continues to pursue the invention of better artificial drivers than the average human driver. The game industry implements artificial opponents that have to deliberately make mistakes to let people win in an otherwise fair, real time game.
  
  Clearly AI is evolving faster than the DNA components that affect the human brain.
  
  People are less startled today than they would have been ten years ago by the proposition that, some time during this century, driving a car will be illegal in some jurisdictions, when the human and property loss statistics prove automated drivers to be substantially safer than nearly all manual ones. The bar for driving safety set by humans is not very high, with day dreaming, texting, occasional tiredness or inebriation slowing an already insufficient reaction time for many street events.
  
  If the driving computing agent panics because there is determines the trajectory of a dog, a child, and an elderly person is intersecting with the car's trajectory, it may resolve the panic and plot a safe course in a millisecond (perhaps avoiding all three or perhaps sacrificing the dog to save the two people), whereas the human may resolve the panic only after hitting some one.
  
  In summary, it is not infallibility that determines the proper balance or volume of AI deployment but the comparison of the distribution of human performances compared with the distribution found with the machine replacements under similar conditions.
  
  
  ---
  
  