Answers.Body: <p>AI is internally limited by model and externally limited by the environment.</p>
  <p>Humans are externally limited by the environment but not necessarily internally limited by a computable model (as AI is).</p>
  <p>So, humans may possess certain skills (e.g. creativity) that an AI may never possess. I had previously asked a related question <a href="https://ai.stackexchange.com/q/10139/2444">Are human brain processes, like creativity, intuition or imagination, computable processes?</a>.</p>
  <p><em>Which research work supports my claims?</em></p>
  <p>Brian Cantwell Smith says that there is no computation without representation (a model).</p>
  <p>In the article <a href="https://www.technologyreview.com/s/511421/the-brain-is-not-computable/" rel="nofollow noreferrer">The Brain Is Not Computable</a>, Miguel Nicolelis, a top neuroscientist at Duke University, also says</p>
  <blockquote>
  <p>The brain is not computable and no engineering can reproduce it</p>
  <p>You can't predict whether the stock market will go up or down because you can’t compute it.</p>
  <p>You could have all the computer chips ever in the world and you won't create a consciousness.</p>
  <p>That's because its most important features are the result of unpredictable, nonlinear interactions among billions of cells</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-11T03:11:04.143Z
Answers.LastActivityDate: 2020-03-11T13:05:38.663Z
Answers.LastEditDate: 2020-06-17T09:57:20.710Z
Answers.UserDisplayName: alamata
Answers.Body: <p>There's a neuroscience theory, known as <a href="https://www.frontiersin.org/research-topics/599/predictive-coding" rel="nofollow noreferrer">predictive coding</a>, which roughly states that the (human) brain is constantly generating and updating a model of the world.</p>
  <blockquote>
  <p>The brain is constantly confronted with a wealth of sensory information that must be processed efficiently to facilitate appropriate reactions. One way of optimizing this processing effort is to predict incoming sensory information based on previous experience so that expected information is processed efficiently and resources can be allocated to novel or surprising information. Theoretical and computational studies led to the formulation of the predictive coding framework (Friston 2005, Hawkins and Blakeslee 2004, Mumford 1992, Rao and Ballard 1999). Predictive coding states that the brain continually generates models of the world based on context and information from memory to predict sensory input. In terms of brain processing, a predictive model is created in higher cortical areas and communicated through feedback connections to lower sensory areas. In contrast, feedforward connections process and project an error signal, i.e. the mismatch between the predicted information and the actual sensory input (Rao &amp; Ballard, 1999). The predictive model is constantly updated according to this error signal.</p>
  </blockquote>
  <p>This theory should not be surprising or unintuitive, given that every person possesses a slightly different perspective (or model) of the world, which is based on her (or his) personal experiences. Of course, this is just a theory, which may not be the most precise one that describes our brain, but this theory is already being <a href="https://www.frontiersin.org/research-topics/599/predictive-coding" rel="nofollow noreferrer">validated by a number of brain imaging studies investigating predictive feedback and the processing of prediction errors.</a></p>
  <p>Therefore, artificial intelligence may not be the only entity that is based on or will be limited by a model of the world. To answer your question more directly, yes, the AI will always be limited by its model and environment (e.g. hardware), in a similar way that <a href="https://en.wikipedia.org/wiki/Flatland" rel="nofollow noreferrer">flatlanders</a> are limited by their 2-dimensional nature and world, but this doesn't necessarily mean we will not be able to create useful (and even sophisticated or human-like) AI systems.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-10T23:52:17.530Z
Answers.LastActivityDate: 2021-02-07T22:20:05.943Z
Answers.LastEditDate: 2021-02-07T22:20:05.943Z
Answers.UserDisplayName: nbro
Body: <p>In section 3 of the paper <a href="https://www.student.cs.uwaterloo.ca/~cs492/11public_html/p18-smith.pdf" rel="nofollow noreferrer">The Limits of Correctness</a> (1985) <a href="https://en.wikipedia.org/wiki/Brian_Cantwell_Smith" rel="nofollow noreferrer">Brian Cantwell Smith</a> writes</p>
  
  <blockquote>
    <p>When you design and build a computer system, you first formulate a model of the problem you want it to solve, and then construct the computer program in its terms.</p>
  </blockquote>
  
  <p>He then writes</p>
  
  <blockquote>
    <p>computers have a special dependence on these models: you write an explicit description of the model down inside the computer, in the form of a set
    of rules or what are called representations - essentially linguistic formulae encoding, in the terms of the model, the facts and data
    thought to be relevant to the system's behavior. It is with respect to these representations that computer systems work. In fact, that's really what computers are (and how they differ from other machines): they run by manipulating representations, and representations are always formulated
    in terms of models. This can all be summarized in a slogan: no computation without representation.</p>
  </blockquote>
  
  <p>And then he says</p>
  
  <blockquote>
    <p>Models have to ignore things exactly because they view the world at a level of abstraction</p>
  </blockquote>
  
  <p>He then writes in section 7</p>
  
  <blockquote>
    <p>The systems that land airplanes are hybrids - combinations of computers and people - exactly because the unforeseeable happens, and because what
    happens is in part the result of human action, requiring human interpretation</p>
  </blockquote>
  
  <p>As quoted above, computers depend on models, which are abstractions (i.e. they ignore a lot of details), which are written inside the computer. Therefore, the true world cannot really be encoded into an algorithm, but only an abstraction and thus simplification of the world can.</p>
  
  <p>So, will AI always depend on models and thus approximations? Can it get rid of or overcome this limitation?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-08T02:01:02.153Z
Comments.Text: @alamata I guess this is more or less open for debate at this point. I mean whether all that brain does is some sort of computation and communication between neurons. Saying that, it certainly does some sort of computation AFAIK.
Comments.UserDisplayName: SpiderRico
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-13T16:10:03.427Z
Comments.Text: We can have pretty good interpretations that are not perfect, but close to it. These are able to be used to form close to perfect results.
Comments.UserDisplayName: Lustwelpintje
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-13T19:03:49.540Z
Comments.Text: @Lustwelpintje Are human brain processes, like creativity, intuition or imagination,  computable processes? If not, I think AI will be limited by the abstractions of the models.
Comments.UserDisplayName: alamata
Tags: <philosophy><agi><models><papers>
Title: Will AI always depend on models and thus approximations?
UserDisplayName: alamata
ThreadText: # Will AI always depend on models and thus approximations?
  
  (Asked by: username_0 on 2020-02-13)
  
  In section 3 of the paper [The Limits of Correctness](https://www.student.cs.uwaterloo.ca/~cs492/11public_html/p18-smith.pdf) (1985) [Brian Cantwell Smith](https://en.wikipedia.org/wiki/Brian_Cantwell_Smith) writes
  
  > When you design and build a computer system, you first formulate a model of the problem you want it to solve, and then construct the computer program in its terms.
  
  He then writes
  
  > computers have a special dependence on these models: you write an explicit description of the model down inside the computer, in the form of a set
  > of rules or what are called representations - essentially linguistic formulae encoding, in the terms of the model, the facts and data
  > thought to be relevant to the system's behavior. It is with respect to these representations that computer systems work. In fact, that's really what computers are (and how they differ from other machines): they run by manipulating representations, and representations are always formulated
  > in terms of models. This can all be summarized in a slogan: no computation without representation.
  
  And then he says
  
  > Models have to ignore things exactly because they view the world at a level of abstraction
  
  He then writes in section 7
  
  > The systems that land airplanes are hybrids - combinations of computers and people - exactly because the unforeseeable happens, and because what
  > happens is in part the result of human action, requiring human interpretation
  
  As quoted above, computers depend on models, which are abstractions (i.e. they ignore a lot of details), which are written inside the computer. Therefore, the true world cannot really be encoded into an algorithm, but only an abstraction and thus simplification of the world can.
  
  So, will AI always depend on models and thus approximations? Can it get rid of or overcome this limitation?
  
  **Tags:** philosophy, agi, models, papers
  
  > **Comments:**
  > **username_2**: We can have pretty good interpretations that are not perfect, but close to it. These are able to be used to form close to perfect results.  
  > **username_0**: @username_2 Are human brain processes, like creativity, intuition or imagination,  computable processes? If not, I think AI will be limited by the abstractions of the models.  
  > **username_1**: @username_0 I guess this is more or less open for debate at this point. I mean whether all that brain does is some sort of computation and communication between neurons. Saying that, it certainly does some sort of computation AFAIK.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_0 on 2020-03-11. Score: 1
  
  AI is internally limited by model and externally limited by the environment.
  
  Humans are externally limited by the environment but not necessarily internally limited by a computable model (as AI is).
  
  So, humans may possess certain skills (e.g. creativity) that an AI may never possess. I had previously asked a related question [Are human brain processes, like creativity, intuition or imagination, computable processes?](https://ai.stackexchange.com/q/10139/2444).
  
  *Which research work supports my claims?*
  
  Brian Cantwell Smith says that there is no computation without representation (a model).
  
  In the article [The Brain Is Not Computable](https://www.technologyreview.com/s/511421/the-brain-is-not-computable/), Miguel Nicolelis, a top neuroscientist at Duke University, also says
  
  > The brain is not computable and no engineering can reproduce it
  >
  > You can't predict whether the stock market will go up or down because you can’t compute it.
  >
  > You could have all the computer chips ever in the world and you won't create a consciousness.
  >
  > That's because its most important features are the result of unpredictable, nonlinear interactions among billions of cells
  
  
  ---
  
  ### Answer by username_3 on 2020-03-10. Score: 1
  
  There's a neuroscience theory, known as [predictive coding](https://www.frontiersin.org/research-topics/599/predictive-coding), which roughly states that the (human) brain is constantly generating and updating a model of the world.
  
  > The brain is constantly confronted with a wealth of sensory information that must be processed efficiently to facilitate appropriate reactions. One way of optimizing this processing effort is to predict incoming sensory information based on previous experience so that expected information is processed efficiently and resources can be allocated to novel or surprising information. Theoretical and computational studies led to the formulation of the predictive coding framework (Friston 2005, Hawkins and Blakeslee 2004, Mumford 1992, Rao and Ballard 1999). Predictive coding states that the brain continually generates models of the world based on context and information from memory to predict sensory input. In terms of brain processing, a predictive model is created in higher cortical areas and communicated through feedback connections to lower sensory areas. In contrast, feedforward connections process and project an error signal, i.e. the mismatch between the predicted information and the actual sensory input (Rao & Ballard, 1999). The predictive model is constantly updated according to this error signal.
  
  This theory should not be surprising or unintuitive, given that every person possesses a slightly different perspective (or model) of the world, which is based on her (or his) personal experiences. Of course, this is just a theory, which may not be the most precise one that describes our brain, but this theory is already being [validated by a number of brain imaging studies investigating predictive feedback and the processing of prediction errors.](https://www.frontiersin.org/research-topics/599/predictive-coding)
  
  Therefore, artificial intelligence may not be the only entity that is based on or will be limited by a model of the world. To answer your question more directly, yes, the AI will always be limited by its model and environment (e.g. hardware), in a similar way that [flatlanders](https://en.wikipedia.org/wiki/Flatland) are limited by their 2-dimensional nature and world, but this doesn't necessarily mean we will not be able to create useful (and even sophisticated or human-like) AI systems.
  
  
  ---
  
  