Answers.Body: <p>Judea Pearl's 2018 comment on ACM.org, in his <a href="https://cacm.acm.org/news/227877-to-build-truly-intelligent-machines-teach-them-cause-and-effect/fulltext" rel="nofollow noreferrer"><em>To, Build Truly Intelligent Machines, Teach Them Cause and Effect</em></a> is piercing truth.</p>
  <blockquote>
  <p>All the impressive achievements of deep learning amount to just curve fitting.</p>
  </blockquote>
  <p>It may be less sensational and more technically correct to state that it is not, &quot;Just curve fitting,&quot; but rather, &quot;sophisticated surface fitting.&quot; Nonetheless, his general assessment indicates the need to look beyond tuning nonlinear functions to fit a surface in <span class="math-container">$\mathbb{R}^n$</span> and consider whether cognition is achievable with a deep network. The split in answers to this question is odd. We have two conflicting assertions, often strongly stated.</p>
  <ol>
  <li>Artificial networks cannot perform logic.</li>
  <li>Artificial networks are the best approach to AI.</li>
  </ol>
  <p>How can rationality be excluded from the list of important human features of intelligence, which is what these two assertions are taken together would mean?</p>
  <p>Is the human brain a network of sophisticated curve fitters?  Marvin Minsky's famous quote, &quot;The brain happens to be a meat machine,&quot; was offered without proof, and neither a proof of his trivialization of the human brain nor a proof that the brain is beyond the reach of Turing computability has been offered since.</p>
  <p>When you read these words, are your neural networks doing the following sequence of curve fits?</p>
  <ul>
  <li>Edges from retinal rods and cones</li>
  <li>Lines from edges</li>
  <li>Shapes from lines</li>
  <li>Letters from shapes</li>
  <li>Linguistic elements from groups of letters</li>
  <li>Linguistic structures from elements</li>
  <li>Understanding from linguistic structures</li>
  </ul>
  <p>The case is strong for the affirmation that the first five is a convergence mechanism on a model, and all the machine learning structure is just a method to fit the data to the model.</p>
  <p>Those last two bullet items are where the paradigm breaks down and where many AI researchers and authors have correctly stated that machine learning has significant limitations when based solely on layers of multi-layer perceptrons and convolution kernels.  Furthermore, the last bullet item is grossly oversimplified in its current state, probably by orders of magnitude.  Even if Minsky is correct that a computer can perform what the brain does, the process of reading and understanding this paragraph could easily have a thousand different kinds of unique process components in patterns of internal workflow with massive parallelism.  Imaging technology indicates this probability.  We have computers modelling only the simplest peripheral layers.</p>
  <blockquote>
  <p>Is there any scientific/mathematical argument that prevents deep learning from ever producing strong AI? — No. But there is no such argument that guarantees it either.</p>
  </blockquote>
  <p>Other questions here investigate whether these sophisticated curve fitters can perform elements of cognition or reasoning.</p>
  <ul>
  <li><a href="https://ai.stackexchange.com/questions/3389/could-a-neural-network-detect-primes">Could a neural network detect primes?</a></li>
  <li><a href="https://ai.stackexchange.com/questions/7416/can-deep-networks-be-trained-to-prove-theorems">Can neural networks be used to prove conjectures?</a></li>
  </ul>
  <p>The totem of three in the question's image, seeing, doing, and imagining, is not particularly complete, accurate, or insightful.</p>
  <ul>
  <li>There are at least five sensory paradigms in humans, not one</li>
  <li>Doing preceded human senses by billions of years — bacteria do</li>
  <li>Imagining is not a significantly higher process than scenario replay from models of past experience with some method to apply set functions to combine them and inject random mutations</li>
  <li>Creativity may just be imagining in the previous bullet item followed by weeding out useless imagination results with some market-oriented quality criteria, leaving the impressive creative products that sell</li>
  </ul>
  <p>The higher forms are appreciation, a sense of realities beyond the scope of scientific measurement, legitimate doubt, love, sacrifice for the good of others or humanity.</p>
  <p>Many recognize that the current state of AI technology is nowhere near the procurement of a system that can reliably answer, &quot;How can I make Y happen?&quot; or &quot;If I have acted differently, will X still occur?&quot;</p>
  <p>There is no mathematical proof that some combination of small curve fitting elements can or cannot achieve the ability to answer those questions as well a typical human being can, mostly because there is insufficient understanding of what intelligence is or how to define it in mathematical terms.</p>
  <p>It is also possible that human intelligence doesn't exist at all, that references to it are based on a religious belief that we are higher as a species than other species.  That we can populate, consume, and exterminate is not actually a very intelligent conception of intelligence.</p>
  <p>The claim that human intelligence is an adaptation that differentiates us from other mammals conflicts with whether we adapt well.  We have not been tested.  Come the next meteoric global killer with a shock wave of the magnitude of that of the Chicxulub crater's meteor, followed by a few and a thousand years of solar winter and we'll see whether it is our 160,000-year existence or bacteria's 4,000,000,000 year existence that proves more sustainable.  In the timeline of life, human intelligence has yet to prove itself significant as an adaptive trait.</p>
  <p>What is clear about AI development is that other kinds of systems are playing a role along with deep learners based on the multi-layer perceptron concept and convolution kernels which are strictly surface fitters.</p>
  <p>Q-learning components, attention-based components, and long-short term memory components are all strictly a surface fitter too, but only by stretching the definition of surface fitting considerably.  They have real-time adaptive properties and state, so they can be Turing complete.</p>
  <p>Fuzzy logic containers, rules-based systems, algorithms with Markovian properties, and many other component types also play their role and are not surface fitters at all.</p>
  <p>In summary, there are points made that have a basis in more than plausibility or a pleasing intuitive quality, however, many of these authors do not provide a mathematical framework with definitions, applications, lemmas, theorems, proofs, or even thought experiments that can be scrutinized in a formal way.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-13T13:08:47.460Z
Answers.LastActivityDate: 2021-04-04T15:14:34.153Z
Answers.LastEditDate: 2021-04-04T15:14:34.153Z
Answers.UserDisplayName: Douglas Daseeco
Answers.Body: <p>It is a paradox, but a deep learning machine (defined as a NeuralNet variant) is unable to learn anything. It is a flexible and configurable hardware/software architecture that can be parametrized to solve a lot of problems. But the optimal parameters to solve a problem are obtained by an external system, i.e. back-propagation algorithm. </p>
  
  <p>Back-propagation subsystem uses conventional programming paradigms, it is not a Neural Net. This fact is in absolute opposition to human mind, where learning and use of knowledge is done by the same system (the mind).</p>
  
  <p>If all the real interesting things are done outside the NN, it is difficult to claim that a NN (in any variant) can develop in an AGI.</p>
  
  <p>It is also possible to find some more differences. Neural nets are strongly numerical in its interface and internals. From this point of view, they are an evolution of support vector machines. </p>
  
  <p>Too much differences and restrictions to expect an AGI.</p>
  
  <p>Note: I strongly disagree in the draw included in the original question. "Seeing", "doing", "imaging" are levels absolutely wrong. It ignores from basic and common software concepts as "abstraction" or "program state" (of mind, in Turing words); applied AI ones as "foresee"; and AGI ones as "free will", "objectives and feelings", ... </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-15T15:50:44.420Z
Answers.Text: I think removing back-propagation (or any part of the training framework) from consideration, and claiming that the remaining part **is** the "Deep Learning" part is artificial, and kind of dodging the question. I think it is reasonable to assume that OP means Deep Learning as it is practiced, including the training processes available.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-15T15:58:08.343Z
Answers.Text: @NeilSlater: If we say a DL is a kind of NeuralNet ( in order to remark the similarity with the human mind and, as consequence, its possible power to reach an AGI) , we are excluding the learning part, that is not a NN. If we include the learning subsystem in the definition of a DL, then it is not a NN, it is only conventional programming, with the power of any conventional program, and has the same posibilities to reach AGI than any other program system or paradigm.
Answers.UserDisplayName: pasaba por aqui
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-18T23:40:06.117Z
Answers.Text: Human brains learn by receiving and processing input from external 'systems' exclusively. The optimal parameters to solve problems are obtained via trial and error, applying rules and processing input from external systems. Training starts while the baby is still in the uterus and continues 24/7 thereafter. The current state of AI is almost certainly not comparable to emulating the human brain; but claiming that AI can't learn (or isn't already learning in a similar way to the human brain) assumes knowledge of how the human brain 'learns' and functions that science doesn't know yet.
Answers.UserDisplayName: Dunk
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-03T15:56:39.287Z
Answers.Text: This doesn't make a lot of sense once you think about what "learning" is. Human brains also learn by adjusting connection weights in fixed hardware. So why aren't they disqualified? (Even when new connections are formed, that's just an update of a weight that was previously 0)
Answers.UserDisplayName: Criticize SE actions means ban
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-15T15:03:03.097Z
Answers.LastActivityDate: 2018-06-15T16:16:05.243Z
Answers.LastEditDate: 2018-06-15T16:16:05.243Z
Answers.UserDisplayName: pasaba por aqui
Body: <p>I read Judea Pearl's <a href="https://rads.stackoverflow.com/amzn/click/com/046509760X" rel="nofollow noreferrer" rel="nofollow noreferrer">The Book of Why</a>, in which he mentions that deep learning is just a glorified curve fitting technology, and will not be able to produce human-like intelligence.</p>
  <p>From his book there is this diagram that illustrates the three levels of cognitive abilities:</p>
  <p><a href="https://i.sstatic.net/vxTLx.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/vxTLx.png" alt="Three levels of cognitive abilities" /></a></p>
  <p>The idea is that the &quot;intelligence&quot; produced by current deep learning technology is only at the level of association. Thus the AI is nowhere near the level of asking questions like &quot;how can I make Y happen&quot; (intervention) and &quot;What if I have acted differently, will X still occur?&quot; (counterfactuals), and it's highly unlikely that curve fitting techniques can ever bring us closer to a higher level of cognitive ability.</p>
  <p>I found his argument persuasive on an intuitive level, but I'm unable to find any physical or mathematical laws that can either bolster or cast doubt on this argument.</p>
  <p>So, is there any scientific/physical/chemical/biological/mathematical argument that prevents deep learning from ever producing strong AI (human-like intelligence)?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-06-15T12:29:16.670Z
Comments.Text: One of the problems faced in answering this is, is that "Deep Learning" is a kind of brand name, where things done with neural networks that go beyond statistical curve fitting - e.g. RNNs for learning sequences, and "deep reinforcement learning" - may also be considered part of it by adherents. If you allow for the term to evolve like this, it is very hard to nail down any argument about its capabilities.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-06-15T16:35:15.190Z
Comments.Text: godel's incompleteness theorem
Comments.UserDisplayName: thecomplexitytheorist
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-06-15T23:03:56.753Z
Comments.Text: @thecomplexitytheorist: Godel's incompleteness, entscheidungsproblem and similars , all they aply also to human mind. As conclusion, if they was a reason to do not reach AI, then neither humans are intelligent. Modus tollens, Godels is not a problem in the path to AGI
Comments.UserDisplayName: pasaba por aqui
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-06-18T23:26:06.490Z
Comments.Text: Who is to say that 'Doing' and 'Imagining' are also not simply 'Curve-Fitting' going on in the brain?
Comments.UserDisplayName: Dunk
Tags: <deep-learning><agi><superintelligence>
Title: Is there any scientific/mathematical argument that prevents deep learning from ever producing strong AI?
UserDisplayName: Graviton
ThreadText: # Is there any scientific/mathematical argument that prevents deep learning from ever producing strong AI?
  
  (Asked by: username_0 on 2018-06-15)
  
  I read Judea Pearl's [The Book of Why](https://rads.stackoverflow.com/amzn/click/com/046509760X), in which he mentions that deep learning is just a glorified curve fitting technology, and will not be able to produce human-like intelligence.
  
  From his book there is this diagram that illustrates the three levels of cognitive abilities:
  
  [![Three levels of cognitive abilities](https://i.sstatic.net/vxTLx.png)](https://i.sstatic.net/vxTLx.png)
  
  The idea is that the "intelligence" produced by current deep learning technology is only at the level of association. Thus the AI is nowhere near the level of asking questions like "how can I make Y happen" (intervention) and "What if I have acted differently, will X still occur?" (counterfactuals), and it's highly unlikely that curve fitting techniques can ever bring us closer to a higher level of cognitive ability.
  
  I found his argument persuasive on an intuitive level, but I'm unable to find any physical or mathematical laws that can either bolster or cast doubt on this argument.
  
  So, is there any scientific/physical/chemical/biological/mathematical argument that prevents deep learning from ever producing strong AI (human-like intelligence)?
  
  **Tags:** deep-learning, agi, superintelligence
  
  > **Comments:**
  > **username_1**: One of the problems faced in answering this is, is that "Deep Learning" is a kind of brand name, where things done with neural networks that go beyond statistical curve fitting - e.g. RNNs for learning sequences, and "deep reinforcement learning" - may also be considered part of it by adherents. If you allow for the term to evolve like this, it is very hard to nail down any argument about its capabilities.  
  > **username_2**: godel's incompleteness theorem  
  > **username_3**: @username_2: Godel's incompleteness, entscheidungsproblem and similars , all they aply also to human mind. As conclusion, if they was a reason to do not reach AI, then neither humans are intelligent. Modus tollens, Godels is not a problem in the path to AGI  
  > **username_4**: Who is to say that 'Doing' and 'Imagining' are also not simply 'Curve-Fitting' going on in the brain?  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_5 on 2018-10-13. Score: 3
  
  Judea Pearl's 2018 comment on ACM.org, in his [*To, Build Truly Intelligent Machines, Teach Them Cause and Effect*](https://cacm.acm.org/news/227877-to-build-truly-intelligent-machines-teach-them-cause-and-effect/fulltext) is piercing truth.
  
  > All the impressive achievements of deep learning amount to just curve fitting.
  
  It may be less sensational and more technically correct to state that it is not, "Just curve fitting," but rather, "sophisticated surface fitting." Nonetheless, his general assessment indicates the need to look beyond tuning nonlinear functions to fit a surface in $\mathbb{R}^n$ and consider whether cognition is achievable with a deep network. The split in answers to this question is odd. We have two conflicting assertions, often strongly stated.
  
  1. Artificial networks cannot perform logic.
  2. Artificial networks are the best approach to AI.
  
  How can rationality be excluded from the list of important human features of intelligence, which is what these two assertions are taken together would mean?
  
  Is the human brain a network of sophisticated curve fitters? Marvin Minsky's famous quote, "The brain happens to be a meat machine," was offered without proof, and neither a proof of his trivialization of the human brain nor a proof that the brain is beyond the reach of Turing computability has been offered since.
  
  When you read these words, are your neural networks doing the following sequence of curve fits?
  
  * Edges from retinal rods and cones
  * Lines from edges
  * Shapes from lines
  * Letters from shapes
  * Linguistic elements from groups of letters
  * Linguistic structures from elements
  * Understanding from linguistic structures
  
  The case is strong for the affirmation that the first five is a convergence mechanism on a model, and all the machine learning structure is just a method to fit the data to the model.
  
  Those last two bullet items are where the paradigm breaks down and where many AI researchers and authors have correctly stated that machine learning has significant limitations when based solely on layers of multi-layer perceptrons and convolution kernels. Furthermore, the last bullet item is grossly oversimplified in its current state, probably by orders of magnitude. Even if Minsky is correct that a computer can perform what the brain does, the process of reading and understanding this paragraph could easily have a thousand different kinds of unique process components in patterns of internal workflow with massive parallelism. Imaging technology indicates this probability. We have computers modelling only the simplest peripheral layers.
  
  > Is there any scientific/mathematical argument that prevents deep learning from ever producing strong AI? — No. But there is no such argument that guarantees it either.
  
  Other questions here investigate whether these sophisticated curve fitters can perform elements of cognition or reasoning.
  
  * [Could a neural network detect primes?](https://ai.stackexchange.com/questions/3389/could-a-neural-network-detect-primes)
  * [Can neural networks be used to prove conjectures?](https://ai.stackexchange.com/questions/7416/can-deep-networks-be-trained-to-prove-theorems)
  
  The totem of three in the question's image, seeing, doing, and imagining, is not particularly complete, accurate, or insightful.
  
  * There are at least five sensory paradigms in humans, not one
  * Doing preceded human senses by billions of years — bacteria do
  * Imagining is not a significantly higher process than scenario replay from models of past experience with some method to apply set functions to combine them and inject random mutations
  * Creativity may just be imagining in the previous bullet item followed by weeding out useless imagination results with some market-oriented quality criteria, leaving the impressive creative products that sell
  
  The higher forms are appreciation, a sense of realities beyond the scope of scientific measurement, legitimate doubt, love, sacrifice for the good of others or humanity.
  
  Many recognize that the current state of AI technology is nowhere near the procurement of a system that can reliably answer, "How can I make Y happen?" or "If I have acted differently, will X still occur?"
  
  There is no mathematical proof that some combination of small curve fitting elements can or cannot achieve the ability to answer those questions as well a typical human being can, mostly because there is insufficient understanding of what intelligence is or how to define it in mathematical terms.
  
  It is also possible that human intelligence doesn't exist at all, that references to it are based on a religious belief that we are higher as a species than other species. That we can populate, consume, and exterminate is not actually a very intelligent conception of intelligence.
  
  The claim that human intelligence is an adaptation that differentiates us from other mammals conflicts with whether we adapt well. We have not been tested. Come the next meteoric global killer with a shock wave of the magnitude of that of the Chicxulub crater's meteor, followed by a few and a thousand years of solar winter and we'll see whether it is our 160,000-year existence or bacteria's 4,000,000,000 year existence that proves more sustainable. In the timeline of life, human intelligence has yet to prove itself significant as an adaptive trait.
  
  What is clear about AI development is that other kinds of systems are playing a role along with deep learners based on the multi-layer perceptron concept and convolution kernels which are strictly surface fitters.
  
  Q-learning components, attention-based components, and long-short term memory components are all strictly a surface fitter too, but only by stretching the definition of surface fitting considerably. They have real-time adaptive properties and state, so they can be Turing complete.
  
  Fuzzy logic containers, rules-based systems, algorithms with Markovian properties, and many other component types also play their role and are not surface fitters at all.
  
  In summary, there are points made that have a basis in more than plausibility or a pleasing intuitive quality, however, many of these authors do not provide a mathematical framework with definitions, applications, lemmas, theorems, proofs, or even thought experiments that can be scrutinized in a formal way.
  
  
  ---
  
  ### Answer by username_3 on 2018-06-15. Score: 0
  
  It is a paradox, but a deep learning machine (defined as a NeuralNet variant) is unable to learn anything. It is a flexible and configurable hardware/software architecture that can be parametrized to solve a lot of problems. But the optimal parameters to solve a problem are obtained by an external system, i.e. back-propagation algorithm.
  
  Back-propagation subsystem uses conventional programming paradigms, it is not a Neural Net. This fact is in absolute opposition to human mind, where learning and use of knowledge is done by the same system (the mind).
  
  If all the real interesting things are done outside the NN, it is difficult to claim that a NN (in any variant) can develop in an AGI.
  
  It is also possible to find some more differences. Neural nets are strongly numerical in its interface and internals. From this point of view, they are an evolution of support vector machines.
  
  Too much differences and restrictions to expect an AGI.
  
  Note: I strongly disagree in the draw included in the original question. "Seeing", "doing", "imaging" are levels absolutely wrong. It ignores from basic and common software concepts as "abstraction" or "program state" (of mind, in Turing words); applied AI ones as "foresee"; and AGI ones as "free will", "objectives and feelings", ...
  
  > **Comments:**
  > **username_1**: I think removing back-propagation (or any part of the training framework) from consideration, and claiming that the remaining part **is** the "Deep Learning" part is artificial, and kind of dodging the question. I think it is reasonable to assume that OP means Deep Learning as it is practiced, including the training processes available.  
  > **username_3**: @username_1: If we say a DL is a kind of NeuralNet ( in order to remark the similarity with the human mind and, as consequence, its possible power to reach an AGI) , we are excluding the learning part, that is not a NN. If we include the learning subsystem in the definition of a DL, then it is not a NN, it is only conventional programming, with the power of any conventional program, and has the same posibilities to reach AGI than any other program system or paradigm.  
  > **username_4**: Human brains learn by receiving and processing input from external 'systems' exclusively. The optimal parameters to solve problems are obtained via trial and error, applying rules and processing input from external systems. Training starts while the baby is still in the uterus and continues 24/7 thereafter. The current state of AI is almost certainly not comparable to emulating the human brain; but claiming that AI can't learn (or isn't already learning in a similar way to the human brain) assumes knowledge of how the human brain 'learns' and functions that science doesn't know yet.  
  > **username_6**: This doesn't make a lot of sense once you think about what "learning" is. Human brains also learn by adjusting connection weights in fixed hardware. So why aren't they disqualified? (Even when new connections are formed, that's just an update of a weight that was previously 0)  
  
  ---
  
  