Answers.Body: <p>An excellent book summarizing the development of thought in this area over several hundred years is <a href="https://rads.stackoverflow.com/amzn/click/com/0262581531" rel="nofollow noreferrer" rel="nofollow noreferrer">Mind Design II</a>, edited by John Haugeland. This book contains a collection of essays written by the major thinkers in this area through until about the 1990s. A brief summary of some of the major ideas is:</p>
  
  <ol>
  <li>Descartes: Minds are spirits, brains are bodies. Minds attach to brains in a <a href="https://en.wikipedia.org/wiki/Corpus_callosum" rel="nofollow noreferrer">special part of the brain</a>. Descartes has a number of thought experiments, but they mostly have a Theological root. Most modern scientists do not find these to be satisfying because the resolve metaphysical questions about the mind by assuming you already believe in a particular interpretation of a God. </li>
  <li><a href="https://en.wikipedia.org/wiki/Gilbert_Ryle" rel="nofollow noreferrer">Gilbert Ryle</a> argues in the late 1940s that concepts like the Mind are in some sense vacuous. Asking whether a machine has a <em>mind</em> or not is like asking whether a field contains two cows or a pair of cows: this is a conceptual distinction that has no relation to the real world. Because of this, its study is futile. The parallel movement of <a href="https://en.wikipedia.org/wiki/Behaviorism" rel="nofollow noreferrer">Behaviorism</a> in psychology also viewed the study of minds and mental states as a fool's errand. Rather, this movement focused on studying behaviors that could be directly measured.</li>
  <li><p>Turing publishes his <a href="https://www.csee.umbc.edu/courses/471/papers/turing.pdf" rel="nofollow noreferrer">seminal paper</a> in AI, in 1950. In this paper, Turing anticipates nearly all later objections to AI, and summarizes reasonable refutations of them. This paper contains a number of short thought experiments. One of the more notable ones is the Turing Test. A key observation Turing makes is that if a machine behaves in a way indistinguishable from a human, then any arguments used to reject its claim to intelligence (and even to <em>consciousness</em>) appear to work equally well on other humans. Searle doesn't really address this in his later argument. Basically, I have no way to know for sure whether you have subjective experiences of the same kind I have. The fact that you're made of roughly the same kind of meat as I am seems like a shaky explanation. For example, would humanoid aliens that act like us also be intelligent? What if it were found that their brains took some radically different form from ours? What if it were found that their brains were really networks of transistor-like machines?</p></li>
  <li><p>The <a href="https://en.wikipedia.org/wiki/Cognitivism_(psychology)" rel="nofollow noreferrer">Cognativist revolution</a> spanned a number of fields starting in the 1960s. This movement held that behaviorism was wrong for two basic reasons. First, Behaviorism does not account for the subjective feelings of existence or understanding. Second, Behaviorism could not explain phenomena like language, that appeared to involve reasoning logically about symbols. Notable authors in this period are <a href="https://en.wikipedia.org/wiki/Noam_Chomsky" rel="nofollow noreferrer">Chomsky</a> and <a href="https://en.wikipedia.org/wiki/Jerry_Fodor" rel="nofollow noreferrer">Fodor</a>, who argued that minds were essentially computer programs that happened to be running on brains instead of computers. These ideas dominated AI, Psychology and Linguistics for about 30-40 years. Fodor's works contain a number of thought experiments involving language and programs.</p></li>
  <li>John Searle's <a href="https://plato.stanford.edu/entries/chinese-room/" rel="nofollow noreferrer">Chinese Room argument</a> caused problems for the cognativists, because it presents a convincing chain of logic showing that a program that does complicated tasks (like linguistics) does not subjectively understand what it is doing. Various objections are raised and Searle responds to them in his publication of the argument. Objections to Searle's arguments form the basis of several later movements in philosophy of AI. </li>
  <li>The <a href="https://www.iep.utm.edu/connect/" rel="nofollow noreferrer">Connectionists</a> argue that we can represent the behavior of the brain (and thus, the mind) with a brain-like program. This paralleled increasing interest in artificial neural networks. An example of a cognativist thought experiment is to imagine a simulation of an entire human brain, conducted with perfect fidelity (for instance, we accurately model all the quantum mechanical interactions). If such a machine were not conscious, opponents need to answer the question of where consciousness is hiding (Is it in the meat? Then see Turing. Is it in some yet-undiscovered property of neurons? Then the burden of proof is definitely on the person proposing such a hidden property, and they must explain why we cannot simulate it). </li>
  <li>The <a href="https://en.wikipedia.org/wiki/Paul_Churchland" rel="nofollow noreferrer">Churchlands</a> and others argued, much as Ryle, that minds as the public understand them are a sort of "folk theory", that appear to explain something about reality to lay people, but are actually a dead end. Concepts like "beliefs" and "experiences" are, according to these arguments, akin to aether or phlogiston: things that our current, malformed, understandings of neuroscience seem to require, but that in fact have no basis in reality, even though nearly everyone believes in them at present. This line of work contains many extensions of the kinds of thought experiments Turing hinted at. </li>
  <li><a href="https://en.wikipedia.org/wiki/Rodney_Brooks" rel="nofollow noreferrer">Brooks</a> and others advance the theory of embodied cognition, in which the mind is not confined to the body, but is actually a property of a body plus the environment it is placed in. An example thought experiment from this school (by Andy Clarke if I recall correctly) is to imagine a person with profound loss of the ability to form new long term memories. This person carries around a notebook. The cover says "Your memory". The first page contains instructions explaining that the person has lost the ability to remember things, and that they write new things they want to remember in the book. The person can remember things for, say, 15 minutes at a time, and the book is very well organized. Where is this person's mind exactly? Is it in their brain (which does not remember anything 15 minutes in the past), or is it in the brain <em>and</em> the book, which together can remember and reason about things essentially like a normal person. If it <em>is</em> in the brain and book, what if we replace the book with a computer inside the person's head? </li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-07T20:40:14.017Z
Answers.LastActivityDate: 2019-10-07T20:40:14.017Z
Answers.UserDisplayName: John Doucette
Body: <p>The <a href="https://plato.stanford.edu/entries/chinese-room/" rel="nofollow noreferrer">Chinese Room argument</a> <em>against</em> strong AI overlooks the fact that "the man in the room" is acting as a macro-scale "neurotransmitter" of the larger system in which he resides. It does not rule out strong AI, it simply reduces to an enigmatic question: <em>where does understanding "reside" and how does it epiphenomenally emerge?</em></p>
  
  <p>What are other examples of thought experiments against or in favor of strong AI (apart from the Chinese room argument) or extensions or refutations to known experiments?</p>
  
Tags: <reference-request><philosophy><agi><chinese-room-argument>
Title: What are examples of thought experiments against or in favour of strong AI, apart from the Chinese room argument?
UserDisplayName: respectful
ThreadText: # What are examples of thought experiments against or in favour of strong AI, apart from the Chinese room argument?
  
  (Asked by: username_0 on 2019-10-07)
  
  The [Chinese Room argument](https://plato.stanford.edu/entries/chinese-room/) *against* strong AI overlooks the fact that "the man in the room" is acting as a macro-scale "neurotransmitter" of the larger system in which he resides. It does not rule out strong AI, it simply reduces to an enigmatic question: *where does understanding "reside" and how does it epiphenomenally emerge?*
  
  What are other examples of thought experiments against or in favor of strong AI (apart from the Chinese room argument) or extensions or refutations to known experiments?
  
  **Tags:** reference-request, philosophy, agi, chinese-room-argument
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-10-07. Score: 3 | Accepted answer
  
  An excellent book summarizing the development of thought in this area over several hundred years is [Mind Design II](https://rads.stackoverflow.com/amzn/click/com/0262581531), edited by username_1 Haugeland. This book contains a collection of essays written by the major thinkers in this area through until about the 1990s. A brief summary of some of the major ideas is:
  
  1. Descartes: Minds are spirits, brains are bodies. Minds attach to brains in a [special part of the brain](https://en.wikipedia.org/wiki/Corpus_callosum). Descartes has a number of thought experiments, but they mostly have a Theological root. Most modern scientists do not find these to be satisfying because the resolve metaphysical questions about the mind by assuming you already believe in a particular interpretation of a God.
  2. [Gilbert Ryle](https://en.wikipedia.org/wiki/Gilbert_Ryle) argues in the late 1940s that concepts like the Mind are in some sense vacuous. Asking whether a machine has a *mind* or not is like asking whether a field contains two cows or a pair of cows: this is a conceptual distinction that has no relation to the real world. Because of this, its study is futile. The parallel movement of [Behaviorism](https://en.wikipedia.org/wiki/Behaviorism) in psychology also viewed the study of minds and mental states as a fool's errand. Rather, this movement focused on studying behaviors that could be directly measured.
  3. Turing publishes his [seminal paper](https://www.csee.umbc.edu/courses/471/papers/turing.pdf) in AI, in 1950. In this paper, Turing anticipates nearly all later objections to AI, and summarizes reasonable refutations of them. This paper contains a number of short thought experiments. One of the more notable ones is the Turing Test. A key observation Turing makes is that if a machine behaves in a way indistinguishable from a human, then any arguments used to reject its claim to intelligence (and even to *consciousness*) appear to work equally well on other humans. Searle doesn't really address this in his later argument. Basically, I have no way to know for sure whether you have subjective experiences of the same kind I have. The fact that you're made of roughly the same kind of meat as I am seems like a shaky explanation. For example, would humanoid aliens that act like us also be intelligent? What if it were found that their brains took some radically different form from ours? What if it were found that their brains were really networks of transistor-like machines?
  4. The [Cognativist revolution](https://en.wikipedia.org/wiki/Cognitivism_(psychology)) spanned a number of fields starting in the 1960s. This movement held that behaviorism was wrong for two basic reasons. First, Behaviorism does not account for the subjective feelings of existence or understanding. Second, Behaviorism could not explain phenomena like language, that appeared to involve reasoning logically about symbols. Notable authors in this period are [Chomsky](https://en.wikipedia.org/wiki/Noam_Chomsky) and [Fodor](https://en.wikipedia.org/wiki/Jerry_Fodor), who argued that minds were essentially computer programs that happened to be running on brains instead of computers. These ideas dominated AI, Psychology and Linguistics for about 30-40 years. Fodor's works contain a number of thought experiments involving language and programs.
  5. username_1 Searle's [Chinese Room argument](https://plato.stanford.edu/entries/chinese-room/) caused problems for the cognativists, because it presents a convincing chain of logic showing that a program that does complicated tasks (like linguistics) does not subjectively understand what it is doing. Various objections are raised and Searle responds to them in his publication of the argument. Objections to Searle's arguments form the basis of several later movements in philosophy of AI.
  6. The [Connectionists](https://www.iep.utm.edu/connect/) argue that we can represent the behavior of the brain (and thus, the mind) with a brain-like program. This paralleled increasing interest in artificial neural networks. An example of a cognativist thought experiment is to imagine a simulation of an entire human brain, conducted with perfect fidelity (for instance, we accurately model all the quantum mechanical interactions). If such a machine were not conscious, opponents need to answer the question of where consciousness is hiding (Is it in the meat? Then see Turing. Is it in some yet-undiscovered property of neurons? Then the burden of proof is definitely on the person proposing such a hidden property, and they must explain why we cannot simulate it).
  7. The [Churchlands](https://en.wikipedia.org/wiki/Paul_Churchland) and others argued, much as Ryle, that minds as the public understand them are a sort of "folk theory", that appear to explain something about reality to lay people, but are actually a dead end. Concepts like "beliefs" and "experiences" are, according to these arguments, akin to aether or phlogiston: things that our current, malformed, understandings of neuroscience seem to require, but that in fact have no basis in reality, even though nearly everyone believes in them at present. This line of work contains many extensions of the kinds of thought experiments Turing hinted at.
  8. [Brooks](https://en.wikipedia.org/wiki/Rodney_Brooks) and others advance the theory of embodied cognition, in which the mind is not confined to the body, but is actually a property of a body plus the environment it is placed in. An example thought experiment from this school (by Andy Clarke if I recall correctly) is to imagine a person with profound loss of the ability to form new long term memories. This person carries around a notebook. The cover says "Your memory". The first page contains instructions explaining that the person has lost the ability to remember things, and that they write new things they want to remember in the book. The person can remember things for, say, 15 minutes at a time, and the book is very well organized. Where is this person's mind exactly? Is it in their brain (which does not remember anything 15 minutes in the past), or is it in the brain *and* the book, which together can remember and reason about things essentially like a normal person. If it *is* in the brain and book, what if we replace the book with a computer inside the person's head?
  
  
  ---
  
  