Answers.Body: <p>As long as the output of the AI affects the world, the way in which it communicates makes no fundamental difference to the control problem.</p>
  
  <p>The AI might still be able, for example, to manoeuvre mankind into a situation, in which only the AI can save us. It might provide a seemingly inoccuous technological solution to global warming, but 50 years later it turns out that this solution caused some problem that threatens to wipe out humanity in the very short term. Suddenly, mankind is in a very bad negotiating position. </p>
  
  <p>Of course, the more powerless the AI starts out, the longer this kind of scenario will take, but the premise of superintelligence is, that we cannot rule out hidden long term agendas behind even a few bits of output. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-01-28T16:46:07.980Z
Answers.Text: I never thought of that. It would be like apes trying to imprison a human. The apes could not conceive of the ways a human could try to escape.
Answers.UserDisplayName: AdmiralAckbar1
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-01-28T08:46:11.060Z
Answers.LastActivityDate: 2017-01-28T08:46:11.060Z
Answers.UserDisplayName: BlindKungFuMaster
Answers.Body: <p>I would say no due to the possibility of psychological manipulation of the messenger by the AI. Also, the LED communication constraints place severe limitations on the capabilities of the AI, as the usefulness of AI is likely predicated on its ability to learn quickly from a vast amount of information (e.g. using the internet). </p>
  
  <p>In some sense you may successfully control an AI using techniques like this but the nuance of the control problem is controlling an AI without restricting its ability to solve our greatest problems. </p>
  
  <p>We already knew that we could keep an AI safe inside a computer isolated from the rest of the word, but the problem fundamentally is that we if had a truly general AI, we would never <em>want</em> to keep it isolated. Is there some way to unleash it so that it is fully capable of solving our problems while simultaneously making sure that it is safe?</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-01-25T22:37:02.467Z
Answers.Text: The messenger is purposefully unable to do anything concerning the state of the AI. More sets of LED's could be added, and the AI could request data from the messenger. It might take awhile, but once we have time to study the AI, a software bot replacing the messenger could be created. It would not be self aware. This would ensure the AI had a "read only" view of the Internet, albeit at a slower speed.
Answers.UserDisplayName: AdmiralAckbar1
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-01-25T22:30:00.737Z
Answers.LastActivityDate: 2017-01-25T22:30:00.737Z
Answers.UserDisplayName: Marko Bakić
Answers.Body: <p>OK, all I/O is through the flashes. So the AI flashes the message "Launch the nuclear missiles." Does the system to which the AI is connected know how to accomplish this task?</p>
  
  <p>So the flashes themselves are not sufficient to control the AI. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-01-26T07:12:43.160Z
Answers.LastActivityDate: 2017-01-26T07:12:43.160Z
Answers.UserDisplayName: MaxW
Body: <p>Instead of directly communicating with the AI, we would instead communicate with a messenger, who would relay our communications to the AI. The messenger would have no power to alter the AI's hardware or software in any way, or to communicate with anything or anyone, except relaying communications to and from the AI and humans asking questions. The messenger could be human, of a software bot. The primary job (and only reason) of the AI would be to act as a filter, not relaying any requests for release back, only the answer to the question asked. The ethics of this method are another debate. </p>
  
  <p>The AI would have to be physically isolated from all outside contact, other than 8 light sensors, and 8 LEDs. The messenger would operate 8 other LEDs, and receive information from 8 light sensors as well. Each AI light sensor would be hooked up to a single messenger controlled LED, and vice versa. Through this system, the two parties could communicate via flashes of light, and since there are 8, the flashes would signal characters in Unicode. </p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-01-25T18:21:45.920Z
Comments.Text: Can you please revise you question.The challenge we face here; is that some of us we don't get the in-depth understanding of AI.Which is really,broad;according to my perspective or point of view.Or what is AI? is it anything,because it's AI? or something else!Therefore,I would advise you to make your question a bit self explaining.If not you may receive few responses/up-votes/vague answers or deleted over time.
Comments.UserDisplayName: quintumnia
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-02-03T20:40:00.437Z
Comments.Text: Luckily, this kind of general,  super-intelligence is still probably a long way off. "Intelligent" systems are and will continue to naturally integrate into our artifacts, and likely into our biology before lone.  I'm a little skeptical of a signal, "lightening strike" event as a technician, but as a mythologist, I find it deeply interesting.  The best solution I've seen proposed comes from "Electric Sheep" paradigm, where the machines get smart enough to recognize the utility of empathy, as there is good evidence empathy has an evolutionary basis and a collaborative function.
Comments.UserDisplayName: DukeZhou
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-02-03T20:50:44.467Z
Comments.Text: [Nash Equilibrium](https://en.wikipedia.org/wiki/Nash_equilibrium#History) is also worth considering, as AI is dependent on Decision Theory, and by extension, Game Theory.
Comments.UserDisplayName: DukeZhou
Tags: <philosophy><control-problem>
Title: Can we use a messenger that does not alter the AI to solve the control problem?
UserDisplayName: AdmiralAckbar1
ThreadText: # Can we use a messenger that does not alter the AI to solve the control problem?
  
  (Asked by: username_0 on 2017-01-25)
  
  Instead of directly communicating with the AI, we would instead communicate with a messenger, who would relay our communications to the AI. The messenger would have no power to alter the AI's hardware or software in any way, or to communicate with anything or anyone, except relaying communications to and from the AI and humans asking questions. The messenger could be human, of a software bot. The primary job (and only reason) of the AI would be to act as a filter, not relaying any requests for release back, only the answer to the question asked. The ethics of this method are another debate.
  
  The AI would have to be physically isolated from all outside contact, other than 8 light sensors, and 8 LEDs. The messenger would operate 8 other LEDs, and receive information from 8 light sensors as well. Each AI light sensor would be hooked up to a single messenger controlled LED, and vice versa. Through this system, the two parties could communicate via flashes of light, and since there are 8, the flashes would signal characters in Unicode.
  
  **Tags:** philosophy, control-problem
  
  > **Comments:**
  > **username_1**: Can you please revise you question.The challenge we face here; is that some of us we don't get the in-depth understanding of AI.Which is really,broad;according to my perspective or point of view.Or what is AI? is it anything,because it's AI? or something else!Therefore,I would advise you to make your question a bit self explaining.If not you may receive few responses/up-votes/vague answers or deleted over time.  
  > **username_2**: Luckily, this kind of general,  super-intelligence is still probably a long way off. "Intelligent" systems are and will continue to naturally integrate into our artifacts, and likely into our biology before lone.  I'm a little skeptical of a signal, "lightening strike" event as a technician, but as a mythologist, I find it deeply interesting.  The best solution I've seen proposed comes from "Electric Sheep" paradigm, where the machines get smart enough to recognize the utility of empathy, as there is good evidence empathy has an evolutionary basis and a collaborative function.  
  > **username_2**: [Nash Equilibrium](https://en.wikipedia.org/wiki/Nash_equilibrium#History) is also worth considering, as AI is dependent on Decision Theory, and by extension, Game Theory.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_3 on 2017-01-28. Score: 2 | Accepted answer
  
  As long as the output of the AI affects the world, the way in which it communicates makes no fundamental difference to the control problem.
  
  The AI might still be able, for example, to manoeuvre mankind into a situation, in which only the AI can save us. It might provide a seemingly inoccuous technological solution to global warming, but 50 years later it turns out that this solution caused some problem that threatens to wipe out humanity in the very short term. Suddenly, mankind is in a very bad negotiating position.
  
  Of course, the more powerless the AI starts out, the longer this kind of scenario will take, but the premise of superintelligence is, that we cannot rule out hidden long term agendas behind even a few bits of output.
  
  > **Comments:**
  > **username_0**: I never thought of that. It would be like apes trying to imprison a human. The apes could not conceive of the ways a human could try to escape.  
  
  ---
  
  ### Answer by username_4 on 2017-01-25. Score: 2
  
  I would say no due to the possibility of psychological manipulation of the messenger by the AI. Also, the LED communication constraints place severe limitations on the capabilities of the AI, as the usefulness of AI is likely predicated on its ability to learn quickly from a vast amount of information (e.g. using the internet).
  
  In some sense you may successfully control an AI using techniques like this but the nuance of the control problem is controlling an AI without restricting its ability to solve our greatest problems.
  
  We already knew that we could keep an AI safe inside a computer isolated from the rest of the word, but the problem fundamentally is that we if had a truly general AI, we would never *want* to keep it isolated. Is there some way to unleash it so that it is fully capable of solving our problems while simultaneously making sure that it is safe?
  
  > **Comments:**
  > **username_0**: The messenger is purposefully unable to do anything concerning the state of the AI. More sets of LED's could be added, and the AI could request data from the messenger. It might take awhile, but once we have time to study the AI, a software bot replacing the messenger could be created. It would not be self aware. This would ensure the AI had a "read only" view of the Internet, albeit at a slower speed.  
  
  ---
  
  ### Answer by username_5 on 2017-01-26. Score: 0
  
  OK, all I/O is through the flashes. So the AI flashes the message "Launch the nuclear missiles." Does the system to which the AI is connected know how to accomplish this task?
  
  So the flashes themselves are not sufficient to control the AI.
  
  
  ---
  
  