Answers.Body: <p>Asimov made the three laws <strong>specifically to prove</strong> that no three laws are sufficient, no matter how reasonable they seem at first. I know a guy that knew the guy and he confirmed this.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-25T22:10:25.507Z
Answers.Text: "I know a guy who knew the guy"... Don't you have a slightly more reliable source? I'm not doubting your answer, but there should surely already exist a proper quote on the matter.
Answers.UserDisplayName: 3442
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-26T14:38:17.757Z
Answers.Text: His name was Ben Williams. This is the Google Plus post where he described his opinions about Asimov and the Three Laws: https://plus.google.com/+JohnNewmanIII/posts/HeTnPdhaY9p
Answers.UserDisplayName: Doxosophoi
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-25T16:00:00.820Z
Answers.LastActivityDate: 2016-08-25T16:00:00.820Z
Answers.UserDisplayName: Doxosophoi
Answers.Body: <p>Consider Asimov's first law of robotics:</p>
  
  <blockquote>
    <p>A robot may not injure a human being or, through inaction, allow a
    human being to come to harm.</p>
  </blockquote>
  
  <p>That law is already problematic, when taking into consideration self-driving cars. </p>
  
  <p>What's the issue here, you ask? Well, you'll probably be familiar with the classic thought experiment in ethic known as <a href="https://en.wikipedia.org/wiki/Trolley_problem" rel="nofollow">the trolley problem</a>. The general form of the problem is this:</p>
  
  <blockquote>
    <p>The trolley is headed straight for them. You are standing some
    distance off in the train yard, next to a lever. If you pull this
    lever, the trolley will switch to a different set of tracks. However,
    you notice that there is one person on the side track. You have two
    options: (1) Do nothing, and the trolley kills the five people on the
    main track. (2) Pull the lever, diverting the trolley onto the side
    track where it will kill one person. Which is the correct choice?</p>
    
    <p>source : <a href="https://en.wikipedia.org/wiki/Trolley_problem" rel="nofollow">Wikipedia</a></p>
  </blockquote>
  
  <p>Self-driving cars will actually need to implement real life variations on the trolley problem, which basically means that <a href="https://www.technologyreview.com/s/542626/why-self-driving-cars-must-be-programmed-to-kill/" rel="nofollow">self-driving cars need to be programmed to kill human beings</a>.</p>
  
  <p>Of course that doesn't mean that ALL robots will need to be programmed to kill, but self-driving cars are a good example of a type of robot that will.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-23T18:12:16.680Z
Answers.LastActivityDate: 2016-08-23T18:12:16.680Z
Answers.UserDisplayName: John Slegers
Answers.Body: <p>Asimov's laws are not strong enough to be used in practice. Strength isn't even a consideration, when considering that since they're written in English words would first have to be interpreted subjectively to have any meaning at all. You can find a good discussion of this <a href="https://youtu.be/7PKx3kS7f4A">here</a>.</p>
  
  <p>To transcribe an excerpt:</p>
  
  <blockquote>
    <p>How do you define these things? How do you define "human", without first having to take a stand on almost every issue. And if "human" wasn't hard enough, you then have to define "harm", and you've got the same problem again. Almost any really solid unambiguous definitions you give for those words&mdash;that don't rely on human intuition&mdash;result in weird quirks of philosophy, leading to your AI doing something you really don't want it to do.</p>
  </blockquote>
  
  <p>One can easily imagine that Asimov was smart enough to know this and was more interested in story-writing than designing real-world AI control protocols.</p>
  
  <p>In the novel <a href="https://en.wikipedia.org/wiki/Neuromancer#Plot_summary">Neuromancer</a>, it was suggested that AIs could possibly serve as checks against each other. Ray Kurzweil's impending <a href="https://en.wikipedia.org/wiki/Technological_singularity">Singularity</a>, or the possibility of hyperintelligent AGIs otherwise, might not leave much of a possibility for humans to control AIs at all, leaving peer-regulation as the only feasible possibility.</p>
  
  <p>It's worth noting that Eliezer Yudkowsky and others ran an <a href="http://www.yudkowsky.net/singularity/aibox/">experiment</a> wherein Yudkowsky played the role of a superintelligent AI with the ability to speak, but no other connection outside of a locked box. The challengers were tasked simply with keeping the AI in the box at all costs. Yudkowsky escaped both times.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-05T09:02:30.487Z
Answers.Text: Do you know if any transcript of the Yudkowsky experiments were ever released? The link you give (and the email exchange it then links to) doesn't appear to provide one.
Answers.UserDisplayName: NietzscheanAI
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-05T13:04:13.053Z
Answers.Text: @user217281728 No, I looked for it but it appears he's keeping it, i.e. his strategy under wraps to maintain the possibility of doing it again. I have some ideas about what he said and why I think that, that I'd be happy to share if you DM me.
Answers.UserDisplayName: dynrepsys
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-05T13:15:44.673Z
Answers.Text: I've now done this.
Answers.UserDisplayName: NietzscheanAI
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-05T13:44:16.630Z
Answers.Text: I'm under the impression that Asimov was smart enough to know this, but *the point* was that simple rules don't work. Pointing out negative examples ("you can't just tell it to not harm people, that won't work") is useful real-world AI control work in its own right.
Answers.UserDisplayName: Matthew Gray
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-05T01:55:51.737Z
Answers.LastActivityDate: 2016-08-05T02:00:59.683Z
Answers.LastEditDate: 2016-08-05T02:00:59.683Z
Answers.UserDisplayName: dynrepsys
Body: <p>Isaac Asimov's famous <a href="https://en.wikipedia.org/wiki/Three_Laws_of_Robotics">Three Laws of Robotics</a> originated in the context of Asimov's science fiction stories. In those stories, the three laws serve as a safety measure, in order to avoid untimely or manipulated situations from exploding in havoc.</p>
  
  <p>More often than not, Asimov's narratives would find a way to break them, leading the writer to make several modifications to the laws themselves. For instance, in some of his stories, he <a href="https://en.wikipedia.org/wiki/Three_Laws_of_Robotics#First_Law_modified">modified the First Law</a>, <a href="https://en.wikipedia.org/wiki/Three_Laws_of_Robotics#Zeroth_Law_added">added a Fourth (or Zeroth) Law</a>, or even <a href="https://en.wikipedia.org/wiki/Three_Laws_of_Robotics#Removal_of_the_Three_Laws">removed all Laws altogether</a>.</p>
  
  <p>However, it is easy to argue that, in popular culture, and even in the field of AI research itself, the Laws of Robotics are taken quite seriously. Ignoring the side problem of the different, subjective, and mutually-exclusive interpretations of the laws, are there any arguments proving the laws themselves intrinsically flawed by their design, or, alternatively, strong enough for use in reality? Likewise, has a better, stricter security heuristics set being designed for the purpose?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-08-05T13:17:01.320Z
Comments.Text: Beware the Killbot Hellscape! https://xkcd.com/1613/
Comments.UserDisplayName: NietzscheanAI
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-08-05T14:33:36.913Z
Comments.Text: We musn't forget this possibility: http://www.xkcd.com/1450/
Comments.UserDisplayName: dynrepsys
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-08-23T18:57:19.313Z
Comments.Text: Asimov's laws are just a small attempt at starting the discussion. For a much more thorough discussion, read "Superintelligence" by Nick Bostrom - that's as exhaustive as it gets based on current knowledge and understanding.
Comments.UserDisplayName: Florin Andrei
Tags: <ai-safety><asimovs-laws>
Title: Are Asimov's Laws flawed by design, or are they feasible in practice?
UserDisplayName: 3442
ThreadText: # Are Asimov's Laws flawed by design, or are they feasible in practice?
  
  (Asked by: username_0 on 2016-08-04)
  
  Isaac Asimov's famous [Three Laws of Robotics](https://en.wikipedia.org/wiki/Three_Laws_of_Robotics) originated in the context of Asimov's science fiction stories. In those stories, the three laws serve as a safety measure, in order to avoid untimely or manipulated situations from exploding in havoc.
  
  More often than not, Asimov's narratives would find a way to break them, leading the writer to make several modifications to the laws themselves. For instance, in some of his stories, he [modified the First Law](https://en.wikipedia.org/wiki/Three_Laws_of_Robotics#First_Law_modified), [added a Fourth (or Zeroth) Law](https://en.wikipedia.org/wiki/Three_Laws_of_Robotics#Zeroth_Law_added), or even [removed all Laws altogether](https://en.wikipedia.org/wiki/Three_Laws_of_Robotics#Removal_of_the_Three_Laws).
  
  However, it is easy to argue that, in popular culture, and even in the field of AI research itself, the Laws of Robotics are taken quite seriously. Ignoring the side problem of the different, subjective, and mutually-exclusive interpretations of the laws, are there any arguments proving the laws themselves intrinsically flawed by their design, or, alternatively, strong enough for use in reality? Likewise, has a better, stricter security heuristics set being designed for the purpose?
  
  **Tags:** ai-safety, asimovs-laws
  
  > **Comments:**
  > **username_1**: Beware the Killbot Hellscape! https://xkcd.com/1613/  
  > **username_2**: We musn't forget this possibility: http://www.xkcd.com/1450/  
  > **username_3**: Asimov's laws are just a small attempt at starting the discussion. For a much more thorough discussion, read "Superintelligence" by Nick Bostrom - that's as exhaustive as it gets based on current knowledge and understanding.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_2 on 2016-08-05. Score: 12 | Accepted answer
  
  Asimov's laws are not strong enough to be used in practice. Strength isn't even a consideration, when considering that since they're written in English words would first have to be interpreted subjectively to have any meaning at all. You can find a good discussion of this [here](https://youtu.be/7PKx3kS7f4A).
  
  To transcribe an excerpt:
  
  > How do you define these things? How do you define "human", without first having to take a stand on almost every issue. And if "human" wasn't hard enough, you then have to define "harm", and you've got the same problem again. Almost any really solid unambiguous definitions you give for those words—that don't rely on human intuition—result in weird quirks of philosophy, leading to your AI doing something you really don't want it to do.
  
  One can easily imagine that Asimov was smart enough to know this and was more interested in story-writing than designing real-world AI control protocols.
  
  In the novel [Neuromancer](https://en.wikipedia.org/wiki/Neuromancer#Plot_summary), it was suggested that AIs could possibly serve as checks against each other. Ray Kurzweil's impending [Singularity](https://en.wikipedia.org/wiki/Technological_singularity), or the possibility of hyperintelligent AGIs otherwise, might not leave much of a possibility for humans to control AIs at all, leaving peer-regulation as the only feasible possibility.
  
  It's worth noting that Eliezer Yudkowsky and others ran an [experiment](http://www.yudkowsky.net/singularity/aibox/) wherein Yudkowsky played the role of a superintelligent AI with the ability to speak, but no other connection outside of a locked box. The challengers were tasked simply with keeping the AI in the box at all costs. Yudkowsky escaped both times.
  
  > **Comments:**
  > **username_1**: Do you know if any transcript of the Yudkowsky experiments were ever released? The link you give (and the email exchange it then links to) doesn't appear to provide one.  
  > **username_2**: @user217281728 No, I looked for it but it appears he's keeping it, i.e. his strategy under wraps to maintain the possibility of doing it again. I have some ideas about what he said and why I think that, that I'd be happy to share if you DM me.  
  > **username_1**: I've now done this.  
  > **username_6**: I'm under the impression that Asimov was smart enough to know this, but *the point* was that simple rules don't work. Pointing out negative examples ("you can't just tell it to not harm people, that won't work") is useful real-world AI control work in its own right.  
  
  ---
  
  ### Answer by username_4 on 2016-08-25. Score: 1
  
  Asimov made the three laws **specifically to prove** that no three laws are sufficient, no matter how reasonable they seem at first. I know a guy that knew the guy and he confirmed this.
  
  > **Comments:**
  > **username_0**: "I know a guy who knew the guy"... Don't you have a slightly more reliable source? I'm not doubting your answer, but there should surely already exist a proper quote on the matter.  
  > **username_4**: His name was Ben Williams. This is the Google Plus post where he described his opinions about Asimov and the Three Laws: https://plus.google.com/+JohnNewmanIII/posts/HeTnPdhaY9p  
  
  ---
  
  ### Answer by username_5 on 2016-08-23. Score: 0
  
  Consider Asimov's first law of robotics:
  
  > A robot may not injure a human being or, through inaction, allow a
  > human being to come to harm.
  
  That law is already problematic, when taking into consideration self-driving cars.
  
  What's the issue here, you ask? Well, you'll probably be familiar with the classic thought experiment in ethic known as [the trolley problem](https://en.wikipedia.org/wiki/Trolley_problem). The general form of the problem is this:
  
  > The trolley is headed straight for them. You are standing some
  > distance off in the train yard, next to a lever. If you pull this
  > lever, the trolley will switch to a different set of tracks. However,
  > you notice that there is one person on the side track. You have two
  > options: (1) Do nothing, and the trolley kills the five people on the
  > main track. (2) Pull the lever, diverting the trolley onto the side
  > track where it will kill one person. Which is the correct choice?
  >
  > source : [Wikipedia](https://en.wikipedia.org/wiki/Trolley_problem)
  
  Self-driving cars will actually need to implement real life variations on the trolley problem, which basically means that [self-driving cars need to be programmed to kill human beings](https://www.technologyreview.com/s/542626/why-self-driving-cars-must-be-programmed-to-kill/).
  
  Of course that doesn't mean that ALL robots will need to be programmed to kill, but self-driving cars are a good example of a type of robot that will.
  
  
  ---
  
  