Answers.Body: <p>Ethics involves the relationships of <em>needs</em> between two or more parties. As Matthew Graves said, if the AI lacks the sufficient human context (understanding of needs), it will produce seemingly perverse ethical behavior.</p>
  
  <p>And let's be honest, some <em>people</em> would cut of other people's arms and put them on pressure plates. Even the best of us will not be able to sympathize with the needs of others with 100% accuracy - at best, we're guessing. And then there are those rare situations where I actually <em>want</em> you to cut off my arm and put it on a pressure plate, perhaps to save a loved one.</p>
  
  <p>If we could make a thing that could sympathize with what a human <em>might</em> need in any given arbitrary situation, then we will have created either A) an artificial <em>human</em> intelligence (AHI) (which could be more or less fallible, like a human), or B) an oracle that can reason about <em>all possible human needs</em> on much faster than human time-scales - in which case you wouldn't <em>need</em> a conscious AI, as all human needs and solutions could be pre-computed via formal specification, which is probably absurd to consider.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-09-30T18:40:58.730Z
Answers.LastActivityDate: 2016-09-30T18:40:58.730Z
Answers.UserDisplayName: Doxosophoi
Answers.Body: <p>You may consider the programming as an ethical part of the design as well. AI will act based on what has been instructed to it as ethically important or not.
  It may/should even be part of the parameters that forge the process of finding solutions, which could allow for a more refine and creative solution.</p>
  
  <p>We understand the basics of ethic in normal circumstances, but if we can't predict how any human will behave in an ethical conundrum we can enforce what an AI wouldn't do.</p>
  
  <p>As long as we have control over the mechanism that drive an AI we sure have a responsability to inject ethical failsafes.
  The problem lies in self taught AI with an ability to overrides directives.
  (CF Asimov Laws.)</p>
  
  <p>The way the AI is creative seems irrelevant in that case.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-09-27T22:27:38.610Z
Answers.LastActivityDate: 2016-09-27T22:27:38.610Z
Answers.UserDisplayName: Vinh NPL
Answers.Body: <p>A lot of this depends on the breadth of consideration. For example, what would the medium and long term effects of the lateral thinking be? The robot could sever an arm for a pressure plate but it would mean that the person no longer had an arm, a functional limitation at best, that the person might bleed out and die/be severely constrained, and that the person (and people in general) would both no longer cooperate and likely seek to eliminate the robot. People can think laterally because consider these things - ethics are really nothing more than a set of guidelines that encompass these considerations. The robot could as well, were it to be designed to consider these externalities.</p>
  
  <p>If all else fails,</p>
  
  <p>Asimov's Laws of Robotics: (0. A robot may not harm humanity, or, by inaction, allow humanity to come to harm.) 1. A robot may not injure a human being or, through inaction, allow a human being to come to harm. 2. A robot must obey orders given it by human beings except where such orders would conflict with the First Law. 3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-09-28T00:32:47.923Z
Answers.LastActivityDate: 2016-09-28T00:32:47.923Z
Answers.UserDisplayName: John
Answers.Body: <p><strong>No</strong>, with a <em>but</em>. We can have creative yet ethical problem-solving if the system has a complete system of ethics, but otherwise creativity will be unsafe by default.</p>
  
  <p>One can classify AI decision-making approaches into two types: interpolative thinkers, and extrapolative thinkers. </p>
  
  <p>Interpolative thinkers learn to classify and mimic whatever they're learning from, and don't try to give reasonable results outside of their training domain. You can think of them as interpolating between training examples, and benefitting from all of the mathematical guarantees and provisos as other statistical techniques.</p>
  
  <p>Extrapolative thinkers learn to manipulate underlying principles, which allows them to combine those principles in previously unconsidered ways. The relevant field for intuition here is <a href="https://en.wikipedia.org/wiki/Mathematical_optimization">numerical optimization</a>, of which the simplest and most famous example is <a href="https://en.wikipedia.org/wiki/Linear_programming">linear programming</a>, rather than the statistical fields that birthed machine learning. You can think of them as extrapolating beyond training examples (indeed, many of them don't even require training examples, or use those examples to infer underlying principles).</p>
  
  <p>The promise of extrapolative thinkers is that they can come up with these 'lateral' solutions much more quickly than people would be able to. The problem with these extrapolative thinkers is that they only use the spoken principles, not any unspoken ones that might seem too obvious to mention.</p>
  
  <p>An attribute of solutions to optimization problems is that the feature vector is often 'extreme' in some way. In linear programming, at least one vertex of the feasible solution space will be optimal, and so simple solution methods find an optimal vertex (which is almost infeasible by nature of being a vertex).</p>
  
  <p>As another example, the minimum-fuel solution for moving a spacecraft from one position to another is called '<a href="https://en.wikipedia.org/wiki/Bang%E2%80%93bang_control">bang-bang</a>,' where you accelerate the craft as quickly as possible at the beginning and end of the trajectory, coasting at maximum speed in between.</p>
  
  <p>While a virtue when the system is correctly understood (bang-bang <em>is</em> optimal for many cases), this is catastrophic when the system is incorrectly understood. My favorite example here is <a href="https://resources.mpi-inf.mpg.de/departments/d1/teaching/ws14/Ideen-der-Informatik/Dantzig-Diet.pdf">Dantzig's diet problem</a> (discussion starts on page 5 of the pdf), where he tries to optimize his diet using math. Under his first constraint set, he's supposed to drink 500 gallons of vinegar a day. Under his second, 200 bouillon cubes. Under his third, two pounds of bran. The considerations that make those obviously bad ideas aren't baked into the system, and so the system innocently suggests them.</p>
  
  <p>If you can completely encode the knowledge and values that a person uses to judge these plans into the AI, then extrapolative systems are as safe as that person. They'll be able to consider and reject the wrong sort of extreme plans, and leave you with the right sort of extreme plans.</p>
  
  <p>But if you can't, then it does make sense to not build an extrapolative decision-maker, and instead build an interpolative one. That is, instead of asking itself "how do I best accomplish goal X?" it's asking itself "what would a person do in this situation?". The latter might be much worse at accomplishing goal X, but it has much less of the tail risk of sacrificing other goals to accomplish X.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-09-27T17:50:12.263Z
Answers.LastActivityDate: 2016-09-27T19:32:53.843Z
Answers.LastEditDate: 2016-09-27T19:32:53.843Z
Answers.UserDisplayName: Matthew Gray
Body: <p>In the recent PC game <em><a href="http://www.theturingtestgame.com/">The Turing Test</a></em>, the AI ("TOM") needs help from Ava to get through some puzzle rooms. TOM says he is unable to solve the puzzles because he is not allowed to "<a href="https://en.wikipedia.org/wiki/Lateral_thinking">think laterally</a>." Specifically, he says he would not have thought to throw a box through a window to solve the first room. His creators, the story goes, turned that capability off because such thinking could produce "ethically suboptimal" solutions, like chopping off an arm to leave on a pressure plate.</p>
  
  <p>Would all creative puzzle-solving abilities need to be removed from an AI to keep its results reasonable, or could we get some benefits of lateral thinking without losing an arm?</p>
  
Tags: <agi><problem-solving>
Title: Could an AI think laterally while avoiding "ethically suboptimal" choices?
UserDisplayName: Ben N
ThreadText: # Could an AI think laterally while avoiding "ethically suboptimal" choices?
  
  (Asked by: username_0 on 2016-09-27)
  
  In the recent PC game *[The Turing Test](http://www.theturingtestgame.com/)*, the AI ("TOM") needs help from Ava to get through some puzzle rooms. TOM says he is unable to solve the puzzles because he is not allowed to "[think laterally](https://en.wikipedia.org/wiki/Lateral_thinking)." Specifically, he says he would not have thought to throw a box through a window to solve the first room. His creators, the story goes, turned that capability off because such thinking could produce "ethically suboptimal" solutions, like chopping off an arm to leave on a pressure plate.
  
  Would all creative puzzle-solving abilities need to be removed from an AI to keep its results reasonable, or could we get some benefits of lateral thinking without losing an arm?
  
  **Tags:** agi, problem-solving
  
  ---
  
  ## 4 answers
  
  ### Answer by username_4 on 2016-09-27. Score: 16 | Accepted answer
  
  **No**, with a *but*. We can have creative yet ethical problem-solving if the system has a complete system of ethics, but otherwise creativity will be unsafe by default.
  
  One can classify AI decision-making approaches into two types: interpolative thinkers, and extrapolative thinkers.
  
  Interpolative thinkers learn to classify and mimic whatever they're learning from, and don't try to give reasonable results outside of their training domain. You can think of them as interpolating between training examples, and benefitting from all of the mathematical guarantees and provisos as other statistical techniques.
  
  Extrapolative thinkers learn to manipulate underlying principles, which allows them to combine those principles in previously unconsidered ways. The relevant field for intuition here is [numerical optimization](https://en.wikipedia.org/wiki/Mathematical_optimization), of which the simplest and most famous example is [linear programming](https://en.wikipedia.org/wiki/Linear_programming), rather than the statistical fields that birthed machine learning. You can think of them as extrapolating beyond training examples (indeed, many of them don't even require training examples, or use those examples to infer underlying principles).
  
  The promise of extrapolative thinkers is that they can come up with these 'lateral' solutions much more quickly than people would be able to. The problem with these extrapolative thinkers is that they only use the spoken principles, not any unspoken ones that might seem too obvious to mention.
  
  An attribute of solutions to optimization problems is that the feature vector is often 'extreme' in some way. In linear programming, at least one vertex of the feasible solution space will be optimal, and so simple solution methods find an optimal vertex (which is almost infeasible by nature of being a vertex).
  
  As another example, the minimum-fuel solution for moving a spacecraft from one position to another is called '[bang-bang](https://en.wikipedia.org/wiki/Bang%E2%80%93bang_control),' where you accelerate the craft as quickly as possible at the beginning and end of the trajectory, coasting at maximum speed in between.
  
  While a virtue when the system is correctly understood (bang-bang *is* optimal for many cases), this is catastrophic when the system is incorrectly understood. My favorite example here is [Dantzig's diet problem](https://resources.mpi-inf.mpg.de/departments/d1/teaching/ws14/Ideen-der-Informatik/Dantzig-Diet.pdf) (discussion starts on page 5 of the pdf), where he tries to optimize his diet using math. Under his first constraint set, he's supposed to drink 500 gallons of vinegar a day. Under his second, 200 bouillon cubes. Under his third, two pounds of bran. The considerations that make those obviously bad ideas aren't baked into the system, and so the system innocently suggests them.
  
  If you can completely encode the knowledge and values that a person uses to judge these plans into the AI, then extrapolative systems are as safe as that person. They'll be able to consider and reject the wrong sort of extreme plans, and leave you with the right sort of extreme plans.
  
  But if you can't, then it does make sense to not build an extrapolative decision-maker, and instead build an interpolative one. That is, instead of asking itself "how do I best accomplish goal X?" it's asking itself "what would a person do in this situation?". The latter might be much worse at accomplishing goal X, but it has much less of the tail risk of sacrificing other goals to accomplish X.
  
  
  ---
  
  ### Answer by username_1 on 2016-09-30. Score: 1
  
  Ethics involves the relationships of *needs* between two or more parties. As Matthew Graves said, if the AI lacks the sufficient human context (understanding of needs), it will produce seemingly perverse ethical behavior.
  
  And let's be honest, some *people* would cut of other people's arms and put them on pressure plates. Even the best of us will not be able to sympathize with the needs of others with 100% accuracy - at best, we're guessing. And then there are those rare situations where I actually *want* you to cut off my arm and put it on a pressure plate, perhaps to save a loved one.
  
  If we could make a thing that could sympathize with what a human *might* need in any given arbitrary situation, then we will have created either A) an artificial *human* intelligence (AHI) (which could be more or less fallible, like a human), or B) an oracle that can reason about *all possible human needs* on much faster than human time-scales - in which case you wouldn't *need* a conscious AI, as all human needs and solutions could be pre-computed via formal specification, which is probably absurd to consider.
  
  
  ---
  
  ### Answer by username_2 on 2016-09-27. Score: 0
  
  You may consider the programming as an ethical part of the design as well. AI will act based on what has been instructed to it as ethically important or not.
  It may/should even be part of the parameters that forge the process of finding solutions, which could allow for a more refine and creative solution.
  
  We understand the basics of ethic in normal circumstances, but if we can't predict how any human will behave in an ethical conundrum we can enforce what an AI wouldn't do.
  
  As long as we have control over the mechanism that drive an AI we sure have a responsability to inject ethical failsafes.
  The problem lies in self taught AI with an ability to overrides directives.
  (CF Asimov Laws.)
  
  The way the AI is creative seems irrelevant in that case.
  
  
  ---
  
  ### Answer by username_3 on 2016-09-28. Score: -1
  
  A lot of this depends on the breadth of consideration. For example, what would the medium and long term effects of the lateral thinking be? The robot could sever an arm for a pressure plate but it would mean that the person no longer had an arm, a functional limitation at best, that the person might bleed out and die/be severely constrained, and that the person (and people in general) would both no longer cooperate and likely seek to eliminate the robot. People can think laterally because consider these things - ethics are really nothing more than a set of guidelines that encompass these considerations. The robot could as well, were it to be designed to consider these externalities.
  
  If all else fails,
  
  Asimov's Laws of Robotics: (0. A robot may not harm humanity, or, by inaction, allow humanity to come to harm.) 1. A robot may not injure a human being or, through inaction, allow a human being to come to harm. 2. A robot must obey orders given it by human beings except where such orders would conflict with the First Law. 3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law
  
  
  ---
  
  