Answers.Body: <p>If a machine learning-based AI is &quot;sufficiently smart enough&quot; to be able to lie, then there is nothing preventing it from lying. This does not mean it can't be persuaded from lying.</p>
  <p>So just make the AI simple enough to not be able to lie.</p>
  <p>The reasoning here is that in order for a system to be able to lie, a system must be able to recognize an incentive to lie. Recognizing this incentive is a challenging function and would be impossible to code manually into a computer. Machine learning can be applied to problems such as these where the function is hard to code manually. Although there has been promising work on understanding what the representations/features learned by machine learning actually represent, it may not be possible in general to have an understanding of what a lie in the agent's representation looks like. Because of this, having a hand-coded rule to catch when an agent is lying is not possible and thus being able to prevent an agent from (or catch an agent when) lying isn't possible when using machine learning.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-07T09:17:48.157Z
Answers.Text: The advantage of Robots that they can't lie, is gone with coming of AI ???
Answers.UserDisplayName: TontyTon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-07T11:07:50.410Z
Answers.Text: Yes however lies need not always be bad. Telling a kid that their drawings are good when they aren't is also a lie but encourages the kid to continue drawing. Having AI that can descern this kind of nuance is important to Artificial General Intelligence.
Answers.UserDisplayName: Jaden Travnik
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-07T13:52:42.717Z
Answers.Text: Good distinguishment between beneficial lying and its harmful variant.
Answers.UserDisplayName: FreezePhoenix
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-08T07:42:48.377Z
Answers.Text: @JadenTravnik "Good distinguishment between beneficial lying and its harmful variant", I support this. But, we can't be sure that they will use lie for only other's benefit.
Answers.UserDisplayName: TontyTon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-07T03:09:12.633Z
Answers.LastActivityDate: 2022-01-03T09:49:28.400Z
Answers.LastEditDate: 2022-01-03T09:49:28.400Z
Answers.UserDisplayName: Jaden Travnik
Answers.Body: <p>You may be interested in the utility functions of deception:</p>
  
  <p>From the abstract of <a href="https://www.ncbi.nlm.nih.gov/pubmed/17109314" rel="nofollow noreferrer">Why Animals Lie: How Dishonesty and Belief can Coexist in a Signaling System. (NIH, 2006)</a>"</p>
  
  <blockquote>
    <p>We develop and apply a simple model for animal communication in which signalers can use a nontrivial frequency of deception without causing listeners to completely lose belief. This common feature of animal communication has been difficult to explain as a stable adaptive outcome of the options and payoffs intrinsic to signaling interactions. Our theory is based on two realistic assumptions. (1) Signals are "overheard" by several listeners or listener types with different payoffs. The signaler may then benefit from using incomplete honesty to elicit different responses from different listener types, such as attracting potential mates while simultaneously deterring competitors. (2) Signaler and listener strategies change dynamically in response to current payoffs for different behaviors. The dynamic equations can be interpreted as describing learning and behavior change by individuals or evolution across generations. We explain how our dynamic model differs from other solution concepts from classical and evolutionary game theory and how it relates to general models for frequency-dependent phenotype dynamics. We illustrate the theory with several applications where deceptive signaling occurs readily in our framework, including bluffing competitors for potential mates or territories. We suggest future theoretical directions to make the models more general and propose some possible experimental tests.</p>
  </blockquote>
  
  <p>A degree of deceptive capability seems to be beneficial from the standpoint of evolution.  </p>
  
  <p>We humans are not always known for veracity, so the ability understand deception might be a critical component in Artificial General Intelligence's ability to interact with humans.  (Specifically, you can't always believe what humans tell you.)</p>
  
  <p>Based on recent human history, the recognition of the unreliability of humans (versus data and as-objective-as-possible analysis) may become critical to the survival of our own species.  </p>
  
  <p><strong>More importantly, it will be essential for strong AI to understand that the "data can lie"</strong> (faulty parameters, inaccurate data, unawareness of incomplete information.)</p>
  
  <p>JT's answer is a great functional overview on why it's not possible with current methods.  <strong>This answer might be regarded in the sense that, aside from very limited special cases such as solved games where true objectivity can be achieved, reality is subjective and "truth" is a subjective function of the parameters and data.</strong></p>
  
  <p>Again, understanding that last bit is likely much more important than trying to code AI's not to "lie". </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-08T07:49:24.363Z
Answers.Text: Pls., 'Answers' should contain answers only, not related topics. Your Answer does not imply to my Question.
Answers.UserDisplayName: TontyTon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-08T18:23:32.890Z
Answers.Text: @TontyTon Thanks for your comment.  I've amended slightly to reinforce my point.
Answers.UserDisplayName: DukeZhou
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-07T18:20:50.980Z
Answers.LastActivityDate: 2018-05-08T18:21:21.590Z
Answers.LastEditDate: 2018-05-08T18:21:21.590Z
Answers.UserDisplayName: DukeZhou
Body: <p>According to <a href="https://www.popsci.com/scitech/article/2009-08/evolving-robots-learn-lie-hide-resources-each-other" rel="nofollow noreferrer">this blog post</a>, it seems that AI systems can lie. However, can an AI be programmed in such a way that it never lies (even after learning new things)?</p>
  
Tags: <machine-learning><ethics>
Title: Can an AI be programmed not to lie?
UserDisplayName: TontyTon
ThreadText: # Can an AI be programmed not to lie?
  
  (Asked by: username_0 on 2018-05-06)
  
  According to [this blog post](https://www.popsci.com/scitech/article/2009-08/evolving-robots-learn-lie-hide-resources-each-other), it seems that AI systems can lie. However, can an AI be programmed in such a way that it never lies (even after learning new things)?
  
  **Tags:** machine-learning, ethics
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2018-05-07. Score: 4
  
  If a machine learning-based AI is "sufficiently smart enough" to be able to lie, then there is nothing preventing it from lying. This does not mean it can't be persuaded from lying.
  
  So just make the AI simple enough to not be able to lie.
  
  The reasoning here is that in order for a system to be able to lie, a system must be able to recognize an incentive to lie. Recognizing this incentive is a challenging function and would be impossible to code manually into a computer. Machine learning can be applied to problems such as these where the function is hard to code manually. Although there has been promising work on understanding what the representations/features learned by machine learning actually represent, it may not be possible in general to have an understanding of what a lie in the agent's representation looks like. Because of this, having a hand-coded rule to catch when an agent is lying is not possible and thus being able to prevent an agent from (or catch an agent when) lying isn't possible when using machine learning.
  
  > **Comments:**
  > **username_0**: The advantage of Robots that they can't lie, is gone with coming of AI ???  
  > **username_1**: Yes however lies need not always be bad. Telling a kid that their drawings are good when they aren't is also a lie but encourages the kid to continue drawing. Having AI that can descern this kind of nuance is important to Artificial General Intelligence.  
  > **username_2**: Good distinguishment between beneficial lying and its harmful variant.  
  > **username_0**: @username_1 "Good distinguishment between beneficial lying and its harmful variant", I support this. But, we can't be sure that they will use lie for only other's benefit.  
  
  ---
  
  ### Answer by username_3 on 2018-05-07. Score: 0
  
  You may be interested in the utility functions of deception:
  
  From the abstract of [Why Animals Lie: How Dishonesty and Belief can Coexist in a Signaling System. (NIH, 2006)](https://www.ncbi.nlm.nih.gov/pubmed/17109314)"
  
  > We develop and apply a simple model for animal communication in which signalers can use a nontrivial frequency of deception without causing listeners to completely lose belief. This common feature of animal communication has been difficult to explain as a stable adaptive outcome of the options and payoffs intrinsic to signaling interactions. Our theory is based on two realistic assumptions. (1) Signals are "overheard" by several listeners or listener types with different payoffs. The signaler may then benefit from using incomplete honesty to elicit different responses from different listener types, such as attracting potential mates while simultaneously deterring competitors. (2) Signaler and listener strategies change dynamically in response to current payoffs for different behaviors. The dynamic equations can be interpreted as describing learning and behavior change by individuals or evolution across generations. We explain how our dynamic model differs from other solution concepts from classical and evolutionary game theory and how it relates to general models for frequency-dependent phenotype dynamics. We illustrate the theory with several applications where deceptive signaling occurs readily in our framework, including bluffing competitors for potential mates or territories. We suggest future theoretical directions to make the models more general and propose some possible experimental tests.
  
  A degree of deceptive capability seems to be beneficial from the standpoint of evolution.
  
  We humans are not always known for veracity, so the ability understand deception might be a critical component in Artificial General Intelligence's ability to interact with humans. (Specifically, you can't always believe what humans tell you.)
  
  Based on recent human history, the recognition of the unreliability of humans (versus data and as-objective-as-possible analysis) may become critical to the survival of our own species.
  
  **More importantly, it will be essential for strong AI to understand that the "data can lie"** (faulty parameters, inaccurate data, unawareness of incomplete information.)
  
  JT's answer is a great functional overview on why it's not possible with current methods. **This answer might be regarded in the sense that, aside from very limited special cases such as solved games where true objectivity can be achieved, reality is subjective and "truth" is a subjective function of the parameters and data.**
  
  Again, understanding that last bit is likely much more important than trying to code AI's not to "lie".
  
  > **Comments:**
  > **username_0**: Pls., 'Answers' should contain answers only, not related topics. Your Answer does not imply to my Question.  
  > **username_3**: @username_0 Thanks for your comment.  I've amended slightly to reinforce my point.  
  
  ---
  
  