Answers.Body: <p>Neither AI alignment nor AI governance are important yet. We are so far away from AGI that we don't even know what is missing.</p>
  
  <p>We don't set up safety instructions for interstellar travel, so why should we do it for AGI? I can also come up with a lot of dangers of that...</p>
  
  <p>There are real dangers of AI, though. Including <strong>societal issues</strong>:</p>
  
  <ol>
  <li><strong>Blind trust</strong>:  Trusting a machine, although it is not perfect. Just because it is good in most cases and seems to be rational/objective. Example: <a href="https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing" rel="nofollow noreferrer">Machine Bias</a></li>
  <li><strong>Unemployment</strong>: AI has the potential to replace a lot of low-skill jobs with very few super high skilled jobs. For example, self-driving cars (&lt; 100 people for one manufacturer, I guess) could replace all jobs in trasportation (1,076,200 of 31,373,700 in Germany (<a href="https://www.destatis.de/DE/Publikationen/StatistischesJahrbuch/Arbeitsmarkt.pdf?__blob=publicationFile" rel="nofollow noreferrer">source</a>) - that is 3.4%!). See <a href="http://www.cgpgrey.com/blog/humans-need-not-apply" rel="nofollow noreferrer">CG gray video</a>.</li>
  <li><strong>Weapon Systems</strong>: The thread of <a href="https://en.wikipedia.org/wiki/Lethal_autonomous_weapon" rel="nofollow noreferrer">leathal autonomous weapons</a> is real. There is the <a href="https://autonomousweapons.org/slaughterbots/" rel="nofollow noreferrer">danger of efficient slaughterbots</a> and the danger of errors - similar to the <a href="https://en.wikipedia.org/wiki/Wech_Baghtu_wedding_party_airstrike" rel="nofollow noreferrer">Wech Baghtu wedding party airstrike</a> or <a href="https://www.nytimes.com/2018/04/23/world/middleeast/yemen-wedding-bombing.html" rel="nofollow noreferrer">that one</a></li>
  </ol>
  
  <p>There are a couple of other problems, but I think they are less severe:</p>
  
  <ul>
  <li>Better fakes: <a href="https://lyrebird.ai/" rel="nofollow noreferrer">lyrebird.ai</a> gives a good demo</li>
  <li>Spam: I think it's easier to improve the spam filters, but humans might have a harder time</li>
  <li>Data is dominance: Companies like Amazon will have a dominant position on the market as they were the first to aquire important customer information</li>
  <li>Bubbles / false hopes: AI (or let's better say Machine learning)  is often seen as a golden hammer. It is not. </li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-11T13:25:28.037Z
Answers.Text: I don't think you can say `Neither AI alignment nor AI governance are important yet`. Here: https://youtu.be/yQE9KAbFhNY?t=287, Robert Miles makes the point that most experts in the field take those concerns seriously. Source: https://arxiv.org/pdf/1705.08807.pdf at page 13.
Answers.UserDisplayName: Benjamin Crouzier
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-25T05:04:38.193Z
Answers.LastActivityDate: 2018-09-25T06:59:42.090Z
Answers.LastEditDate: 2018-09-25T06:59:42.090Z
Answers.UserDisplayName: Martin Thoma
Body: <p>I have trouble finding material (blog, papers) about this issue, so I'm posting here.</p>
  
  <p>Taking a recent well known example: Musk has tweeted and warned about the potential dangers of AI, saying it is "potentially more dangerous than nukes", referring the issue of creating a superintelligence whose goals are not aligned with ours. This is often illustrated with the <a href="https://wiki.lesswrong.com/wiki/Paperclip_maximizer" rel="nofollow noreferrer">paperclip maximiser though experiment</a>. Let's call this first concern "AI alignment".</p>
  
  <p>By contrast, in <a href="https://www.youtube.com/watch?v=ycPr5-27vSI" rel="nofollow noreferrer">a recent podcast</a>, his concerns seemed more related to getting politicians and decision makers to acknowledge and cooperate on the issue, to avoid potentially dangerous scenarios like an AI arms race. In a paper co-authored by Nick Bostrom: 
  <a href="https://www.fhi.ox.ac.uk/wp-content/uploads/Racing-to-the-precipice-a-model-of-artificial-intelligence-development.pdf" rel="nofollow noreferrer">Racing to the Precipice: a Model of Artificial Intelligence Development</a>, the authors argue that developing AGI in a competitive situation incentivises us to skim on safety precautions, so it is dangerous. Let's call this second concern "AI governance".</p>
  
  <p>My question is about the relative importance between these two issues: AI alignment and AI governance.</p>
  
  <p>It seems that most institutions trying to prevent such risks (<a href="https://intelligence.org/" rel="nofollow noreferrer">MIRI</a>, <a href="https://www.fhi.ox.ac.uk/" rel="nofollow noreferrer">FHI</a>, <a href="https://futureoflife.org/" rel="nofollow noreferrer">FLI</a>, <a href="http://openai.com/" rel="nofollow noreferrer">OpenAI</a>, <a href="https://deepmind.com/" rel="nofollow noreferrer">DeepMind</a> and others) just state their mission without trying to argue about why one approach should be more pressing than the other.</p>
  
  <p>How to assess the relative importance of those two issues? And can you point me any literature about this?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-09-24T21:04:59.447Z
Comments.Text: Welcome to AI!  Informally, most of the scholars I've been reading seem to suggest the social impacts strongly outweigh the technical issues at present.  Specifically, we know that automation is going to replace many, many jobs in the near future, we know that strong forms of AI have more and more control over our behaviors (social media engagement strategies, as an example.) By contrast, apocalyptic scenarios involving AI are still hypothetical.  (Doesn't mean the technical issue is not important, just less immediate.)
Comments.UserDisplayName: DukeZhou
Tags: <agi><social><neo-luddism><value-alignment><risk-management>
Title: Should we focus more on societal or technical issues with AI risk
UserDisplayName: Benjamin Crouzier
ThreadText: # Should we focus more on societal or technical issues with AI risk
  
  (Asked by: username_0 on 2018-09-24)
  
  I have trouble finding material (blog, papers) about this issue, so I'm posting here.
  
  Taking a recent well known example: Musk has tweeted and warned about the potential dangers of AI, saying it is "potentially more dangerous than nukes", referring the issue of creating a superintelligence whose goals are not aligned with ours. This is often illustrated with the [paperclip maximiser though experiment](https://wiki.lesswrong.com/wiki/Paperclip_maximizer). Let's call this first concern "AI alignment".
  
  By contrast, in [a recent podcast](https://www.youtube.com/watch?v=ycPr5-27vSI), his concerns seemed more related to getting politicians and decision makers to acknowledge and cooperate on the issue, to avoid potentially dangerous scenarios like an AI arms race. In a paper co-authored by Nick Bostrom:
  [Racing to the Precipice: a Model of Artificial Intelligence Development](https://www.fhi.ox.ac.uk/wp-content/uploads/Racing-to-the-precipice-a-model-of-artificial-intelligence-development.pdf), the authors argue that developing AGI in a competitive situation incentivises us to skim on safety precautions, so it is dangerous. Let's call this second concern "AI governance".
  
  My question is about the relative importance between these two issues: AI alignment and AI governance.
  
  It seems that most institutions trying to prevent such risks ([MIRI](https://intelligence.org/), [FHI](https://www.fhi.ox.ac.uk/), [FLI](https://futureoflife.org/), [OpenAI](http://openai.com/), [DeepMind](https://deepmind.com/) and others) just state their mission without trying to argue about why one approach should be more pressing than the other.
  
  How to assess the relative importance of those two issues? And can you point me any literature about this?
  
  **Tags:** agi, social, neo-luddism, value-alignment, risk-management
  > **Comments:**
  > **username_1**: Welcome to AI!  Informally, most of the scholars I've been reading seem to suggest the social impacts strongly outweigh the technical issues at present.  Specifically, we know that automation is going to replace many, many jobs in the near future, we know that strong forms of AI have more and more control over our behaviors (social media engagement strategies, as an example.) By contrast, apocalyptic scenarios involving AI are still hypothetical.  (Doesn't mean the technical issue is not important, just less immediate.)  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2018-09-25. Score: 1
  
  Neither AI alignment nor AI governance are important yet. We are so far away from AGI that we don't even know what is missing.
  
  We don't set up safety instructions for interstellar travel, so why should we do it for AGI? I can also come up with a lot of dangers of that...
  
  There are real dangers of AI, though. Including **societal issues**:
  
  1. **Blind trust**: Trusting a machine, although it is not perfect. Just because it is good in most cases and seems to be rational/objective. Example: [Machine Bias](https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing)
  2. **Unemployment**: AI has the potential to replace a lot of low-skill jobs with very few super high skilled jobs. For example, self-driving cars (< 100 people for one manufacturer, I guess) could replace all jobs in trasportation (1,076,200 of 31,373,700 in Germany ([source](https://www.destatis.de/DE/Publikationen/StatistischesJahrbuch/Arbeitsmarkt.pdf?__blob=publicationFile)) - that is 3.4%!). See [CG gray video](http://www.cgpgrey.com/blog/humans-need-not-apply).
  3. **Weapon Systems**: The thread of [leathal autonomous weapons](https://en.wikipedia.org/wiki/Lethal_autonomous_weapon) is real. There is the [danger of efficient slaughterbots](https://autonomousweapons.org/slaughterbots/) and the danger of errors - similar to the [Wech Baghtu wedding party airstrike](https://en.wikipedia.org/wiki/Wech_Baghtu_wedding_party_airstrike) or [that one](https://www.nytimes.com/2018/04/23/world/middleeast/yemen-wedding-bombing.html)
  
  There are a couple of other problems, but I think they are less severe:
  
  * Better fakes: [lyrebird.ai](https://lyrebird.ai/) gives a good demo
  * Spam: I think it's easier to improve the spam filters, but humans might have a harder time
  * Data is dominance: Companies like Amazon will have a dominant position on the market as they were the first to aquire important customer information
  * Bubbles / false hopes: AI (or let's better say Machine learning) is often seen as a golden hammer. It is not.
  
  > **Comments:**
  > **username_0**: I don't think you can say `Neither AI alignment nor AI governance are important yet`. Here: https://youtu.be/yQE9KAbFhNY?t=287, Robert Miles makes the point that most experts in the field take those concerns seriously. Source: https://arxiv.org/pdf/1705.08807.pdf at page 13.  
  
  ---
  
  