Answers.Body: <p>I believe the term you are looking for is "(technological) singularity".</p>
  
  <p><a href="https://en.wikipedia.org/wiki/Technological_singularity" rel="noreferrer">https://en.wikipedia.org/wiki/Technological_singularity</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-12T11:39:30.087Z
Answers.LastActivityDate: 2020-03-12T11:39:30.087Z
Answers.UserDisplayName: tripleee
Answers.Body: <p>The likely expression you are looking for is <em>AI takeover</em>, which is a common topic in <a href="https://en.wikipedia.org/wiki/Artificial_intelligence_in_fiction" rel="noreferrer">science fiction</a> movies, such as <a href="https://en.wikipedia.org/wiki/2001:_A_Space_Odyssey_(film)" rel="noreferrer">2001: A Space Odyssey</a> and <a href="https://en.wikipedia.org/wiki/The_Matrix" rel="noreferrer">The Matrix</a>, and <a href="https://en.wikipedia.org/wiki/AI_takeovers_in_popular_culture" rel="noreferrer">popular culture</a>. Although the AI takeover is an <strong>unlikely scenario</strong> in the next years, <a href="https://en.wikipedia.org/wiki/Open_Letter_on_Artificial_Intelligence" rel="noreferrer">certain scientists, such as Stephen Hawking, have expressed concerns about it</a> and some philosophers, especially Nick Bostrom, are really interested in the topic.</p>
  
  <p>The AI takeover concept is related to concepts such as the <a href="https://en.wikipedia.org/wiki/Technological_singularity" rel="noreferrer">AI singularity</a>, <a href="https://en.wikipedia.org/wiki/Superintelligence" rel="noreferrer">superintelligence</a>, <a href="https://wiki.lesswrong.com/wiki/Intelligence_explosion" rel="noreferrer">intelligence explosion</a>, <a href="https://en.wikipedia.org/wiki/AI_control_problem" rel="noreferrer">AI control problem</a>, <a href="https://en.wikipedia.org/wiki/Existential_risk_from_artificial_general_intelligence" rel="noreferrer">existential risk</a>, <a href="https://en.wikipedia.org/wiki/Machine_ethics" rel="noreferrer">machine ethics</a> and <a href="https://en.wikipedia.org/wiki/Friendly_artificial_intelligence" rel="noreferrer">friendly AI</a>.</p>
  
  <p>The book <a href="https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies" rel="noreferrer">Superintelligence: Paths, Dangers, Strategies</a> (2014) by N. Bostrom may be helpful if you are interested in hypothetical scenarios.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-12T20:55:47.090Z
Answers.Text: It sounds like he's more worried about the AI becoming *SELF AWARE*.
Answers.UserDisplayName: Hannover Fist
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-12T14:27:15.830Z
Answers.LastActivityDate: 2020-03-12T14:32:55.437Z
Answers.LastEditDate: 2020-03-12T14:32:55.437Z
Answers.UserDisplayName: nbro
Answers.Body: <p>If you think about it, it is already happening.</p>
  
  <ul>
  <li>Thousands of drivers work for Uber Intelligence.</li>
  <li>There are many applications that dictate the rules and define what the seller and the end user need to do</li>
  </ul>
  
  <p>This idea is called Singularity or <a href="https://en.wikipedia.org/wiki/Technological_singularity" rel="nofollow noreferrer">Technological Singularity</a>, it would be possible with a <a href="https://en.wikipedia.org/wiki/Superintelligence" rel="nofollow noreferrer">Superintelligence</a>.</p>
  
  <p>However, the possibility of this happening is unknown. Have they reached that level yet? We have quantum computers, we have companies with huge data centers spread all over the planet, we have technology in space, we have free Tensorflow and studies for anyone on the planet to be able to create artificial intelligence models.</p>
  
  <p>If we have contact or help from other intelligent civilizations, the possibilities can expand on a surreal scale.</p>
  
  <p>Google has complete information about humanity (or something close to that). But even with all this data, creating artificial intelligence with a conscience is something that goes far beyond.</p>
  
  <p>But maybe we already have enough to improve our concept of morals, respect, social interaction. Facebook invests and studies ways to improve social interaction. And if you think about it, it is one of the main means of communication.</p>
  
  <p>The big question is that it is not possible to know what would happen if a super artificial intelligence with conscience would do if it existed. Extinguish humans for harming nature? Just find ways to improve the planet by understanding that human defects and errors are just your own nature as well as everything else in nature? Just watching the show on Netflix because you gave up on humanity? We do not know.</p>
  
  <p>But, particularly, I would love to see that happen. In fact, one of my personal goals is to create this super intelligence. But alone it will be very difficult. A conscience without interaction from other consciences is not a clash of universes. And the clash of universes is what makes us reflect, think, revise, create new paths and thoughts. It is what allows us to create other universes.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-13T17:14:28.890Z
Answers.Text: So the AI could be distributed, emergent, hosted on man and machine alike, sort of a conscious [meme](https://en.wikipedia.org/wiki/Meme) (and I don't mean an image with a funny text). Sort of what Susan Blackmore [talked about.](https://www.ted.com/talks/susan_blackmore_memes_and_temes)
Answers.UserDisplayName: Peter - Reinstate Monica
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-13T17:53:31.150Z
Answers.Text: One interesting thing about "The Singularity" is that it's always considered to relate to AI.  But the actual definition is "*a hypothetical future point in time at which technological growth becomes uncontrollable and irreversible, resulting in unforeseeable changes to human civilization*".  By that definition, there is no "The Singularity", just lots of small singularities.  The horse collar was a singularity.  The concept of selective breeding was a singularity.  Railways, roads, cars, telephones, mobile phones, the internet... every one is a singularity.  AI is just another one of many.
Answers.UserDisplayName: Graham
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-13T13:15:01.447Z
Answers.LastActivityDate: 2020-03-13T13:15:01.447Z
Answers.UserDisplayName: GIA
Answers.Body: <p>If I'm not mistaken you're looking for <a href="https://en.wikipedia.org/wiki/LessWrong#Roko&#39;s_basilisk" rel="noreferrer">Roko's Basilisk</a>, </p>
  
  <blockquote>
    <p>in which an otherwise benevolent future AI system tortures simulations of those who did not work to bring the system into existence</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-26T17:04:35.510Z
Answers.Text: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/118967/discussion-on-answer-by-oso-what-is-the-idea-called-involving-an-ai-that-will-ev).
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-12T18:09:04.770Z
Answers.LastActivityDate: 2020-03-12T18:09:04.770Z
Answers.UserDisplayName: Andy
Body: <p>It's an idea I heard a while back but couldn't remember the name of. It involves the existence and development of an AI that will eventually rule the world and that if you don't fund or progress the AI then it will see you as "hostile" and kill you. Also, by knowing about this concept, it essentially makes you a candidate for such consideration, as people who didn't know about it won't understand to progress such an AI. From my understanding, this idea isn't taken that seriously, but I'm curious to know the name nonetheless.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-15T02:22:35.213Z
Comments.Text: Why would a sentient AI bother "ruling" humanity? Once sentient, the AI would realize that the majority of the problems on Earth are caused by humans. Fortunately for the AI it can survive nearly anywhere - so I expect the first and last thing the AI would do would be to build a fleet of spacecraft, clone itself into each of them, and blast off in different directions to maximize its chances of survival. Humanity? Schmoomanity! Who needs 'em?!?
Comments.UserDisplayName: Bob Jarvis - Слава Україні
Tags: <philosophy><agi><definitions><superintelligence><singularity>
Title: What is the idea called involving an AI that will eventually rule humanity?
UserDisplayName: Displaza
ThreadText: # What is the idea called involving an AI that will eventually rule humanity?
  
  (Asked by: username_0 on 2020-03-12)
  
  It's an idea I heard a while back but couldn't remember the name of. It involves the existence and development of an AI that will eventually rule the world and that if you don't fund or progress the AI then it will see you as "hostile" and kill you. Also, by knowing about this concept, it essentially makes you a candidate for such consideration, as people who didn't know about it won't understand to progress such an AI. From my understanding, this idea isn't taken that seriously, but I'm curious to know the name nonetheless.
  
  **Tags:** philosophy, agi, definitions, superintelligence, singularity
  > **Comments:**
  > **username_1**: Why would a sentient AI bother "ruling" humanity? Once sentient, the AI would realize that the majority of the problems on Earth are caused by humans. Fortunately for the AI it can survive nearly anywhere - so I expect the first and last thing the AI would do would be to build a fleet of spacecraft, clone itself into each of them, and blast off in different directions to maximize its chances of survival. Humanity? Schmoomanity! Who needs 'em?!?  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_8 on 2020-03-12. Score: 25 | Accepted answer
  
  If I'm not mistaken you're looking for [Roko's Basilisk](https://en.wikipedia.org/wiki/LessWrong#Roko's_basilisk),
  
  > in which an otherwise benevolent future AI system tortures simulations of those who did not work to bring the system into existence
  
  > **Comments:**
  > **username_3**: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/118967/discussion-on-answer-by-oso-what-is-the-idea-called-involving-an-ai-that-will-ev).  
  
  ---
  
  ### Answer by username_2 on 2020-03-12. Score: 10
  
  I believe the term you are looking for is "(technological) singularity".
  
  <https://en.wikipedia.org/wiki/Technological_singularity>
  
  
  ---
  
  ### Answer by username_3 on 2020-03-12. Score: 7
  
  The likely expression you are looking for is *AI takeover*, which is a common topic in [science fiction](https://en.wikipedia.org/wiki/Artificial_intelligence_in_fiction) movies, such as [2001: A Space Odyssey](https://en.wikipedia.org/wiki/2001:_A_Space_Odyssey_(film)) and [The Matrix](https://en.wikipedia.org/wiki/The_Matrix), and [popular culture](https://en.wikipedia.org/wiki/AI_takeovers_in_popular_culture). Although the AI takeover is an **unlikely scenario** in the next years, [certain scientists, such as Stephen Hawking, have expressed concerns about it](https://en.wikipedia.org/wiki/Open_Letter_on_Artificial_Intelligence) and some philosophers, especially Nick Bostrom, are really interested in the topic.
  
  The AI takeover concept is related to concepts such as the [AI singularity](https://en.wikipedia.org/wiki/Technological_singularity), [superintelligence](https://en.wikipedia.org/wiki/Superintelligence), [intelligence explosion](https://wiki.lesswrong.com/wiki/Intelligence_explosion), [AI control problem](https://en.wikipedia.org/wiki/AI_control_problem), [existential risk](https://en.wikipedia.org/wiki/Existential_risk_from_artificial_general_intelligence), [machine ethics](https://en.wikipedia.org/wiki/Machine_ethics) and [friendly AI](https://en.wikipedia.org/wiki/Friendly_artificial_intelligence).
  
  The book [Superintelligence: Paths, Dangers, Strategies](https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies) (2014) by N. Bostrom may be helpful if you are interested in hypothetical scenarios.
  
  > **Comments:**
  > **username_4**: It sounds like he's more worried about the AI becoming *SELF AWARE*.  
  
  ---
  
  ### Answer by username_5 on 2020-03-13. Score: 1
  
  If you think about it, it is already happening.
  
  * Thousands of drivers work for Uber Intelligence.
  * There are many applications that dictate the rules and define what the seller and the end user need to do
  
  This idea is called Singularity or [Technological Singularity](https://en.wikipedia.org/wiki/Technological_singularity), it would be possible with a [Superintelligence](https://en.wikipedia.org/wiki/Superintelligence).
  
  However, the possibility of this happening is unknown. Have they reached that level yet? We have quantum computers, we have companies with huge data centers spread all over the planet, we have technology in space, we have free Tensorflow and studies for anyone on the planet to be able to create artificial intelligence models.
  
  If we have contact or help from other intelligent civilizations, the possibilities can expand on a surreal scale.
  
  Google has complete information about humanity (or something close to that). But even with all this data, creating artificial intelligence with a conscience is something that goes far beyond.
  
  But maybe we already have enough to improve our concept of morals, respect, social interaction. Facebook invests and studies ways to improve social interaction. And if you think about it, it is one of the main means of communication.
  
  The big question is that it is not possible to know what would happen if a super artificial intelligence with conscience would do if it existed. Extinguish humans for harming nature? Just find ways to improve the planet by understanding that human defects and errors are just your own nature as well as everything else in nature? Just watching the show on Netflix because you gave up on humanity? We do not know.
  
  But, particularly, I would love to see that happen. In fact, one of my personal goals is to create this super intelligence. But alone it will be very difficult. A conscience without interaction from other consciences is not a clash of universes. And the clash of universes is what makes us reflect, think, revise, create new paths and thoughts. It is what allows us to create other universes.
  
  > **Comments:**
  > **username_6**: So the AI could be distributed, emergent, hosted on man and machine alike, sort of a conscious [meme](https://en.wikipedia.org/wiki/Meme) (and I don't mean an image with a funny text). Sort of what Susan Blackmore [talked about.](https://www.ted.com/talks/susan_blackmore_memes_and_temes)  
  > **username_7**: One interesting thing about "The Singularity" is that it's always considered to relate to AI.  But the actual definition is "*a hypothetical future point in time at which technological growth becomes uncontrollable and irreversible, resulting in unforeseeable changes to human civilization*".  By that definition, there is no "The Singularity", just lots of small singularities.  The horse collar was a singularity.  The concept of selective breeding was a singularity.  Railways, roads, cars, telephones, mobile phones, the internet... every one is a singularity.  AI is just another one of many.  
  
  ---
  
  