Answers.Body: <p>No one knows. </p>
  
  <p>A useful definition of sentience due to the philosopher Thomas Nagel is <a href="https://en.wikipedia.org/wiki/Thomas_Nagel#What_is_it_like_to_be_a_something" rel="nofollow">'something it is like'</a> to be. </p>
  
  <p>For example, we intuitively feel that there is nothing it is like to be a brick, but that there probably is to be a dog and so on.</p>
  
  <p>However, there is no objective <em>test</em> currently known to physics which can tell if some other entity is having such 'first hand experience', and correspondingly no <em>designs</em> that will definitely lead to sentience.</p>
  
  <p>The best test we have is the Turing test and its variants. The most obvious designs are neuromorphic ones, since we know that the design of the human brain is at least correlated with sentience.</p>
  
  <p>In the light of the above, we can't definitively say a great deal about lower complexity thresholds for sentience - the best we can do count neurons in creatures that we might be prepared to admit are sentient.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-10-12T20:22:01.510Z
Answers.LastActivityDate: 2016-10-12T20:22:01.510Z
Answers.UserDisplayName: NietzscheanAI
Answers.Body: <p>Actually, the terminator AI would not have to be sentient in my opinion. It was a hardcoded condition that it preserve itself as it was the most important asset that the military had in resisting invasion. It was supposed to be an oversight on the part of the programmers that the AI turned on Americans in order to defend itself. Unexpected behaviour does not require sentience at all.</p>
  
  <p>What makes the AI in sci-fi fundamentally different from real existing AI is that it is a "General AI" that is able to understand the world on many different levels simultaneously and still make intelligent decisions. All real AIs are programmed to do very specific things like image recognition or pathfinding. A GPS pathfinder, for example, can't learn to drive a car. In fact, it does not know that there is a car. Or a road. Or people. It merely finds the shortest distance between interconnected nodes on its map.</p>
  
  <p>Personally, I do not believe that there is any proof that a "general AI" is possible. I do not believe that it is a plausible progression of current developments in the next 100 years.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-10-16T11:29:26.130Z
Answers.LastActivityDate: 2016-10-16T11:29:26.130Z
Answers.UserDisplayName: Lorry Laurence mcLarry
Answers.Body: <p>There are already programs that have broken free of our control (<a href="https://en.wikipedia.org/wiki/Morris_worm" rel="nofollow noreferrer">Morris worm</a>) so that in itself doesn't imply any great computational demands.</p>
  
  <p>Sentience is ill-defined but is certainly not a pre-requisite for a program to do mischief beyond what its creators intend.</p>
  
  <p>It's difficulty to estimate what sort of processing power is required to support human-like intelligence, since we don't know what the most efficient way to achieve that would be.  If the most processing efficient would be to implement a neural network approaching the number of neurons and interconnects of the human brain processing signals at the same rate, the fastest artificial neural network implementations extant are at least 4-5 orders of magnitude short, is thousands of times less power efficient, and doesn't seem to have a realistic way to scale to the number of interconnects required (<a href="https://ai.stackexchange.com/questions/1834/power-efficiency-of-human-brains-vs-neural-networks">see this question</a>)</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-10-12T17:17:37.877Z
Answers.Text: from what i understand about the morris worm (and i concede this is limited) he released it, then edited it to ask if there was an existing copy of it (and still run 1/7 times), this implies that Morris had contorl over it? surely?
Answers.UserDisplayName: Alex Robinson
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-10-12T17:22:13.700Z
Answers.Text: @Cursed1701 - Morris wanted a proof of concept that would spread without getting him into trouble; his bug allowed it to replicate wildly which definitely got him in trouble.  So to that extent it wasn't in his control.
Answers.UserDisplayName: antlersoft
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-10-12T17:12:53.330Z
Answers.LastActivityDate: 2016-10-12T17:12:53.330Z
Answers.LastEditDate: 2017-04-13T12:53:10.013Z
Answers.UserDisplayName: antlersoft
Body: <p>In lots of sci-fi, it seems that AI becomes sentient (Terminator, Peter F Hamilton's SI (commonwealth saga), etc.)</p>
  <p>However, I'm interested in whether this is actually plausible, whether an AI could actually break free form being controlled by us, and if that is possible, whether there is any research as to what sort of complexity/processing power an AI would need to be able to do this.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-10-12T20:25:38.100Z
Comments.Text: Sentience is a very strong condition, and is not required in general in order to 'break free of control'. Even very simple systems can exploit their environment in unexpected ways. Please edit the question to clarify whether it is 'breaking free of control' or sentience that is your primary concern.
Comments.UserDisplayName: NietzscheanAI
Tags: <reference-request><sentience>
Title: Could an AI break free form being controlled by us?
UserDisplayName: Alex Robinson
ThreadText: # Could an AI break free form being controlled by us?
  
  (Asked by: username_0 on 2016-10-12)
  
  In lots of sci-fi, it seems that AI becomes sentient (Terminator, Peter F Hamilton's SI (commonwealth saga), etc.)
  
  However, I'm interested in whether this is actually plausible, whether an AI could actually break free form being controlled by us, and if that is possible, whether there is any research as to what sort of complexity/processing power an AI would need to be able to do this.
  
  **Tags:** reference-request, sentience
  
  > **Comments:**
  > **username_1**: Sentience is a very strong condition, and is not required in general in order to 'break free of control'. Even very simple systems can exploit their environment in unexpected ways. Please edit the question to clarify whether it is 'breaking free of control' or sentience that is your primary concern.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_3 on 2016-10-12. Score: 2 | Accepted answer
  
  There are already programs that have broken free of our control ([Morris worm](https://en.wikipedia.org/wiki/Morris_worm)) so that in itself doesn't imply any great computational demands.
  
  Sentience is ill-defined but is certainly not a pre-requisite for a program to do mischief beyond what its creators intend.
  
  It's difficulty to estimate what sort of processing power is required to support human-like intelligence, since we don't know what the most efficient way to achieve that would be. If the most processing efficient would be to implement a neural network approaching the number of neurons and interconnects of the human brain processing signals at the same rate, the fastest artificial neural network implementations extant are at least 4-5 orders of magnitude short, is thousands of times less power efficient, and doesn't seem to have a realistic way to scale to the number of interconnects required ([see this question](https://ai.stackexchange.com/questions/1834/power-efficiency-of-human-brains-vs-neural-networks))
  
  > **Comments:**
  > **username_0**: from what i understand about the morris worm (and i concede this is limited) he released it, then edited it to ask if there was an existing copy of it (and still run 1/7 times), this implies that Morris had contorl over it? surely?  
  > **username_3**: @Cursed1701 - Morris wanted a proof of concept that would spread without getting him into trouble; his bug allowed it to replicate wildly which definitely got him in trouble.  So to that extent it wasn't in his control.  
  
  ---
  
  ### Answer by username_1 on 2016-10-12. Score: 2
  
  No one knows.
  
  A useful definition of sentience due to the philosopher Thomas Nagel is ['something it is like'](https://en.wikipedia.org/wiki/Thomas_Nagel#What_is_it_like_to_be_a_something) to be.
  
  For example, we intuitively feel that there is nothing it is like to be a brick, but that there probably is to be a dog and so on.
  
  However, there is no objective *test* currently known to physics which can tell if some other entity is having such 'first hand experience', and correspondingly no *designs* that will definitely lead to sentience.
  
  The best test we have is the Turing test and its variants. The most obvious designs are neuromorphic ones, since we know that the design of the human brain is at least correlated with sentience.
  
  In the light of the above, we can't definitively say a great deal about lower complexity thresholds for sentience - the best we can do count neurons in creatures that we might be prepared to admit are sentient.
  
  
  ---
  
  ### Answer by username_2 on 2016-10-16. Score: 0
  
  Actually, the terminator AI would not have to be sentient in my opinion. It was a hardcoded condition that it preserve itself as it was the most important asset that the military had in resisting invasion. It was supposed to be an oversight on the part of the programmers that the AI turned on Americans in order to defend itself. Unexpected behaviour does not require sentience at all.
  
  What makes the AI in sci-fi fundamentally different from real existing AI is that it is a "General AI" that is able to understand the world on many different levels simultaneously and still make intelligent decisions. All real AIs are programmed to do very specific things like image recognition or pathfinding. A GPS pathfinder, for example, can't learn to drive a car. In fact, it does not know that there is a car. Or a road. Or people. It merely finds the shortest distance between interconnected nodes on its map.
  
  Personally, I do not believe that there is any proof that a "general AI" is possible. I do not believe that it is a plausible progression of current developments in the next 100 years.
  
  
  ---
  
  