Answers.Body: <p>At the highest level, all it needs is for the various systems already discussed to incorporate code objects. If it can interpret its source code / model architecture from the formatted text objects underpinning them, can 'understand' them in terms of having a useful ML model, and alter the code with its reaction, then it can self-program. </p>
  
  <p>That is, the basic loop behind a recursively improving intelligence is simple. It examines itself, writes a new version, and then that new version examines itself and writes a new version, and so on.</p>
  
  <p>The difficult component comes at lower levels. We don't need to invent a new concept like 'sensor,' what we need to do is build very, very sophisticated sensors that are equal to the task of understanding code well enough to detect and write improvements.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-01T07:26:26.713Z
Answers.Text: Realistically the halting problem only really applies to 'turning machines' which are pure mathematical constructs that can't really exist (they require an infinite tape for unlimited memory for example) and can run for an infinite time. Real world computers have limited amounts of memory. There are ways to write software that can be formally verified (Idris, Coq). Using dependent types. Limit the size of an array (ie < the amount or ram). Not allowing a program to modify itself in memory in a way that could violate the formal proofs. No infinite loops. No byte loop/divide by zero. Etc...
Answers.UserDisplayName: David C. Bishop
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-18T20:06:36.767Z
Answers.Text: Although the knee-jerk computer science reaction to statements about systems that understand their own code is often to cite the halting problem, it turns out that AI approaches do have *something* useful to say about that: http://cs.stackexchange.com/questions/62393/machine-learning-to-predict-the-halting-problem
Answers.UserDisplayName: NietzscheanAI
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-18T20:18:50.670Z
Answers.Text: Right, the Halting Problem is a no-go theorem for *fully* understanding *all possible* code, but doesn't stop one from having a good understanding of most code that you actually come across.
Answers.UserDisplayName: Matthew Gray
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-18T19:51:21.833Z
Answers.LastActivityDate: 2016-08-18T19:51:21.833Z
Answers.UserDisplayName: Matthew Gray
Body: <p>An AI agent is often thought of having "sensors", "a memory", "machine learning processors" and "reaction" components. However, a machine with these does not necessarily become a self-programming AI agent. Beyond the parts mentioned above, is there any other elements or details necessary to make a machine capable of being a self-programming AI agent?</p>
  
  <p>For example, <a href="http://www.iiim.is/wp/wp-content/uploads/2011/05/goertzel-agisp-2011.pdf" rel="noreferrer">a paper from 2011</a> declared that solving the optimization problem of maximizing the intelligence is a must-have feature for the self-programming process, as quoted below:</p>
  
  <blockquote>
    <p>A system is said to carry out an instance of self-programming when it undergoes learning regarding some element of its "cognitive infrastructure", where the latter is defined as the fuzzy set of "intelligence-critical" features of the system; and the intelligence-criticality of a system feature is defined as its "feature quality," considered from the perspective of solving the optimization problem of maximizing the intelligence of a multi-feature system.</p>
  </blockquote>
  
  <p>However, this description of "optimization of intelligence" is vague. Can anyone give a clear definition or better summary for the necessary components for self-programming agents?</p>
  
  <p><sub>This question is from the 2014 closed beta, with the asker having a UID of 23.</sub></p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-08-18T20:26:42.447Z
Comments.Text: Here's pretty much the last word on self-optimization: [Goedel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements](http://arxiv.org/abs/cs/0309048) by Juergen Schmidhuber.
Comments.UserDisplayName: NietzscheanAI
Tags: <machine-learning><intelligent-agent><computer-programming>
Title: What are the necessary components to make an AI agent capable of self-programming?
UserDisplayName: Mithical
ThreadText: # What are the necessary components to make an AI agent capable of self-programming?
  
  (Asked by: username_0 on 2016-08-18)
  
  An AI agent is often thought of having "sensors", "a memory", "machine learning processors" and "reaction" components. However, a machine with these does not necessarily become a self-programming AI agent. Beyond the parts mentioned above, is there any other elements or details necessary to make a machine capable of being a self-programming AI agent?
  
  For example, [a paper from 2011](http://www.iiim.is/wp/wp-content/uploads/2011/05/goertzel-agisp-2011.pdf) declared that solving the optimization problem of maximizing the intelligence is a must-have feature for the self-programming process, as quoted below:
  
  > A system is said to carry out an instance of self-programming when it undergoes learning regarding some element of its "cognitive infrastructure", where the latter is defined as the fuzzy set of "intelligence-critical" features of the system; and the intelligence-criticality of a system feature is defined as its "feature quality," considered from the perspective of solving the optimization problem of maximizing the intelligence of a multi-feature system.
  
  However, this description of "optimization of intelligence" is vague. Can anyone give a clear definition or better summary for the necessary components for self-programming agents?
  
  This question is from the 2014 closed beta, with the asker having a UID of 23.
  
  **Tags:** machine-learning, intelligent-agent, computer-programming
  
  > **Comments:**
  > **username_1**: Here's pretty much the last word on self-optimization: [Goedel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements](http://arxiv.org/abs/cs/0309048) by Juergen Schmidhuber.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2016-08-18. Score: 5 | Accepted answer
  
  At the highest level, all it needs is for the various systems already discussed to incorporate code objects. If it can interpret its source code / model architecture from the formatted text objects underpinning them, can 'understand' them in terms of having a useful ML model, and alter the code with its reaction, then it can self-program.
  
  That is, the basic loop behind a recursively improving intelligence is simple. It examines itself, writes a new version, and then that new version examines itself and writes a new version, and so on.
  
  The difficult component comes at lower levels. We don't need to invent a new concept like 'sensor,' what we need to do is build very, very sophisticated sensors that are equal to the task of understanding code well enough to detect and write improvements.
  
  > **Comments:**
  > **username_1**: Although the knee-jerk computer science reaction to statements about systems that understand their own code is often to cite the halting problem, it turns out that AI approaches do have *something* useful to say about that: http://cs.stackexchange.com/questions/62393/machine-learning-to-predict-the-halting-problem  
  > **username_2**: Right, the Halting Problem is a no-go theorem for *fully* understanding *all possible* code, but doesn't stop one from having a good understanding of most code that you actually come across.  
  > **username_3**: Realistically the halting problem only really applies to 'turning machines' which are pure mathematical constructs that can't really exist (they require an infinite tape for unlimited memory for example) and can run for an infinite time. Real world computers have limited amounts of memory. There are ways to write software that can be formally verified (Idris, Coq). Using dependent types. Limit the size of an array (ie < the amount or ram). Not allowing a program to modify itself in memory in a way that could violate the formal proofs. No infinite loops. No byte loop/divide by zero. Etc...  
  
  ---
  
  