Answers.Body: <p>I think most answers that come to mind are going to be connected to the control problem in some way---e.g. sandboxing frameworks, AI behavior monitoring protocols, etc.</p>
  
  <p>However, if you aren't interested in those for now, that pretty much leaves two possible needs such technology could satisfy: the technical challenges of AI itself, and the need to optimize the social impact on AI. </p>
  
  <p>The technologies satisfying the first need are relatively "obvious" once you have the AI implementation figured out---depending on what that is, there will be certain hardware and software requirements (massively parallel processing, neural network frameworks, etc). Most of these technologies seem likely to arise "naturally" from the status quo---if we just keep developing our hardware and software as we are now, we will eventually get there.</p>
  
  <p>Coming up with technologies satisfying the second need is a challenge, and far more interesting in my opinion. We need to start a discussion about this, so you have a great question here. I don't know that I can contribute much, but maybe I can provide some starter ideas:</p>
  
  <ul>
  <li>More robust encryption schemes, so that the massive amount of information handled by AI is secure even in the face of AI-capable adversaries. We'd want these encryption schemes to be information-theoretic secure, so that it is mathematically impossible to recover the plaintext, rather than just computationally difficult, since an AI may be able to improve it's own computational efficiency with unprecedented ease. Quantum cryptography may help.</li>
  <li>Improved robotics, which would require improved battery technology, to multiply by a great factor the ability of AI to manipulate the physical world and assist with physical tasks (obviously requires a fantastic solution to the control problem). </li>
  <li>If you consider it separate from the control problem (and I think there is a useful distinction here), lots of AI-monitoring technology will be necessary. Think of technologies acting as regulators for newly automated tasks---e.g. traffic monitoring algorithms to watch for undesirable behavior among automated vehicles.</li>
  </ul>
  
  <p>Answering this question is very tough, particularly because you are asking for technologies which we'd have to develop <em>before</em> we have AI. Of course, once we have general AI we can instruct it to develop any of these technologies for us, supposing the control problem is adequately solved. Nevertheless it seems wise to solve develop improved cryptography immediately since even with weak AI (or a weak control problem solution) the need is urgent. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-15T05:58:33.893Z
Answers.Text: Great answer, thx, upvoted! I will leave the question open a little longer, in case your answer inspired others as well. The best answer after the weekend will be accepted.
Answers.UserDisplayName: Demento
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-15T01:24:40.493Z
Answers.LastActivityDate: 2017-09-15T01:24:40.493Z
Answers.UserDisplayName: Marko Bakić
Body: <p>The sequence in which new technologies are developed can be crucial for it's success and it's safe use. For example, before developing a nuclear reactor you want to have technologies like reliable cooling systems and secure ultimate disposal zones available and tested. Otherwise we might have a bad time.</p>
  
  <p>I see AI with general superhuman intelligence as important predecessor for dangerous technologies like nanobots or generators using nuclear fusion, because superhuman intelligence can be helpful in foreseeing dangers that might lead to catastrophe using these technologies in an insecure way.</p>
  
  <p>But what about important technologies we want to have <strong>before</strong> inventing an AI with general superhuman intelligence? Solving the control-problem is obvious, but are there other technologies we should have in advance?</p>
  
Tags: <agi><control-problem><superintelligence>
Title: Which technologies should humanity develop before developing an AI with general superhuman intelligence?
UserDisplayName: Demento
ThreadText: # Which technologies should humanity develop before developing an AI with general superhuman intelligence?
  
  (Asked by: username_0 on 2017-09-07)
  
  The sequence in which new technologies are developed can be crucial for it's success and it's safe use. For example, before developing a nuclear reactor you want to have technologies like reliable cooling systems and secure ultimate disposal zones available and tested. Otherwise we might have a bad time.
  
  I see AI with general superhuman intelligence as important predecessor for dangerous technologies like nanobots or generators using nuclear fusion, because superhuman intelligence can be helpful in foreseeing dangers that might lead to catastrophe using these technologies in an insecure way.
  
  But what about important technologies we want to have **before** inventing an AI with general superhuman intelligence? Solving the control-problem is obvious, but are there other technologies we should have in advance?
  
  **Tags:** agi, control-problem, superintelligence
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2017-09-15. Score: 3 | Accepted answer
  
  I think most answers that come to mind are going to be connected to the control problem in some way---e.g. sandboxing frameworks, AI behavior monitoring protocols, etc.
  
  However, if you aren't interested in those for now, that pretty much leaves two possible needs such technology could satisfy: the technical challenges of AI itself, and the need to optimize the social impact on AI.
  
  The technologies satisfying the first need are relatively "obvious" once you have the AI implementation figured out---depending on what that is, there will be certain hardware and software requirements (massively parallel processing, neural network frameworks, etc). Most of these technologies seem likely to arise "naturally" from the status quo---if we just keep developing our hardware and software as we are now, we will eventually get there.
  
  Coming up with technologies satisfying the second need is a challenge, and far more interesting in my opinion. We need to start a discussion about this, so you have a great question here. I don't know that I can contribute much, but maybe I can provide some starter ideas:
  
  * More robust encryption schemes, so that the massive amount of information handled by AI is secure even in the face of AI-capable adversaries. We'd want these encryption schemes to be information-theoretic secure, so that it is mathematically impossible to recover the plaintext, rather than just computationally difficult, since an AI may be able to improve it's own computational efficiency with unprecedented ease. Quantum cryptography may help.
  * Improved robotics, which would require improved battery technology, to multiply by a great factor the ability of AI to manipulate the physical world and assist with physical tasks (obviously requires a fantastic solution to the control problem).
  * If you consider it separate from the control problem (and I think there is a useful distinction here), lots of AI-monitoring technology will be necessary. Think of technologies acting as regulators for newly automated tasks---e.g. traffic monitoring algorithms to watch for undesirable behavior among automated vehicles.
  
  Answering this question is very tough, particularly because you are asking for technologies which we'd have to develop *before* we have AI. Of course, once we have general AI we can instruct it to develop any of these technologies for us, supposing the control problem is adequately solved. Nevertheless it seems wise to solve develop improved cryptography immediately since even with weak AI (or a weak control problem solution) the need is urgent.
  
  > **Comments:**
  > **username_0**: Great answer, thx, upvoted! I will leave the question open a little longer, in case your answer inspired others as well. The best answer after the weekend will be accepted.  
  
  ---
  
  