&gt; Feynman in particular was quite able to continually accomplish his goals, and he purposely chose divergent goals to test himself (his whole &quot;I&#x27;ll be a biologist this summer&quot; thing).<p>His goals pertained to himself. He never influenced the masses and never amassed much power.<p>&gt; it takes intelligence to walk into the DAP meeting and join as member 55 and come out conquering mainland Europe.<p>I didn&#x27;t say it doesn&#x27;t, but it doesn&#x27;t take <i>super</i> intelligence to do that. Just more than a baseline. Hitler was no genius.<p>&gt; What&#x27;s imminently dangerous about AI is that all it really takes is a few innovations that might be in seemingly unrelated areas enable probably several people who see the pattern to go and implement AI.<p>That could be said just about anything. A psychologist could accidentally discover a fool-proof mechanism of brainwashing people; a microbiologist could discover an un-killable deadly microbe; an archeologist could uncover a dormant spaceship from a hostile civilization. There&#x27;s nothing that shows that such breakthroughs in AI are any more imminent than in other fields.<p>&gt; Our infrastructure is barely secure against human attackers, and it could be totally obliterated by an AI that chose to do that<p>Why?<p>&gt; but it could almost certainly make it seem to any nuclear power that another nuclear power had launched a nuclear attack<p>Why can an AI do that but a human can&#x27;t?<p>&gt; limitless capability for action<p>God has limitless capability for action. But we have no reason whatsoever to believe that either God or true AI would reveal themselves in the near future.<p>&gt; near instantaneous ability to act<p>No. Again,<p>&gt; there&#x27;s nearly nothing that would make shit hit the fan FASTER than a hostile AI.<p>There&#x27;s nothing that would make shit hit the fan FASTER than a hostile spaceworm devouring the planet. But both the spaceworm and the AI are currently speculative sci-fi.<p>&gt; I&#x27;m confused as to how you can read three comments of &quot;intelligence is the ability to accomplish goals&quot;<p>There are a couple of problems with that: one, that is not the definition that is commonly used today. Britney Spears has a lot of ability to achieve her goals, but no one would classify her as especially intelligent. Two, that is not where AI research is going. No one is trying to make computers able to &quot;achieve goals&quot;, but able to carry out certain computations. Those computations are very loosely correlated with actual ability to achieve goals. You could define intelligence as &quot;the ability to kill the world with a thought&quot; and then say AI is awfully dangerous, but that definition alone won&#x27;t change AI&#x27;s actual capabilities.<p>&gt; A generally intelligent person (say, Feynman) could learn to manipulate people and would almost certainly be successful at it<p>I disagree. We have no data to support that prediction. We know that manipulation requires intelligence, but we do not know that added intelligence translates to added ability to manipulate and that that relationship scales.<p>&gt; what would take less than a week to totally destroy our current civilization?<p>That is a strange question, because you have no idea how long it would take an AI. I would say that whatever an AI could achieve in a week, humans could achieve in a similar timeframe and much sooner. In any case, as someone who worked with neural-networks in the nineties, I can tell you that we haven&#x27;t made as much progress as you think. We are certainly not at any point where a sudden discovery could yield true AI any more than a sudden discovery would create an unkillable virus.