&gt; which means something along the lines of &quot;ability to accomplish goals,&quot; with the social construct of &quot;intelligence&quot;<p>Perhaps. But if that is the case, the people who are most intelligent by this definition are far from the ones recognized as intelligent by the AI-fearing community. Let me put it this way: Albert Einstein and Richard Feynman would not be among them. Adolf Hitler, on the other hand, would be a genius.<p>&gt; You&#x27;re wrong, and super-human AI is a massive issue, regardless of how you pattern-match it as &quot;religious&quot;<p>If you think I don&#x27;t always presume that everything I say is likely wrong then you misunderstand me. I do, however, don&#x27;t understand what you mean by &quot;a massive issue&quot;. Do you mean imminent danger? Yes, I guess it&#x27;s possible, but being familiar with the state of the art, I can at least discount the &quot;imminent&quot; part.<p>&gt; You&#x27;re not creative enough (one might dare say &quot;intelligent&quot; but that would be too snarky) to imagine all the ways in which an AI could devastate humanity without it having much intelligence or much charm<p>I can imagine many things. I can even imagine an alien race destroying our civilization tomorrow. What I fail to see is compelling arguments why AI is any more dangerous or any more imminent than hundreds of bigger, more imminent threats.<p>&gt; In fact, most hyper-successful people are almost certainly good at both.<p>I would gladly debate this issue if I believed you genuinely believed that. If you had a list ordered by social power of the top 100 most powerful people in the world, I doubt <i>you</i> would say their defining quality is intelligence.<p>&gt; it&#x27;s very possible to apply the scientific method and empirical problem solving to finding them, and then exploiting humans that way. This is a huge subfield of psychology (persuasion) and the basis of marketing.<p>Psychology is one of the fields I know most about, and I can tell you that the people most adept at exploiting others are not the ones you would call super-intelligent. You wouldn&#x27;t say they are of average intelligence, but I don&#x27;t think you&#x27;d recognize their intelligence as being superior.<p>&gt; It&#x27;s nice to think the world is a safe place, but the reality is that our social order is increasingly precarious and an AI could easily disrupt that.<p>There are so many things that could disrupt that, and while AI is one of them, it is not among the top ten.