(Sorry, this is a bit rambling. Hopefully it&#x27;ll still be interesting to some of you. Have had a few pints at this point...)<p>EDIT: Actually, this is nearing &quot;crazy&quot; levels. Just ignore unless you really enjoy stream-of-consciousness. Sorry about this, HN! :)<p>I know I&#x27;m <i>really</i> late to the party here, but there&#x27;s a premise in this whole discussion that I&#x27;m not sure I understand.<p>Why <i>should</i> we prevent AI from taking over? I mean, I &quot;get it&quot;... it wouldn&#x27;t be <i>HI</i> and that <i>feels</i> kind of weird, but what&#x27;s objectively special about HI? Why are we treating &quot;HI==good&quot; as axiomatic? I mean even us tribal, overly-emotional (&amp;c) humans value <i>DI</i> (Dog Intelligence) even if we&#x27;re pretty sure that it can&#x27;t contemplate the fact that we&#x27;re all made of the remnants of supernovae. There&#x27;s no evidence as of yet that a greater intelligence a) exists[1], even in principle, or b) would be any less benevolent towards us. Perhaps they would even create nice little simulations for us to exist in. Though, I wonder what the purpose of <i>my</i> simulation is, given current circumstances :).<p>Yes, a transition from HI-&gt;AI would inevitably lead to a lot of human death (unless we&#x27;re talking <i>really</i> out-there take-over plans involving disease and such), but would AI really be <i>worse</i>? And for <i>whom</i> and <i>why</i>? Humans themselves have caused a lot of death and we seem to value ourselves pretty highly overall (and undeservedly, IMO).<p>It might be that HI is the &quot;end of the road&quot; just like the Turing Machine appears to be the end of the road in terms of what you <i>can</i> compute... but not in terms of <i>how fast</i> you can compute it. Would &quot;faster&quot; automatically mean &quot;better&quot; (see footnote)? I dunno.<p>[1] The existence of a &quot;higher&quot; (&quot;faster&quot; is probable) intellige is a interesting question. How would you judge such a thing? Is there more &quot;power&quot;[2] to be gained through something other than being able to reflect at yourself? AFAIUI self-reflection is one of <i>the</i> distinuguishing features of intelligence, but given that we&#x27;re &quot;better&quot; than Chimps -- who have an idea of &quot;self&quot;, thus self-reflection -- it may not be the decider. And even so, such reflection is still subject to Physics and thus without &quot;free will&quot;.<p>[2] Not just faster, but &quot;better&quot;, in some non-linear sense.