&gt; I mean, the paperclip example in the Medium article... did that factory ever need AI involved?<p>That example got a bit lost in translation.  The standard example of the paperclipping AI is not about building an AI to run a paperclip factory.  It&#x27;s about providing an appropriate value system for an AI, so that it can evaluate courses of action.  The standard example: you use a paperclip as an example for an AI as the smallest unit of incremental value, such that having a paperclip is epsilon better than not having a paperclip.  The AI files that information away, and tiles the universe with paperclips.  Related examples include using a smiling human face as an example of happiness and having the AI tile the universe with the smallest possible object that matches its &quot;smiling human face&quot; recognizer.<p>&gt; And they also kind of assume that there&#x27;s only one AI at a time<p>Yes.  If you&#x27;re going to build a strong AI, then either you got its value system <i>right</i>, and it will take over the world (in the good, &quot;solves all problems at once&quot; way), or you got its value system <i>wrong</i>, and it will destroy the world.  There is no plausible scenario in which we build strong AI (as in, capable of self-improvement to better satisfy its value function) and it <i>doesn&#x27;t</i> take over the world, or in which it allows other strong AIs to exist that are not effectively subroutines of itself (any AI that shares the same value system is the same AI, and any AI that doesn&#x27;t share the same value system is too dangerous to exist).<p>&gt; that everything is connected to the internet in some way, that humanity couldn&#x27;t simply wipe out anything that poses a threat (and no, an AI in a factory environment wouldn&#x27;t get access to anything that would allow for nuclear&#x2F;chemical&#x2F;biological weapons).<p>You can&#x27;t keep a strong AI capable of self-improvement in a &quot;box&quot;.  Random example: a security conference this year demonstrated using nothing but DRAM to successfully transmit short-range GSM signals.  That&#x27;s something <i>humans</i> thought up and implemented.<p>That&#x27;s leaving aside intentional human intervention or development-and-release.  Not least of which because a strong AI done <i>right</i> provides a massive reduction in both existential risk for humanity and day-to-day risk for humans.