&quot;If Dr. Good finishes an AI first, we get a good AI which protects human values. If Dr. Amoral finishes an AI first, we get an AI with no concern for humans that will probably cut short our future.&quot;<p>AI advanced enough to be &quot;good&quot; or &quot;evil&quot; won&#x27;t be developed instantaneously, or by humans alone. We&#x27;ll need an AI capable of improving itself. I believe the authors argument falls apart at this point; surely any AI able to evolve will undoubtedly evolve to the same point, regardless of it being started with the intention of doing good or evil. Whatever ultra-powerful AI we end up with is just an inevitability.