If that were the case, I would agree we would be quite far away from simulated human-level cognition.<p>My main point was that either we have the computational capacity, or we do not. If we do, the chances of overshooting the human-level mark by a large margin are quite high, hence the risk.<p>Another angle on this is that perhaps human cognition has an unnecessarily high degree of computational complexity, a degree that&#x27;s far above the threshold required for an AGI to be dangerous.<p>The Paperclip Maximizer[0] is an excellent example of this. Perhaps due to low computational complexity of its architecture, the thing wouldn&#x27;t even be sentient in any sense of the word, and yet it&#x27;d still be perfectly capable of destroying the world.<p>Why? Because its initial intelligence would be determined by the amount of computing power allocated to it. Allocate enough computing power, and assuming you&#x27;ve passed the magic threshold, it would start allocating itself computing power in short order.<p>Perhaps what we should fear most is not sentient AGI capable of thought similar to humans, but AGI that&#x27;s a structured amalgamation of existing AI technology—if only because the latter is far more likely to be feasible first.<p>[0] <a href="http:&#x2F;&#x2F;wiki.lesswrong.com&#x2F;wiki&#x2F;Paperclip_maximizer" rel="nofollow">http:&#x2F;&#x2F;wiki.lesswrong.com&#x2F;wiki&#x2F;Paperclip_maximizer</a>