Why AI is harder than we think
I think another common fallacy is to assume that humans have more general intelligence than we actually do. This is manifest in three ways:<p>1. Assuming that we can build an AI than can do what a human does, without being embedded in the physical world and in constant communication with other humans in the same way we are. I think this is covered by &quot;Intelligence is all in the brain&quot; in the article<p>2. Not considering that intelligent breakthroughs by humans isn&#x27;t partly a product of chance.. of millions, and now billions of humans trying semi-random things. Obviously there&#x27;s SOME intelligence behind it, otherwise we wouldn&#x27;t have achieved more than other animals. But maybe we&#x27;re overestimating how much of our results are product of intelligence alone.<p>3. Not considering that we use pretty dumb heuristics to come to decisions. I think the paperclip maximizer is a silly example, because the decision of whether an AI should kill or cooperate with humanity to maximize the production of paperclips is probably undecidable. Coming to a clear decision probably requires more computing power than you could have on a single planet. We humans don&#x27;t need to be certain about the outcome to make a decision. We have emotions like fear, anger, pride and jealousy to nudge us towards decisions like going to war with other people to grab their resources. We need to remember that we&#x27;re a product of evolving in an environment where we had to compete with other humans for resources, so that&#x27;s why we&#x27;re prone to those kinds of decisions. AIs will be product of an environment of competing to please humans to gain computational resources. So the heuristics they develop will probably be strongly tied to achieving that goal.<p>Not that AIs couldn&#x27;t be dangerous, put probably more because humans make active decisions to instruct AIs to harm other humans.