Greedy, Brittle, Opaque, and Shallow: The Downsides to Deep Learning
Humans, as opposed to deep learning, have embodiment. We can move about, push and prod, formulate ideas and test them in the world. A deep net can&#x27;t do any of that in the supervised learning setting. The only way to do that is inside an RL agent. The problem is that any of our RL agents so far need to run inside a simulated environment, which is orders of magnitude less complex than reality. So they can&#x27;t learn because they can&#x27;t explore like us.<p>The solution would be to improve embodiment for neural nets and to equip RL agents with internal world simulators (a world model) they could use to plan ahead. So we need simulation both outside and inside agents. Neural nets by themselves are not even the complete answer. But what is missing is not necessarily a new algorithm or data representation, it&#x27;s the whole world-agent complex.<p>Not to mention that a human alone is not much use - we need society and culture to unlock our potential. Before we knew the cause, we believed disease was caused by gods, and it took many deaths to unlock the mystery. We&#x27;re not perfect either, we just sit on top of the previous generations. Another advantage we have - we have a builtin reward system that guides learning, which was created by evolution. We have to create this reward system for RL agents from scratch.<p>In some special cases like board games, the board is a perfect simulation in itself (happens to be trivial, just observe the rules, play against a replica of yourself). In that case RL agents can reach superhuman intelligence, but that is mostly on account of having a perfect playground to test ideas in.<p>In the future simulation and RL will form the next step in AI. The current limiting factor block is not the net, but the simulator. I think everyone here has noticed the blooming of many game environments used for training RL agents from DeepMind, OpenAI, Atari, StarCraft, Dota2, GTA, MuJoCo and others. It&#x27;s a race to build the playground for the future intelligences.<p>Latest paper from DeepMind?<p>&gt; IMPALA: Scalable Distributed DeepRL in DMLab-30. DMLab-30 is a collection of new levels designed using our open source RL environment DeepMind Lab. These environments enable any DeepRL researcher to test systems on a large spectrum of interesting tasks either individually or in a multi-task setting.<p>Before we build an AI, we need to build a world for that AI to be in.