 Dear Fellow Scholars, this is Two Minute Papers with Dr. Karo Zsolnai-Fehir. Today, we are going to see how an AI can win a complex game that it has never seen before. Zero prior training on that game. Yes, really. Now, before that, for context, have a look at this related work from 2019, where scientists at OpenAI built a super-fun high-density game for their AI agents to play. And boy, did they do some crazy stuff. Now, these agents learned from previous experiences and to the surprise of no one, for the first few million rounds we start out with Pandemonium. Everyone just running around aimlessly. Then, over time, the hiders learned to lock out the seekers by blocking the doors off with these boxes and started winning consistently. I think the coolest part about this is that the map was deliberately designed by the OpenAI scientists in a way that the hiders can only succeed through collaboration. But then, something happened. Did you notice this pointy door-stop-shaped object? Are you thinking what I am thinking? Well, probably. And not only that, but later, the AI also discovered that it can be pushed near a wall and be used as a ramp and... Tada! Got him! Then, it was up to the hiders again to invent something new. So, did they do that? Can this crazy strategy be defeated? Well, check this out. These resourceful little critters learned that since there is a little time at the start of the game when the seekers are frozen, apparently, during this time, they cannot see them, so why not just sneak out and steal the ramp and lock it away from them? Absolutely incredible. Look at those happy eyes as they are carrying that ramp. But, today is not 2019, it is 2021, so I wonder what scientists at the other amazing AI lab deep-mind have been up to? Can this paper be topped? Well, believe it or not, they have managed to create something that is perhaps even crazier than this. This new paper proposes that these AI agents look at the screen just like a human wood and engage in open-ended learning where the tasks are always changing. What does this mean? Well, it means that these agents are not preparing for an exam. They are preparing for life. Hopefully, they learn more general concepts and, as a result, maybe excel at a variety of different tasks. Even better, these scientists at deep-mind claim that their AI agents not only excel at a variety of tasks, but they excel at new ones they have never seen before. Those are big words, so let's see the results. The red agent here is the Heider and the blue is the Seeker. They both understand their roles, the red agent is running, and the blue is seeking. Look, its viewing direction is shown with this lightsaber-looking line pointing at the red agent. No wonder it is running away. And look, it manages to get some distance from the Seeker and finds a new, previously unexplored part of the map and hides there. Excellent. And you would think that the Star Wars references and here, no, not even close. Look, in a more advanced variant of the game, this green Seeker lost the two other Heiders, and what does he do? Ah, yes, of course, grabs his lightsaber and takes the high ground. Then, it spuds the red agent and starts chasing it, and all this without ever having played this game before. That is excellent. In this cooperative game, the agents are asked to get as close to the purple pyramid as they can. Of course, to achieve that, they need to build a ramp, which they successfully realize. Excellent. But it gets better. Now, note that we did not say that the task is to build a ramp, the task is to get as close to the purple pyramid as we can. Does that mean... yes, yes it does. Great job bending the rules, little AI. In this game, the agent is asked to stop the purple ball from touching the red floor. At first, it tries its best to block the rolling of the ball with its body. Then, look, it realizes that it is much better to just push it against the wall. And it gets even better, look, it learned that best is to just chuck the ball behind this slab. It is completely right, this needs no further energy expenditure, and the ball never touches the red floor again. Great. And finally, in this king of the here game, the goal is to take the white floor and get the other agent out of there. As they are playing this game for the first time, they have no idea where the white floor is. As soon as the blue agent finds it, it stays there, so far so good. But this is not a cooperative game. We have an opponent here, look, boom, a quite potent opponent indeed who can take the blue agent out, and it understands that it has to camp in there and defend the region. Again, awesome. So, the goal here is not to be an expert in one game, but to be a journeyman in many games. And these agents are working really well at a variety of games without ever having played them. So, in summary, open AI's agent, expert in an narrower domain, deep minds agent, journeyman in a broader domain. Two different kinds of intelligence, both doing amazing things. Loving it. What a time to be alive. Scientists at DeepMind have knocked it out of the park with this one. They have also published AlphaFull this year a huge breakthrough that makes an AI-predict protein structures. Now, I saw some of you asking why we didn't cover it. Is it not an important work? Well, quite the opposite. I am spellbound by it, and I think that paper is a great gift to humanity. However, I try my best to educate myself on this topic, however, I don't feel that I am qualified to speak about it. Not yet, anyway. So, I think it is best to let the experts who know more about this take the stage. This is, of course, bad for views, but no matter, we are not maximizing views here, we are maximizing meaning. This episode has been supported by Lambda GPU Cloud. If you are looking for inexpensive Cloud GPUs for AI, check out Lambda GPU Cloud. You recently launched Quadro RTX 6000, RTX 8000, and V100 instances, and hold onto your papers because Lambda GPU Cloud can cost less than half of AWS and Asia. Plus, they are the only Cloud service with 48GB RTX 8000. Join researchers at organizations like Apple, MIT, and Caltech in using Lambda Cloud instances, workstations, or servers. Make sure to go to lambdaleps.com, slash papers to sign up for one of their amazing GPU instances today. Our thanks to Lambda for their long-standing support, and for helping us make better videos for you. Thanks for watching, and for your generous support, and I'll see you next time.