 Dear Fellow Scholars, this is two-minute papers with Dr. Karo Zsolnai-Fehir in 2017 scientists at OpenAI published a paper where virtual humans learned to tackle each other in a sumo competition of sorts and found out how to rock a stable stance to block others from tackling them. This was a super interesting work because it involved self-play or in other words copies of the same AI were playing against each other and the question was how do we pair them with each other to maximize their learning. They found something really remarkable when they asked the algorithm to defeat an older version of itself. If it can reliably pull that off, it will lead to a rapid and predictable learning process. This kind of curriculum-driven learning can supercharge many different kinds of AI's. For instance, this robot from a later paper is essentially blind as it only has proprioceptive sensors which means that the only thing that the robot senses is its own internal state and that's it. No cameras, no depth sensors, no light or nothing. And at first it behaves as we would expect it. Look, when we start out, the agent is very clumsy and can barely walk through a simple terrain. But as time passes, it grows to be a little more confident and with that the terrain also becomes more difficult over time in order to maximize learning. So how potent is this kind of curriculum in teaching the AI? Well, it learned a great deal in the simulation and a scientist deployed it into the real world just look at how well it traversed through this rocky mountain, stream and not even this nightmare-ish snowy descent gave it too much trouble. This new technique proposes a similar curriculum-based approach where we would teach all kinds of virtual life forms to navigate on stepping stones. The examples include a virtual human, a bipedal robot called Cassie and this sphere with toothpick legs too. The authors call it monster, so you know what? Monster it is. The fundamental question here is how do we organize the stepping stones in this virtual environment to deliver the best teaching to this AI? We can freely choose the heights and orientations of the upcoming steps and, of course, it is easier said than done. If the curriculum is too easy, no meaningful learning will take place and if it gets too difficult too quickly, well then in the better case this happens. And in the worst case, whoops! This work proposes an adaptive curriculum that constantly measures how these agents perform and creates challenges that progressively get harder, but in a way that they can be solved by the agents. It can even deal with cases where the AI already knows how to climb up and down and even deal with longer steps. But that does not mean that we are done because if we don't build the Spires right, this happens. But after learning 12 to 24 hours with this adaptive curriculum learning method, they become able to even run, deal with huge step height variations, high-step tilt variations, and let's see if they can pass the hardest exam. Look at this mess, my goodness, lots of variation in every perimeter. And yes, it works. And the key point is that the system is general enough that it can teach different body types to do the same. If there is one thing that you take home from this video, it shouldn't be that it takes from 12 to 24 hours, it should be that the system is general. Normally, if we have a new body type, we need to write a new control algorithm, but in this case, whatever the body type is, we can use the same algorithm to teach it. Absolutely amazing. What a time to be alive. However, I know what you're thinking. Why teach them to navigate just stepping stones? This is such a narrow application of locomotion. So why this task? A great question. And the answer is that the generality of this technique, we just talked about also means that the stepping stone navigation truly was just a stepping stone. And here it is. We can deploy these agents to a continuous terrain and expect them to lean on their stepping stone chops to navigate well here too. Another great triumph for curriculum based AI training environments. So what do you think? What would you use this technique for? Let me know in the comments or if you wish to discuss similar topics with other fellow scholars in a warm and welcoming environment, make sure to join our Discord channel. The link is available in the video description. This episode has been supported by Lambda GPU Cloud. If you're looking for inexpensive Cloud GPUs for AI, check out Lambda GPU Cloud. They've recently launched Quadro RTX 6000, RTX 8000 and V100 instances. And hold onto your papers because Lambda GPU Cloud can cost less than half of AWS and Azure. Plus they are the only Cloud service with 48GB RTX 8000. Join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to LambdaLabs.com slash papers to sign up for one of their amazing GPU instances today. Our thanks to Lambda for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.