 Dear Fellow Scholars, this is two minute papers with Karojona Ifeher. Earlier, we had an episode about OpenAI's absolutely amazing algorithm that mastered Dota 2, a competitive online battle arena game and managed to beat some of the best players in the world in a relatively limited one versus one game mode. While the full 5 versus 5 version of this learning algorithm is still in the works, scientists at OpenAI experimented with some self-play in other games and have found some remarkable results. You can see here that most of these amusing experiments take place in a made up 3D game with simulated physics. For instance, performing well with these humanoid creatures means controlling 17 actuated joints properly. These agents use a reinforcement learning algorithm to maximize a reward, for instance, a sumo warrior gets a thousand points for pushing their opponent out of the ring. The first interesting thing is that a learning curriculum was used, which means that the algorithm was allowed to explore on their own by relaxing the strict scores that are given only one winning. This is combined with the fact that these agents play against themselves led to some remarkable emergent behaviors. Here you can see with the score how much of a difference this curriculum makes. And you also see that whenever a plot is symmetric, that means that they are zero sum games. So if one agent wins a given number of points, the other loses the same amount. The self-play part is also particularly interesting, as many agents are being trained in parallel at the same time. And if we are talking about one versus one games, we have to create some useful logic to decide who to pair with whom. It seems that training against an older version of a previously challenged opponent was the best strategy. This makes sense because they are running a similar algorithm. And for self-play, this means that the algorithm is asked to defeat an older version of itself. If it can reliably do that, it will lead to a smooth and predictable learning process. It is kind of incredible to think about the fact that we have a virtual world with a bunch of simulated learning creatures, and we are omnipotent beings trying to craft the optimal learning experience for them. The perks of being a researcher in machine learning. And we are even being paid for this. Isn't this incredible? Ssh, don't tell anyone about this. There are so many interesting results here and so much to talk about. For instance, we haven't even talked about transfer learning, where these creatures learn to generalize their knowledge learned from previous tasks to tackle new challenges more efficiently. Make sure to have a look at the paper and the source code is available for everyone free of charge. If you're one of those fellow thinkers, you'll be more than happy to look into the video description. If you wish to hear more about transfer learning, subscribe and turn on notifications because the next episode is going to be about some really cool results in this area. Thanks for watching and for your generous support, and I'll see you next time.