 Dear Fellow Scholars, this is Two Minute Papers with Karojona Ifehir. This episode has been sponsored by Lambda Labs. Not so long ago, we talked about DeepMind's Alpha Star, an AI that was able to defeat top tier human players in Starcraft 2, a complex real-time strategy game. Of course, I love talking about AI's that are developed to challenge pro gamers at a variety of difficult games, so this time around we'll have a look at another major milestone, Open AI 5, which is an AI that plays Dota 2, a multiplayer online battle arena game with a huge cult following. As this game requires long-term strategic planning, it is a classic nightmare scenario for any AI. But, Open AI is no stranger to Dota 2, in 2017 they showed us an initial version of their AI that was able to play one versus one games with only one hero and was able to reliably be Dandy, a world champion player. That was quite an achievement, however, of course this was meant to be a stepping stone towards playing the real Dota 2. Then in 2018 they unveiled Open AI 5, an improved version of this AI that played 5 vs 5 games with a limited hero pool. This team was able to defeat competent players but was still not quite at the level of a world champion human team. In a 1 hour interview, the Open AI research team mentioned that due to the deadline of the international event, they had to make quite a few concessions. And this time several things have changed. First, they didn't just challenge some local team of formidable players, no no, they flat out challenged OG, the reigning world champion team, an ambitious move that exudes confidence from their side. Second, this time around, there was no tight deadline as the date of the challenge was chosen by Open AI. Let's quickly talk about the rules of the competition and then see if Open AI's confident move was justified. These learning agents don't look at the pixels of the game and as a result they see the world as a big bunch of numbers. And this time around, it was able to play a pool of 17 heroes and trained against itself for millions and millions of games. And now, let's have a look at what happened in this best of 3 series. In match 1, right after picking the roster of heroes, the AI estimated its win probability to be 67% so it was quite a surprise that early on, it looked like Open AI's bots were running around aimlessly. Over time, we found out that it was not at all the case. It plays unusually aggressively from the gap goal and uses buybacks quite liberally at times where human players don't really consider it to be a good choice. These buybacks resurrect a perished hero quickly but in return cost money. Later, it became clearer that these bots are no joke. They know exactly when to engage and when to back out from an engagement with the smallest sliver of health left. I will show quite a few examples of those to you during this video, so stay tuned. A little less than 20 minutes in, we had a very even game 1, if anything, Open AI seemed a tiny bit behind and someone noted that we should perhaps ask the bots what they think about their chances. And then the AI said, yeah, no worries, we have a higher than 95% chance to win the game. This was such a pivotal moment that was very surprising for everyone. Of course, if you call out to win with confidence, you better go all the way and indeed win the game. Right? Right. And sure enough, they wiped out almost the entire world champion team of the human players immediately after. No tell's pushed all the way back. Officials will come out to hold them down but Open AI, they've got two more kills. And then noted, you know what? Remember that we just said? Forget that. We estimate our chances to win to be above 99% now. And shortly after, they won match number one. Can you believe this? This is absolutely amazing. Interestingly, one of the developers said that the AI is great at assessing whether a fight is worth it. As an interesting corollary, if you engage with it and it fights you, it probably means that you are going to lose. That must be quite confusing for the players. Some mind games for you. Love it. At the event, it was also such a joy to see such a receptive audience that understood and appreciated high level plays. One words to match number two. Right after the draft, which is the process of choosing the heroes for each team, the AI predicted a win percentage that was much closer this time around, around 60%. In this game, the AI turned up the heat real fast and said just five minutes into the game, which is nothing that it has an over 80% chance to win the game. And now, watch this. In the game, you can see a great example of where the AI just gets away with a sliver of health. Look at this guy. They will find the follow-up wrap around Killon towards him, at least they are trying, but with that walk right out, he is able to run away. Another fish comes out. Surely this kill is going to be there, but no, a stun from the span holds back the shaker and he teepes out on 30 HP. Open AI gets out of there with the span, they cannot get that kill OG. Look at that. This is either an accident or some unreal level foresight from the side of this agent. I'd love to hear your opinion on which one you think it is. By the 9.5 minute mark, which is still really early, Open AI 5 said, yes, we got this one too. Over 95%. Here you see an interesting scenario where the AI loses one hero, but it almost immediately kills two of the human heroes and comes out favorably, at which point we wonder whether this was a deliberate bait it pulled on the humans. They do it the disabled. Hex comes out from the hotel, they'll kill up the crystal maiden, but Open AI Viper, diving past the tower onto us, tops this span, coming back in on minimal HP to throw that stun and secure the kill, as Open AI again getting the favourable trade another tower taken, they are playing at a ferocious speed here in the second game. By the 15 minute mark, the human players lost a barracks and were heavily underfunded and outplayed with seemingly no way to come back. And sure enough, by the 21 minute mark, the game was over. There is no other way to say it, this second game was a one sided beat down. 1-1 was a strategic back and forth where Open AI 5 waited for the right moment to win the game in a big team fight, where here they pressure the human team from the get go and never let them reach the end game where they might have an advantage with their picks. Also have a look at this. Unreal The final result is 2-0 for Open AI. In the post-match interview, No-Tale, one of the human players noted that he is confident that from 5 games they would take at least 1 and after 15 games they would start winning reliably. Very reminiscent of what we heard from players playing against DeepMind's AI in Starcraft 2 and I hope this will be tested. However, in the end he agreed that it is inevitable that this AI will become unbeatable at some point. It was also noted that in 5 vs 5 fights, they seem better in planning than any human team is and there is quite a lot to learn from the AI for us humans. They were also trying to guess the reasoning for all of these early buybacks. According to the players, initially they flat out seemed like misplaced. Perhaps the reason for this instant and not really great buybacks might have been that the AI knows that if the game goes on for much longer, statistically their chances with their given composition to win the game do windows, so it needs to immediately go and win right now whatever the cost. And again, an important lesson is that in this project, open AI is not spending so much money and resources to just play video games. Dota 2 is a wonderful test bed to see how their AI compares to humans at complex tasks that involve strategy and teamwork. However, the ultimate goal is to reuse parts of this system for other complex problems outside of video games. For instance, the algorithm that you've seen here today can also do this. But wait, there's more. Players after these show matches always tend to get these messages from others on Twitter telling them what they did wrong and what they should have done instead. Well, luckily these people were able to show their prowess as open AI gave the chance for anyone in the world to challenge the open AI 5 competitively and play against them online. This way not only team OG, but everyone can get crushed by the AI. How cool is that? This arena event has concluded with over 15,000 games played where open AI 5 had a 99.4% win rate. There are still ways to beat it, but given the rate of progress of this project, likely not for long. Insanity. As always, if you're interested in more details, I put a link to a Reddit AMA in the video description and I also can't wait to pick the algorithm apart for you, but for now we have to wait for the full paper to appear. And note that what happened here is not to be underestimated. Huge respect to the open AI team, to OG for the amazing games and congratulations to the humans who were able to beat these beastly bots online. So there you go. Another long video that's not two minutes and it's not about the paper. Yet. Welcome to two minute papers. If you're doing deep learning, make sure to look into Lambda GPU systems. Lambda offers workstations, servers, laptops, and the GPU cloud for deep learning. You can save up to 90% over AWS, GCP, and Azure GPU instances. Every Lambda GPU system is pre-installed with TensorFlow, PyTorch, and Keras. Just plug it in and start training. Lambda customers include Apple, Microsoft, and Stanford. Go to LambdaLabs.com, slash, papers, or click the link in the description to learn more. Big thanks to Lambda for supporting two minute papers and helping us make better videos. Thanks for watching and for your generous support, and I'll see you next time.