 Dear Fellow Scholars, this is 2 Minute Papers with Karo Isholaifahir. I think this is one of the more important things that happened in AI research lately. In the last few years we have seen Deep Mind defeat the best goal players in the world, and after Open AI's venture in the game of Dota 2, it's time for Deep Mind to shine again, as they take on Starcraft 2 a real-time strategy game. The depth and the amount of skill required to play this game is simply astounding. The search space of Starcraft 2 is so vast that it exceeds both chess and even go by a significant margin. Also, it is a game that requires a great deal of mechanical skill, split second decision-making, and we have imperfect information as we only see what our units can see. A nightmare situation for any AI. Deep Mind invited a beloved pro player, TLO, to play a few games against their new Starcraft 2 AI that goes by the name Alpha Star. Note that TLO is a professional player who is easily in the top 1% of players or even better, mid-grandmaster for those who play Starcraft 2. This video is about what happened during this event, and later I will make another video that describes the algorithm that was used to create this AI. The paper is still under review, so it will take a little time until I can get my hands on it. At the end of this video, you will also see the inner workings of this AI. Let's dive in. This is an AI that looked at a few games played by human players, and after that initial step, it learned by playing itself for about 200 years. In our next episode, you will see how this is even possible, so I hope you are subscribed to the series. You see here that the AI controls the blue units, and TLO, the human player, plays red. Right at the start of the first game, the AI did something interesting. In fact, what is interesting is what it didn't do. It started to create new buildings next to its nexus, instead of building a wall-off that you can see here. Using a wall-off is considered standard practice in most games, and the AI used these buildings to not wall-off the entrance, but shield away the workers from possible attacks. Now note that this is not unheard of, but this is also not a strategy that is widely played today and is considered non-standard. It also built more worker units than what is universally accepted as standard, we found out later that this was partly done in anticipation of losing a few of them early on. Very cool. One, almost before we even knew what happened, it won the first game a little more than 7 minutes in, which is very quick, noting that in-game time is a little faster than real time. The thought process of TLO at this point is that that's interesting, but okay, well, the AI plays aggressively and managed to pull this one off. No big deal. We will fire up the second game, and in the meantime, a few interesting details. The goal of setting up the details of this algorithm was that the number of actions performed by the AI roughly matches a human player, and hopefully it still plays as well or better. It has to make meaningful strategic decisions. You see here that this checks out for the average actions every minute, but if you look here, you see around the tail end here that there are times when it performs more actions than humans, and this may enable play styles that are not accessible for human players. However, note that many times it also does miraculous things with very few actions. Now, what about another important detail? Reaction time. The reaction time of the AI is set to 350 milliseconds, which is quite slow. That's excellent news because this is usually a common angle of criticism for game AI's. The AI also sees the whole map at once, but it is not given more information than what its units can see. This perhaps is the most commonly misunderstood detail, so it is worth noting. So in other words, it sees exactly what a human would see if the human would move the camera around very quickly, but it doesn't have to move the camera, which adds additional actions and cognitive load to the human, so one might say that the AI has an edge here. The AI plays these games independently. What's more, each game was played by a different AI, which also means that they do not memorize what happened in the last game, like a human would. Early in the next game, we can see the utility of the wall of inaction, which is able to completely prevent the AI's early attack. Later that game, the AI used disruptors, the unit which, if controlled with such level of expertise, can decimate the army of the opponent with area damage by killing multiple units at once. It has done an outstanding job picking away at the army of TLO. Then, after getting a significant advantage, Alpha Star loses it with a few sloppy plays and by deciding to engage aggressively while standing in tight choke points. You can see that this is not such a great idea. This was quite surprising as this is considered to be Starcraft 101 knowledge right there. During the remainder of the match, the commentators mentioned that they play and watch games all the time and the AI came up with an army composition that they have never seen during a professional match. And the AI won this one too. After this game, it became clear that these agents can play any style in the game, which is terrifying. Here you can see an alternative visualization that shows a little more of the inner workings of the neural network. We can see what information it gets from the game, the visualization of neurons that get activated within the network, what locations and units are considered for the next actions, and whether the AI predicts itself as the winner or the loser of the game. If you look carefully, you will also see the moment when the agent becomes certain that it will win this game. I could look at this all day long and if you feel the same way, make sure to visit the video description. I have a link to the source video for you. The final result against TLO was 5-0. I actually lost everything all the fire matches. So that's something. And he mentioned that Alpha Star played very much like a human does and almost always managed to outmaneuver him. However, TLO also mentioned that he is confident that upon playing more training matches against these agents, he would be able to defeat the AI. I hope he will be given a chance to do that. This AI seems strong, but still beatable. I would also note that many of you would probably expect the later versions of Alpha Star to be way better than this one. The good news is that the story continues and we'll see whether that's true. So at this point, the DeepMind scientists said maybe we could try to be a bit more ambitious and asked, can you bring us someone better? And in the meantime, pressed that training button on the AI again. In comes mana, a top tier pro player, one of the best pro task players in the world. This was a nerve-wracking moment for DeepMind scientists as well because their agents played against each other, so they only knew the AI's win rate against a different AI. But they didn't know how they would compete against a top pro player. It may still have holes in its strategy. Who knows what would happen. Understandably, they had very little confidence in winning this one. What they didn't expect is that the new AI was not slightly improved or somewhat improved. No, no, no, no. This new AI was next level. This set of improved agents, among many other skills, had incredibly crisp micromanagement of each individual unit. In the first game, we've seen it pulling back injured units, but still letting them attack from afar masterfully, leading to an early win for the AI against mana in the first game. He and the commentators were equally shocked by how well the agent played. And I will add that I remember from watching many games from a now inactive player by the name Marine King a few years ago. And I vividly remember that he played some of his games so well, the commentator said that there is no better way to put it, he played like a god. I am almost afraid to say that this micromanagement was even more crisp than that. This AI plays phenomenal games. In later matches, the AI did things that seemed like blunders, like attacking on ramps and standing in choke points, or using unfavorable unit compositions and refusing to change it and get this. It's still one all of those games 5-0 against a top pro player. Let that sink in. The competition was closed by a match where the AI was asked to also do the camera management. The agent was still very competent, but somewhat weaker and as a result lost this game, hence the 0 or 1 part in the title. My impression is that it was asked to do something that it was not designed for and expect a future version to be able to handle this use case as well. I will also commend Mana for his solid game plan for this game and also huge respect for deep-mind for their sportsmanship. Interestingly, in this match, Mana also started a worker over saturation strategy that I mentioned earlier. This he learned from Alpha Star and used it in his winning game. Isn't that amazing? Deep-mind also offered a Reddit AMA where anyone could ask them questions to make sure to clear up any confusion. For instance, the actions per minute part has been addressed. I've included a link to that for you in the video description. To go from a turn-based, perfect information game, like Go, to a real-time strategy game of imperfect information in about a year sounds like science fiction to me. And yet, here it is. Also, note that Deep-mind's goal is not to create a godlike Starcraft II AI. They want to solve intelligence, not Starcraft II, and they use this game as a vehicle to demonstrate its long-term decision-making capabilities against human players. One more important thing to emphasize is that the building blocks of Alpha Star are meant to be reasonably general AI algorithms, which means that parts of this AI can be reused for other things. For instance, Demis Asabi mentioned weather prediction and climate modeling as examples. If you take only one thought from this video, let it be this one. I urge you to watch all the matches because what you are witnessing may very well be history in the making. I put a link to the whole event in the video description plus plenty more materials, including other people's analysis, Manus' personal experience of the event, his breakdown of his games, and what was going through his head during the event. I highly recommend checking out his fifth game, but really, go through them all. It's a ton of fun. I made sure to include a more skeptical analysis of the game as well to give you a balanced portfolio of insights. Also, huge respect for Deep-mind and the players who practice their chops for many, many years and have played really well under immense pressure. Thank you all for this delightful event. It really made my day. And the ultimate question is, how long did it take to train these agents? Two weeks. Wow. And what's more, after the training step, the AI can be deployed on an inexpensive consumer desktop machine. And this is only the first version. This is just a taste, and it would be hard to overstate how big of a milestone this is. And now, scientists that Deep-mind have sufficient data to calculate the amount of resources they need to spend to train the next even more improved agents. I am confident that they will also take into consideration the feedback from the Starcraft community when creating this next version. What a time to be alive. What do you think about all this? Any predictions? Is this harder than Dota 2? Let me know in the comments section below. And remember, we humans build up new strategies by learning from each other, and of course, the AI, as you have seen here, doesn't care about any of that. It doesn't need intuition and can come up with unusual strategies. The difference now is that these strategies work against some of the best human players. Now it's time for us to finally start learning from an AI. GG. Thanks for watching and for your generous support, and I'll see you next time.