 Dear Fellow Scholars, this is 2 Minute Papers with Karojona Ifahir. After having a look at OpenAI's effort to master the Dota 2 game, of course, we all know that scientists at DeepMind are also hard at work on an AI that beats the Capture the Flag Game Mode in Quake 3. Quake 3 Arena is an iconic first-person shooter game and Capture the Flag is a fun game mode where each team tries to take the other team's flag and carry it to their own base while protecting their own. This game mode requires good aiming skills, map presence, reading the opponents well, and tons of strategy, a nightmare situation for any kind of AI. Not only that, but in this version, the map changes from game to game, therefore the AI has to learn general concepts and be able to pull them off in a variety of different, previously unseen conditions. This doesn't seem to be within the realm of possibilities to pull off. The minimaps here always show the location of the players, each are color coded to blue or red to indicate their teams. Much like humans, these AI agents learned by looking at the video output of the game and have never been told anything about the game or what the rules are. These scientists at DeepMind ran a tournament with 40 human players who were matched up against these agents randomly, both as opponents and teammates. In this tournament, a team of average human players had a win probability of 43% where a team of strong players won slightly more than half 52% of their games. And now hold on to your papers because the agents were able to win 74% of their games. So the difference between the average and the strong human players' win rate is 9%. And the difference between the strongest humans and the AI is more than twice that margin 22%. This is insanity. And as you see, it barely matters what the size or the layout of the map is or how many teammates there are, the AI's win rate is always remarkably high. These agents showcase many human-like behaviors such as staying at their own base to defend it, camping within the opponent's base or following teammates. This builds on a new architecture by the name, for the win, FTW in short, good workfox, instead of training one agent, it uses a population of agents that train and evolve from each other to make sure that the diverse set of playstyles are discovered. This uses recurrent neural networks. These are neural network variants that are able to learn and produce sequences of data. Here, two of these are used, a fast and a slow one that operate on different time scales, but share a memory module. This means that one of them has a very accurate look at the near past and the other one has a more coarse look that can look back more into the past in return. If these two work together correctly, decisions can be made that are both good locally at this point in time and globally to maximize the probability of winning the whole game. This is really huge because this algorithm can perform long-term planning, which is one of the key reasons why many difficult games and tasks remain unsolved. Well, as it seems now, not for long. An additional challenge is that the game score is not necessarily subject to maximization like in most games, but there is a mapping from the scores into an internal reward, which means that the algorithm has to be able to predict its own progress towards winning. And note that even though Quake III and Capture the Flag is an excellent way to demonstrate the capabilities of this algorithm, this architecture can be generalized to other problems. I am going to give you a few more tidbits that I have found super interesting, but before, if you are enjoying this episode and would like to pick up some cool perks like early access, ending the topic of future episodes or getting your name listed in the video description as a key supporter, why not support the show on Patreon. With this, you can also help us make better videos in the future. You can find us at patreon.com slash 2 minute papers and we also support Bitcoin and other crypto currencies. The addresses are available in the video description. And now, onwards to the cool tidbits. A human plus agent team has been able to defeat an agent plus agent team 5% of the time, indicating that these AIs are able to coordinate and play together with anyone they are given. I get goosebumps from this. Love it. The reaction time and accuracy of the agents is better than that of humans, but not nearly perfect as many people would think. However, they outclass humans even if we artificially reduce their accuracy and reaction times. In another experiment, two agents were paired up against two professional game tester humans who could freely communicate and train against the same agents for 12 hours to see if they can learn their patterns and force them to make mistakes. Even with this, humans had only 125% of these games. Given the other numbers we have, it is very likely that this unfair advantage made no difference whatsoever. How about that? If there are any more questions, make sure to have a look at the paper that describes every possible tidbit you can possibly imagine. Thanks for watching and for your generous support, and I'll see you next time.