 Dear Fellow Scholars, this is 2 Minute Papers with Dr. Károly Zsolnai-Fehir in computer games and all kinds of applications where we are yearning for realistic animation, we somehow need to tell our computers how the different joints and body parts of these virtual characters are meant to move around over time. Since the human eye is very sensitive to even the tiniest inaccuracies, we typically don't program these motions by hand, but instead we often record a lot of real-time motion capture data in a studio and try to reuse that in our applications. Previous techniques have tackled quadruped control and we can even teach bipeds to interact with their environment in a realistic manner. Today we will have a look at an absolutely magnificent piece of work where the authors carved out a smaller subproblem and made a solution for it that is truly second to none. And this subproblem is simulating virtual characters playing basketball. Like with previous works, we are looking for realism in the movement and for games it is also a requirement that the character responds to our controls well. However, the key challenge is that all we are given is 3 hours of unstructured motion capture data. That is next to nothing and from this next to nothing, a learning algorithm has to learn to understand these motions so well that it can weave them together even when a specific movement combination is not present in this data. That is quite a challenge. Compared to many other works, this data is really not a lot so I am excited to see what value we are getting out of these 3 hours. At first I thought we would get only very rudimentary motions and boy was I dead wrong on that one. We have control over this character and can perform these elaborate maneuvers and it remains very responsive even if we mesh the controller like a madman producing these sharp turns. As you see it can handle these cases really well. And not only that but it is so well done we can even dribble through a set of obstacles leading to a responsive and enjoyable gameplay. Now about these dribbling behaviors, do we get only one boring motion or not? Not at all it was able to mine out not just one but many kinds of dribbling motions and is able to weave them into other moves as soon as we interact with the controller. This is already very convincing especially from just 3 hours of unstructured motion capture data. But this paper is just getting started. Now hold on to your papers because we can also shoot and catch the ball, move it around that is very surprising because it has looked at so little shooting data. Let's see yes less than 7 minutes. My goodness and it keeps going what I have found even more surprising is that it can handle unexpected movements which I find to be even more remarkable given the limited training data. These crazy corner cases are typically learnable when they are available in abundance in the training data which is not the case here. Amazing. When we compare these motions to a previous method we see that both the character and the ball's movement is much more lively. For instance here you can see that the face function neural network pfnn in short almost makes it seem like the ball has to stick to the hand of the player for an unhealthy amount of time to be able to create these motions. It doesn't happen at all with the new technique and remember this new method is also much more responsive to the player's controls and thus more enjoyable not only to look at but to play with. This is an aspect that is hard to measure but it is not to be underestimated in the general playing experience. Just imagine what this research area will be capable of not in a decade but just two more papers down the line. Loving it. Now at the start of the video I noted that the authors carved out a small use case which is training an AI to weave together basketball motion capture data in a manner that is both realistic and controllable. However, many times in research we look at a specialized problem and during that journey we learn general concepts that can be applied to other problems as well. That is exactly what happened here as you see parts of this technique can be generalized for quadruped control as well. Because good boy is pacing and running around beautifully. And you guessed right our favorite biped from the previous paper is also making an introduction. I am absolutely spellbound by this work and I hope that now you are too. Can't wait to see this implemented in newer games and other real time applications. What a time to be alive. This episode has been supported by weights and biases. In this post they show you how to use their system with PyTorch lightning and decouple your science code from your engineering code and visualize your models. You can even try an example in an interactive notebook through the link in the video description. Weight and biases provides tools to track your experiments in your deep learning projects. Their system is designed to save you a ton of time and money and it is actively used in projects at prestigious labs such as OpenAI, Toyota Research, GitHub and more. And the best part is that if you have an open source academic or personal project you can use their tools for free. It really is as good as it gets. Make sure to visit them through wnb.com slash papers or click the link in the video description to start tracking your experiments in five minutes. Thanks for watching and for your generous support and I'll see you next time.