 Dear Fellow Scholars, this is two-minute papers with Dr. Karojona Ifehir. Today, we are going to perform pose estimation with an amazing twist. You'll love it. But wait, what is pose estimation? Well, simple, a video of people goes in and the posture they are taking comes out. Now, you see here that previous techniques can already do this quite well even for videos. So, by today, the game has evolved. Just pose estimation is not that new. We need pose estimation plus something else. We need a little extra, if you will. So what can that little extra be? Let's look at three examples. For instance, one Nvidia has a highly advanced pose estimation technique that can refine its estimations by putting these humans into a virtual physics simulation. Without that, this kind of foot sliding often happens, but after the physics simulation, not anymore. As a result, it can understand even this explosive sprinting motion. This dynamic serve too, you name it, all of them are very close. So, what is this good for? Well, many things, but here is my favorite. If we can track the motion well, we can put it onto a virtual character so we ourselves can move around in a beautiful, imagined world. So, that was one. Pose estimation plus something extra, where the something extra is a physics simulation. Nice. Now, two, if we allow an AI to read the Wi-Fi signals bouncing around in a room, it can perform pose estimation even through walls. Kind of. Once again, pose estimation with something extra. And three, this is pose estimation with inertial sensors. This works when playing a friendly game of table tennis with a friend, or, wait, maybe a not so friendly game of table tennis. And this works really well, even in the dark. So all of these are pose estimation plus something extra. And now, let's have a look at this new paper which performs pose estimation plus, well, pose estimation as it seems. Okay, I don't see anything new here, really. So why is this work on two-minute papers? Well, now, hold on to your papers and check this out. Yes. Oh, yes. So what is this? Well, here's the twist we can give a piece of video to this AI, write a piece of text as you see up here, and it will not only find what we are looking for in the video, market, but then even track it over time. Now that is really cool. We just say what we want to be tracked over the video, and it will do it automatically. It can find the dog and the capybara. These are rather rudimentary descriptions, but it is by no means limited to that. Look, we can also say a man wearing a white shirt and blue shorts riding a surfboard. And yes, it can find it. And also we can add a description of the surfboard and it can tell which is which. And I like the tracking too. The scene has tons of high frequency changes, lots of occlusion, and it is still doing really well. Loving it. So I am thinking that this helps us take full advantage of the text descriptions. Look, we can ask it to mark the parrot and the cacotoo, and it knows which is which. So I can imagine more advanced applications where we need to find the appropriate kind of animal or object among many others, and we don't even know what to look for. Just say what you want, and it will find it. I also liked how this is done with a transformer neural network that can jointly process the text and video in one elegant solution. That is really cool. Now of course, every single one of you fellow scholars can see that this is not perfect, not even close. Depending on the input, tempero coherence issues may arise. These are the jumpy artifacts from frame to frame. But still, this is swift progress in machine learning research. We could only do this in 2018, and we were very happy about it. And just a couple of papers down the line, we just say what we want, and the AI will do it. And just imagine what we will have a couple more papers down the line. I cannot wait. So what would you use this for? Please let me know in the comments below. And wait, the source code, an interactive demo, and a notebook are available in the video description. So, you know what to do? Yes, let the experiments begin. This video has been supported by weights and biases. Check out the recent offering fully connected, a place where they bring machine learning practitioners together to share and discuss their ideas, learn from industry leaders, and even collaborate on projects together. You see, I get messages from you fellow scholars telling me that you have been inspired by the series, but don't really know where to start. And here it is. Fully connected is a great way to learn about the fundamentals, how to reproduce experiments, get your papers accepted to a conference, and more. Make sure to visit them through wnb.me slash papers, or just click the link in the video description. Our thanks to weights and biases for their longstanding support, and for helping us make better videos for you. Thanks for watching and for your generous support, and I'll see you next time.