 Dear Fellow Scholars, this is two minute papers with Dr. Karo Jolai-Fehir. In June 2020, OpenAI published an incredible AI-based technique by the name ImageGPT. The problem here was simple to understand, but nearly impossible to actually do, so here goes, we give it an incomplete image, and we ask the AI to fill in the missing pixels. That is, of course, an immensely difficult task because these images made the picked any part of the world around us. It would have to know a great deal about our world to be able to continue the images, so how did it do? Let's have a look. This is undoubtedly a cat, but look, see that white part that is just starting? The interesting part has been sneakily cut out of the image. What could that be? A piece of paper, something else? Now, let's leave the dirty work to the machine and ask it to finish it. Oh yeah, that makes sense. Now, even better, let's have a look at this water droplet example too. We humans know that since we see the remnants of ripples over there too, there must be a splash, but the question is, does the AI know that? Oh yes, yes it does. Amazing. And the true image for reference. But wait a second, if image GPT could understand that this is a splash and finish the image like this, then here is an absolutely insane idea if a machine can understand that this is a splash, could it maybe not only finish the photo, but make a video out of it? Yes, that is indeed an absolutely insane idea, we like those around here. So, what do you think? Is this a reasonable question or is this still science fiction? Well, let's have a look at what this new learning base method does when looking at such an image. It will do something very similar to what we would do, look at the image, estimate the direction of the motion, recognize that these ripples should probably travel outwards and based on the fact that we've seen many splashes in our lives, if we had the artistic skill, we could surely fill in something similar. So, can the machine do it too? And now, hold on to your papers because this technique does exactly that. Whoa! Please meet Eulerian motion synthesis and it not only works amazingly well, but look at the output video, it even loops perfectly. Yum yum yum, I love it. And it works mostly on fluid and smoke. I like that. I like that a lot because fluids and smoke have difficult, but predictable motion. That is an excellent combination for us, especially given that you see plenty of those simulations on this channel. So, if you are a long time fellow scholar, you already have a key knife for them. Here are a few example images paired with the synthesized motion fields, these define the trajectory of each pixel or in other words regions that the AI thinks should be animated and how it thinks should be animated. Now, it gets better, I have found three things that I did not expect to work, but was pleasantly surprised that they did. One, reflections, kind of work. Two, fire, kind of works. And now, if you have been holding on to your paper so far, now squeeze that paper because here comes the best one. Three, my beard works too. Yes, you heard it right. Now, first things first, this is not any kind of beard, this is an algorithmic beard that was made by an AI. And now, it is animated as if it were a piece of fluid using a different AI. Of course, this is not supposed to be a correct result, just a happy accident, but in any case, this sounds like something straight out of a science fiction movie. I also like how this has a nice Obi-Wan Kenobi quality to it, loving it. Thank you very much to my friend Oliver Wong and the authors for being so kind and generating these results only for us. That is a huge honor. Thank you. This previous work is from 2019 and creates high quality motion, but has a limited understanding of the scene itself. And of course, let's see how the new method fares in these cases. Oh yeah, this is a huge leap forward. And what I like even better here is that no research techniques often provide different trade-offs than previous methods, but are really strictly better than them. In other words, competing techniques usually do some things better and some other things worse than their predecessors, but not this. Look, this is so much better across the board. That is such a rare sight. Amazing. Now, of course, not even this technique is perfect. For example, this part of the image should have been animated, but remains stationary. Also, even though it did well with reflections, reflection is a tougher nut to crack. Finally, thin geometry also still remains a challenge, but this was one paper that made the impossible possible and just think about what we will be able to do two more papers down the line. My goodness. What a time to be alive. This episode has been supported by weights and biases. In this post, they show you how to use their tool to get a neural network to generate captions for your images using an attention model. During my PhD studies, I trained a ton of neural networks which were used in our experiments. However, over time, there was just too much data in our repositories and what I am looking for is not data, but inside. And that's exactly how weights and biases helps you by organizing your experiments. It is used by more than 200 companies and research institutions, including OpenAI, Toyota Research, GitHub, and more. And get this. Weight and biases is free for all individuals, academics, and open source projects. Make sure to visit them through wnba.com slash papers or just click the link in the video description and you can get a free demo today. Our thanks to weights and biases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.