 Dear Fellow Scholars, this is Two Minute Papers with Dr. Karojona Ifehir. This new learning-based method is capable of generating new videos, creating lava from water, enhancing your dance videos, adding new players to a football game, and more. This one will feel like it can do absolutely anything. For instance, have a look at these videos. Believe it or not, some of these were generated by an AI from this new paper. Let's try to find out together which ones are real and which are generated. Now, before we start, many of the results that you will see here will be flawed, but I promise that there will be some gems in there too. This one doesn't make too much sense. One down. Two, the bus appears to be changing length from time to time. Three, the bus pops out of existence. Four, wait, this actually looks pretty good until we find out that someone is about to get rear-ended. This leaves us with two examples. Are they real or are they fixed? Now, please stop the video and let me know in the comments what you think. So, this one is real footage. Okay, what about this guy? Well, get this, this one is actually one of the generated ones. Just this one was real and this is what this new technique is capable of. We give it a sample video and it re-imagines variations of it. Of course, the results are hit or miss. For instance, here, people and cars just pop in and out of existence, but if we try it on the billiard balls, wow, now this is something. Look at how well it preserved the specular highlights and also the shadows move with the balls in the synthetic variations too. Once again, the results are generally hit or miss, but if we look through a few results, we often find some jumps out there. But this is nothing compared to what is to come. Let's call this first feature video synthesis and we have four more really cool applications with somewhat flawed but sometimes amazing results. Now, check this out. Here comes number two video analogies. This makes it possible for us to mix two videos that depict things that follow a similar logic. For instance, here only four of the 16 videos are real, the rest are all generated. Now, here comes feature number three time retargeting. For instance, we can lengthen or shorten videos. Well, that sounds simple enough, no need for an AI for that, but here's the key. This does not mean just adding new frames to the end of the video. Look, they are different. Yes, the entirety of the video is getting redesigned here. We can use this to lengthen videos without really adding new content to it. Absolutely amazing. We can also use it to shorten a video. Now, of course, once again, this doesn't mean that it just chops off the end of the video. No, no, it means something much more challenging. Look, it makes sure that all of your killer moves make it into the video, but in a shorter amount of time. It makes the video tighter if you will. Amazing feature number four, sketch to video. Here, we can take an input video, add a crew drawing, and the video will follow this drawing. The result is, of course, not perfect, but in return, this can handle many number to number transitions. And now, feature number five, video in painting on steroids. Previous techniques can help us delete part of an image or even a video and generate data in these holes that make sense given their surroundings. But this one does something way better. Look, we can cut out different parts of the video and mark the missing region with the color, and then what happens? Oh, yes. The blue regions will contain a player from the blue team, the white region, a player from the white team, or if we wish to get someone out of the way, we can do that too. Just mark them green. Here, the white regions will be impainted with clouds and the blues with birds, loving it. But, wait a second, we noted that there are already good techniques out there for image impainting. There are also good techniques out there for video time retargeting. So, what is so interesting here? Well, two things. One, here, all of these things are being done with just one technique. Normally, we would need a collection of different methods to perform all of these. But here, just one AI. Two, now hold on to your papers and look at this. Whoa! Previous techniques take forever to generate these results. 144p means an extremely crude, pixelated image, and even then they take from hours to my goodness, even days. So, what about this new one? It can generate much higher resolution images and in a matter of minutes. That is an incredible leap in just one paper. Now, clearly, most of these results aren't perfect. I would argue that they are not even close to perfect. But some of them already show a lot of promise. And, as always, dear fellow scholars, don't not forget to apply the first law of papers. Which says, don't look at where we are, look at where we will be, two more papers down the line. And two more papers down the line, I am sure that not two, but five, or maybe even six out of these six videos, will look significantly more real. This video has been supported by weights and biases. They have an amazing podcast by the name Gradient Descent, where they interview machine learning experts who discuss how they use learning based algorithms to solve real world problems. They've discussed biology, teaching robots, machine learning in outer space, and a whole lot more. Perfect for a fellow scholar with an open mind. Make sure to visit them through wnb.me slash gd or just click the link in the video description. Our thanks to weights and biases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support. Thanks for watching and for your generous support. And I'll see you next time.