 Dear Fellow Scholars, this is 2 Minute Papers with Károly Zsolnai-Fehér. I am really excited to show this to you as I was looking to make this episode for quite a while. You'll see lots of beautiful slow motion footage during the narration, and at first it may seem disconnected from the narrative, but by the end of the video you'll understand why they look the way they do. Now before we proceed, let's talk about the difference between interpolation and extrapolation. One means that we have measurement points for a given quantity and would like to know what happened between these points. For instance, we have two samples of a person's location at 4 and at 5 o'clock, and we'd like to know where the guy was at 4.30. However, if we are doing extrapolation, we are interested in guessing a quantity beyond the reach of our sample points. For instance, extrapolation would be predicting what happens after the very last frame of the video. For our earlier episode, we talked about financial extrapolation, make sure to have a look it was super fun. The link is in the video description. Optical flow is really useful because it can do this kind of interpolation and extrapolation for images. So let's do one of them right now. You'll see which it will be in a second. So this is a classical scenario that we often encounter when producing a new 2 minute paper episode. This means that roughly every other frame is duplicated and offers no new information. You can see this as I step through these individual frames. And the more astute fellow scholars immediately would point out that wait. We have a lot of before and after image pairs, so we could do a lot better. Why don't we try to estimate what happened between these images? And that is exactly what we call frame interpolation. Interpolation because it is something between two known measurement points. And if we run the optical flow algorithm that can accomplish this, we can fill in these doubled frames with new ones that actually carry new information. So the ratio here was roughly 2 to 1. Roughly every other frame provides new information. Super cool. So what are the limits of this technique? What if we artificially slow the video down so that it's much longer, so not only every other, but most of the frames are just duplicates? This results in a boring and choppy animation. And we fill those in too, note that the basic optical flow equations are written for tiny little changes in position, so we shouldn't expect it to be able to extrapolate or interpolate any quantity over a longer period of time. But of course, it always depends on the type of motion we have at hand, so let's give it a try. As you can see with optical flow, the algorithm has an understanding of the motions that take place in the footage, and because of that, we can get some smooth, buttery, slow motion footage that is absolutely mesmerizing. Almost like shooting with a slow motion camera. And note that the majority of these frames were not containing any information, and this motion was synthesized from these distant sample points that are miles and miles away from each other. Really cool. However, it is also important to point out that optical flow is not a silver bullet, and it should be used with moderation and special care, as it can also introduce nasty artifacts like the one that you see here. This is due to an abrupt, high frequency change that is more difficult to predict than a slow and steady translation or rotation motion. To avoid these cases, we can use a much simpler frame interpolation technique that we call frame blending. This is a more naive technique that doesn't do any meaningful guesswork and computes the average of the two results. Why don't we give this one a try too? Or even better, let's have a look at the difference between the original choppy footage and the interpolated versions with frame blending and optical flow. If we do that, we see that frame blending is unlikely to give us nasty artifacts, but in return, the results are significantly more limited compared to optical flow because it doesn't have an understanding of the motion taking place in the footage. So the question is when to use which? Well, until we get an algorithm that is able to adaptively decide when to use what, it still comes down to individual judgment and sometimes quite a bit of trial and error. I'd like to make it extremely sure that you don't leave this video thinking that this is the only application of optical flows. It's just one of the coolest ones. But this motion estimation technique also has many other uses. For instance, if we have an unmanned aerial vehicle, it is really great if we can endow it with an optical flow sensor because then it will be able to know in which direction it needs to rotate to avoid a tree or whether it is stable or not at a given point in time. And with your support on Patreon, we were not only able to bump up the resolution of future two-minute paper's episodes to 4K, but we're also running them at true 60 frames per second, which means that every footage can undergo either a frame blending or optical flow step to make the animation smoother and more enjoyable for you. This takes a bit of human labor and is computationally expensive, but our new two-minute paper's rig is now capable of handling this. It is fantastic to see that you fellow scholars are willing to support the series, and through this we can introduce highly desirable improvements to the production pipeline. This is why we thank you at the end of every episode for your generous support. You fellow scholars are the best YouTube audience anywhere. And who knows, maybe one day, will be at a point where two-minute papers can be a full-time endeavor and will be able to make even more elaborate episodes. As I am tremendously enjoying making these videos, that would be absolutely amazing. Have you found any of these disturbing optical flow artifacts during this episode? Have you spotted some of these in other videos on YouTube? Let us know in the comments section so we can learn from each other. Thanks for watching and for your generous support. I'll see you next time.