 Dear Fellow Scholars, this is two-minute papers with Karo Zsolnai-Fehir. Have you heard of the Can Burns effect? If you have been watching this channel, you have probably seen examples where a still image is shown and a zooming and panning effect is added to it. It looks something like this. Familiar, right? The fact that there is some motion is indeed pleasing for the eye, but something is missing. Since we are doing this with 2D images, all the depth information is lost, so we are missing out on the motion parallax effect that we will see in real life when moving around the camera. So in short, this is only 2D. Can this be done in 3D? Well, to find out, have a look at this. Wow, I love it. Much better, right? Well, if we would try to perform something like this without this paper, we'd be met with bad news. And that bad news is that we have to buy an RGBD camera. This kind of camera endows the 2D image with depth information, which is specialized hardware that is likely not available in our phones as of the making of this video. Now since depth estimation, from this simple, monocular 2D images without depth data is a research field of its own, the first step sounds simple enough. Take one of those neural networks, then ask it to try to guess the depth of each pixel. Does this work? Well, let's have a look. As we move our imaginary camera around, oh oh, this is not looking good. Do you see what the problems are here? Problem number 1 is the presence of geometric distortions. You see it if you look here. Problem number 2 is referred to as semantic distortion in the paper, or in other words, we now have missing data. But this poor, tiny human's hand is also… Ouch. Let's look at something else instead. If we start zooming in into images, which is a hallmark of the Can Burns effect, it gets even worse. We get artifacts. So how does this new paper address these issues? After creating the first, coarse depth map, an additional step is taken to alleviate the semantic distortion issue, and then this depth information is up-sampled to make sure that we have enough fine details to perform the 3D Can Burns effect. Let's do that. Unfortunately, we are still nowhere near done yet. We have previously occluded parts of the background suddenly become visible, and we have no information about those. So, how can we address that? Do you remember image in painting? I hope so, but if not, no matter, I'll quickly explain what it is. Both learning based and traditional handcrafted algorithms exist to try to fill in dismissing information in images with sensible data by looking at its surroundings. This is also not as trivial as it might seem first, for instance, just filling in sensible data is not enough, because this time around we are synthesizing videos, it has to be temporally coherent, which means that there must not be too much of a change from one frame to another, or else we'll get a flickering effect. As a result, we finally have these results that are not only absolutely beautiful, but the user study in the paper shows that they stack up against handcrafted results made by real artists. How cool is that? It also opens up really cool workflows that would normally be very difficult if not impossible to perform. For instance, here you see that we can freeze this lightning bolt in time, zoom around and marvel at the entire landscape. Love it. Of course, limitations still apply if we have really thin objects such as this flagpole, it might be missing entirely from the death map, or there are also cases where the image in painter cannot fill in useful information. I cannot wait to see how this work evolves a couple of papers down the line. One more interesting tidbit, if you have a look at the paper, make sure to open it in an Adobe reader, you will likely be very surprised to see that many of these things that you think are still images are actually animations. Papers are not only getting more mind-blowing by the day, but also more informative and beautiful as well. What a time to be alive. This video has been supported by you on Patreon. If you wish to support the series and also pick up cool perks in return like early access to these episodes, or getting your name immortalized in the video description, make sure to visit us through patreon.com slash 2 minute papers. Thanks for watching and for your generous support and I'll see you next time.