 Dear Fellow Scholars, this is Two Minute Papers with Dr. Karo Zsolnai-Fehir. Today we get to be a paper historian and witness the amazing progress in machine learning research together and learn what is new in the world of Nerf's. But first, what is a Nerf? In March of 2020, a paper appeared describing an incredible technique by the name Mural Radiance Fields or Nerf in short. This work enables us to take a bunch of input photos and their locations, learn it, and synthesize new previously unseen views of not just the materials in the scene, but the entire scene itself. And here we are talking not only digital environments, but also real scenes as well. Now, that's quite a value proposition, especially given that it also supported refractive and reflective surfaces as well. These are both quite a challenge. However, of course, Nerf had its limitations. For instance, in many cases it had trouble with scenes with variable lighting conditions and lots of occluders. And to my delight, only five months later, in August of 2020, a follow-up paper appeared by the name Nerf in the world or Nerf W in short. This speciality was tourist attractions that a lot of people take photos of and we then have a collection of photos taken during a different time of the day and, of course, with a lot of people around. And lots of people, of course, means lots of occluders. Nerf W improved the original algorithm to excel more in cases like this. A few months later, on 2020, November 25th, another follow-up paper appeared by the name deformable Neural Radiance Fields, the Nerf. The goal here was to take a selfie video and turn it into a portrait that we can rotate around freely. This is something that the authors call a Nerfie. If we take the original Nerf technique to perform this, we see that it does not do well at all with moving things and that's where the new deformable variant really shines. And today's paper not only has some nice video results embedded in the front page, but it offers a new take on this problem and offers, quote, space-time view synthesis of dynamic scenes. Whoa! That is amazing. But what does that mean? What does this paper really do? The space-time view synthesis means that we can record a video of someone doing something. Since we are recording movement in time and there is also movement in space or in other words, the camera is moving. Both time and space are moving. And what this can do is one frees one of those variables in other words pretend as if the camera didn't move. Or two pretend as if time didn't move. Or three generate new views of the scene while movement takes place. My favorite is that we can pretend to zoom in and even better zoom out even if the recorded video looked like this. So how does this compare to previous methods? There are plenty of nerve variants around. Is this really any good? Let's find out together. This is the original nerve we already know about this and we are not surprised in the slightest that it's not so great on dynamic scenes with a lot of movement. However, what I am surprised by is that all of these previous techniques are from 2020 and all of them struggle with these cases. These comparisons are not against some ancient technology from 1985. No, no. All of them are from the same year. For instance, this previous work is called Consistent Video Depth Estimation and it is from August 2020. We showcased it in this series and marveled at all of these amazing augmented reality applications that it offered. The snowing example here was one of my favorites. And today's paper appeared just three months later in November 2020. And the authors still took the time and effort to compare against this work from just three months ago. That is fantastic. As you see, this previous method kind of works on this dog but the lack of information in some regions is quite apparent. This is still maybe usable but as soon as we transition into a more dynamic example, what do we get? Well, pandemonium. This is true for all previous methods. I cannot imagine that the new method from just a few months later could deal with this difficult case and look at that. So much better. It is still not perfect. You see that we have lost some detail but witnessing this kind of progress in just a few months is truly a sight to behold. It really consistently outperforms all of these techniques from the same year. What a time to be alive. If you, like me, find yourself yearning for more quantitative comparisons, the numbers also show that the two variants of the new proposed technique indeed outpace the competition. And it can even do one more thing. Previous video stabilization techniques were good at taking a shaky input video and creating a smoother output. However, these results often came at the cost of a great deal of cropping. Not this new work. Look at how good it is at stabilization and it does not have to crop all this data. Praise the papers. This episode has been supported by Lambda GPU Cloud. If you're looking for inexpensive Cloud GPUs for AI, check out Lambda GPU Cloud. They've recently launched Quadro RTX 6000, RTX 8000 and V100 instances and hold onto your papers because Lambda GPU Cloud can cost less than half of AWS and Asia. Plus, they are the only Cloud service with 48GB RTX 8000. Join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.com slash papers to sign up for one of their amazing GPU instances today. Thanks to Lambda for their long-standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.