 This episode has been supported by Lambda Labs. Dear Fellow Scholars, this is two-minute papers with Karo Jorna Ifahir. In our earlier paper, Gaussian material synthesis, we made a neural renderer, and what this neural renderer was able to do is reproduce the results of a light transport simulation within 4 to 6 milliseconds in a way that is almost pixel-perfect. It took a fixed camera and scene, and we were able to come up with a ton of different materials, and it was always able to guess what the output would look like if we changed the physical properties of a material. This is a perfect setup for material synthesis, where these restrictions are not too limiting. Trying to perform high-quality neural rendering has been a really important research problem lately, and everyone is asking the question, can we do more with this? Can we move around with the camera and have a neural network predict what the scene would look like? Can we do this with animations? Well, have a look at this new paper, which is a collaboration between researchers at the Technical University of Munich and Stanford University, where all we need is some video footage of a person or object. It takes a close look at this kind of information and can offer three killer applications. One, it can synthesize the object from new viewpoints. Two, it can create a video of this scene and imagine what it would look like if we reorganized it, or can even add more objects to it. And three, perhaps everyone's favorite, performing facial reenactment from a source to a target actor. As with many other methods, these neural textures are stored on top of the 3D objects, however, a more detailed, high-dimensional description is also stored and learned by this algorithm, which enables it to have a deeper understanding of intricate light-transport effects to create these new views. For instance, it is particularly good at reproducing specular highlights, which typically change rapidly as we change our viewpoint for the object. One of the main challenges was building a learning algorithm that can deal with this kind of complexity. The synthesis of mouth movements was always the Achilles' heel of these methods, so have a look at how well this one does with it. You can also see with the comparisons here that in general, this new technique smokes the competition. So how much training data do we need to achieve this? I would imagine that this would take hours and hours of video footage, right? No, not at all. This is what the results look like as a function of the amount of training data. On the left, you see that it already kind of works with 125 images, but contains artifacts, but if we can supply 1000 images, we're good. Note that 1000 images sounds like a lot, but it really isn't, it's just half a minute worth of video. How crazy is that? Some limitations still apply. You see one failure case here, and the neural network typically needs to be retrained if we wish to use it on new objects, but this work finally generalizes to multiple viewpoints, animation, scene editing, lots of different materials and geometries, and I can only imagine what we'll get two more papers down the line. Respect to used tools for accomplishing this, and in general, make sure to have a look at Matthias Neesner's lab who just got tenured as a full professor and he's only 32 years old. Congratulations. If you have AI-related ideas and you would like to try them, but not do it in the cloud because you wish to own your own hardware, look no further than Lambda Labs. Lambda Labs offers sleek, beautifully designed laptops, workstations and servers that come pre-installed with every major learning framework and updates them for you, taking care of all the dependencies. Look at those beautiful and powerful machines. This way, you can spend more if your time with your ideas and don't have to deal with all the software maintenance work. Make sure to go to lambdalabs.com, slash papers, or click their link in the video description and look around, and if you have any questions, you can even call them for advice. Big thanks to Lambdalabs for supporting this video and helping us make better videos for you. Thanks for watching and for your generous support, and I'll see you next time.