 Dear Fellow Scholars, this is two-minute papers with Dr. Karo Zsolnai-Fehir. About two years ago, we worked on a neuro-rendering system which would perform light transport on this scene and guess how it would change if we would change the material properties of this test object. It was able to closely match the output of a real light simulation program and it was near instantaneous as it took less than 5 milliseconds instead of the 40 to 60 seconds the light transport algorithm usually requires. This technique went by the name Gaussian material synthesis and the learned quantities were material properties. But this new paper sets out to learn something more difficult and also more general. We are talking about a 5D Neural Radiance Field Representation. So what does this mean exactly? What this means is that we have the three dimensions for location and two for view direction or in short the input is where we are in space and what we are looking at and the resulting image of this view. So here we take a bunch of this input data, learn it and synthesize new previously unseen views of not just the materials in the scene but the entire scene itself. And here we are talking not only digital environments but also real scenes as well. Now that's quite a value proposition so let's see if it can live up to this promise. Wow! So good! Love it! But what is it really that we should be looking at? What makes a good output here? The most challenging part is writing an algorithm that is able to reproduce delicate high frequency details while having temporal coherence. So what does that mean? Well, in simpler words we are looking for sharp and smooth image sequences. Perfectly matte objects are easier to learn here because they look the same from all directions while glossier, more reflective materials are significantly more difficult because they change a great deal as we move our head around and this highly variant information is typically not present in the learned input images. If you read the paper you'll see these referred to as non-lumbarian materials. The paper and the video contains a ton of examples of these view dependent effects to demonstrate that these difficult scenes are handled really well by this technique. Refractions also look great. Now if we define difficulty as things that change a lot when we change our position or view direction a little, not only the non-lumbarian materials are going to give us headaches, occlusions can also be challenging as well. For instance you can see here how well it handles the complex occlusion situation between the ribs of the skeleton here. It also has an understanding of depth and this depth information is so accurate that we can do these nice augmented reality applications where we put a new virtual object in the scene and it correctly determines whether it is in front of or behind the real objects in the scene. End of what these new iPads do with their lidar sensors but without the sensor. As you see this technique smokes the competition. So what do you know? Entire real world scenes can be reproduced from only a few views by using neural networks and the results are just out of this world. Absolutely amazing. What you see here is an instrumentation of this exact paper we have talked about which was made by weights and biases. I think organizing these experiments really showcases the usability of their system. Also weights and biases provides tools to track your experiments in your deep learning projects. Their system is designed to save you a ton of time and money and it is actively used in projects at prestigious labs such as OpenAI, Toyota Research, GitHub and more. And the best part is that if you are an academic or have an open source project you can use their tools for free. What really is as good as it gets. Make sure to visit them through wnbe.com slash papers or just click the link in the video description and you can get a free demo today. Our thanks to weights and biases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.