 Dear Fellow Scholars, this is Two Minute Papers with Dr. Karo Zonai-Fehir. Today we are going to have a look at this insane light transport simulation paper and get our minds blown in three chapters. Chapter 1. Radiosity. Radiosity is an old, old light transport algorithm that can simulate the flow of light within a scene. And it can generate a scene of this quality. Well, not the best clearly, but this technique is from decades ago. Back in the day, this was a simple and intuitive attempt at creating a light simulation program and quite frankly, this was the best we could do. It goes something like this. Slice up the scene into tiny surfaces and compute the light transport between these tiny surfaces. This worked reasonably well for diffuse light transport, matte objects if you will. But it was not great at rendering shiny objects. And it gets even worse. Look, you see these blocky artifacts here? These come from the fact that the scene has been subdivided into these surfaces and the surfaces are not fine enough for these boundaries to disappear. But, yes, I hear you asking, Karoy, why talk about this ancient technique? I'll tell you in a moment, you'll see, I promise. So, yes, radiosity is old. Some professors still teach it to their students. It makes an interesting history lesson, but I haven't seen any use of this technique in the industry for decades now. If radiosity would be a vehicle, it would be a horse carriage in the age of high-tech Tesla cars. So, I know what you're thinking. Yes, let's have a look at those Teslas. Chapter 2. Neural rendering. Many modern light simulation programs can now simulate proper light transport which shiny objects, none of these blocky artifacts, they are, as you see, absolutely amazing. They can render all kinds of material models, detailed geometry, caustics, color bleeding, you name it. However, they seem to start out from a noisy image and as we compute the path of more and more light rays, this noisy image clears up over time. But, this still takes a while. How long? Well, from minutes to days. Ouch. And then, a neural rendering entered the frame. Here you see our earlier paper that replaced the whole light simulation program with a neural network that learned how to do this. And it can create these images so quickly that it easily runs not in minutes or days, but as fast as you see here. Yes, real time on a commodity graphics card. Now note that this neural render is limited to this particular scene. With this, I hope that it is easy to see that we are now so far beyond radiosity that it sounds like a distant memory of the olden times. So, once again, why talk about radiosity? Well, check this out. Chapter 3. Neural Radiosity Excuse me, what? Yes, you heard it right. This paper is about neural radiosity. This work elevates the old, old radiosity algorithm by using a similar formulation to the original technique, but also infusing it with a powerful neural network. It is using the same horse carriage, but strapping it onto a rocket, if you will. Now you have my attention. So, let's see what this can do together. Look. Yes, it can render really intense, specular highlights. Hold on to your papers and… Wow. Once again, the results look like a nearly pixel-perfect copy of the reference simulation. And we now understand the limitations of the old radiosity, so let's strike where it hurts the most. Yes, perfectly specular, mirror-like surfaces. Let's see what happens here. Well, I can hardly believe what I am seeing here. No issues whatsoever. Still close to pixel-perfect. This new paper truly elevates the good old radiosity to the next level. So good. Loving it. But wait a second. I hear you asking, yes, Karoi, this is all well and good. But if we have the reference simulation, why not just use that? Good question. Well, that's right. The reference is great, but that one takes up to several hours to compute, and the new technique can be done super quickly. Yet they look almost exactly the same. My goodness. In fact, let's look at an equal time comparison against one of the Tesla techniques past tracing. We give the two techniques the same amount of time and see what they can produce. Let's see. Now that is no contest. Look, this is Eric Vitch's legendary scene where the light only comes from the neighboring room through a door that is only slightly a jar. This is notoriously difficult for any kind of light transport algorithm. And yet, look at how good this new one is entangling it. Now, note that not even this technique is perfect. It has two caveats. One training takes place per scene. This means that we need to give the scenes to the neural network in advance. So it can learn how light bounces off of this place. And this can take from minutes to hours. But, for instance, if we have a video game with a limited set of places that you can go, we can train our neural network on all of them in advance and deploy it to the players who can then enjoy it for as long as they wish. No more training is required after that. But the technique would still need to be a little faster than it currently is. And two, we also need quite a bit of memory to perform all this. And yes, I think this is an excellent place to invoke the first law of papers which says that research is a process. Do not look at where we are, look at where we will be two more papers down the line. And two more papers down the line, who knows, maybe we get this in real time and with a much friendlier memory consumption. Now, there is one more area that I think would be an excellent direction for future work. And that is about the per scene training of the neural network. In this work, it has to get a feel of the scene before the light simulation happens. So how does the knowledge learn on one scene transfer to others? I imagine that it should be possible to create a more general version of this that does not need to look at a new scene before the simulation takes place. And in summary, I absolutely love this paper. It takes an old old algorithm, blows the dust off of it and completely reinvigorates it by infusing it with a modern learning based technique. What a time to be alive! So what do you think? What would you use this for? I'd love to hear your thoughts. Please let me know in the comments below. And when watching all these beautiful results, if you feel that this light transport thing is pretty cool and you would like to learn more about it, I had a master-level course on this topic at the Technical University of Vienna. Since I was always teaching it to a handful of motivated students, I thought that the teachings shouldn't only be available for the privileged few who can afford a college education, but the teachings should be available for everyone. Free education for everyone, that's what I want. So the course is available free of charge for everyone, no strings attached, so make sure to click the link in the video description to get started. We write a full-light simulation program from scratch there and learn about physics, the world around us, and more. If you watch it, you will see the world differently. Percepti Labs is a visual API for TensorFlow carefully designed to make machine learning as intuitive as possible. This gives you a faster way to build out models with more transparency into how your model is architected, how it performs, and how to debug it. And it even generates visualizations for all the model variables and gives you recommendations both during modeling and training and thus all this automatically. I only wish I had a tool like this when I was working on my neural networks during my PhD years. Visit perceptilabs.com, slash papers, and start using their system for free today. Thanks to perceptilabs for their support and for helping us make better videos for you. Thanks for watching and for your generous support, and I'll see you next time.