 Dear Fellow Scholars, this is 2 Minute Papers with Dr. Karo Jolna Ifehir. Finally, this is my Happy Episode. Well, of course, I am happy in every episode, but this is going to be my Happy Happy Episode, if you will. Why is that? Well, buckle up, because today we are going to use OpenAI's DOLI2 a text to image AI, and we will see what it is made of. Can it create beautiful light transport effects or not? We will see through four beautiful experiments. For instance, this may sound like science fiction, but today we will also see if it can recreate this scene from a true master of digital 3D art. So, what is this light thing I keep talking about? A light transport simulation means a computer program that is able to compute the path of light rays to create beautiful images like this, and this, and this. And our key problem is that initially we only get noisy images, and it can take a long time for the simulator to eliminate this noise. So, can DOLI2 help with that? Well, how? For instance, it can perform a variant generation where in goes one image and the AI synthesizes other similar images. This is really cool, as it means that the AI has a good understanding of what it sees and can create different variations of it. And, wait a minute, are you thinking what I am thinking? Oh, yes, experiment number one. De-noising. Let's give this noisy input image from a light transport simulator, give it to the variant generator, and see if it is able to recreate the essence of the image, but without the noise. Let's see. Well, that is super interesting. It did not denoise the image, but it did something else. It tried to understand what the noise is in this context and found it to be some sort of gold powder. How cool is that? Based on the insights gained here, let's try again with a little less noise. Oh, yes, this difficult scene would normally take even up to days to compute correctly. Do you see these light tricks here? We would need to clean those up. So, variant generation, it's your turn again. And, look at that. Wow, we get a noise free image that captured the essence of our input. I cannot believe it. So good. Interestingly, it did not ignore the light tricks, but it thought that this is the texture of the object and synthesize the new ones accordingly. This actually means that Dolly too does what it is supposed to be doing, faithfully reproducing the scene and putting a different spin on it. So cool. And I think this concept could be supercharged by generating such a noisy input quickly, then denoising it with one of those handcrafted techniques for these images. These are typically not perfect, but they may be just good enough to kickstart the variant generator. I would love to see some more detailed experiments in this direction. Now, what else can this do? Well, experiment number two, my favourite, caustics. Oh, yes, these are beautiful patterns of reflected light that we see a lot of in real life and they produce some of the most beautiful images, any light transport simulation can offer. Yes, that's right. With such a simulation, we can compute these two. How cool is that? So now, let's ask Dolly too to create some of these for us. And the results are truly sublime. So regular caustics checkmark. And what about those fun, hard-shaped caustics when we put a ring in the middle of an open book? My goodness, the AI understands that and it really works. Loving it. However, if you look at those beautiful volumetric caustics, when running variant generation on that, it only kind of works. There are some rays of hope here, but otherwise, I feel that the AI thinks that this is some sort of laser experiment instead. And also, don't forget about Daniel Baskin's amazing results who created these drinks. But wait, we are light transport researchers here, so we don't look at the drink. What do we look at? Yes, of course, the caustics. Beautiful. And if we are looking at beautiful things, time for experiment number three, subsurface scattering. What is that? Oh boy, subsurface scattering is the beautiful effect of light penetrating our skin, milk, and other materials, and bouncing inside before coming out again. The lack of this effect is why the skin looks a little plasticky in older video games. However, light transport simulation researchers took care of that too. This is from our earlier paper with the Activision Blizzard Game Development Company. This is the same phenomenon, a simulation without subsurface scattering. And this one is with simulating this effect. And in real time. Beautiful. You can find the link to this paper in the video description. So, can an AI pull this off today? That's impossible, right? Well, it seems so. If I plainly ask for subsurface scattering from Dolly2, I did not get any of that. However, when prompting a text to image AI, we have to know not only what we wish to see, but how to get it out of the algorithm. So, if we ask for translucent objects with strong backlighting, bingo, Dolly2 can do this too. So good. Loving it. And now, hold onto your papers, because now is the time for our final experiment. Experiment number four, reproducing the work of a true master. If the previous experiment was nearly impossible, I really don't know what this is. Here is a beautiful little virtual world from Reynante Martinez, and it really speaks for itself. Now, let's put it into the variant generator, and see what Dolly2 is made of. Wow! Look at that. These are incredibly good. Not as good as the master himself, but I think the first law of papers should be invoked here. Wait, what is that? The first law of papers says that research is a process. Don't not look at where we are, look at where we will be two more papers down the line. And two more papers down the line, I have to say. I can imagine that we will get comparable images. I also love how it thinks that fingerprints are part of the liquid. It is a bit of a limitation, but a really beautiful one. What a time to be alive! And we haven't even talked about indirect illumination, dispersion, and many other amazing light transport effects. I really hope we will see some more experiments perhaps from you fellow scholars in this direction too. By the way, I have a master level light transport simulation course for all of you, free of charge, no strings attached, and we write a beautiful little simulator that can create this image and more. The link is in the video description. This episode has been supported by Ranway, professional and magical AI video editing for everyone. I often hear you fellow scholars asking, okay, these AI techniques look great, but when do I get to use them? And the answer is, right now, Ranway is an amazing video editor that can do many of the things that you see here in this series. For instance, it can automatically replace the background behind the person. It can do in-painting for videos amazingly well, and can do even text to image, image to image, you name it. No wonder it is used by editors, post-production teams, and creators at companies like CBS, Google, Vox, and many other. Make sure to go to RanwayML.com, slash papers, sign up, and try it for free today. And here comes the best part, use the code two minute at checkout, and get 10% off your first month. Thanks for watching and for your generous support, and I'll see you next time.