 Friendly greetings to everyone, my name is Karo Jornai and I promise to you that the pronunciation of my name is going to be the most complicated thing in this unofficial talk. This piece of work is a collaboration between Activision Blizzard, the University of Saragusa and the Technical University of Vienna. The quest here was to render images with really high quality subsurface scattering in real time on commodity hardware. To render photorealistic images, we populated scene with objects. Add the camera and the light source and start tracing rays from the camera to determine the incoming gradients. Now even though there is a large volume of research going on on how to do this efficiently, this is still a really time consuming process. What's more, this figure here shows light transport only between surfaces, meaning that we suppose that rays of light propagate only in vacuum. If we extend our program to support participating media, we can render volumetric effects like smoke, haze and many others and also translucent materials such as skin, plant leaves, marble, wax and so on. However, this extension bumps up the dimensionality of the integral we need to solve, making the process even more time consuming. But the reward for this is immense. Here on the left you can see how our skin would look like without subsurface scattering. It is indeed a very important factor in the visual appearance of many translucent materials. It is not a surprise that the motion picture and the gaming industries are yearning for a real time solution for this. There are fortunately simplified models to render subsurface light transport in optical lithic materials. What we do here is take an infinite hog space of a chosen translucent material, light an infinite azimoli thin pencil beam from above in normal incidence. This beam will penetrate the surface of the material and will start to attenuate as it becomes more and more submerged into the medium. During this process, these photons undergo many scattering events and eventually exit somewhere away from the origin. Counting up these photons exiting at different distances, we can build a histogram that we call diffusion profile and we'll denote as RD. This is an actual simulated diffusion profile, what it looks like if we look from above. Another important bit of preliminary knowledge is that we can directly use these diffusion profiles by convolving them with an input irradiance map to add subsurface scattering to it as a post-processing step. This is how the result looks like after the convolution is applied. Now this is remarkable as we don't have to run a fully ray-traced simulation with participating media. However, these signals are stored as images, so normally this means that we compute a 2D convolution between them. Unfortunately this is very costly, but there are techniques to reduce this problem to several much cheaper 1D convolutions. One example is Dion's excellent technique. He takes into consideration that in a homogenous and isotropic medium, the diffusion profiles are radially symmetric, therefore it is possible to take a 1D slice of this profile as shown below here and trying to fit it with a sum of Gaussians which are individually also radially symmetric. This means that we can use a cheaper set of 1D convolutions instead of using the 2D profile directly. This is an example input signal and the results with Dion's technique with different number of Gaussians compared to the true diffusion kernel. It is important to point out that even the mathematical properties of Gaussians, this technique requires one horizontal and one vertical convolution per Gaussian. These are 1D convolutions. This also means that if we'd like to obtain high quality subsurface scattering, we need at least 4 Gaussians and therefore 8 convolutions. This is not suitable for most real-time applications. However, it is a really smart idea and hasn't been really improved since 2007. And honestly, when we started this project, we didn't think anyone could realistically come up with something that's better than this. Our quest was nonetheless to obtain high fidelity results with a separable kernel using only 2 convolutions which is marked with green up there. Visualizing the SVD of the diffusion profile, it is clear that the signal is non-separable. It is not possible to write this 2D function as a mere product of 1D functions. However, the sami-lock plot tells us that the higher ranked singular values decay rapidly, meaning that most of the information here is not random. It has a lot of structure, therefore a rank 1 approximation sounds like a good starting point. The plan was to treat the diffusion profile here on the right as a matrix for which we compute the SVD. Here you can see the one singular value that we're taking and the corresponding left and right singular vectors that we hear denote by 8. We then compute one horizontal and one vertical convolution using these singular vectors to reconstruct the diffusion kernel and obtain the output. This is the input and the rank 1 SVD reconstruction. This would be the ground truth and now we can see that the separable SVD approximation is indeed looking very grim. There is a world of a difference between the two. So wow, this is surprising, especially considering the fact that the Eckhart Young theorem teaches us that the SVD is the best reconstruction in terms of the Frobenius norm which corresponds here to the RMS error. This is the absolute best reconstruction we can obtain with respect to the RMS error. Very disappointing. Here is the own algorithm, the one with the one D slice and fitting with one Gaussian. This means the same amount of convolutions and hence the same execution time as the rank 1 SVD and the ground truth. This is how the SVD looks like on the real world scene compared to using the true kernel. Looking at it again in our disappointment, we notice that the SVD yields an overall darker image therefore the reconstruction is not energy conserving. A new idea came up. Maybe we would try this before putting the project on ice and calling it a day. What if we would solve a minimization problem where the reconstructed kernel would still be as close as possible to the diffusion profile but would also be energy conservant. This should definitely be a viable option. And the results are dreadful. Herendus, I just don't know what to say. Look at the notes. Somehow as if the inputty radiance signal showed up as a really nasty ringing artifact. And we had the same around the ear. We visualized the actual kernel on a disc of light to see what went wrong here. And yes we see that it is indeed dreadful, nothing like the simulated diffusion kernel. But we hadn't the slightest idea why this happened. Visualizing the kernel itself in a simple 1D plot and staring at it for a while, it looks like a great separable approximation with respect to the RMS error. Most of the energy of the signal is close to the origin and the optimizer tries to reconstruct these details as closely as possible. Please note that the kernel plots are deceiving. These signals indeed have the same amount of energy, but the tail of the fit is extending really far away and this makes up for the seemingly less energy of the signal. This very delicate thing took a week of my life and I kind of wanted back. So what if we would minimize not only the RMS error by itself, which forces the optimizer to concentrate the energy to the origin of the signal where the energy spike is. But we would add a guide function which behaves like an envelope that tells the optimizer to possibly reconstruct regions not close to the origin but focus a bit more on far end scattering. This is the fit we originally had. And this is a very simple distance weighted guide function I had in mind. Imagine that we now have a more general model for which we used k equals 0, a constant envelope to obtain these horrendous results. I will now tell the optimizer to use k equals 1, which means that we would give higher weight to the regions further away from the origin. This is what we obtained. Very intuitive, we have a signal with the same amount of energy as if we pushed it from the top to neglect the origin and add this energy to the tail of the signal to focus on the reconstruction of far end scattering. And now we can even go k equals 2, which is essentially squishing the signal a bit more to emphasize far end scattering at the price of neglecting sharp close range details. Back to the original fit. That's weighted by distance a bit by plugging k equals 1 in the optimizer. Almost there. Okay, let's go k equals 2, a bit more emphasis on far end scattering. Now this looks remarkably close to the ground truth. This is the journey behind the guided optimization technique that is separable, requires only two convolutions and is one of the techniques we propose for applications with strict real-time constraints. We also propose an other technique which we have mathematically derived and for which I admit not having an intuitive story. So before I dare showing you the next slide, take a big breath and let's go. So sorry this is how it looks like and what is remarkable about it that this follows a completely different paradigm. What we're aiming for here is not to make our kernel close to the original diffusion kernel, but trying to make the result of the convolution the same. It is almost like minimizing the L2 distance of the resulting convolved images, not the kernels themselves. Stealing for images, not kernels. This is impossible to solve for a general case, so in order to accomplish this, one needs to confine the solution to a class of input-eradient signals, input images where it would work well. In our derivation, we plugged in one of these signals as inputs. This means that the technique should behave analytically on signals like this. And the most remarkable thing that this is a mere rank-1 separable approximation that is really analytic for these class of signals. This means that it mimics the effect of the true kernel perfectly. Let's take a look at this on a practical case, of course, not all signals are one these signals. This is our result with the analytic preintegrated kernel and the ground truth. Very close to being indistinguishable. Furthermore, it is really simple to implement and it has a closed form solution that does not require any kind of optimization procedure. One more interesting detail for the more curious minds, this technique is analytic for a greater class than only one of these signals, a class that we call additively separable signals. So what about artistic editing? The preintegrated technique is great, but it does not offer any kind of artistic control over the output. The guided approximation requires optimization, but in return, it also offers some degree of artistic freedom on how the desired output should look like. We also have this technique, which is simply using two separable Gaussians of different variance values, one each to provide perfect artistic freedom in adjusting the magnitudes of close and faring scattering. Note that these two Gaussians are not the same as in Dian's approach with the two Gaussians, as we do not use the radially symmetric one this signal directly. A real-world example, this is the input irradiance, heavily exaggerated faring scattering, heavily exaggerated close range scattering, and a more conservative, really good looking mixture of the two. Rapping it up, the SVD is great for applications that can afford higher rank reconstructions, the kernel preintegration is a simple technique that is analytic for additively separable signals, guided optimization, a more general version of the preintegration that can be conveniently tuned with one parameter, manual approximation, many degrees of freedom for artists, while it has a quite reasonable accuracy that is comparable to four Gaussians with previous techniques. Different techniques with different levels of scientific rigor and different target audiences ranging from scientists to artists working in the industry. Now even though we used examples with skin to demonstrate our techniques, it is important to point out that they work for a variety of translucent media such as plants, marble, roads, in the steel life, and milk. The most important take home message from this project, at least for me, is that it's entirely possible to do academic research together with companies and create results that can make it to multimillion dollar computer games, but also having proven results that are useful for the scientific community. Thank you.