 This episode has been supported by Lambda. Dear Fellow Scholars, this is Two Minute Papers with Karo Zsolnai-Fehir. In this series, we talk about research on all kinds of physics simulations, including fluids, collision physics, and we have even ventured into hair simulations. We mostly talk about how the individual hair strands should move and how they should look in terms of color and reflectance. Having these beautiful videos takes getting many, many moving parts right, for instance, before all of that, the very first step is not any of those steps. First, we have to get the 3D geometry of these hair styles into our simulation system. In a previous episode, we have seen an excellent work that does this well for human hair. But what if we would like to model not human hair, but something completely different? Well, hold on to your papers, because this new work is so general that it can look at an input image or video and give us not only a model of the human hair, but human skin, garments, and of course, my favorite, smoke plumes and more. But if you look here, this part begs the following question. The input is an image and the output also looks like an image, and we need to make them similar. So, what's the big deal here? A copying machine can do that. No? Well, not really. Here's why. To create the output, we are working with something that indeed looks like an image, but it is not an image. It is a 3-dimensional cube in which we have to specify color and opacity values everywhere. After that, we simulate rays of light passing through this volume, which is a technique that we call ray marching, and this process has to produce the same 2D image through ray marching as what was given as an input. That's much, much harder than building a copying machine. As you see here, normally, this does not work well at all, because, for instance, a standard algorithm sees lights in the background and assumes that these are really bright and dense points. That is kind of true, but they are usually not even part of the data that we would like to reconstruct. To solve this issue, the authors propose learning to tell the foreground and background images apart, so they can be separated before we start the reconstruction of the human. And this is a good research paper, which means that if it contains multiple new techniques, each of them are tested separately to know how much they contribute to the final results. We get the previously seen dreadful results without the background separation step. Here are the results with the learned backgrounds. We can still see the lights due to the way the final image is constructed, and the fact that we have so little of this halo effect is really cool. Here, you see the results with the true background data where the background learning step is not present. Note that this is cheating, because this data is not available for all cameras and backgrounds, however, it is a great way to test the quality of this learning step. The comparison of the learned method against this reveals that the two are very close, which is exactly what we are looking for. And finally, the input footage is also shown for reference. This is ultimately what we are trying to achieve, and as you see, the output is quite close to it. The final algorithm excels at reconstructing volume data for toys, small plumes, and humans alike. And the coolest part is that it works for not only stationary inputs, but for animations as well. Wait, actually, there is something that is perhaps even cooler with the magic of neural networks and latent spaces we can even animate this data. Here you see an example of that where an avatar is animated in real time by moving around this magenta dot. A limiting factor here is the resolution of this reconstruction. If you look closely, you can see that some fine details are missing, but you know the saying, given the rate of progress in machine learning research, two more papers down the line, and this will likely be orders of magnitude better. And if you feel that you always need to take your daily dose of papers, my statistics show that many of you are subscribed, but didn't use the bell icon. If you click this bell icon, you will never miss a future episode and can properly engage in your paper addiction. This episode has been supported by Lambda. If you are a researcher or a startup looking for cheap GPU compute to run these algorithms, check out Lambda GPU Cloud. I've talked about Lambda's GPU workstations in other videos and I'm happy to tell you that they are offering GPU cloud services as well. The Lambda GPU Cloud can train ImageNet to 93% accuracy for less than $19. Lambda's web-based IDE lets you easily access your instance right in your browser. And finally, hold onto your papers because the Lambda GPU Cloud costs less than half of AWS and Azure. Make sure to go to LambdaLabs.com slash papers or click the link in the video description and sign up for one of their amazing GPU instances today. Our thanks to Lambda for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.