 Dear Fellow Scholars, this is two-minute papers with Dr. Karo Zona Ifehir. Today, we are going to see how we can use our hands, but not our fingers to mingle with objects in virtual worlds. The promise of virtual reality VR is indeed truly incredible. If one day it comes to fruition, doctors could be trained to perform surgery in a virtual environment, expose astronauts to virtual zero-gravity simulations, work together with telepresence applications, you name it. The dream is getting closer and closer, but something is still missing. For instance, this previous work uses a learning-based algorithm to teach a head-mounted camera to tell the orientation of our hands at all times. One more paper down the line, this technique appeared that can deal with examples with challenging hand-hand interactions, deformations, lots of self-contact, and self-occlusion. This was absolutely amazing because these are not gloves. No, no. This is the reconstruction of the hand by the algorithm. Absolutely amazing. However, it is slow, and mingling with other objects is still quite limited. So, what is missing? What is left to be done here? Let's have a look at today's paper and find out together. This is its output. Yes, mingling that looks very natural. But, what is so interesting here? The interesting part is that it has realistic finger movements. Well, that means that it just reads the data from the sensors on the fingers, right? Now, hold on to your papers and we'll find out once we look at the input. Oh my, is this really true? No sensors on the fingers anywhere. What kind of black magic is this? And with that, we can now make the most important observation in the paper, and that is that it reads information from only the wrist and the objects in the hand. Look, the sensors are on the gloves, but none are on the fingers. Once again, the sensors have no idea what we are doing with our fingers. It only reads the movement of our wrist and the object, and all the finger movement is synthesized by it automatically. Whoa! And with this, we can not only have a virtual version of our hand, but we can also manipulate virtual objects with very few sensor readings. The rest is up to the AI to synthesize. This means that we can have a drink with a friend online, use a virtual hammer, too, depending on our mood, fix, or destroy virtual objects. This is very challenging because the finger movements have to follow the geometry of the object. Look, here, the same hand is holding different objects, and the AI knows how to synthesize the appropriate finger movements for both of them. This is especially apparent when we change the scale of the object. You see, the small one requires small and precise finger movements to turn around. These are motions that need to be completely resynthesized for bigger objects. So cool. And now comes the key. So does this only work on objects that it has been trained on? No, not at all. For instance, the method has not seen this kind of teapot before, and still it knows how to use its handle, and to hold it from the bottom, too, even if both of these parts look different. Be careful, though, who knows, maybe virtual teapots can get hot, too. Once more, it handles the independent movement of the left and right hands. Now, how fast is all this? Can we have coffee together in virtual reality? Yes, absolutely. All this runs in close to real time. There is a tiny bit of delay, though, but a result like this is already amazing, and this is typically the kind of thing that can be fixed one more paper down the line. However, not even this technique is perfect, it might still miss small features on an object, for instance, a very thin handle might confuse it. Or if it has an inaccurate reading of the hand pose and distances, this might happen. But for now, having a virtual coffee together, yes, please, sign me up. This episode has been supported by Lambda GPU Cloud. If you're looking for inexpensive Cloud GPUs for AI, check out Lambda GPU Cloud. They've recently launched Quadro RTX 6000, RTX 8000, and V100 instances, and hold onto your papers because Lambda GPU Cloud can cost less than half of AWS and Azure. Plus, they are the only Cloud service with 48GB RTX 8000. And researchers at organizations like Apple, MIT, and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to LambdaLabs.com slash papers to sign up for one of their amazing GPU instances today. Our thanks to Lambda for their long-standing support and for helping us make better videos for you. Thanks for watching and for your generous support, and I'll see you next time.