 Dear Fellow Scholars, this is 2 Minute Papers with Károly Zsolnai-Fehér. Today, we are going to talk about OpenAI's robot hand that dexterously manipulates and solves Rubik's cube. Here you can marvel at this majestic result. Now, why did I use the term dexterously manipulates Rubik's cube? In this project, there are two problems to solve. One, finding out what kind of rotation we need to get closer to a soft cube, and two, listing the finger positions to be able to execute these prescribed rotations. And this paper is about the letter, which means that the rotation sequences are given by a previously existing algorithm, and OpenAI's method manipulates the hand to be able to follow this algorithm. To rephrase it, the robot hand doesn't really know how to solve the cube and is told what to do, and the contribution lies in the robot figuring out how to execute these rotations. If you take only one thing from this video, let it be this thought. Now, to perform all this, we have to first solve a problem in a computer simulation, where we can learn and iterate quickly, and then transfer everything the agent learned there to the real world, and hope that it obtained general knowledge that indeed can be applied there. This task is one of my favorites. However, no simulation is as detailed as the real world, and as every experienced student knows very well, things that are written in the textbook might not always work exactly the same in practice. So the problem formulation naturally emerges. Our job is to prepare this AI in this simulation, so it becomes good enough to perform well in the real world. Well, good news. First, let's think about the fact that in a simulation, we can train much faster, as we are not bound by the physical limits of the robot hand. In a simulation, we are bound by our processing power, which is much, much more vast, and is growing every day. So this means that the simulated environments can be as grueling as we can make them be, what's even more, we can do something that OpenAI refers to as automatic domain randomization. This is one of the key contributions of this paper. The domain randomization part means that it creates a large number of random environments, each of which are a little different, and the AI is meant to learn how to account for these differences and hopefully, as a result, obtain general knowledge about our world. The automatic part is responsible for detecting how much randomization the neural network can shoulder and hence the difficulty of these random environments is increased over time. So how good are the results? Well, spectacular. In fact, hold onto your papers because it can not only dexterously manipulate and solve the cube, but we can even hamstring the hand in many different ways and it will still be able to do well. And I am telling you, scientists at OpenAI got very creative in tormenting this little hand. They added a rubber glove, tied multiple fingers together, threw a blanket on it, and pushed it around with a plastic jar of paint. It still worked. This is a testament to the usefulness of the mentioned automatic domain randomization technique. What's more, if you have a look at the paper, you will even see how well it was able to recover from a randomly breaking joint. What a time to be alive. As always, some limitations apply. The hand is only able to solve the cube about 60% of the time for simpler cases and the success rate drops to 20% for the most difficult ones. If it gets stuck, it typically does in the first few rotations. But so far, we have been able to do this 0% of the time. And given that the first steps towards cracking the problem are almost always the hardest, I have no doubt that two more papers down the line, this will become significantly more reliable. But, you know what? We are talking about OpenAI. Make it one paper. This episode has been supported by weights and biases. weights and biases provides tools to track your experiments in your deep learning projects. It can save you a ton of time and money in this project and is being used by OpenAI, Toyota Research, Stanford and Berkeley. Here you see a right up of theirs where they explain how to visualize the gradients running through your models and illustrate it through the example of predicting protein structure. They also have a live example that you can try. Make sure to visit them through WendeeB.com slash papers, W-A-D-B.com slash papers, or just click the link in the video description and you can get a free demo today. Or thanks to weights and biases for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.