 Dear Fellow Scholars, this is two-minute papers with Károly Zsolnai-Fehér. Since the latest explosion in AI research, virtually no field of science remains untouched by neural networks. These are amazing tools that help us solve problems where the solutions are easy to identify but difficult to explain. For instance, we all know a backflip when we see one, but mathematically defining all the required forces, rotations and torque is much more challenging. Neural networks excel at these kinds of tasks provided that we can supply them a large number of training samples. If we pick inside these neural networks, we see more and more layers and more and more neurons within these layers as years go by. The final decision depends on what neurons are activated by our inputs. They are highly efficient, however, trying to understand how a decision is being made by these networks is going to be a fruitless endeavor. This is especially troublesome when the network gives us a wrong answer that we, without having access to any sort of explanation, may erroneously accept without proper consideration. This piece of work is about distillation, which means that we take a neural network and try to express its inner workings in the form of a decision tree. Decision trees take into consideration a series of variables and provide a clear roadmap towards the decision based on them. For instance, they are useful in using the age and the amount of free time of people to try to guess whether they are likely to play video games or deciding who should get a loan from the bank based on their age, occupation and income. Yeah, this sounds great. However, the main issue is that decision trees are not good substitutes for neural networks. The theory says that we have a generalization versus interpretability trade off situation, which means that trees that provide us good decisions overfit the training data and generalize poorly and the ones that are easy to interpret are inaccurate. So in order to break out of this trade off situation, a key idea of this piece of work is to ask the neural network to build a decision tree by taking an input dataset for training, trying to generate more training data that follows the same properties and feed all this to the decision tree. Here are some results for the classical problem of identifying digits in the MNIST dataset. As each decision is meant to cut the number of output options in half, it shows really well that we can very effectively perform the classification in only four decisions from a given input. And not only that, but it also shows what it is looking for. For instance, here we can see that the final decision before we conclude whether the input number is a 3 or 8, it looks for the presence of a tiny area that joins the ends of the 3 to make an 8. A different visualization of the Connect 4 game dataset reveals that the neural network quickly tries to distinguish two types of strategies. One, where the players start playing on the inner and one with the outer region of the board. It is shown that these trees perform better than traditional decision trees. Once more, they are only slightly worse than the corresponding neural networks, but can explain their decisions much more clearly and are also faster. In summary, the rate of progress in machine learning research is truly insane these days, and I am all for papers that try to provide us a greater understanding of what is happening under the hood. I am loving this idea in particular. We had an earlier episode on how to supercharge these decision trees via tree boosting. If you are interested in learning more about it, the link is available in the video description. Thanks for watching and for your generous support and I'll see you next time.