 Dear Fellow Scholars, this is two-minute papers with Karoizhona Ifehe. I have recently been witnessing a few heated conversations regarding the submission of deep-learning papers to computer vision conferences. The forums are up in arms about the fact that despite some of these papers showcase remarkable results, they were rejected on the basis of, from what I have heard, not adding too much to the tree of knowledge. They argue that we don't understand what is going on in these neural networks and cannot really learn anything new from them. I'll try to understand and rephrase their argument differently. We know exactly how to train a neural network. It's just that, as an output of this process, we get a model of something that resembles a brain as a collection of neurons and circumstances under which these neurons are activated. These stories in a file that can take up to several gigabytes and the best solutions are often not intuitively understandable for us. For instance, in this video, we are training a neural network to classify these points correctly, but what exactly can we learn if we look into these neurons? Now imagine that in practice, we don't have a handful of these boxes, but millions of them, and more complex than the ones you see here. Let's start with a simple example that hopefully helps getting a better grip of this argument. Now I'll be damned if this video won't be more than a couple minutes, so this is going to be one of those slightly extended two-minute paper's episodes. I hope you don't mind. The grammatical rules of my native language, a lot of them, are contained in enormous tomes that everyone has to go through during their school years. Rules are important. They give the scaffoldings for constructing sentences that are grammatically correct. Can we explain or even enumerate these rules? Well unless you are a linguist, the answer is no. Almost no one really remembers more than a few rules, but every native speaker knows how their language should be spoken. And it is because we have heard a lot of sentences that are correct and learned it by heart what makes a proper sentence and what is gibberish. This is exactly what neural networks do. They are trained in a very similar way. In fact they are so effective at it that if we try to forcefully insert some of our knowledge in there, the solutions are going to get worse. It is therefore an appropriate time to ask questions like what merits a paper and what do we define a scientific progress. What if we have extremely accurate algorithms where we don't know what is going on under the hood or simpler, more intuitive algorithms that may be subpar in accuracy? If we have a top tier scientific conference where only a very limited number of papers get accepted, which ones shall we accept? I hope that this question will spark a productive discussion and hopefully scientific research venues will be more vigilant about this question in the future. Okay, so the question is crystal clear, knowledge or efficiency. How about possible solutions? Can we extract scientific insights out of these neural networks? Global compression is a way to essentially compress the information in this brainish thing, this collection of neurons we described earlier. To demonstrate why this is such a cool idea, let's quickly jump to this program by deep mind that plays Atari games at an amazingly high level. In breakout, the solution program that you see here is essentially an enormous table that describes what the program should do when it sees different inputs. It is so enormous that it has many millions of records in there. A manual of many thousand pages, if you will. It is easy to execute for a computer that completely impossible for us to understand why and how it works. However, if we intuitively think about the game itself, we could actually write a super simple program in one line of code that would almost be as good as this. All we need to do is try to follow the ball with the pedal. One line of code and pretty decent results. Not optimal, but quite decent. From such a program, we can actually learn something about the game. Essentially, what we could do with these enormous tables is compressing them into much, much smaller ones. Once there are so tiny that we can actually build an intuition from them. This way, the output of a machine learning technique wouldn't only be an extremely efficient program. But finally, the output of the procedure would be knowledge. Insight. If you think about it, such an algorithm would essentially do research by itself. At first, it would randomly try experimenting and after a large amount of observations are collected, these observations would be explained by a small number of rules. That is exactly the definition of research. And perhaps this is one of the more interesting future frontiers of machine learning research. And by the way, earlier we have talked about a fantastic paper on neural programmer interpreters that also aim to output complete algorithms that can be directly used and understood. The link is available in the description box. Thanks for watching and for your generous support, and I'll see you next time.