 Dear Fellow Scholars, this is 2 Minute Papers with Karo Zsolnai-Fehir. In 2017, scientists at OpenAI embarked on an AI project where they wanted to show a neural network a bunch of Amazon product reviews and wanted to teach it to be able to generate new ones or continue a review when given one. Now, so far, this sounds like a nice hobby project, definitely not something that would justify an entire video on this channel. However, during this experiment, something really unexpected happened. Now it is clear that when the neural network reads these reviews, it knows that it has to generate new ones, therefore it builds up a deep understanding of language. However, beyond that, it used surprisingly few neurons to continue these reviews and scientists were wondering why is that? Usually, the more neurons, the more powerful the AI can get, so why use so few neurons. The reason for that is that it has learned something really interesting. I'll tell you what in a moment. This neural network was trained in an unsupervised manner, therefore it was told to do what the task was but was given no further supervision, no label datasets, no additional help, nothing. Upon closer inspection, they noticed that the neural network has built up a knowledge of not only language, but also built a sentiment detector as well. This means that the AI recognized that in order to be able to continue a review, it needs to be able to efficiently detect whether the new review seems positive or not. And thus, it dedicated the neuron to this task which we were referred to as the sentiment neuron. However, it was no ordinary sentiment neuron, it was a proper state of the art sentiment detector. In this diagram, you see this neuron at work. As it reads through the review, it starts out detecting a positive outlook which you can see with green and then, uh oh, it detects that the review has taken a turn and is not happy with the movie at all. And all this was learned on a relatively small dataset. Now, if we have this sentiment neuron, we don't just have to sit around and be happy for it. Let's play with it. For instance, by overwriting this sentiment neuron in the neuron at work, we can force it to create positive or negative reviews. Here is a positive example. Quote. Just what I was looking for, nice fitted pants exactly matched seem to color contrast with other pants I own, highly recommended and also very happy. And if we overwrite the sentiment neuron to negative, we get the following. The package received was blank and has no barcode, a waste of time and money. There are some more examples here on the screen for your pleasure. This paper teaches us that we should endeavor to not just accept these AI-based solutions but look under the hood and sometimes a gold mine of knowledge can be found within. Absolutely amazing. If you have enjoyed this episode and would like to help us make better videos for you in the future, please consider supporting us on patreon.com slash 2 minute papers or just click the link in the video description. In return, we can offer you early access to these episodes or even add your name to our key supporters so you can appear in the description of every video and more. We also support cryptocurrencies like Bitcoin, Ethereum and Litecoin. The majority of these fans is used to improve the show and we use a smaller part to give back to the community and empower science conferences like the Central European Conference on Computer Graphics. This is a conference that teaches young scientists to present their work at bigger venues later and with your support it's now the second year we've been able to sponsor them which warms my heart. This is why every episode ends with you know the drill. Thanks for watching and for your generous support and I'll see you next time.