 Dear Fellow Scholars, this is two-minute papers with Karo Zsolnai-Fehir. OpenAI GPT2 is a learning-based technique which can perform common natural language processing operations, for instance, answering questions, completing text, reading comprehension, summarization, and more. What is absolutely amazing about this technique is that it is able to perform these tasks with as little supervision as possible. This means that we unleash the algorithm that they call GPT2 to read the internet, and it will learn our language by itself. To perform this, of course, we need a lot of training data, and here the AR reads 40GB of internet text, which is 40GB of non-binary plain-text data, which is a stupendously large amount of text. You see here an example of how it fares with text completion, and the results were absolutely amazing. And not only that, because hold onto your papers, we can now try it ourselves, and my results show that it can be as wise as a scholar, and it can also cheat like nobody's business. So, let's try something simple here. First, as everyone is talking about job security, I'd like to give it the start of this two-minute paper script, and see if I am about to be replaced. Let's see. It continues the intro with, the history of Eastern Europe since 1989 has been an interesting and complex story spanning four decades. On one side you have a regime of Western-backed dictators from the mid-1990s, and so on. Very scholarly indeed. However, I think I might be safe for now. And now, something that I wanted to try since the inception of GPT-2, which is, feed it a snippet of something that I said about wavelet turbulence, which is one of the best papers ever written, and have the AI finish it. I tried to frame it as an exam question, and was not disappointed by the results. So, the text is, there was a similar technique by the name wavelet turbulence, which is, I hope you see what I did there, and look what it did. There was a similar technique by the name wavelet turbulence, which is not at all what we are talking about. But I let you decide whether it is worth your time to read it. Wow! GPT-2 has been caught cheating, red-handed. I love it. Let's continue what it had to say. Anyway, my aim here is to describe how we use a fluid simulation to simulate smoke and liquid effects using just a few simple equations. Solving a few functions gives us a set of derivatives of the particle trajectories, that is, equations that describe how and where the particles will come and go across the liquid in the simulation. For simplicity, as a good teacher of high school students should know, we can use a simple equation which simulates two particles that collide. I am stunned. It recognized that we are talking about fluid simulations, which is already remarkable, but it went much further. The completion is not bad at all, and is not only coherent, on-popic, but has quite a bit of truth to it. I will have to rethink my previous claim about my job security. The even crazier thing is that the size of this model is about 750 million parameters, which is only half of the size of the original full model, which is expected to be even better. I put a link to this website in the video description for your pleasure. Make sure to play with it. This is mad fun. And GPT2 will also see so many applications that we cannot even fathom yet. For instance, here you can see that one can train it on many source code files on GitHub, and it will be able to complete the code that we write on the fly. Now, nobody should think of this as GPT2 writing programs for us. This is, of course, unlikely. However, it will ease the process for novice and expert users alike. If you have any other novel applications in mind, make sure to leave a comment below. For now, Bravo OpenAI and a big thank you for Danielle King and the HuggingFace company for this super convenient public implementation. Let the experiments begin. This episode has been supported by weights and biases. Wates and biases provides tools to track your experiments in your deep learning projects. It can save you a ton of time and money in these projects and is being used by OpenAI, Toyota Research, Stanford, and Berkeley. It is very easy to set up. In fact, this blog post shows how we can use their framework to visualize our progress using XG Boost, a popular library for machine learning models. Get ready, because this is quite possibly the shortest blog post that you have seen. Yep, that was basically it. I don't think it can get any easier. Make sure to visit them through WendeeB.com slash papers, www.wendeeB.com slash papers, or just click the link in the video description and sign up for a free demo today. Our thanks to weights and biases for helping us make better videos for you. Thanks for watching and for your generous support, and I'll see you next time.