 Dear Fellow Scholars, this is Two Minute Papers with Dr. Carter-Jorna Ifehid. I got so excited by the amazing results of this paper. I will try my best to explain why, and by the end of this video, there will be a comparison that blew me away, and I hope you will appreciate it too. With the rise of neural network-based learning algorithms, we are living the advent of image generation techniques. What you see here is a set of breathtaking results created with a technique called StyleGand2. This can generate images of humans, cars, cats, and more. As you see, the progress in machine learning-based image generation is just stunning. And don't worry for a second about the progress in text processing, because that is also similarly amazing these days. A few months ago, OpenAI published their GPT-3 model that they unleashed to read the internet and learned not just our language, but much, much more. For instance, the internet also contains a lot of computer code, so it learned to generate website layouts from a written description. But that's not all, not even close, to the joy of technical PhD students around the world. It can properly type-set mathematical equations from a plain English description as well. And get this, it can also translate a complex legal text into plain language, or the other way around. And it does many of these things nearly as well as humans. So, what was the key to this work? One of the keys of GPT-3 was that it uses a neural network architecture that is called the Transformer Network. This really took the world by storm in the last few years, so our first question is, why Transformers? One, Transformer Networks can typically learn on stupendously large data sets, like the whole internet, and extract a lot of information from it. That is a very good thing. And two, Transformers are attention-based neural networks, which means that they are good at learning and generating long sequences of data. Ok, but how do we benefit from this? Well, when we ask OpenEAS GPT-3 to continue our sentences, it is able to look back at what we have written previously. And it looks at not just a couple of characters. No, no, it looks at up to several pages of writing backwards to make sure that it continues what we write the best way it can. That sounds amazing. But what is the lesson here? Just use Transformers for everything, and off we go? Well, not quite. They are indeed good at a lot of things when it comes to text processing tasks, but they don't excel at generating high resolution images at all. Can this be improved somehow? Well, this is what this new technique does, and much, much more. So, let's dive in and see what it can do. First, we can give it an incomplete image and ask it to finish it. Not bad, but OpenEAS image GPT could do that too, so what else can it do? Oh boy, a lot more. And by the way, we will compare the results of this technique against image GPT at the end of this video. Make sure not to miss that. I almost fell off the chair, you will see in a moment why. Two, it can do one of my favorites, depth to image generation. We give it a depth map, which is very easy to produce, and it creates a photorealistic image that corresponds to it, which is very hard. We do the easy part, the AI does the hard part. Great. And with this, we not only get a selection of these images, but since we have their depth maps, we can also rotate them around as if they were 3D objects. Nice. Three, we can also give it a map of labels, which is, again, very easy to do. We just say, here goes the sea, put some mountains there, and the sky here, and it will create a beautiful landscape image that corresponds to that. I can't wait to see what amazing artists all over the world will be able to get out of these techniques, and these results are already breathtaking, but research is a process and just imagine how good they will become two more papers down the line. My goodness. Four, it can also perform super resolution. This is the CSI thing where in goes a blurry image and out comes a finer, more detailed version of it. Which craft? And finally, five, we can give it a pose and it generates humans that take these poses. Now, the important thing here is that it can supercharge transformer networks to do these things at the same time with just one technique. So, how does it compare to OpenAI's image completion technique? Well, remember that technique was beyond amazing and set a really high bar. So, let's have a look together. They were both given the upper half of this image and had to fill in the lower half. Remember, as we just learned, transformers are not great at high resolution image synthesis. So, here for OpenAI's image GPT, we expect heavily pixelated images, and, oh yes, that's right. So, now, hold on to your papers and let's see how much more detailed the new technique is. Holy matter of papers. Do you see what I see here? Image GPT came out just a few months ago and there is already this kind of progress. So, there we go. Just imagine what we will be able to do with these supercharge transformers just two more papers down the line. Wow! And that's where I almost fell off the chair when reading this paper. Hope you held on to yours. It truly feels like we are living in a science fiction world. What a time to be alive. What you see here is a report of this exact paper we have talked about which was made by weights and biases. I put a link to it in the description. Make sure to have a look. I think it helps you understand this paper better. Weight and biases provides tools to track your experiments in your deep learning projects. Their system is designed to save you a ton of time and money, and it is actively used in projects at prestigious labs such as OpenAI, Toyota Research, GitHub, and more. And the best part is that weights and biases is free for all individuals, academics, and open source projects. It really is as good as it gets. Make sure to visit them through wnb.com slash papers or just click the link in the video description and you can get the free demo today. Our thanks to weights and biases for their longstanding support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.