 Dear Fellow Scholars, this is two-minute papers with Karo Jolene Fahir. In this series, we often discuss a class of techniques by the name Image Impainting. Image impainting methods are capable of filling in missing details from a mostly intact image. You see the legendary patch match algorithm at work here, which is more than 10 years old and it is a good old computer graphics method with no machine learning insight and after so much time, 10 years is an eternity in research years, it still punches way above its weight. However, with the ascendancy of neural network based learning methods, I am often wondering whether it would be possible to take a more difficult problem, for instance, impainting not just images, but movies as well. For instance, let's take an old, old black and white movie that suffers from missing data, flickering, blurriness and interestingly, even the contrast of the footage has changed as it faded over time. Well, hold onto your papers because this learning based approach fixes all of these and even more. Step number one is restoration, which takes care of all of these artifacts and contrast issues. You can not only see how much better the restored version is, but it is also reported what the technique did exactly. However, it does more. What more could be possibly asked for? Well, colorization. What it does is that it looks at only six colorized reference images that we have to provide and uses this as our direction and propagated to the remainder of the frames and it does an absolutely amazing work at that. It even tells us which reference image it is looking at when colorizing some of these frames, so if something does not come out favorably, we know which image to recolor. The architecture of the neural network that is used for all this also has to follow the requirements appropriately. For instance, beyond the standard spatial convolution layers, it also makes ample use of temporal convolution layers, which helps smearing out the colorization information from one reference image to multiple frames. However, in research, a technique is rarely the very first at doing something and sure enough. This is not the first technique that does this kind of restoration and colorization. So, how does it compare to previously published methods? Well, quite favorably. With previous methods, in some cases, the colorization just appears and disappears over time while it is much more stable here. Also, fewer artifacts make it to the final footage and since cleaning these up is one of the main objectives of these methods, that's also great news. If we look at some quantitative results or in other words numbers that describe the difference, you can see here that we get a 3-4 decibels cleaner image, which is outstanding. Note that the decibel scale is not linear, but a logarithmic scale, therefore if you read 28 instead of 24, it does not mean that it is just approximately 15% better. It is a much, much more pronounced difference than that. I think these results are approaching a state where they are becoming close to good enough so that we can revive some of these old masterpiece movies and give them a much deserved facelift. What a time to be alive! This episode has been supported by weights and biases. Wates and biases provides tools to track your experiments in your deep learning projects. It can save you a ton of time and money in these projects and is being used by OpenAI to your research, Stanford and Berkeley. They also wrote a guide on the fundamentals of neural networks where they explain in simple terms how to train a neural network properly, what are the most common errors you can make, and how to fix them. It is really great you got to have a look. So make sure to visit them through wendeeb.com slash papers or just click the link in the video description and you can get a free demo today. Our thanks to weights and biases for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.