 Dear Fellow Scholars, this is two-minute papers with Dr. Karoizhou-nai-Fehir. When watching science fiction movies, we often encounter crazy devices and technologies that don't really exist or sometimes, once, that are not even possible to make. For instance, reconstructing sound from vibrations would be an excellent example of that and could make a great novel with the Secret Service trying to catch dangerous criminals. Except that it has already been done in real-life research. I think you can imagine how surprised I was when I first saw this paper in 2014 that showcased a result where a camera looks at this bag of chips and from these tiny, tiny vibrations, it could reconstruct the sounds in the room. Let's listen. Mary had a little lamb whose fleece was white as snow and everywhere that Mary went, that lamb was stored to go. I was wrong in looking at these beautiful photos of her and everywhere that Mary was, I was stored to go. Yes, this indeed sounds like science fiction. But 2014 was a long, long time ago and since then we have a selection of powerful learning algorithms and the question is, was the next idea that sounded completely impossible a few years ago, which is now possible? Well, what about looking at silent footage from a speaker and trying to guess what they were saying? Checkmark that sounds absolutely impossible to me, yet this new technique is able to produce the entire tube of this speech after looking at a video footage of the leap movements. Let's listen. Between the wave light, the frequency and the speed of electromagnetic radiation. In fact, the product of the wavelength at the frequency is its speed. Wow. So, the first question is, of course, what was used as the training data? It used a dataset with lecture videos and chess commentary from five speakers and make no mistake, it takes a ton of data from these speakers, about 20 hours from each, but it uses video that was shot in a natural setting, which is something that we have in abundance on YouTube and other places on the internet. Note that the neural network works on the same speakers it was trained on and was able to learn their gestures and leap movements remarkably well. However, this is not the first work attempting to do this, so let's see how it compares to the competition. Set white with i7 soon. Set white with i7 soon. The new one is very close to the true spoken sentence. Let's look at another one. Erfendowance of action-famed decision. Eight field guns were captured in position. Note that there are gestures, a reasonable amount of head movement and other factors that play and the algorithm still does amazingly well. Potential applications of this could be video conferencing in zones where we have to be silent, giving a voice to people with the inability to speak, do to a phonia or other conditions, or potentially fixing a piece of video footage where parts of the speech signal are corrupted. In these cases, the gaps could be filled with such a technique. Look. Let's look at a cell potential of 0.5 volts for an oxidation of bromide by permanganate. The question I have is what pH would cause this voltage? Would it be a pH? Now, let's have a look under the hood. If we visualize the activations within this neural network, we see that it found out that it mainly looks at the mouth of the speaker. That is, of course, not surprising. However, what is surprising is that the other regions, for instance, around the forehead and eyebrows, are also important to the attention mechanism. Perhaps this could mean that it also looks at the gestures of the speaker and uses that information for the speech synthesis. I find this aspect of the work very intriguing and would love to see some additional analysis on that. There is so much more in the paper, for instance, I mentioned giving a voice to people with a phonia which should not be possible because we are training these neural networks for a specific speaker, but with an additional speaker embedding step, it is possible to pair up any speaker with any voice. This is another amazing work that makes me feel like we are living in a science fiction world. I can only imagine what we will be able to do with this technique two more papers down the line. If you have any ideas, feel free to speculate in the comments section below. What a time to be alive! This episode has been supported by Snap Inc. What you see here is Snap ML, a framework that helps you bring your own machine learning models to Snapchat's A or Lenses. You can build augmented reality experiences for Snap Chets hundreds of millions of users and help them see the world through a different lens. You can also apply to Snap's AR Creator Residency program with a proposal of how you would use Lens Studio for a creative project. If selected, you could receive a grant between 1 to $5,000 and work with Snap's technical and creative teams to bring your ideas to life. It doesn't get any better than that. Make sure to go to the link in the video description and apply for their residency program and try Snap ML today. Our thanks to Snap Inc for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.