 This might look like a GPU, smell like a GPU, stick in a PCI Express 16X slot like a GPU, and even talk like a GPU. But I assure you, its intended purpose is very different and to some potentially terrifying. It's dubbed the ASUS IoT AI Accelerator PCIe card or CRL-G160U-P3DF for short. And it's designed for artificial intelligence computation. Ooh. Speaking of intelligence, you'd be intelligent to know I'm gonna tell you about our sponsor, Honey. Honey is the free to use shopping tool that helps search for the best promo codes on tons of your favorite sites. Get it today at joinhoney.com slash LTT. ["Honey on the Drive"] ["Honey on the Drive"] Historically, we've mostly shied away from covering AI stuff. And there's a couple of good reasons for it. One is we're not AI or machine learning developers. So finding practical use cases that we are able to set up and demo for you guys can be a little tricky. And second, a lot of those use cases are extremely technical behind the scenes kinds of things, at least for now. So hardware like this isn't too applicable to end users like me and you. That is until today. While doing research for and testing the home automation setup at my new house, get subscribed by the way, you don't wanna miss any future content around revamping that place. We stumbled into a bit of a roadblock regarding presence detection or the ability for the systems in the house to be aware of whether or not anyone is actually home. Presence detection provides us with a number of benefits. For example, we can turn off lights and set the HVAC to be more energy efficient when there's nobody at home. And this is relatively simple to implement on a whole house level. But therein lies the challenge. We put a lot of effort into making my house a lot more granularly controllable. Practically every room has its own independent HVAC and lighting, which is awesome for personal preferences. Like the kids might want their rooms cooler than my wife and I do, for example, or vice versa. But it can also be used to improve efficiency and save cost. If no one's in the giant rec room downstairs, what's the point of heating and cooling it? I mean, surely that can't be that hard to automate. Well, as we found out, it can be really freaking hard. One idea we had was to install Bluetooth low energy beacons in every room and read people's presence through their phones. The only problem with that is that people have this tendency when they're at home to put their phone on a charger and, oh, I don't know, walk around their house. So that won't work. Another idea was to use motion sensors, but those kind of crap the bed as soon as you decide to take a nap on the couch and then you wake up in a sauna of a room in the summer, which sounds like a pretty bad time to me. So this is where our AI card comes into play. This is an Asus designed product, but all of the important AI bits are actually made by Google of all people under the brand Coral. We've got both of them linked down in the description below. Let's take this thing apart. This specific card has 16 onboard edge TPUs, as Google calls them. And the TPU stands for Tensor Processing Unit. In the most simple terms, these TPUs are hardware processors that are physically designed and optimized to run a specific application. You may have heard of something called an ASIC as something that's really good at mining as specific cryptocurrency. Well, these are the same idea, except that instead of mining Bitcoin, they're for running machine learning inference calculations. So what we're looking at here is essentially just a PCI Express 16 X interface here going. You can actually see the traces going right into this PCI Express switch, which appears to be just splitting out these lanes into an M.2 2X interface for each one of these little dual TPUs here. I was expecting something less kind of modular and science fair projecty. Now each of the edge TPUs, of which there are two on these M.2 cards, can do around four trillion operations per second with a power draw of only two watts. So two watts times 16 TPUs brings us to 32 watts for the card. You add in your PCIe switch and fans, and that brings you to 52 watts, which is well within the 75 watts that this PCI Express slot could power. But for some reason, it still requires a six pin power connector, and it's got this gigantic copper fin cooler on it. Now you're probably thinking, wow, that's really cool Linus, but most of this sounds like the boring technical stuff that you were talking about earlier. But if you were following closely, while we were researching better ways to detect presence in my new house, we stumbled upon some software called Frigate that uses cameras plus this hardware to really efficiently detect presence. It's really freaking cool. Let's take it for a test drive. I gotta put this back together. And we're up. This test bench is running the latest version of Unraid, and it's gonna stand in for my home NAS. Now you might notice there is also a GPU installed in it. So we need that as a video output because this is a non-APU Ryzen processor, but it's also useful for offloading the camera video decode process from the CPU. Now in my home deployment, I'll probably just use the CPU because I'm gonna have 24 epic cores that are otherwise gonna go unused. But for now, this is the better solution. Now once Unraid's booted up, you can see that we have just a single array disk with no redundancy. That's not what I'd recommend as a production deployment, but it's good enough for this video. Now we just pop over to the community apps tab and installed apps. So we've got our Coral Accelerator module drivers. We don't need this if we're using a USB version of Google's AI processor. We only need it for the PCI Express version. And then we've also got our Nvidia drivers, which are for our GPU. Now it should be noted that you can use your CPU for Frigate, but performance is not good. Next we install Frigate. Now we can't just launch it. We've got to actually configure Frigate. Otherwise it's gonna have no idea what devices it's supposed to be using. So step one, we're gonna copy the GPU UUID from our Nvidia driver and paste that there. Then for our TPU mapping, we've mapped it to our Apex underscore zero device. So that is only one of the 16 TPUs that are on that card. Because we're only gonna have a couple of cameras, we're not gonna bother mapping all of them, but we did map five of them. So that should be enough for our demo for now. Wow, this is a great password, Jake, I like it. I see right here, oh gee. Don't get overwhelmed. Most of this is included in the template, but there are a couple of things that we had to tweak. So one is we had to set up our camera. So you can see we've pulled our camera from the writer's den and we've got our network path for it right here. We're just using the one camera for now. We actually didn't take all five of our NPUs. We've just got four of them configured here. Again, this is total overkill for what we're doing. And then we've also got this right here. So we are limiting our frames per second for the camera feed that we're using to five FPS. That is all we're gonna need in 99% of situations to have functional detection. And adding more processing to this would serve no real purpose. One other critical thing back up at the top here is that you can plumb this into an MQTT server, which is basically its way of communicating back to your other home automation devices. So this MQTT server 10.20.0.71 is running on our office home assistant instance. Hey, five FPS of green. I can see why you said we didn't need more FPS. There's not a whole lot of movement there. It's peanut butter demo time, peanut butter demo time, peanut butter demo time. What are you at? What are you at? Okay. That one camera is set up. We have another one too, but let's just- Okay. Hi there. Oh, wait, hold on. I screwed up. Oh, all right. Do we not have the detection on? We do, but I didn't turn on like the bounding boxes. Bounding boxes are just little boxes that show us what the software is detecting and the degree of certainty that it has for what that object is. Humans aren't the only thing that it can detect, but they're kind of the only thing that we care about for home automation presence detection. You're telling me we don't have to detect how many apples you have in your fruit bowl? Oh yeah. Oh yeah. Oh yeah, you are there, buddy. Oh, it's kind of detecting your feet. Does it know who I am? That part we have not done yet. What if it only sees like this much of me? Yeah, it still thinks you're there. What if my legs show? Oh, it lost you. It lost me? What about a hand? Oh yeah, it sees you again. No, really? Okay, what if it just saw an elbow? Oh yeah, it's detecting you. What, really? Yeah. Can it see me? It's confused right now. It's a little confused. What about if there's just like a casual like hand there? It's kind of freaking out. So sometimes it actually sees two people when he's kind of obscured, but that's totally fine. Oh, like it thinks I might be like two people having a sex orgy behind the curtain. Uh, yeah, sure. But it's fine because all we need to know is if there's anyone, right? Right. Yeah, and sex orgy aside. You don't want the heat to cut out during your sex orgy if it's gonna get cold. Man, you can do so much cool stuff with this. But wait, there's more. What if you have a common area, like a kitchen or a rec room, and you wanna be able to apply personalized preferences when you're present? Like say, have your Spotify music with your specific playlist play when you're in the room and not when Yvonne's in the room, right? Oh, I never even thought of that. You could walk around the house and have Spotify playing here, oh, I'm over here now. And now there's my Spotify over here. Oh my God, that would be so cool. Turns out someone had that exact thought and created Double Take, which is a piece of software that allows you to determine the face. This is more creepy. Yeah. But it's all local. I mean, we could just configure this to not log any of this shit, right? Yeah. Did a line disappear? Oh, wow, yeah. Nice. Now Double Take isn't available on the Unread Community App Store, at least not yet. So we need to manually set up the Docker container ourselves. Then once it's set up, we need to configure it to listen to Frigate over MQTT. It should be noted that Double Take itself doesn't do facial recognition. It's just an API and interface to make hooking into and training facial recognition software easier. So the first thing we need to do is grab some pictures of me, which are going to be training data for our facial recognition. So I guess I need to go over there, right? We could go over there, or we can just upload some photos we already have. Double Take currently supports DeepStack, Compreface, and Facebox. They're all machine learning-based facial recognition models. DeepStack is on the Community App Store, so that's the easiest to set up. But I tried it out and it wasn't really that great with the limited training data I had. Compreface, on the other hand, it requires Docker Compose, so I have to run it in a VM. But it's scary good. I put a couple photos of Linus and secretly kind of had him walk by without telling him, and it was like Linus. And then I walked by and it was like Jake, with like 10 photos of each of us. All right, let's try it. Why do you have so many pictures of me on your hard drive? Ha ha ha ha ha. Well, you make my drive hard. Why CPU when I can see you pee? Why talk to you when I can leave? Can I zoom? How do I use a Mac? Do you know? No, these stills are horrible. It's scary to see you without a beard. Ba-dum-ba-dum-ba-da. Oh wow, yeah, yeah, yeah. Oh my God, 95.8%. How can it be that sure on unbearded Linus training? There was only one that it is actually confident about. I mean, 94% seems pretty damn confident to me. And you know what you can do is take all of these and say, train off of those data. Now you wanna walk over there again? Yeah, for sure. Okay, that's plenty. There's so many Linus's now. Linus, Linus, Linus, Linus, Linus. There's a couple of times it thinks it's James. I think like once you trained a good model with Compreface, I'd probably say that like it would only accept it as that person unless the confidence was like 99%. Some of these other models, Madison, I also made with people's photos with them wearing a mask. So it like literally has the eyes. Okay, go back, run over there. All right, I'm going, I'm going, I'm going. See if it makes more mistakes. I'm gonna pull up a chair Linus style. It's very confident about Linus now. What, who else would behave like this? I don't think that's what it's looking for, but. Man, so you could just kind of wander around your house for 10 minutes, find anything that's wrong, boom, train it. Part of it is also gonna be specific cameras because you know, each camera looks a little different. One thing I asked Jake about, and he said it doesn't have this functionality for now is can it morph over time and can it continue to self train? The thing with facial recognition is like the more data you feed it, the more uncertain it becomes. All of this leaves us with some questions. Do you even need this thing? Could a bog standard CPU do the same calculations? The answer is yes, it could, but way slower. To put it in perspective, your average quad core CPU would probably only be able to handle a few frames per second on a quad core. But because these edge TPUs are so optimized for running these specific calculations, they are stupid fast at them. A single coral edge TPU can handle around a hundred frames per second of people detection in Frigate. And we've got 16 of them. So if my rough math is correct, that's around 1600 frames per second. We might actually be pushing that once we've got all the cameras in the new place. But we're only taking five FPS from each. Still, it's a lot of cameras. No, no, no, five FPS. Yeah, five, okay, yeah. Man, that's crazy. So a single one of these TPUs could probably handle 15 to 20 cameras at five FPS. Another question this raises is, are we worried about the security implications? Fortunately, since we have all of our own hardware and all of this is open source and locally hosted, none of the data goes outside of my house. So I'm actually pretty happy with the level of security and safety. As for being worried about AI taking over, no one's asking that question. So it's in here anyway. The answer is no. Leaving only one question then, who's our sponsor? FreshBooks. FreshBooks is the simple accounting solution that is designed specifically with you in mind, the small business owner. It features built-in automation that allows you to spend less time tracking projects and more time growing your business. So whether you're a trades person, creative agency, or a YouTuber, you can choose the plan that's right for you with FreshBooks. They have an award-winning Toronto-based support team who are always happy to help you if you need it. And you don't have to take my word for it. You can try FreshBooks for free for 30 days today with no credit card required at freshbooks.com slash Linus. If you enjoyed this video, maybe go check out the video where we built the server that is gonna be running all this stuff. We're gonna have to get a little bit creative with PCI Express expansion on it. But that's the machine that's gonna somehow run this full height card. Yeah, yeah. So good luck everybody. Are we gonna do an external PCIe box? We should totally do it. Yeah, we should totally do it. Yeah, that'll be a video. Get subscribed. We're gonna do a water cooler and everything.