 We have looked at a lot of ball in GPUs over the years, whether it's the six Titan Vs we had for the six editors project, three GV100 quadros for 12K ultra wide gaming, or even this unreleased mining GPU, the CMP178X. There are not a lot of cards out there that we have not been able to get our hands on in one way or another, except for one, until now. The Nvidia A100, this is their absolute top dog AI enterprise, high performance compute, big data analytics monster, and they refused to send it to me. Well, I got one anyway Nvidia, so deal with it. Just like everyone's got to deal with my segues. Smart Deploy provides out of the box Windows imaging support for over 1500 computer models. You can deploy one Windows image to any hardware model with ease, and you can get free licenses worth over $500 at smartdeploy.com slash Linus. The first two questions on your mind are probably why we weren't able to get one of these and what ultimately changed that resulted in me holding one in my hands right now. The answer to the first one is that Nvidia just plain doesn't see these things to reviewers, and at a cost of about $10,000, it's not the sort of thing that I would just, you know, buy because I got that swagger. You know what I'm saying? As for how we got one, I can't tell you. And in fact, we even blacked out the serial number to prevent the fan who reached out offering to get us one from getting identified. This individual agreed to let us do anything we want with it. So you can bet your butt we're gonna be taking it apart. And all we had to offer in return was that we would test Ethereum mining on it, send a shroud that'll allow them to actually cool the thing and reassemble it before we return it. So let's compare it really quickly to the CMP170HX, which is the most similar card that we have. It's the silver metal, and it's not ribbed for my pleasure. All right. And we actually have one other point of comparison. This isn't a perfect one. This is an RTX 3090, and what would have been maybe more apt is the Quadro, or rather they dropped the Quadro branding, but the A6000. Unfortunately, that's another really expensive card that I don't have a legitimate reason to buy, and Nvidia wouldn't send one of those for the comparison either. So the specs on this are pretty similar. We're gonna use it as a stand-in, since we're not really looking at any workstation loads anyway. So the A100 then. This is a 40 gigabyte card. I'm gonna let that sink in for a second. And the craziest part is that 40 gigs is not even enough for the kinds of workloads that these cards are used to crunch through. We're talking enormous data sets to the point where this 40 gig model is actually obsolete now, replaced by an 80 gig model, and these NVLink bridge connectors on the top here, let's go ahead and pull these off. These are used to link up multiples of these cards so they can all pool memory and work on even larger data sets. Now the die at the center of it is a seven nanometer TSMC manufactured GPU called the GA100. We're gonna pop this shroud off. We're gonna take a look at it. It has a base clock of just 765 megahertz, but it'll boost up to 1410. That memory runs at a whopping one and a half terabytes a second of bandwidth on a massive 5,120 bit bus. It's got 6,912 CUDA cores, and a, what is it, 250 watt TDP. She's packing. Oh, you're just going right for it. I'm going right for it. Oh, geez. This is Linus Tech Tips. And basically every part of this is identical to the CMP card. It kind of looks that way. I mean, the color's obviously different. Yeah, but it looks like the clamshell is two pieces in the same manner. There's no display outputs. The fins look the same. Now here's something. The CMP card specifically didn't even contain the hardware for video encode, if I recall correctly. Yeah, this doesn't have NVENC. Okay, so it's not that it was fused off. It's that it's just plain not on the chip. Not on GA100, yeah. Okay, but- So GA102, which is like 3090, does have it. Got it. And A6000. Okay, you ready? Oh, God. So yeah, it's like exactly the same on the outside. Same jank power connector. Wow, that is super jank. Check this out, guys. It uses a single eight pin EPS power connector, which you might think is a PCIe power connector. So here, look, I'll show you guys. This is an eight pin, like normal GPU connector, but watch, he no go in. But if we take the connector out of our CPU socket on the motherboard, there you go. Oh, well, the clips are interfering a little bit. I mean, what the heck is going on here, ladies and gentlemen? You need more power. Yeah, exactly. So you can combine two PCIe connectors into that. Can't remember how to get it out of here. I see the fingerprint of the technician who assembled the card, though. Think we have to unclip this part first? Oh, there's a little screw, right? Yeah, there's a little screw. Ha ha, third type of screws. But you didn't see that one, nerd. You're a nerd. Your face is a nerd. Your butt's a nerd. Whoa. This is not coming off, Jake. But you gotta like tilt it out, buddy. Whoa, whoa, whoa, don't pull the cooler off. See, it's like it's caught back here. Oh, I wouldn't raise my hand. Oh, hey, hey, how you doing? Jesus. Stressful. Look, maybe if we break it, you'll actually have to buy one. I don't wanna buy one. That's not the goal. What? I thought you put your hand up for a high five. I was like, whoa, what are you talking about? I don't want to buy one. Why not? Whoa, what is going on here? You see that? It looks like there was a thermal pad there or something, but there isn't. It actually- It's like greasy. No, look at it closer. It's not greasy. It's like, you see how this is like brushed almost? Or like, looks like somebody sandblasted it. That part's not. I don't remember that on this card. All right, so the spring loading mechanism is just from the bend of the back plate. That's kinda cool. So I checked the CMP thing. Doesn't look like it. I wonder why they would have like a map of that. This doesn't look brushed at all. What did we, last time we twisted. No, I don't think we did. Yeah, we did. I just looked at the video. I'm pretty sure I just reamed on it. Oh my God. No, you were against reaming on it. And then we were like, just twist a little. I'm for real. Oh God. Ah, it has an IHS. It looks basically the same. Yeah. We're gonna have to clean that off and see. Yeah. There's not much alcohol. No, I like to go in dry first. So, yep, that's the same thing. All right. I mean, this isn't the first time Nvidia has used the same silicon in two different products with two different capabilities. We see the same thing with their Quadro lineup versus their GeForce lineup, where things will just be disabled through drivers or fusing off different functional units on the chip. What I wanna know then is besides the lack of NVLink connectors on this one. Well, they are in there. They're just not accessible and they probably don't work. Right. What is the actual difference in function between them? Well, this one doesn't have full PCIe 16X. Right. It has less memory. It's, I think it has way less transistors, but it is still a GA100. Yeah. So the transistors are there. Yeah, they're probably just not functional. Let me see what the chip number is on that one. Yeah, cause weren't we not even able to find a proper Nvidia.com reference to this one anyway? So we're just relying on someone else's spec sheet. So the transistor count could just be wrong. Okay. So this is, so the CMP card was a GA- Look at this guy. Yeah, what a weirdo. GA100 105F. And this is a GA100 833. If it's a GA100, I guess it could be a different GA100. I don't know. I mean, it used to be back in the day, you would assume that it's just using the same silicon as the GeForce cards because Nvidia's data center business hadn't gotten that big yet, but nowadays they can totally justify an individual, like new die design for a particular lineup of enterprise cards. And interestingly enough, the SXM version doesn't have an IHS, at least it seems that way, but the SXM version is also like 400 Watts, and this is like 250. Yeah, totally different classes of capabilities. All right, let's put it back together then, shall we? I got you a new goop. Goop me. I brought two goops. I'm going for the no look catch. Oh yeah, baby. X marks the spot, baby. My finest work. Maybe it'll perform better now. Probably not. Oh, boop, boop, boop, boop. Backing it up. Cool story, bro. Thanks. Thanks, bro. Where's our back plate? Did you take it? Oh, shoot. It's black. I thought it was gold. I was looking for gold. Aren't we all? I don't know about you, but I found my gold. What's that? Yvonne. Shut up. All right, all right, let's get going here. Which one do you want to put on the bench first? What do you mean? We're not going to compare it to that thing. Oh. It doesn't do anything. Okay. We don't need this thing. Here we go, boys. We can't put this in the first slot because we don't have a display output. You like the bottom one? Your bottom? Sure. This, okay, this is how you flex IT style. Now you might've noticed at some point that the A100 doesn't have any sort of cooling fan. It's just one big, fat, long heat sink with a giant vapor chamber under it to spread the heat from that massive GPU. So Jake actually designed what we call the shroudinator. It allows us to take these two screws that are on the back of the card for securing it in a server chassis because that's how it's designed to be used. So it's passive, but there's lots of airflow going through the chassis. And then lets us take those screw holes and mount a fan to the back of the card. It's frankly not amazing. What? No. But that is aerodynamics at its peak. You should hire me to work on F1 cars, okay? Yeah, not so much. Yeah, it only blows probably more air out this end from the back pressure than it does out this end. But it's enough to cool it, I swear. It is. Let's go ahead and turn on the computer, shall we? Okay, so a couple of interesting points here. It wouldn't boot right off the bat. You have to enable above 4G decoding. And then I also had to go in and I think it's called like 4G MMIO or something like that. I had to set that to 42. Okay. The answer to the universe. Yes, thank you. And they are both here, A100 PCIe 40 fricking gigabytes. I installed the like game ready driver for the 3090. And then I installed the data center driver and I think it overrode it. But the game ready driver, it still showed as like active and you could do stuff with A100 and vice versa. So it's probably fine. Now, interestingly, the A100 doesn't show up in task manager at all. Did the CMP? I can't remember. No, no, I don't think it did actually. Anyways. What do you wanna do in blender? Faster in BMW? BMW is probably too short. Yeah, let's do classroom. I think BMW on a 3090 is like 15 seconds or something like that. Okay, yeah, let's do classroom. That's also like the spiciest 3090. That you can get, yeah, pretty much. It's just so thick. Why would you ever use it? Yeah, because you wanna do- Is it even doing anything? Here's one reason, because you can do classroom renders in a minute and 18 seconds. That's why. Okay, well, what about the A100? Oh, you didn't plug the fan in, hey? Oh, whoops. How hot is this? Probably warm. Fortunately, it hasn't been doing anything. Time to beat is a minute and 18 seconds. So let's go ahead and see how it does. It feels like this is the intake. I mean, it's hot, so like- Oh yeah, it's going, it's going, Jake. It's going. You did good. It works enough. This should be, like this is all- This should be way faster. Way huger GPU, right? It's actually slower. How much? Not by much. It's like a few seconds, but it's slower. So it's worse in CUDA. What about optics? So the interesting thing is this card doesn't have ray tracing cores. The 3090 does. So you'd think that optics would only work on the 3090, right? Do you want me to just try the A100? Yeah, sure. Yeah, it's still GPU compute. I mean, you've got to give it to it in terms of efficiency. For real though, even running two renders to the 3090s one, the average power consumption here is still lower. Yeah, well, and looking at while it's running, it's like 150 watts. Yeah. Versus 350 or whatever it was on the 3090. Yeah. Ready to go again? Yep. Oh my God. Man, this thing is fast. What's the power consumption? Three, 353. And the fan is still like, just, I want one of these. This thing's sick. Okay. It's way faster. Yeah, there's no question. We don't even need to finish this. It's gonna be like 30 seconds. Yeah, not even close. So do you want to know why? I would love to know why. You said it earlier. You just weren't really thinking about it. This has half the CUDA cores of a 3090. It's like 7,000-ish, I think. So it's just full of like machine learning stuff. Yeah. So it has basically half the CUDA cores. So the fact that it is even close is kind of crazy in CUDA mode. But in Optics, what I found out is Optics will use the tensor cores for like AI denoising. But nothing else. Which you'll see in there. So I think it's falling back to CUDA for the other stuff. Got it. But the 3090 has ray tracing and tensor cores. So it just demolishes. Where's the thing where you can select apps and then tell it which GPU to use? Yeah, here we go. No. So it will not allow you to select the A100 to run games, even if we could pipe it through our onboard or through a different graphics card, like we did with that mining card ages ago. No DirectX support whatsoever. Let's check it in GPUZ. So way fewer CUDA cores. You can see that. We go from over 10,000 to a lot less than 10,000. The pixel fill rate's actually higher. I guess that's your HBM2 memory talking. 1.5 gigabytes per second. What's a 39? 1.5 terabytes per second. It's like almost 60% almost. Holy bananas. But what about the supported tech? Yeah. So we can do CUDA, OpenCL, PhysX. Sure. We should set it as the PhysX card. Dedicated PhysX card. All the rag dolls everywhere. And OpenGL, but not Direct anything or Vulkan even. OpenGL. Now that's interesting. Go to the advanced tab. Cause you can select like a specific DirectX version at the top under general. Like what about like DX12? What does it say? Device not found. It's the same as the mining card. It'll do OpenCL. So we can mine on it. All right. I mean, should we try that? Yeah. We could do mining or folding or. Sure. I have a feeling that's going to kind of suck for that too. There's no AI in mining. I don't think so. It's still a big GPU, dude. So you can't. Well suck is relative, right? Like for the price you'd never buy it. Oh, I think it might be better than the CMP card though. Just a little bit. Shut up. I think so. So the only thing you can adjust, I think this is the same with the CMP card is the core clock and the power limit. You can't mess with the memory speed. And you can move the power limit only down it looks like. Yeah. Top is the 3090, bottom is the A100. Wow. That is a crap ton faster than the 3090. It's pretty much the same as the CMP, but look at the efficiency. 714 kilo hash per watt. And I bet you if we lower the power limit to like 80, it's a little bit lower speed. Yeah. Maybe we can go, I don't know. We probably don't have to tinker with this too much. I mean, it doesn't draw that much power to begin with, I guess. Yeah. I think it's pretty fricking efficient right out of the box. I mean, the efficiency is better. It's a little bit better. But before it was doing 175 mega hash roughly at 250 watts. So it's pretty damn good. 3090, you can probably do like 300 watts with 120 mega hash. We're running the folding client now. I've had it running for a few minutes and it's kind of hard to say. The thing with folding is based on whatever project you're running, which is whatever job the server has sent you to process, your points per day will be higher or lower. So it's possible that the A100 got a job that rewards less points than the 3090 did. It does look like it's a bit higher, but you can see our 39, this is like a little like comparison app thing, is 31% lower than the average. So it's probably just that this job doesn't give you that many points. Got it. The interesting part is the 3090 is drawing 400. 400, holy shnikes. The A100 is drawing 240. Man, that's efficient. And performance per watt, maybe gamers don't care that much. Actually, we know for a fact, gamers don't care that much. In the data center, that's everything because the cost of the card is trivial compared to the cost of power delivery and cooling on a data center scale. Especially when you have eight of these with a 400 watt power budget, like you would get on the SXM cards, in a single chassis times 50 chassis. Like that's a lot of power. Let's try something machine learning. Unfortunately, for obvious reasons, most machine learning or deep learning, whatever you wanna call it, benchmarks, don't run on Windows. So instead I've switched over to Ubuntu and we've set up the CUDA toolkit, which is gonna include our GPU drivers that we need to even run the thing, as well as Docker and the Nvidia Docker container, which will allow us to run the benchmark. We're gonna be running the ResNet 50 benchmark, which runs within TensorFlow 2. This is a really, really common benchmark for big data clusters and stuff, except our cluster, it's just one GPU. In a separate window, I've got Nvidia SMI running. It's kinda like the Linux version of MSI Afterburner, but it's made by Nvidia, so not quite. But what it's good for is at least telling us our power and the memory usage, which we should see spike a lot when we run this benchmark. I took the liberty of pre-creating a command to run the benchmark. So we're gonna be running with XLA on to hopefully bump the numbers a bit. We will do that for the A100 as well. So no worries there, it should be the same. As well as using a, what do you want? Look, he left, cause he didn't have time for this, and now he's back. This is the world's most expensive lint roller. I don't even remember what I was saying, damn it. Distractions aside, we're gonna be running with XLA on. That'll probably give us a bit higher number than you would normally, but it is still accurate. And we're gonna be running the same settings on the A100 as well, so no concerns there. We'll also be using a batch size of 512, as well as FP16 rather than FP32. So if you wanna recreate these tests yourself, you totally can. Let's see what our 3090 can do. Look at that, 24 gigs of VRAM completely used. God, I don't know if there's any application aside from like Premiere that will use all that VRAM. I'm sure Andy can attest to that. Okay, 1400 images a second, that's pretty respectable. I think like a V100, which is the predecessor to the A100 does like less than a thousand. So the fact that a 3090, which is a consumer gaming card, can pull off those kinds of numbers is huge. Mind you, the wattage, 412 watts, that's a lot of power. It'll be interesting to see how much more efficient the A100 is when we try that after. The test is done now and the average total images per second is 1,435, that's pretty good. I've gone ahead and added our A100 so we can run the benchmarks on that instead. I'm expecting this is gonna be substantially more performant. So it's the same test. I'm just gonna run the command here. Gotta wait a few seconds. We got Nvidia SMI up again. You can see that it's just running on the A100. The RAM on the 3090 is not getting filtered, just using that as a display output. See, all 40 gigabytes used, that's crazy. If we thought the 3090 was fast, look at that, Andy. That's like a full thousand images more. We're getting like 2,400 instead of 1,400. And the icing on the cake, if you look at Nvidia SMI, we're using like 250 watts instead of 400 while getting like almost double the performance. That is nuts. Probably the coolest thing about this whole experience though is seeing the Ampere architecture on a seven nanometer manufacturing process. Cause you gotta remember, while none of this is applicable to our daily business, what this car does do is excite me for the next generation of Nvidia GPUs. Because even though the word on the street is that the upcoming Ada Lovelace architecture is not going to be that different from Ampere, consider this. Nvidia's gaming lineup is built on Samsung's eight nanometer node, while the A100 is built on TSMC's seven nanometer node. Now we've talked a fair bit about how nanometers from one fab to another can't really be directly compared in that way. But what we can do is say that it is rumored that Nvidia will be building the newer Ada Lovelace gaming GPUs on TSMC's five nanometer node, which should perform even better than their seven nanometer node. And if the efficiency improvements are anything like what we're seeing here, we are expecting those cards to be absolute freaking monsters. So good luck buying one. Hey, at least you can buy one of these. We've got new pillows. That's right. This is the, what are we calling it? The couch ripper. It's an AMD themed version of our CPU pillow with alpaca and regular filling blend. And this video is brought to you by our sponsor, IDAgent. 90% of data breaches start with a phishing email. So you can reduce your organization's chance of experiencing a cybersecurity disaster by up to 70% with security awareness training that includes phishing simulation. Bullfish ID by IDAgent is a phishing simulation platform that transforms your biggest attack surface into your biggest defensive asset. You can add every employee to your security team with security awareness training that empowers them to spot and stop phishing threats. You can automate training campaigns and reporting for stress-free, consistent training that gets results. Choose from a rich set of plug and play phishing campaign kits and video lessons accompanied by short quizzes, or you can create your own phishing campaigns and training materials easily. Bullfish ID provides effective, affordable one-stop phishing resistance training that fits any business and budget. Get two months for free and 50% off setup at bullfish ID at it.idagent.com slash Linus. If you guys enjoyed this video, maybe go check out our previous video looking in more depth at the CMP170HX. I like the silver better. We were smart, we'd be mining on this, but we're not that smart. Well, you know, mining is bad.