 Is this just like a- Oh yeah, yeah, yeah. Stop, stop, stop, stop, stop. But where are they? Really? By an order of magnitude, this will be the single most expensive unboxing we've ever done. Inside these boxes, this crate, this server rack, this additional box, and this final box over here, this monster server from Supermicro is everything that we need to deploy a server with a petabyte of SSDs. Kyoksia provided the SSDs, they sponsored this video, it has been a year in the making, and we are about to see performance, the likes of which we have never seen and may well never see again. Realistically, like- Yeah. In our lifetime. They're not even just SSDs, they're NVMe SSDs. Let's bring back, holy sh- That's a series we haven't done in a while. Holy sh- Holy sh- Holy sh- Let's bring back, holy sh- I think that server's worth like $100,000. I think that server's worth like $100,000. $100,000 server you said. Something like that. Well, let's move it gently then. There's gonna be a whole follow-up video detailing the exact architect. Follow up videos. Yeah, there's gonna be multiple videos detailing the setup and architecture of a petabyte of flash because while Kyoksia does make such high capacity NVMe SSDs now, that you could pretty easily build a server with a petabyte of flash accessible to it. But in order to get the full performance out of it. Oh my goodness. You gotta do something a little special. I don't think I'm gonna let you pick this one up. You know, you gotta give Nvidia credit. It looks pretty special. Does their hardware have sex appeal or does it have sex appeal? I think this is just the module. This actually slots into the main server. So what are we looking at here? So each of these is an Nvidia A100, 80 gig. So we've got eight of them, NVLink together. That's 640 gigabytes of VRAM, video memory. These cards alone are probably worth more than any other individual server we own. This thing, this unit is almost 100,000 US dollars. Just this part of it. And what are these? Are these just like some kind of- I think that's just power delivery. They asked us not to take them off, but I mean- Oh really? We already have it, so like we could just do it anyways. I mean, Nvidia can't hate us anymore, can they? Yeah, well. They probably can. You know what? Let's not piss off the enterprise guys. Because- This looks like power delivery. Can I just say that it's all sponsored by Kioxia who provided the flash, but Nvidia has been absolute bros and broettes about this whole thing. Cause we wouldn't have been able to do it without them. So I guess, yeah, let's pick it up. Why don't we move this off of the server so we at least have somewhere to put this. Well, I'm gonna know what's under it. The server. Oh wait, that's it. Ah, yes. Oh, that's why the box was so big. Yeah. Oh, look at those rails. I know, right? These are chunk-tastic. Let's just, yeah, let's just lift this out of the way. Put it on top of that? Sure. Sure. That's safe. So how did this even come to be now? I've been babysitting this for- A year. A year, but I wasn't there for the first conversation. What happened was Kioxia has worked with us a number of times and they were looking for, you know, some kind of cool SSD related video to do. Cool, you say. I took an absolute long shot cause this was after we'd already seen the appetite for a petabyte project on the channel. Right. But they're a flash vendor. And I was like, well, it would have to be SSDs and it would have to be, and I thought they were gonna come back with something way smaller. Don't tell Kioxia, but they would have gotten away with like a quarter of this. But I was like, well, it's gonna, it can't be smaller than what we've done before. And they were like, oh, this is actually a very interesting engineering challenge from our standpoint. Let's see who we can partner with to try and make this happen. And the answer was Super Micro, AMD, Weka, freaking everyone you could possibly name in the server space. So how does this freaking work? Okay, let's lift this boy up. We're gonna start at the back, which is a little unusual. This is our head unit. This is PCIe over fiber because the SSDs don't actually go in the control unit. This is where all the compute lives. In order to do this kind of throughput, you need a very, very exotic, very powerful solution. The SSDs themselves. This is the size of an ATX power supply. Are gonna live in additional units that we're gonna unbox in a few minutes here. How many watts are these? Holy sh. This is a 3000 watt power supply. Can we even power this? I've never actually seen one of these plugs properly used. I'm just curious what is in here. Oh, wait, what? Is this just like a- Oh yeah, yeah, yeah. This will be like a- Stop, stop, stop, stop, stop. So you gotta unscrew it then. No, sorry. Like I meant this part. Oh yeah, okay. No, you're right. Stop though. You gotta take this side out. Well, I thought you were gonna. I don't have a screwdriver. What are you doing? What are you doing over here? You're just- Waiting for you to hand me the screwdriver. Well. Okay. Oh my God. What even is that? So here's what we're looking at. We've got our PCIe over fiber cards here. These are just Nvidia Kinect X6 200 gig cards. Wow. There's eight of them. And then we've got more PCIe. So you can see we've got, there you go right there. Okay, we've got a little bit more PCIe to these NVMe bays at the back here. So you got four more of those. And then you've got a little bit more PCIe, okay, up here. And that's going to this, which is probably like a dual 10 gig card or something like that. That's just for- It might just be gigabit like for management. You know what? I know you like that much PCIe. I gotta tell you, just with, there's more PCIe. Oh, are you serious? So this unit can take a total of 10 NVMe drives. And then it has another eight PCIe 16X slots. Then there were those NVMe connectors I didn't recognize to the sleds at the back. And then there's all the A100s. Oh my goodness. So do we slide it in? I think we should put it in the rack first. They have to be epics of some sort. Oh, they are. Okay. Dual 64 core. But where are they? Oh, they're in the top. They're in like the very top. Yeah. So there's the PCIe lanes for the A100s that just sit under it. Very cool. I don't know how to take this out. I wonder if it got hit a little actually in shipping. It looks a little long. It looks a little bowed. Oh boy. Oh boy. Oh, there's a screw right here. Oh, there's your problem. Oh, you got this? I got it. Don't drop it. I got it. Dual 64 core epics. I have to assume that that's like, is this two terabytes of Samsung memory? The spec sheet said 256, but I don't actually know who sent it. Okay, here, let's just pop one out. Oh boy. They're not gonna like this. That's a 64 gig DIMM. Dual rank, 3,200 megahertz Samsung. Well, it's super micro memory. So this is their qualified. So 64 gigs, that would make this, yeah. 32, two terabytes. Two terabytes of RAM. Ah, look at those freaking real. These things must scream. Yeah, these fans. What are those, 40 mils? Gotta be absolutely ridiculous. This is a lot of hardware in one box. Unfricking real. Love the airflow management here. Gotta put basically all of it through the CPUs. The RAMs. The RAMs just like. I can have a little bit. Yeah, come on. Maybe there's a little bit of cooling. Look, just like a tiny little spot there. These VRM heatsinks are like, I don't need much. There's business level hardware. And then there's enterprise level hardware. This thing is unbelievably heavy. There is a lot of freaking cooling in it. It's like a solid ass block of aluminum. It's very heavy. Yes, I understand. I'm getting the goddamn thing. Huh. How many watts is this? Oh, look at that. They got flappers. What the hell? Holy crap, the fan itself is this thick. Yeah. Wait, no, is there two in there? Crazy. Maybe two in there. So there's two blades. See this? Yeah, there's two blades. So you've got this blade is swept this way. They spin independently. And then this one is swept this way because if you want more static pressure, you can't put two fans that spin the same way. There's some kind of aerodynamics thing that makes it. So that doesn't work very well. They have to be spinning counter to each other. We didn't really consider that when we assembled this up here, we were going to have to bring it down. Yeah. Okay, plan. Where are we gonna put it? We're gonna put it on top of the rack for now. I'll take the lights out. I see. We ready? Oh my God. Okay, if we drop this, like it's game over. No, no, you know, we're good. Here's what we're gonna do. This thing already got beat up to hell in shipping. So we're gonna bring it down like this. Go ahead and put it on. Yep. There you go, little guy. Cool. I don't know if little guy is the right word for it. Absolute unit guy. This is when you have all the budget cause it's not just powerful. It's dense. You got to remember in the data center, density has a ton of value because building a data center is the most expensive thing about building a data center. So each of these OneU servers are gonna be our storage hosts. There's six of them. Six? Six. We gotta put 72 drives in here. Now you might think, hey guys, this is Austin. You might think, hey guys, why didn't you just pick servers that could take more NVMe SSDs? Well, here's the thing. Once you've got high performance enough SSDs and these are all Kyoksia CD6 PCIe Gen 4x4 NVMe drives, you run out of CPU power as well as PCIe for networking as well. So you can't just put as many drives as you want in each server. You gotta add more compute. You gotta add more networking. Yeah, it's dual processor. So these are, oh gosh. 32 cores each and a far more reasonable, I'm guessing this is 512 gigs of RAM. It says 256, but I think this is wrong. Yeah, only 256. And then of course we've got the Nvidia Connect X6 200 gigabit network cards that are gonna handle our NVMe over, I guess actually they're not fiber, are they? Cause they're so close. I think we're gonna be using direct attach. Direct attach. You call it NVMe over fiber. It's NVMe over fiber. Just with fiber, you can go farther. Is that the boot SSD? Oh, it's adorable. Oh my God, it just slots right into the. It's a little SATA thumb drive. Come on out. I think there's probably a lock on it. Oh yeah, there is. Look at that. These are super cool. Yeah. Oh look, they even included a USB type A port so you could run un-rate on this thing. Yeah, so the reason that people use these instead of an internal USB drive is just that the grade of NAND flash that gets used for a device like this tends to be higher. And while honestly in a real enterprise deployment, a boot drive failing is probably not the end of the world. A lot of the time they're booted over fiber too. Yeah, it's just why, right? Just use something better. Now you might be wondering why we went AMD Epic, which means you probably haven't been paying close attention over the last few years, but basically these will boost up to 3.7 gigahertz, giving us tons of compute, 64 total cores, and they have lightning fast, not to mention a ton of PCIe connectivity. So every device in pretty much this whole rig, I think is gonna be running at maximum speed. Definitely will. We don't want any bottlenecks. We can't have bottlenecks. Before we move on, a couple more fun things I noticed about this. These are 20 watt fans. Each of them. 80 watts. That's 150 watts in fans. Oh yeah, they're like the same type. Epic has so much PCIe that you can just put a PCIe 16X slot just chilling in the middle of your server. Just hang in there. Yes, this is a 200 gigabit card. One of two. Each of these is gonna have a 400 gigabit per second connection to our head unit. The best part, there's that slot. There's another one back here. Oh wow, something went over there. Well that one's at least accessible from the outside. Yeah, yeah, yeah. This is just hanging. They're like, you wanna put some NVMe in there? Yeah, just a... You only live once, man. Now for the next part. You guys saw this crate we were using as a table? Ah! Oh, it's very sharp. What the heck is all this? Wait, hold on, go down. What? I think there might be more servers in here. Keoksia. Oh, these must be the drives. Oh, what's this from Mellanox? Oh, excuse me, excuse me. The box says Mellanox, but I mean to say Nvidia. Nvidia acquired Mellanox somewhere in between this box being printed and us doing this project. What could be in this box? Hmm. Oh! Wait, what? Whoa, what? We originally asked Micron to send memory for this project, but Super Marco was like, oh, it's not qualified yet, blah, blah, blah, blah. Wait, what? I think these are just 8 gig GIMPs. 8 gig 3,200. Can I take some of this home? Is it all 8 gig? I don't know. Well, thanks Micron. What's behind this door? In fairness though, there are some really cool cables in here. There's power cables. Those aren't the cool ones. And then there's these guys. These will do 200 gigabit per second. And these are the direct attach copper cables that we're gonna be running between our head unit and our SSD units. Now there's networking gear that's like pretty cool. And then there's this. This is a Mellanox, well, Nvidia, 3,700 series, 200 gigabit switch. That's right. Each of these ports, 200 gigabit per second. And it can do it all at full speed at the same time. This thing is bananas. They will all do 200 gigabit per second. It's always fun when you start measuring your network switches in terabits. The fans are 30 watt. 30 watt fans. I wanna see how big are the power supplies. Really? I just looked at the spec sheet of that switch. 12.8 terabit. Oh, they are. Oh, that's hilarious. We were like, Kyoksia, are you guys new to working with- No, it matters. The position matters. All the position matters. Yeah, don't switch them around. Oh, I was gonna be all like, Kyoksia, come on. What's the fun of, you know, unboxing all the SSDs if you don't put them in a box? But okay, the position actually matters a lot for this configuration. So we'll just take a look at this drive and carefully put it back in here. Each of these is 15.36 terabytes and can deliver a total throughput of six gigabytes per second, which is impressive, considering that we're gonna have 12 of them per server. So that's 72 gigabytes a second per server. Absolutely mind boggling numbers, but there's even better stuff. So it's obviously using a completely custom solution. Everything from the controller to the flash is all Kyoksia through and through. And this puppy can do a million IOPS per drive. So by the time we're done with this, we are going to be putting everything we had ever done before to absolute shame. If you don't watch our channel regularly, you might not have heard of Kyoksia, but we've actually worked with them a fair bit. They were the first to deliver a U.3 capable drive, the first to deliver PCIe Gen4 drives. They are formerly Toshiba's flash division. So they've got literally decades of experience working with this kind of technology. And I'm really excited to see what these drives can do at their absolute peak. Of course, speed's not the entire story and I'm obligated to tell you that they are of course using the same high quality flash that they use in their high-end enterprise SaaS and other NVMe products with the same reliability features. And of course also they've got support for everything from non-encrypted drives to sanitized instant array, self-encrypting drive, SCD FIPS to level two certified. If you need it and it's an SSD, Kyoksia has something for you. Or something better than you would think. Or something better. The final piece of the puzzle is our server rack sort of. Let's talk about how we're powering this. Oh my goodness. What the heck is going on here? Is this the most whack power cord you guys have ever seen? Here it is. Next to a human hand for scale, okay? I mean, I don't know about human. There's a normal hand. Freaking ridiculous. These are L630s, so that's 240 volt, 30 amp. Each of them. We didn't actually have a place to plug these in. Yeah. We're gonna be stealing, I think one of the plugs from the welder and then one of the plugs from the mill. I got Infinite Cables to make these giant extension cords. They're like just as thick, maybe thicker. So yeah, we can't use the welder or the router when we run this rack, but I think that'll be okay. Infinite Cables also provided a 100 meter, 100 gig cable so we can plug it into the switch back there. No. I guess we'll have to like run it outside, but yeah. Guys, this is gonna be a really fun series. We're gonna go through the hardware, the software. We're gonna show you AI benchmarks. Yeah, it's gonna be great. Four times 12. 384. 300. Plus another 128. Plus another 128. So 512 cores. That's the same amount of RAM we have in our main editing server. We're gonna have 512 cores and a terabyte of flash. It's a little bit more than that. This is gonna be a terabyte of flash. Petabyte. Like 1.1 petabytes. Sorry, my brain broke. That's not possible. Petabyte of flash. 12.8. Petabyte of flash. That's crazy. No one would be crazy enough. Final shout out to everyone who helps. Keoksia, Nvidia slash Mellanox, whatever you decide you wanna be. AMD. Micron. Supermicro, FS.com. Infinite Cables. Infinite Cables. And Jake Tyvey. Jake Tyvey, who's been core. And all of our contacts at those places. I'm taking all this home. You guys have been so patient. This has been. It's gonna be a trade show at my house. If you guys are enjoying this series, maybe go check out the original Petabyte project series. That's pretty cool too. I actually don't think the panel in my house has enough capacity to run this. I don't think mine does either.