 When Dream needed a server that could store unlimited gameplay footage with a matching budget, he knew there was only one place to go. Yo. And we're gonna be building it in true Dream fashion. That's right, we're gonna be speedrunning the creation of a 1.2 petabyte hard drive storage server with a 200 terabyte all NVMe SSD server. All we need is a little bit of help. Alright! I think we're going for a world record here, guys! Keoksia sponsored the SSD, so that's taken care of. 45 Drive's practically built the server for us, and you guys are all here to witness it, starting with the big face reveal. You ready? Cha! Okay, it's more of a face play. But look, it's a really cool one, it has Dream on it. Let's go, Jakey! Oh, Jesus. Alright. Got a server to build here, boys! Oh, this is a lot of drives to shuck. Because the server's for Dream, some of the hardware did end up getting sent to us, but the hard drives were not one of those things, meaning that, yes, my friends, Dream actually did shell out for 60, the full 60, 20 terabyte hard drives. That's going to be a total of 1.2 petabytes of raw storage, which should work out to somewhere in the neighborhood of one petabyte of usable storage when we're done here. I think we're going to play this one a little bit safer. I'll put some hotspare, so it might be more like 900. Okay, sure. That's fair enough. I don't want to get a tech support call realistically from like wherever the crap Dream is from. The server hardware itself was provided by 45 drives, and this is their TurboTier server. It's got a 26-core Xeon Gold, 512 gigs of DDR4 memory, which we'll be using as a ZFS read cache. We've got four LSI HBAs, which are allowing us to hook up all of these freaking drives, and we actually got them to send a dual-port 25 gigabit per second network card, which sorry 45 drives, we're actually going to steal from this server and put in the SSD server. And there is a reason for this. The fancy switch. Hey, I need more drives, Jake. Hold on. Shush. World record. The fancy switch that Ubiquiti sent happens to have two 25 gig ports, so I'm thinking we put them together aggregate 50 gig to the storage server. Wow. It's probably excessive, but we're going to do it anyways. Honestly, though, outside of our own server room, he is getting the finest storage setup out of probably anyone else on YouTube or Twitch. By the way, Jake, can I just shout you out for putting these in exactly the correct orientation for me to just slot them right in? Are you being sarcastic? Are you not trying to? No, no, no. I was actually... No, it's like the most ergonomic thing ever. I just grabbed them and put them in. Yeah, I'll be sure to do it the other way now. Oh, you piece of crap. Why did I say anything? I was not trying at all. This just happened to be how they were getting pulled out. That's it. Are we done? 1.2 petabytes. You know, it's not that fun when it's this easy. What? I mean, it's... No, I'm still into it. We haven't found out if it works yet. We're replacing this dual 25 gigabit NIC with this dual 10 gigabit NIC. And the reason we're doing that is that, I mean, they're hard drives. Realistically, if we manage to get, you know, what? Any gigabit trunk, so two gigabytes a second out of it, that'll be fine. I mean, it'll do a lot more than that, but you're just using this for archives. So yes. Well, yes and no. I think it's possible that they could end up editing partially off of it if they were working on a project. It'll have plenty for that. Called back from, you know, a previous video or something along those lines. Come on, Jake. What? What was the ergonomic rationale behind putting this server here? I'm not saying you're wrong. I'm just saying I don't see how you're right. Are you complaining because you're short? No, I'm complaining because tall people don't consider my short person needs. That's what I'm complaining about. Right, right, right. Very important distinction. Unlike the petabyte of hard drive base storage, this one actually is going to have a ton of room for expansion. And that is thanks to high density drives like the CM6 from Kyoksia. These puppies are available in up to what? 15.3 terabytes of pop? This is a 30 terabyte SSD. What? Shut up. It's not. Not only is it a 30 terabyte SSD, it's a 30 terabyte TLC SSD. It's not even QLC. These are crazy. Kyoksia just sent these for Dream? Where's mine? They sent 15 for Dream and I was like, yeah, we'll just give him like seven. Really? Yeah. It's not like he paid for them. Yeah. That still means he's going to end up with 120 terabytes of usable space with two parity drives and a hot spare. Wrap your brain around it, right? This is the state of the art. I mean, I think they're looking at what? 22 to 26 in the next year. And those aren't even commensal. Those are shingled. Exactly. This has 50% more capacity than this in a fraction of the size. Mind you, it's probably quite literally 20 times the cost. Not to mention this is PCIe Gen 4. These will do almost seven gigabytes a second read on a single drive. And that's why we're keeping some. Yes, you should. Okay. So we're going to do a four node. Wait, what? Oh my, oh my God. Oh my God. How do you do that? You just literally, holy. Oh my God. Oh my God. Oh my God. Did I put it there? Are you kidding me? Dude, what the actual is wrong with you? Oh my God. Oh, oh, I want to throw up. I really, oh, do you think it still works? No. Did you hear the noise it made? It's a pretty, it's a pretty bouncy floor. Okay. The video's done. I'm leaving. Goodbye. It might still work. Okay. Lunch says it still works. You know, it might actually still work. Here's the thing. I've actually talked to Seagate engineers about the kind of drop testing that they do on these things. There's a solid chance that thing still works. What happened? What? Nothing. You talking ****? No, no. I was just saying that you're a fine young man and I'm really proud of the way you've turned out. I'm not proud of the way you've turned out. This is pretty gorgeous. This is the new version of the one we have, right? Yeah. That's specifically designed for Epic Milan. Milan. So that's a 32 core 7543P. So single processor. It boosts to 3.8 gigahertz, I think. So not quite as good as the 75F3. Yeah, but that's pretty freaking incredible. This thing is going to rip. Are you putting the RAM in? Are you going to talk about the RAM at all? I'm just going to put it in. Really? For speed running. Okay. Well, it's going to be 512 gigs of RAM. Nope. What? 128. 128. That's it? They're 8 gig DIMMs. These are from the $1 million PC that we didn't use. Right. Micron sent us literally like flats. 96. Yeah. Flats of these 8 gig ECC 3200 modules. They're fast. Yeah. They're exactly what you want, just not the capacity that you want. But for this, it's perfect because we're not using RAM caching since they're NVMe drives. Don't need that for ZFS, but he's going to get 128 gigs anyway. So we just want a nice fast CPU, which for Epic, high speed memory is really important. And then we're basically not going to hit this at all. Yeah. I mean, stuff will go through it, but- That's it. That's pretty much it. 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59. That's it. Shut up. It is not. There's only 59. Yeah. Cause it starts at zero. Oh, hold on. There might be 60 actually. Hold on. It starts at zero, doesn't it? It might start at zero. That it does. You might be okay. That it does. Are you screen recording? There's the moment. Yeah. I'm not screen recording, but it does start at zero. There is 60. That doesn't necessarily mean one of them isn't completely buggered. Okay. Cooler time? Yeah. I mean, it's cooler for a 2U. Yeah. The nice thing about a 2U is you get double the cooler capacity, which means your fans get to spend a lot slower. Yeah. Which is going to be good for at your house. Yeah. One user like, ugh. We're using a similar machine to this at my place, right? Yeah. Okay. Just the less cool one. Oh, really? So Dream's going to have a better server than me. Yeah. I'm feeling pretty inadequate right now. I love these tool-less sleds, man. When I built the first NVMe Wannic, it took fricking forever and I didn't even put all the screws in. Ugh. These are so much easier to deal with. I can't believe that every one of these is 30 fricking terabytes. I just installed 60 terabytes of SSD storage. Hold on. Here comes number 90. 90. Like, fricking what? I mean, we do have more. We can put more. Man, they're dense. Yeah. I was going to grab one of the CD6s, which are like the not enterprise ones. They are substantially heavier. These things are dense as hell. Here, David, catch. I can't. Hold on a second. No, I want to open it. What? No. Yeah. No. Yeah, chill. Jesus. Just relax. What's the worst that can happen? I lose the screws and we can't put it back together? Yeah. It's got a multi-PCB design. Oh my God. Of course it does. Oh my God. These caps, though. That's going to be the difference. These are their actual enterprise capacity. They're willing to say it's enterprise. This is for power loss protection. These capacitors ensure that if any data is sitting in a cache or in a DRAM cache, for example, it can be flushed to NAND before the drive actually powers down, even in the event of a sudden power loss to the server. That's freaking cool. And those are huge. Look at the little thermal pads. Oh, so do we have little thermal pads? Wait. They have cooling for the caps. That's amazing. Wow. I'm so happy right now. That's so much NAND. How much NAND is that NAND? Hold on. We got to find more NAND. We're going on a voyage. Oh, there's a clip. There's a clip here. No. Hold on. Hold on. Oh my God. This is so cool. I never get a chance to play with stuff like this. God damn it. Oh, there's another layer. Hold on. So they got thermal pads between. Oh my God, Jake. I think it's three layers. Jake. No. Stop. Please, please. I think it's three layers. I can't do this. I think it's three layers. Hold on. How do we get this layer? My heart. There it goes. Holy balls. Look at that guy. It's got four DRAM cache dies. What about that controller? That thing is huge. Wow. Hold on. Here it goes. Here it goes. Here it goes. Eight. Wait, there's more? Eight DRAM cache dies. Well, that's a lot. That's a lot. That's a lot. That's a lot. That's a lot. That's a lot. That's a lot. That's a lot. That's a lot. Well, that's the thing. Is the higher capacity you go for your NAND flash, the more DRAM cache you need. And if you're going to have 30 fricking terabytes of NAND, you're going to need a lot of DRAM. Eight gigabit, one G by eight. So it's one gig. So it'd be eight gigs, I guess. Eight gigs of flipping RAM. This SSD has as much as most people's gaming machines. There's a new update. I brought out the UniFi switch. We were kind of originally thinking we might build a whole rack and send it to him, but then apparently his installers are doing a bunch of crap. So we're just going to give him the switch. This switch has two 25 gig ports that we're going to aggregate together to be 50 gig for NVMe server. Totally, probably overkill for what he's doing. And then we're going to do two 10 gigs aggregated together for 20 gig to the hard drives. Then all these other ports are 10 gig to the rest of the house. So that means if for whatever reason they expanded and had like a whole team of editors there, they could have five people concurrently hitting this boy for 50 gigabits a second and this boy for 20 gigabits a second. That is a lot of throughput for game play. And if you really wanted, you could switch this to a four by 10 card and then you could have 40 gig. Each editor will get its own 10 gig. It's pretty sick. It's a nice little switch. You were going to show me something cool. There's a new view. Yeah. Hey, look, so they added this new port view. So now instead of having to edit a port one by one, you can go, Hey, I want to edit that one, that one, that one, that one, and maybe this one over here. Oh, that's cool. And you can set, bam, I only want this. Now security minded folks might put Mac address like limits. So only that one Mac address can connect to that port. It's a nice security thing. You can also see what device is connected to a specific port a lot easier. So I can just click on it. So I plugged the BMC into that one port and then I just click on that. There's the IP. Here's a cool feature for Ubiquiti. It'd be really nice to be able to take this go. Yes, that's the device that I want and just set a rule that only that Mac address can connect to that port just with one click here. Yeah. If you just go like a limit lock Mac address lock, that would be cool. I bet they'll do it. They're probably going to watch this and they're probably going to do it. Yeah, they might do it. Hey, guess what? It works. Now I got a VGA cable because I thought the IPMI was broken. Turns out they screwed up. I love you Gigabyte, but they sent a server that doesn't support Milan chips. No, it's a Rome server. It's a Rome server. Do we have any Rome chips? We do. So we had a 75 or 7402 P, which is a 24 core. Still going to be plenty for this. Yeah, for the way cut down drive configuration that we put in here anyway. Yeah. Okay. It's going to be great. Okay. So now we'll actually be great. Yeah. We're in true now on both of them. We're in true NAS core as much as I like scale core is a little more production ready. Yeah. And we want this to just work and him not to have to screw with it. And this is actually shockingly user friendly to use these days. Back in the early free NAS days, you had to kind of be nerdy enough that you could have basically done it yourself in Linux anyway. Now man, this GUI is so pretty. Okay. Line MTI, which one do you want to start with? I want to start with the one that definitely has a working drive in this slot right here. I will admit there are 60 functioning drives in the server presently. I will not admit that I don't think, or that didn't make sense. I will not admit that I don't think. I think that drive will die faster than the other ones. That's nice. But that wasn't the bet. I'll buy you lunch. Whatever. Yes. Why would you do that? It's fine. It's fine. Don't touch it yet. It's still spinning. I'm letting it spin. It's still spinning. I'm letting it spin. Wiggle wobbling. Fairly. I can't believe he bought a petabyte of storage. I can't believe you threw that hard drive. Can we just cut back to that for a second? Holy crap. I can't believe you did that. Okay. So we want some pretty chonktastic RAID zeniths. I'm figuring maybe not 15 drive V-Devs like we do. I was thinking to do 10 drive Z2s. Or I mean, for like performance, apparently the optimal is six drives, but that's a lot of capacity loss. Yeah, we'd have to do Z1s at that point and I really don't want to do that. So RAID Z2 is kind of like RAID 6, where you can lose two of your drives before you actually lose any data. But it means that two drives out of whatever the V-Dev size you have are lost in terms of capacity. So RAID Z1 sets aside only one of those drives for parity data, but that means you can only sustain one loss before your next loss is going to cost you data. So for Dream, you're thinking 10? Well hold on. I just realized we want to have some hot spares. Yeah. We don't want 10 hot spares. Why don't we do this? Why don't we do four hot spares, one for each bank, and then we can do seven drive V-Devs. Of RAID Z2, we're going to lose a lot of capacity. Why don't we go 14 drive V-Devs then, RAID Z3. So that would allow three drives lost per one of these rows with a spare. Okay, so if we do RAID Z3's 14 drives, we get 800 terabytes usable. I'm about to blow your mind. How much is each of these drives worth? Like 600? 600? Yeah, 600 bucks. 600 dollars, okay? Let's do some quick math real quick here, okay? 600, 1200, 2400, okay? 5000, 10,000. 10,000 dollars worth of drives is sitting doing nothing. There will be no data actually stored on them, okay? They're just holding data just in case these ones fail, or standing there ready to go in case these fail. 10 grand, that level of redundancy costs in a server like this. Let's call this big data. Big data. We're going to set it to be an SMB share. We're going to go into advance. I'm going to do Z standard compression because it's really great. Record size of one meg because it's all footage. Bam, now we can start stuff. Big data. So it works out to be 740 tibby bytes usable after formatting. That's some big data. All right, let's do the NVMe. We're going to do it pretty similar to the petabyte server where we have a hot spare, but we're just doing one and it's a RAID Z2, which means we have two disks of parity protection. I think that is so overkill. It is? With a hot spare, we need a RAID Z1. Do you want to get a phone call? Hold on. The problem with SSDs, hear me out, the problem with SSDs, they're not like hard drives where they kind of staggered fail. Usually when you have like 20 SSDs and they're all from the same batch, they'll just start failing at the same time because they're so much more dependent on like write endurance. It's better to be safer. He's got 100 tibby bytes. That's so much. That's more than we have. I know, but this is like 90 terabytes of SSD storage just not being used. Well, it's being used. Jake, I think it's fine. RAID Z1 and a hot spare. I know. They're enterprise drives. He's barely going to touch these compared to the workloads they're designed for. You know I'm not wrong. I see Wendell like through the camera. I'm calling him. Hey there, Mr. Wilson. It's me, Linus the Menace. How's it going? Well Jake and I are having a bit of a debate right now. We've got seven of these 30 terabyte Kyoksia drives and I'm not going to tell you who is in which position so that you're unbiased, but there are two potential configurations for them in TrueNAS here. Are you willing to weigh in on which one you think is proper? One of us wants to do a RAID Z2 with a hot spare and one of us wants to do a RAID Z1 with a hot spare. I would probably do a RAID Z1 honestly. It's a little bit more dangerous, but those drives are so fast and if you're on top of it, it's probably okay. But what if they're not on top of it? If they're not on top of it, then you've got to go RAID Z2. Do you really think it's likely that we're going to hit these drives hard enough that we should be concerned about their lifespan? If you got a hot spare on the flash side, I would probably just make it RAID Z2 instead of having a hot spare. It's only because flash wears differently than mechanical hard drives now. What about a RAID Z2 and a hot spare? Oh my god, Jake. Okay, all right, let's just go RAID Z2 then with seven drives. We need to put some footage on this thing. Nice. It's going through my workstation, so it's not going to be that fast. Wow, even through your workstation, it's doing 400 plus megabytes a second? You know what? I should go jump onto mine and hit it at the same time and see if that even goes down. The bottleneck is going to be your station, not this machine. Okay, well, I'm still copying at 400 to 450 megabytes a second. How about you? Okay, you ready for me to press go? Oh, you haven't pressed go yet? J.K., I already did. Yeah, that's what I figured. And I'm going at 530, 540 megabytes a second. Yeah, the one thing to consider is because we're copying a folder full of a bunch of little files, Windows will never really get there because it's like starting, stopping, starting, stopping, starting, stopping. But that's still super awesome. This is really important. Just because we have redundancy, okay, so we have that extra couple of drives that can fail, doesn't mean that we have backup. So if this is where they're going to store their active projects, then if something were to go wrong, let's say a power supply blew. I mean, theoretically, those are also redundant. So you could just, you could just yank one in the middle of operation. But let's say it blew in such a catastrophic way that it lit on fire. Okay, the server is dead, but somehow the fire doesn't spread outside the steel chassis and light the other server on fire. We need a backup. That backup is going to go automatically from this server to this one. Unfortunately, eating up some of our capacity. But not that much. It's going to be incremental. So each time it backs up, it's keeping what's already on there and just adding the new stuff. And it will cycle automatically. We can set it so it keeps like a daily backup for two weeks and it automatically deletes the old ones when it makes the new ones and doesn't store a bunch on here. And then as soon as they delete something from here, that will eventually fade out of those daily or weekly backups until the only copy that exists is the archival copy, which like us, Dream has said is not super essential. It's a nice to have, not a need to have because you can always go re-download it off of YouTube if it comes to that. That's your offsite backup, essentially. Thanks YouTube. Okay, I started the backup. It's backing up from this server over to this server and it's running at 400 megabytes a second. I mean, that's a lot of data. You can probably get it to go faster by disabling like secure ciphers on SSH, but right now we just had it in the default configuration just to show that it works. And since by default, this is only going to be a daily backup, it's got 24 hours to move whatever's on here. That's plenty enough. If you guys enjoyed this video, check out Kyocya's products. They've got everything from enterprise to consumer grade SSD products. We're going to have them linked down below. Also major shout out to Gigabyte who sent over the wrong server, but they did send a server so kudos for that. Micron for sending over the RAM, AMD for providing the CPU that we couldn't use. Thanks Gigabyte. And also 45 drives for sending over this epic store-inator that happens to color match the green tabs on here. Oh wow. This is a dream front plate. They can do like literally anything custom you want. Yeah, it looks amazing. If you guys enjoyed this video, go check out, I don't know, one of the other times Jake and I worked on a server thing. He'll find something. Petabyte of flash. Yeah. Oh yeah, we need to get back going. We got to finish that. But Jake is too busy helping me with my house. He moved in early. He's an in-demand kind of boy. Yeah.