 So I've got a problem. When it comes to building a storage server, there are two main options for your drives. You can either go with SSDs, which are compact and have amazing performance, or you can go with traditional mechanical storage, which has the benefit of lower prices and outstanding capacity with the drawback of being an order of magnitude or more slower, especially in highly random workloads, like if they were loaded up with a bunch of virtual machines, or in our case, if they're pulling network storage duty for a whole bunch of heavy users who are all trying to edit 8k videos at the same time. If only there were some way to merge the benefits of both with the drawbacks of neither. Well, there is. It's called the Apple Fusion Drive. And this episode is brought to you by iFixit. The Marlin screwdriver set from iFixit features five specialty precision screwdrivers. Check it out at iFixit.com forward slash Linus at the link below. The concept of tiered storage is nothing new. It's been commonplace in the data center for many years and has even made its way down to the consumer level in the form of Intel's Rapid Storage Technology or RST, which uses a small, fast SSD to accelerate a larger, slower mechanical drive and, more recently, Intel's Optane Memory Technology, which does pretty much the exact same thing, but with even faster Optane rather than NAND SSDs. The problem that I'm running into here is that most of the tiering solutions out there have one of two problems. They are either inordinately complicated to set up, requiring extensive knowledge of not just Linux, but even broader storage architecture, or they are expensive because you are paying the folks who have the aforementioned knowledge to create a turnkey solution for you. There are some exceptions to this, however, and I would like to explore a couple of them. So the popular FreeBSD-based FreeNAS software from iX Systems uses your system's RAM and its SSDs to cache frequently used data that's stored on your mechanical hard drives. But the issue is that ZFS's particular implementation is geared more towards accelerating performance of large databases and doesn't look like it's going to be great for our workload because the file sizes that we want cached that we're storing on our hard drives are so large, like these are 8K red video files, and there are practical limits to how much level 2 ARC that would be the SSD tier. I mean, there's also practical limits to how much RAM you can put in a given system. So then, the one we're going to try today is the built-in tiering mode for storage spaces in Windows Server, and what better way to test this out than to set up a tiered storage space on the network. So I'm going to use this test server right here, and then have our team of editors just switch over to it and see what happens. And it's at this point then that we wondered whether or not Optane would make a difference for some of the heavy tasks that we deal with around the office. No. No, it does not. Now, our regular editing server uses 31 1.2 TB NVMe drives, and it was really expensive for us to build, and they've been complaining that there's not enough space on it because we've only got about 24 TB of usable space after all the lost space to redundancy on it. And they were like, oh, well, you could just upgrade it. I went, yeah, okay. Technically, I do have another 17 bays that I could fill up, but that's going to cost me like seven grand, eight grand, even if I buy used drives on eBay. So I was looking for another way. So our test server here, the config is pretty straightforward. I've got four 10 TB drives, then because I want to see exactly how much cash you need. This is what really got me thinking about it. Intel uses just 32 GB of cash for large multi TB hard drives, and we've seen real world huge increases in performance. So I went, okay, let's try this. So I took two of their 900 series Optane PCI Express drives, and then I've got a total of 40 TB of mechanical storage, and I want to see if for a fraction of the price, we can get good enough performance that the editors aren't going to run into a bottleneck. So you can see I've got my four mechanical drives, I've got my two SSDs. Allocation is all going to be automatic, so I go ahead and I try to create a pool. Okay, I've created a pool. Now I try to create a virtual disk, create storage tiers on this virtual disk. It should automatically detect which ones are your fast drives, and which ones are your capacity drives. So we're going to use mirroring, because that way we're protected from a drive failure. We're going to use fixed provisioning, and now we get to specify the size of our faster and our standard tier. We're going to go ahead and select the maximum sizes. Failed to create virtual disk, not supported. So the issue is that the GUI version of storage spaces doesn't have some very important options, like selecting how many columns you want. So when you only have two drives and the default is three columns, it spits out this stupid error. How did nobody at Microsoft notice this for years? Anyway, doesn't matter. I called in a lifeline, LeadEater over on the forum is a guru when it comes to this stuff, so I'm just going to pull up his DM over there and follow his instructions to hopefully do this in PowerShell. So this will be fun. You guys are going to be learning right along with me. I haven't done this before the video. So I can still use the GUI to create my storage pool, apparently. Go ahead and do that. Then I need to fire up PowerShell and change the default number of columns. Okay. Well, I don't know what I just did, but I'm going to change it to mirror number of columns default one. I think I think we did that. Let's find out. So I think the command I just entered changed the mirror layout default to one column. It did not. Okay. Okay, so I'm using PowerShell ISE now and this seems to be working better. Mirror number of columns default one. Come on! So mad right now. I just want to change a stupid setting. It's definitely called mirror. It's definitely called storage pool one. Like I create these commands using their own thing. We'll figure this out at some point. The good news is there's a backup plan. Swear I had this working at one point. What is this? Oh, okay. Oops. Actually, yeah, sure. Oops. Close. All right. Here we go. So right now we've only got a 10 gig link. I can put a faster card in here if it ends up being a networking bottleneck. But for now what we need to do is grab a bunch of just like video projects that are actually current and copy them to our tiered storage drive here. The idea is that I want to bring over more than would comfortably fit in the cache and then I want the editors during our test to open up first a whole set of completely different projects from each other then close those open up a new set go back open the other ones and evaluate the performance for themselves. So we're back and we settled into a steady transfer speed of anywhere from 200 to 300 plus megabytes a second and all of our projects are over on here. So I guess all that's left for me to do then is go interrupt the editing team. No, no, no, no. So I just need to know if you guys all hit this and try to edit 8k footage at the same time. Will you notice slowdowns? Because I know some people are sensitive to slowdowns. So here's the idea. I don't even know what a network share is. This is the network share. But I don't know what that is. So everyone kind of pick a project and open it and start doing video editor things. Whatever those are. Work the way you would normally work. Because full is going to be affected by like your system as well. So the idea is that once it arrives, it gets cached in the fast tier. So I just wanted to get a feel for what this is like. Ed, are you opening a project? Oh, you're encoding. What you're encoding it? Oh, exporting probably isn't a very stressful test. Okay, what are you doing, Taron? I don't know. Can you try doing some more editor things? Pretend to edit this. I'm moving stuff around. Keep pretending. Pretend to do your job. You mean like the normal thing? Yes. Can you go back and play that sequence again? And see if it's any better. How fast would you expect it to be? Like, is this normal or? It's a little bit faster. It's a little bit faster. It's not playing. Okay, just hold on a second. I haven't given up on saving money yet. So the thing I want to find out now is if you guys keep doing stuff, will it get any better? So I'm starting to come around to the realization that maybe tiered storage solutions aren't grab and go. Because in every environment, you'd want your tiers to behave in a completely different manner. Like if there was some programmability in storage spaces where I could say, okay, take all the newest files that come in and assume they are important until such time as they do not get accessed for a very long period or until such time as something new comes in and bumps it out. Then it would be less likely for those guys to open up a project and have it chug until storage spaces goes, oh, this is important and can promote it from the slow mechanical tier. So it's possible that if I had a much, much larger fast NVMe tier, we wouldn't run into this issue because this is really a very heavy load for four hard drives and two SSDs. But the problem is that that would require me to basically build that entire server without even knowing that it would work. So you are dropping... I'm dropping 90% light frames. Can you rate your editing experience out of 10 for me here? There's like little moments of seven out of 10, but then overall it's more like a three out of 10. Sorry, you're saying this is better than Wannock? A little bit, maybe. All right, everyone just go back to Wannock, un-map the drive, throw it all in the garbage. So that's pretty much it. I am coming to the acceptance stage in the stages of grief here, where I'm realizing that if we want more capacity of our high performance network storage, I'm just gonna have to pull out the wallet and make it happen. Thankfully, to take some of the sting out of it is our sponsor for this video, Squarespace. With Squarespace, you can make your own beautiful website without the hassle. They're all in one platform, makes it easy to get up and running quickly, and they've got all kinds of great features, from their logo designer to their built-in e-commerce, to the ability to publish in Apple News format, so that your blog will be available immediately to millions of potential readers worldwide. And they offer all kinds of support, so you can build the best website you can build, whether it's their seminars or contacting their 24-7 tech support via live chat and email. So don't take my word for it. Go try out Squarespace at the link in the video description below, and when you decide to sign up for real, go to squarespace.com slash LTT, and you can get 10% off your first purchase. So thanks for watching, guys. If you disliked this video, you can hit that button, but if you liked it, hit like, get subscribed, or maybe consider checking out where to buy the stuff we featured at the link in the video description. Also down there is our merch store, which has cool shirts... It's winter. Like this one, and our community forum, which you should totally join.