 We have a desperate lack of internet because we are about to host over 200 gamers over at Lab 2 for the whale land party. And we are stuck with an internet connection that is slower than the one at my freaking house. I mean, okay, fine. It's not that slow. But still, two gigabit down, split up between 250 people. That's only like eight megabits per person. That's one megabyte a second. You're not gonna be downloading Warzone if you're on that kind of speed. Fortunately, we have an ace up our sleeves. Oh, it's a big sleeve, big ace. Because today, with the help of Intel, who sponsored today's video and a portion of our whale land party, along with Super Micro who sent over this server, ubiquity, I guess, gets a shout at infinite cables, we are gonna build the ultimate Steam Battle.net origin Windows update, Epic game store, any game you can think of caching server to give us local high speed copies of as many games as possible. And we're gonna test this thing with over 200 gamers to see just how hard we can push it. It's gonna be freaking epic and steam and good old games and Battle.net. Damn it. That's not in the script. ["Skyrim 2.0 Remake"] Caching game downloads and updates on your local network is nothing new. The basic principle revolves around DNS or domain name system. So when you go into your browser and you type in any website, let's say lttstore.com, your computer contacts a DNS server which converts that text-based URL into the numerical IP address of the website that you want to go to. That way your machine knows where to send the request. Now the caching solution that we're gonna be deploying today sits in between the DNS servers like Google's 8.8.8.8 or CloudFlare's 1.1.1.1. And it will actually redirect those requests to the NGINX web server software that is running on it. NGINX then checks the request against a list of cacheable content. In our case, Windows updates, Steam, Origin, Battle.net and basically all the other game stores. And if it matches, it will cache any incoming content from those stores onto the machine storage. That means that the next time someone wants to download that same thing, say at a LAN party where everyone wants to download the same game at the same time, NGINX will automatically redirect that download to the local cache of the game rather than download it from the internet. Now for a home gamer with a single system, it's not gonna matter much as you likely aren't re-downloading the same games very often. But when you've got 200 plus systems on a small connection, the benefit is huge. Now, if you're an OG, you'll probably remember the video that we shot deploying a solution like this in our office nearly four years ago. But quite a few things have changed since then. For one, the process of deploying a Steam caching server has gotten a lot easier and a lot more robust. Gone are the days of running a separate Docker container for every service that you intend to cache. It is all built into one with a sweet piece of software called LAN cache, creative name. And two, this time we actually have 200 plus people and their computers at an actual event where we can coordinate a test of our caching server and see what sorts of wild speeds we can push from it. We're getting a little bit ahead of ourselves here though. Why don't we talk a little bit about the hardware? This is the Super Server SYS 510P-WTR from Super Micro. It's actually a fairly simple and modest one-use server. Up front, we've got four three and a half inch bays, which by default are configured for SATA drives, but can be upgraded to SAS or NVMe like you see here. These are gonna be spicy. And then around back. And then around back, it's equally boring. There's the two full height and one half height PCI express Gen 4 by 16 slots alongside dual 10 gig networking management, some USB and dual redundant power supplies. Let's pull one of these boys out. Whoa, those are long, but only 500 watts. That sounds pretty weak these days, but it's actually perfectly reasonable since this system is only designed to house up to a 270 watt CPU along with the four storage drives up front. There's no GPUs or anything like that since all we need to run our game caching is a beefy CPU and storage. So it's perfect for our use case. The machine came from the factory at Supermicro with an Intel Xeon Gold 6314U Ice Lake processor. That's a 32 core, 64 thread, 205 watt chip. It's not the beefiest CPU that you could install in this machine by any stretch of the imagination, but it is plenty for what we're trying to do since the caching process is actually fairly efficient. We also have eight sticks of registered ECC memory. And since Linux, our operating system of choice here, will cache our frequently accessed files in system memory, our best bet is to cram this system full of as much RAM as possible to take some of the load off of our storage drives. Fortunately, we've got a whopping 256 gigs of 3,200 megatransfer per second SK Hynix memory to fill up all of the available slots. The truth is our RAM is not really gonna be a major factor for us because we went so flipping overkill on our storage. Originally, our plan was to throw four of Kioxia's ultra badass CM6R 30 terabyte Gen 4 NVMe drives in a software raid zero. Unfortunately, we were on a bit of a tight schedule getting the server here in time for the LAN. In fact, we are shooting this on a Wednesday, the LAN party starts on Friday. And because of the rush, a small oopsie was made on our order and while the sleds say NVMe, the back plane that it's connected to is only plugged in via SATA. So they overnighted some cables. We should have those for the actual event, but in the meantime, we had to get a little bit creative. We're gonna have to wire up just two of these SSDs for now. Oh God, really? This is how we're doing it? Oh, and then here's the other one, it gets better. It's not that bad. You know, I feel like we had a real role reversal moment here. Yeah, yeah, yeah, get creative. Here, look, you can take one of the back plane, Mole X is out there. And then I got a little long boy. Yeah, it works great. See this, look at that. It's mint, like from the factory. What Jake just handed me is a PCI express by four to U.2 adapter card, which is fair enough. Then we've got this other one, which is a PCIe by four to mini SAS HD, but the PCIe version of it, that then goes to U.2 and then plugs in with this Mole X connector to the power that used to go into the back plane. Jake! Don't yell at me, yell at Super Micro. No, I don't want to yell at Super Micro. They did us a big solid- Put that in the- Yeah, doing this for us. So I don't even, whatever, Super Micro, you're chill. You're chill in my book. Oh God, oh God. No, it's not in the slot at all. Well, it's in the spot it needs to be there. There we go. Yes! Mission success. How many LTT people does it take to put in an expansion? Couple. Three. Two to do it and one to film them. Ha ha, cause we make videos. A few, yes. Yeah, that's a joke. Oh wait, we gotta put some screws in that first. You want screws in it, what? Here, I got some screws. What, you don't even need- No, no, put some screws in it, seriously. You don't even need screws. That's like a $10,000 SSD. What? They're 30 terabytes, what are you- I thought you, but the script said we were going to use them. So we are using them. Which one? The 30 terabyte Kyoksia's. Yeah. Oh, bloody hell, Tate. What, I don't understand. Well, no, the script said we were going to use them. We were going to use four. And it sounded like we were- We're only gonna use two. We're only gonna use two for the whole LAN? I mean, I guess they're fast. These things read at like seven gigabytes of second. I mean, that's fair. That's really, really fast. How much network would we need for that? And you got to remember that when you're downloading games, that's a highly sequential activity. Yes, having multiple users hitting them at the same time is going to add some randomness to the requests. You suck. But it's still relatively sequential compared to if we were running this as a database server or something like that. Just do this for you, it's fine. Okay. I mean, that is like your job. Yes, all right, fair. We do have one small problem. It's cool that this server has dual onboard 10 gig network cards. That would have been sufficient if we were gonna use it just internally here. But since we've got 200 people hitting it, oh, and since Ubiquiti provided an enterprise XG24 network switch, which happens to have dual 25 gig uplink ports. Well, I think you can see where I'm going with this. No, you can't. It's right here. This is a dual port 25 gigabit Nullinox ConnectX 5 card, which we should be able to link together for a whopping 50 gigabit combined throughput. That's really spicy. Wait, why did you put this in the... I don't know. You know, we should update your job description to do things for me properly. We don't do things properly here. This is Lime's Mini, you know? No, no, no, we do things properly. I mean, this is gonna work, right? I mean, hopefully I haven't tested anything, so. You haven't? We already pre-cached three and a half terabytes. Nice. But they said we should pre-cache as much as possible. So we're gonna try to pre-cache like 40 terabytes. I have never used command strips before. Okay, so this side's peeled. Really? Yeah. Okay, well. I just put holes in my walls like an animal. If we were gonna be deploying this long-term, then it would be better to use a NAS operating system like TrueNAS. But since we are just using it for the event, we're throwing Ubuntu Server 20.04 on it. You wanna boot it up? I do. I'm convinced this guy likes tinkering with gaming technology like this more than actually playing games. Whoa. Let's see if he's any good when we get there. Dude. Dude. You played MW2 with me. Dude. I smacked everybody, it's fine. Played a lot of MW2. No one calls it MW2. I do. When you have to say it so much you actually play it so much. It's not even faster. M-W-2. MW2. Modern Warfare 2. It's the same number of syllables. Yeah, but say it. It doesn't even save any time. MW2. Modern Warfare 2. That takes way longer. Marginally longer. Once the OS is installed and we're SSH'd in, it's best practice to install any available updates. Then we can go ahead and install Docker IO and Docker Compose. These are the only prerequisites required by LAN cache, the pre-made caching solution that we're gonna be using that comes nicely bundled in a Docker container. Before we download LAN cache though, we're gonna wanna set up our SSD RAID 0. DigitalOcean has a really nice tutorial for doing this on Ubuntu server on their website. So we're just gonna copy past the commands from there. Well, let's see if our array imported that I definitely didn't make earlier. It did not. Oh, it did. Look. So I've already cached 3.3 terabytes. Okay. Of 56. RAID 0, baby. Do we even have enough of this to... Do we even have enough time to fill that? Maybe here. With our storage ready and Docker installed, we can run the download commands from landcache.net, edit the config and we're off to the races. Now, editing the config is where things get a little trickier. It's not that bad. Well, it's trickier than just copy pasting. Yeah, well, the main things you have to do are set your system IP. This is so LAN cache actually knows what IP address you're using. If you don't, I think it actually binds to all the IP addresses and it might still work, but probably just set it anyways. Then you're gonna set your cache route, which is where everything gets stored. Yeah. You really wanna put this in the right spot. Do that. If you don't put it in the right spot, it's just gonna fill your boot SSD and you're gonna have a really bad time. When it gets completely full, your system will probably go super unresponsive to the point where like it will take forever to enter requests. And even I've had that happen before. When you have zero bits of free space on an SSD, it craps itself. Sometimes you can't even uninstall stuff because you can't like write the log file for the uninstall. It's really bad. Bad time. Then you wanna set your cache size as well as your cache index size for the RAM. And what was the ratio? One gig to- 256 megs per terabyte. Per terabyte. Okay, so we've got enough. I set up the equivalent of 60 terabytes. We will need to up that when we add the other drives in. The last thing you'll wanna do is set your time zone. It's not mission critical, but yeah. It's nice for logs. So if you're like having issues and your logs are in a different time zone, and you're like, what the hell is going on? We set that to Vancouver. That's where we're at. Okay. That's all we gotta do, man. We can just click start. Systems on our network won't magically start using the cache. They don't even know it exists, right? So we actually have to go on the system and point it to it. And the easiest way to do this, if you're just testing, is to go into the settings and go enter your IP address. Don't put 8.8.8 in there. That could confuse things. Just put your LAN cache. It will forward everything else. Unless it doesn't. In which case having a secondary one would- A lot of the times, Windows doesn't like properly fail over anyways. Are you serious? I don't know, whatever. Fine. So remember when our internet went down the other day? It's cause the Steam cache was offline and the internet was fine. Just nobody had DNS resolving. And in the DHCP server, I had 8.8.8 is option two, but they just, Windows sucks, okay? If you're doing this as an actual deployment, you'll want to set that IP address for the DNS server in your router. Otherwise, you know, setting it on all the different computers individually is gonna be an absolute nightmare. Especially if the IP ever changes. Okay. It's set. We're good. I'm firing up Steam. Right. Disk management. Yeah. I put a seven terabyte in here. Oh. Wow. You'll have to delete that. It's probably like a ZFS part. See you later, buddy. Yeah. Put D. We'll call it big D. Big D. Why are you in offline mode? Why is it in offline mode? You don't have to, when you go to install the game, it'll prompt you. What the hell? It's confused. What just happened? What? What even just happened? We had internet. We had all the internet. We run out of internet. Where's my series of tubes? Darn it. Damn it, Dale. Yeah. This is sponsored by Intel. We should have an Intel NIC in there when that actually works. If we use the Intel NIC, it would probably be fine. We should just go get one. This is not working. Oh, it's so cute. Hey, Jake, have you seen? Oh, you spoiled it. What? Well, I was trying to ask you on camera. Yes, I have seen. You've seen what logistics did to all of our Intel NICs? They put these adorable little knock-downs on them. I sort of instructed them a little bit on it. I love it. Okay, let's just leave it. We'd have to install drivers. Come on, let's go. Steam, steam. Screen record. Oh, God. Here we go. This computer. Gotta get my... You don't need to do it. Just click install. No. Click install. I won't. I have to add one. This is a nice UI. Oh, really? I hate it. I mean, it looks pretty. I don't know if it actually is. Yeah, the old one was simpler. Wow. You totally could have just done this. No, I couldn't have. It wouldn't have been here. Yes, it would. It'll say, do you want to install a new library? I don't believe you. Okay, go. See, look, quick. It's gonna go too fast and you're gonna miss it. Hold on. Where is it? Where is it? That's not that fast. Yeah, that's not that fast. This is like... Oh, did you remove the DNS? No. I think you might've. No, it's there. It's not using it. No, that's it. Yeah, it's configured properly. Just remove it and then add it again. IP config slash flush DNS. No. Oh. Word slash flush DNS. No. Flush. What? Are you drunk? Go to space, buddy. Flush DNS. Oh. That's one word. There you go. I think you said slash slash. No, I said slash. I was like, what is slash slash? No, I said forward slash. No, you said slash slash. Whatever. Anyways, okay, okay. Let the record show that I need to leave. What is he talking about? Go to Steam, damn it. Oh my God. Why do you have no connection? Oh my God. Plug into the Intel card. You don't even have drivers? What difference does it make? Well, I can get them. I can't even. You're plugged into the Intel card. I can't get drivers. How am I supposed to get drivers, smart guy? 2.5 gig. Man, this is plugged into 10 gig. Why don't we just go to my computer upstairs, which is also plugged into 10 gig. That is legitimately an option. If this doesn't work immediately, that's where we're going. Okay, fine. I'm glad this is gonna be such a pain-free experience for the land attendees. Oh, it will be. See, like, the CPU in this system is like, oh my God. Yeah, we had a 12900KS downstairs, which is what we really wanted to use, because that thing is fast. Look, my SSD is just, 960 Pro is just at 100%. This is as fast, look, it's right at 1.2 gigs a second. Something to consider is that not every game is set up exactly the same in terms of the way that the files are compressed. So some of them are more CPU heavy, some are less CPU heavy. Counter-Strike Global Offensive might not be the best benchmark, but we are getting 230 megabytes a second right now. Oh, I do have a way to demo this, that's more better. So the SteamCache pre-filling thing just downloads to RAM and then just deletes it, right? Because you don't wanna download the things for pre-filling, right? That doesn't make any sense. No one at the land is gonna need to go any faster than that anyway, because they're gonna have two and a half gig max to each station, right? No, gig only. Only gig? Oh, well then. Here, let's try Black Ops 1. This one is historically pretty good download. 3.4, 4.5, 5.3 gigabit? Woo, 5.4? Now we just have to see how it scales to 200 people. It's pretty much done, six gigs. Well, no, the data is downloaded. What it's doing right now is probably just unpacking it, because it's like, yeah, we would have done that on the fly, but you downloaded me too fast. Way to go, hot shot. Your goal then is to start this. No, I just need to select one game, and then it'll already be cached. Arc Survival Evolved, enter. Look, it's already gonna be cached. This is gonna download at 10 gig, like on the dot. You think so? Yeah, look, nine point, there you go. The cache, because this is already cached, and this is just downloading to RAM, there you go, nine gig. And the CPU is at what? 18% only. Whatever. This thing's gonna be able to do like... Probably 30? Yeah. Maybe 40? That's outstanding. Yeah. So stuff went down. The setup here is absolutely wild, but we're finally ready for the big test. Check this out. We had to build a router, okay? The router was only one gig capable. We have a two gigabit connection, so this is just made of another server. Got our surveillance set up, got our main switches here, and this is it, the SteamCache Clayton server. We're ready for the big moment. Okay, so we got 150 people who ended up coming to the LAN, and we need them all to go right now. One, two, three. Whoa, that is a lot more than the two gig internet connection we've got. Freaking awesome. This thing absolutely saved our bacon. And the most wild part is that the CPU usage is not even that high. Like, I'm pretty surprised at how easily this handled this, even with just a couple of NVMe drives. Like, it's barely even working at it. Freaking love this. 261 megabytes per second. Wow. That's mine. Get over here. The speed, though. Thanks, buddy. Thanks, Intel, for sponsoring this video, and thanks for providing the SteamCache. It absolutely saved the day. Everyone was able to download games. I mean, I was downloading games at two and a half gig, basically, the whole time, because I brought myself a little USB-C two and a half gig network adapter. But everyone else was probably at one gig, for the most part. Absolutely no hiccups at all. If you guys enjoyed this video, you can check out a classic where we built ourselves a PF SenseBox way back in the day, but one that was designed for gigabit internet, not two gigabit internet. It was a while ago, but the principles are still pretty much the same.