 11.420 gigabytes per second. Wow. We just copied a hundred gigabytes of data. And what was it like 15 seconds? Looks like 15, 16 grand, depending on what way you want the air to flow. I'm really glad we bought it on eBay. Okay then. It was back in 2018 that we took our first dive into the insanity that is 100 gigabit ethernet with a direct connection using a direct attach cable like this one between two systems, thanks to some network interface cards that the folks over at Mellanox sent. That yielded results that were fairly unimpressive actually due to limitations in both Windows file copy being single threaded and our lack of support for RDMA, a technology that can greatly accelerate file transfer speed over the network and efficiency. Well today that changes because not only do we have our shiny new 100 gigabit switch, we've also got some shiny new fiber optic cabling, a bunch of both 100 gig and 25 gigabit network cards, and we've got both RDMA and multi-threaded file transfer working. Things are about to get very interesting for our videos. Ridge Wallet wants to redefine the wallet with its compact frame and RFID blocking plates. Check out how they can keep your wallet bulge down and use offer code Linus to save 10% and get free worldwide shipping. (*upbeat music*) Performance issues aside, any networking setup where you're relying on having the systems connected to each other is a pain in the butt to set up and manage. So any kind of practical high speed deployment starts with the switch. This is the Dell S5048F-ON. It's actually pretty similar to the S4048T-ON switches that we already have in our server room, except that instead of 48 ports of standard RJ45 that runs at 10 gig with six 40 gigabit QSFP pluses over at the end, this one has 48 ports of SFP28, which are capable of 25 gigabit each and then six QSFP28s that can run at 100 gigabit. We just ran the fiber cabling that we're gonna need for all of our editors to plug into these ports, giving them 25 gigabit per second connectivity. As for the six 100 gigabit ports, we can use these to link to our NVMe storage server, which means theoretically we should be able to do 100 gigabit per second off of that server all freaking day. In fact, we could probably even do 200 gigabit. We've got so many of these. We could like aggregate them and use a dual port card or something like that. Let us know if you wanna see that. Around the other side, we've got dual redundant 750 watt, 80 plus platinum power supplies in a switch that doesn't even have POE. Man, this thing's a chunky boy. But realistically, the reason we need it is to run the fiber optic transceivers that physically light up the laser that shoots out of here and goes into the neck on the other end. And also to handle processing, 3.6 terabit per second of full duplex processing in a single one you rack. Well, yeah, okay, 750 watt power supply. I get it. Conveniently, this came out of the box with the correct airflow orientation. You can actually have it so that the air goes in the back and out the front or in the front and out the back. And it's just like a part that you can buy on Dell to switch it around. Should we open it? We gotta open it, right? Let's open it. We technically have a warranty, but that's never stopped you before. There we go. Ooh. Ooh. Holy bananas. This thing's got upgradable sodium slots, boys. 16 gig M-SATA drive up in here. Got a couple, what is this? Eight gigs of DDR3 memory? Should we throw another eight gigs in it, Jake? I don't think so. I don't think so. But I mean, it must have expandable memory for a reason. Is this heat sink not beautiful? Yeah, I think it's pretty cute. I wanna pull up our video on the other switch because I feel like this one has less cooling in it. It does. It has way less cooling. Because the RJ45s are what get hot. Actually, it's funny you mentioned the RJ45 one using more power. So my understanding is that there's actually more processing involved on the switch side of things for RJ45. And that's one of the reasons that RJ45 switches are more expensive than optical ones where a lot of the cost ends up being, there you go, in the transceivers that you have to plug into them. You guys ready for liftoff? Reeeeeeee! Yeah, we're gonna unplug that for now. Let's talk about the machines we're gonna be running first. One of them is running a 3990X, this one right here. The other is a 32 core Threadripper Pro. We finally got our hands on one. So make sure you're subscribed if you wanna see the video on that. For now, let's set up this machine. Our 3990X is gonna be acting as our file server. It's running Windows Server 2019. And for its storage to make sure that it can actually keep up with 100 gigabit, we will be using a honey badger. If you're not familiar with this puppy, it's from Liquid. And it's an eight slot NVMe card full of four terabyte M.2 drives from Sabrent that's capable of nearly 30 gigabytes per second of sequential read speeds on its own. This way we can avoid any storage bottlenecks in our test without resorting to a RAM disk, which as we discovered because the software is so janky, actually will not alleviate bottlenecks at all at these speeds. Also, since this is the server, we're gonna go with a 100 gigabit connection to it. That's something you don't see every day. We've got our storage and networking plugged into the 16X slots. And then our graphics card is limping along in a, I think this is an 8X slot, isn't it? Yeah, it is. Oh, well. Because we're going over such a short distance, it's a little more cost effective to use direct attach copper cables. So we're gonna use that to go from our 100 gigabit network card to our 100 gig port on the switch. Now we're gonna change gears. This is, oh, this is two, one moment please. Fiber optic cabling. If you get the really baller stuff with like corning indestructible glass inside them, they're not that fragile, but otherwise they can actually be very fragile. We got our cables from Infinite Cables and our transceivers from fs.com. And you can actually see they're slightly different part numbers, I guess, DE. So this is gonna handle 25 gigabit connectivity from our switch to our workstation. We're using this Threadripper Pro workstation from Lenovo to kind of stand in place of what we normally use, which are Threadripper 3960Xs on just like regular Asus gaming motherboards. Start with the honey badger down here in the bottom. This is why it's so important for workstation use to go ATDT rather than with a regular desktop processor, even if you don't need more than 12 or 16 cores. It's because it's all about that connectivity. High speed, baby. Now we're starting with 25 gigabit because this is the use case that we're actually testing for and validating. But before this video is over, we will put a second 100 gigabit card in our workstation here. Now that the switch is fired up for real, we're running at 25 gigabit per second on our client and yep, full fat 100 gigabit. But there's another step. Check this out. If I go into the system info here, you can see we're not running regular Windows 10 or even Windows 10 Pro. This is Windows 10 Pro for workstation. Now that doesn't mean that you're gonna get better performance in Adobe Premiere or Cinebench because it's for workstation. And in fact, it doesn't even require a reinstall. It's just a software key unlock. What it does have though, is some workstation specific features that we need, most importantly, SMB Direct, which is Microsoft's implementation of RDMA over ethernet. RDMA or Remote Direct Memory Access is a technology that allows you to read and write to the system memory of a remote machine without interrupting the processing of the CPU on that system. So it essentially cuts the entire network processing stack, kernel and CPU out of the data transfer equation, meaning that you can do faster transfers at lower latency and at least theoretically, with barely any CPU usage on either system. Now this thing used to be a total maze to set up like a flippin' nightmare. But with the addition of SMB Direct to Windows Server and now to Windows 10 Pro for workstation, it is theoretically plug and play, no configuration required. I guess we will see about that. I'll believe it when I see it. On our server machine, we've pre-configured a Windows storage space in its own special RAID zero type mode here. Is it fast? The answer is yes. We're gonna change that to gigabytes per second. Oh yeah. So it's really nice compared to the stock Windows dynamic disk striping, which was actually deprecated ages ago because it's actually multi-threaded, meaning that it can usually handle much higher speeds. We've got a network share set up on this puppy, so let's go ahead and head over here. WANUC 3, users. There's a big fat 20 gigabyte file. Let's transfer it. 1.47 gigabytes per second. That's well shy of 25 gigabit per second, but part of the problem is that Windows file transfer is single threaded. Now there is a multi-threaded version called RoboCopy, but we're gonna wait till we move to 100 gigabit to try that for you guys. This is reading from the server. We're seeing about 1.45 gigabytes a second, but let's go the other way. 2.7 gigabytes a second. It's flipping nuts. Oh no, we dipped, we dipped. We were at 2.3 gigabytes a second. Oh, this is pleb. This is pleb tier now. Unfreaking real. And get this, if we run crystal disk mark over the network, it's multi-threaded. Let's have a look. Just do the sequential. Okay, here we go. Oh, actually, oh, this is really cool. So I'm gonna show you this. Now normally when you're doing this kind of like crazy amount of data movement, your CPU usage should light up, but here we are doing 1.1 gigabytes a second and it's barely freaking touching it. Let's write to the server now here. Give it a sec. 2.8 gigabytes a second. Oh, it didn't even blink. Even more mysterious though, is the fact that we're running crystal disk mark here and our networking usage shows basically nothing. That is because this transfer actually bypasses the traditional TCP and UDP network stacks, which Windows task manager monitors. So the only way for us to see if RDMA is even working is in a different application called resource monitor. 2.2 gigabytes per second. There it is. Oh, these are probably gigabit actually. Bites per second. Are they gigabytes? Yeah. Okay. And then outbound, there it is. Three gigabytes per second. Where's my other 100 gig card? Is this it? Yeah. We don't have a PCI back plate on this one, but that's okay. Should still work fine. Just be gentle. Yeah. 100 gigabit time, crystal disk mark. Here we go. Oh! Excuse me. 11.420 gigabytes per second. Wow. Okay. The right speeds. No, no, no, not wow. You're supposed to say, holy. Yeah, holy. What the? Ball. Ball. Wow. Ball, ball, ball, ball, ball, ball, ball, ball, ball, ball. Now let's try Robo copy, which is command line only in spite of the fact that it's been around forever, but that's okay. You can get GUIs for it. This one's called Cho-Easy Copy. Easy with a Z. I love you developers. You guys are hilarious. For comparison though, naturally we should run a regular Windows file copy first. So just shy of two gigabytes per second copying from our local machine to the server. That's well shy of a hundred gigabit, but like, dang, it's not like it's slow either. Now let's go the other way from our local machine to, or from the server to our local, oh, God damn it, whatever. It doesn't matter. Going the other way, we get around two gigabytes per second. Enough of that. Let's bring out the big guns. Cho-Easy Copy. Making sure it's running with multi-threading on. Yes. And go. I am expecting some big boy number six. Wow, did that just peak at six gigabytes a second? Just four now. Holy crap. We have already transferred 51 gigabytes. What is even happening here? What is even going on? We have transferred a hundred gigabytes. A hundred gigabytes. It's done. It's not done. It's not done? Well, we've got a hundred gigabytes. It's like pre-allocated. It's almost done. You can see it in the. Oh yeah, there it is. 80%, 90%, Boom. We just copied a hundred gigabytes of data. And what was it? Like 15 seconds? 20 seconds, something like that. 20 tops. Can we try 16 threads? Cause that would allow us to copy all of the files. I tried it and it wasn't better, but sure. Try it. Not better? Oh, let's just send it. 4.2 billion bytes per second. 4.2 gig. Okay. So yeah, not appreciably better. But better than Windows File Transfer. Definitely a lot better than Windows File Transfer. And if we were on Linux, we probably could see full 100 gigabits. So about, what would that work out to? About 12, 12 and a half gigabytes a second? Yeah, something like that. Man, these stations are gonna be like real snappy. Now we're gonna go the other way, copying from this machine to the server and see what we're looking at here. Holy. 14 gigabytes a second? What? No, no, it's only wait. Oh yeah, I was looking wrong. Okay. Yeah, yeah, the 14 million was just, that was the inbound. Okay. I was like, what? Yeah, about 2.7 gigabytes a second. So we're definitely limited somewhat. At least our CPU usage is basically nothing, which is great news for me because it means we don't have to go buy a bunch of 32 or 64 core thread rippers for the workstations. 24 is gonna be fine, just fine. Should we fire up a premiere timeline? Do we have it on here? Yeah, we can if you want. Inbound bytes per second, actually only about 250 megabytes per second running at full resolution, using about 66% CPU. This is red 8K footage running at full res in Adobe Premiere, which is ridiculous. You would never do that. Wow, the timeline is super responsive with BRAW. Wow, that's not too shabby, is it? Wow. What we're looking at now is a mix of 8K and 12K BRAW footage. Okay, so that's well in excess of one gigabyte per second. Of course it doesn't show up here, but poor CPU. Even at one quarter res, this thing is chugging. So let's go ahead and scrub around on that timeline. Holy crap. Okay, we peaked at three gigabytes a second. Yeah, it's not bad. Realistically, what we're after with this is not necessarily more throughput to this workstations. It's more about latency, because what happens is if you have any kind of latency spike on the server side, not just one, but all of the workstations are liable to crash at the same time. And Wannix server was good for a long time with our 10 gigabit RJ45 connection. The problem is that we've added so many more editors that we're starting to run into trouble again. So this is hopefully the last solution that we'll need to that problem. But realistically, probably not. What's in your online security toolkit? Adding a VPN lets you mask your IP and encrypt traffic to and from your devices. And private internet access has reliable service with thousands of servers in dozens of countries. They offer no bandwidth caps, configurable encryption with a kill switch to keep you in control of your connection. And when combined with private browsing, PIA can make websites think you're in a different country. Try it out risk-free with their 30-day money back guarantee at the link below. You can connect up to 10 devices at once with clients for Windows, Mac OS, Android, iOS, and Linux. What are you waiting for? If you guys enjoyed this video, maybe check out... The first time we tried to do 100 gig? Actually, you know what? Go learn more about the Honey Badger. This thing's freaking cool. It's sick. Check it out.