 Now we've yet to actually boot this motherboard with six graphics cards. This could be the thing that kills the entire project. (*beeping*) Crap. The whole system restarted when I bumped that external PCI express cable. My level of pissed off right now is pretty high. That's not what I was expecting. (*music*) Storyblocks Video offers you studio quality clips at a fraction of the cost. Check them out today at the link in the video description. So we found a pattern. VM2 and VM6 are not working. And honestly, I'm at the point now where all I can really do is give up on two and six in their current configurations, and we're gonna try swapping out the graphics cards because we finally got an error and it was something to do with the onboard audio controller for the graphics cards. Makes no sense. Never seen this before on any Pascal card, especially a Founders edition. So here's a Titan V, which is a good choice because this is what we'd like to use for the final build anyway, because it has single slot I.O. Although we have to convince Nvidia to send us six of them and they're like, this is not a supported configuration for the Titan V. I'm like, yeah, yeah, I know. People are gonna know that. Anyway, our Quadro systems are working fine too. So we're gonna swap out our 1070 and our Titan X, which conveniently are right on the top of the pile here with these two puppies. Ow. I could do that a thousand times and I'd never stop dropping things and I'd never stop wearing sandals. Okay, I think it's booted. So it picked that up. So that's fine. And six, needs a GPU as well. And it picked up that one, which might be fine. Oh, sound card. D9. Okay, new update. We swapped in two new graphics cards and then we did end up having to disable their audio controllers, which is weird, but then one of them worked. So we've actually got five lit up right now. So it's just the last one that's not firing up. So I'm wondering if maybe it's just like my display port to HDMI cable or something. No, I don't think so. Cause num lock's not working on here. So we're gonna throw an RTX 2080 in here because nothing's worked yet. I really doubt it's gonna fire up though. No. Yeah, it lit up. All right, let's try the other five. No, I'm not actually doing it. I'm not actually doing it. I wanna see all of them up. I know it's normal when you're installing display drivers for the screen to go blank, but I'm just very on edge about this. Num lock's not working. You have got to be kidding me. Yeah, five didn't fire up. Why? What is the symbolism? Okay, well six fired up, but it's in Windows recovery nonsense mode. All the USB devices are there and now it's off. Bastard, it's trying so hard. You're just capturing my sadness over here. Thanks. So after sleeping on it, I thought maybe we need to get a little bit back to basics here. I pulled out all the graphics cards and re-ran the utility called HW lock. So something's going on with PCI Express that's causing some kind of nonsense in the system. It was causing HW lock to crash and it was causing our VMs not to load. Now with our new BIOS, which took forever to even start posting, now what we can do is start to slowly re-add hardware until we reach a problem. Everything's in. Should be fine. Nvidia confirmed the shipment of our six Titan Vs for this project. There seems to have been a mistake because I never had the briefing call where I came up with some real workloads to use them with. Like realistically, they were gonna support this. It's a cool project. They just jumped the gun a little, that's all. Now I do wanna get in here and turn on above 4G decoding. And it's weird because I've had a really hard time getting into the BIOS ever since I updated it. Like I just go straight from this pre-BIOS loading thing to boom, unraids firing up. Oh, there's a PS2 port. Maybe I should plug into that. The saga continues. It's erroring out on probably PCI resources allocation. But instead of letting me go into the BIOS and reconfigure that it's just hard resetting. You bastard. That's funny, I remember sending Asus a memo about this board saying this is the best behaved dual socket board I've yet seen. Like to issue a retraction. I mean, it's not garbage. It's like really cool, fancy stuff, but like I need it to work. Come on, just output. New plan. We pull out all the GPUs again and we reconfigure the PCI 4G decoding thing with only the onboard connected. Go. Just go. Instead of showing me this whole bar and showing me all the post codes, it just shows me the first one. It's not working correctly. Turned off. Come on. Yes. Yes, okay. So we are on the latest BIOS. Let's just go through here and see what all we need to change. Disable the serial port, CPU storage. Nope, onboard LAN. Disabled. Asus engine booster above 4G decoding on. Okay, see, I don't know what ASPM does. We don't need any of these operums. And we're extract CSM. I mean, maybe if we just start disabling devices, that'll help. This is it. Everything's reconfigured, which means we need to plug these SATA devices into the other controller. Now we are going to add graphics cards. Let's just throw them all in. I mean, why not yellow it again? That worked out so well for us last time. How many times do you live? I don't make everything hard. I mean, just ask the audience. I'm sure tons of them are flaccid right now. Okay, so we're back to square. I don't know what square this is, but we're back to actually being able to boot Unraid, see a display off of the onboard VGA, and we have all six graphics cards installed in the system. What square is that? Hip square. You know, like it's hip to be, I'm gonna go get my laptop. So here's what we're gonna do. We're gonna edit all of our VMs to not have, oh, well they're gone, so that's convenient. Okay, they don't have their USB controllers because those aren't plugged in. So can we start them? Which one's this? Four, let's start number four. I expect this to turn on. I'm using the power of positive speech. Oh, crap. Domain ID three is tainted. This is not a good indicator right now because if it's the graphics cards that are our problem, we are a lot further away from this project working than I hoped. Okay, so it fired up, but I don't see anything here yet, which is also not a good indicator because remember, four worked before. Yeah, we can try that. I don't think so though. Oh, boy, okay. Good work, David. Okay, starting numero six. Internal error, yeah, once again, it is flipping out about the audio. Crap, that's bad. What the hell? Okay, I'm really screwing this up right now. Okay, let's try two just for the lulls. Device 8F00, whatever the crap. What do you mean 8F whatever? What is this? Okay, did it change the, is that the GPU? It changed my GPU IDs. Oh, crap, so, oh, man, which one did? Okay, forget it, we're just gonna create a new one. However much memory, it doesn't matter right now. Let's just get it working. Primary disk location, manual, graphics card, Titan V, create. All right, let's see if that worked. Oh, maybe actually, yes. So there it is. Keyboard and mouse obviously aren't working right now because I don't have my USB controller plugged in, but okay, well, let's keep going then, I guess. Oh, dang it. Well, two's not up. Unless it is when I turn the monitor back on. New plan. We're gonna try two and six as the only things plugged into the system. Okay, I got LS Toppo to run. So the difference between our new LS Toppo run and the old one is now I have all six GPUs installed in the system. So what that allows us to do, if this image viewing program wasn't completely worthless, sure, good enough. What this allows us to do is figure out which GPUs are the problem. So we can see here, six is our Titan X and two is our Titan V. So then if we go into system devices, this tells us that 1D81 right here, and then our Titan X, those are our problem GPUs. Okay, so this confirms a few things. Number one is it confirms that my map of the PCI Express slots is probably right because slot two and slot three, which is where those are, one of them should be CPU two, I believe that's slot two, and one of them should be CPU one, I believe that's slot three. And actually I checked that, that's right. So it confirms that, and it confirms why I think four was fine and six is bad. Both of them are the only subordinate devices on a PLX bridge. And those are the ones I'm having issues with. So it is my belief that either the solution is UEFI tuning, so I've drafted an email to ASUS explaining that these are my problems, or a new UEFI. I don't remember how I started that sentence. My brain is very tired. I'm gonna send them an email. Hopefully it'll be coherent. In the meantime though, is that it? Now what I haven't been having problems with is my U.2 to PCI Express X4 adapters. Like this really is just for testing. I intend to populate these two. Now graphics card can draw up to 75 Watts through the PCI Express slot itself. So obviously this SATA to power adapter thing, like look at these pin or wires on this thing. But we are not going to be loading this thing anyway. I just wanna see them work. I don't need to run Doom for 24 hours straight or anything like that. All right, so here goes one and here goes two. So bad. So I'm gonna put this here. So we're only getting four lanes, but that's four lanes PCIe Gen 3. Like that's not bad. Does anyone else have really mixed feelings when they finally reach the end of a problem? Like where part of them is relieved and happy and the other part of them is just like pissed off that it took so long. If the cringing was real last time, now it's reached a level of hyper realism that we thought only possible with RTX. Okay, so I need this to reach in through here. Ah, yeah. This seems fine. So step number one is finding out if my GPUs are even working in those slots. I mean, theoretically, there's no real difference between this and this. They're just PCI Express over a cable, but you never know. Titan B, boom, there it is. And 1B00, this one's the same. So that's our Titan X. So they're there. So let's try starting it. Started right away. I don't know if we have a display output though. Well, would you look at that? Not only is it working, but what's really odd is it's running in the correct resolution. Let's see if it survives a reboot here. Need I remind you guys that this is the GPU that is plugged into an extension, plugged into a daughter board, is vertical in the case and completely unsupported by anything. Like if this works, it's a small Christmas miracle. Hey, are you working or what? Same behavior. Let's try the Titan B. Let's see what happens. Oh, oh, oh, oh, oh. Okay, yay, obtain. It's going fast at least. Wow, it's going really fast. Now it's slowing down. And of course it's 100% complete and yet here we are. Progress bars, man. How are they this broken? I'm gonna see if I can reboot that other one in the meantime. So I ran LS Topo again and let's have a look at what our topology is now. So here's those two devices. We're still having issues. Why? Why those two specifically? We're actually moving backwards. We're losing VMs now. Four no longer boots. I'm gonna try three and one as well. One's up, five's up, three's up. I don't know if they're gonna stay up. Five, three, and one. Let's try stopping them and then we'll try rebooting them. That's the real test. And none of them have a signal. Okay then. Okay. I had really hoped these adapters were gonna be the solution. We are going to put our GPUs in this thing. Like there's nothing that would lead me to believe that this will be better than using those NBME adapters. All right, maybe we can shove this over to get that HDMI cable in there. Yep, there we go. Okay, so that's in. Let's go ahead and throw our expansion card, doodad, back in here. Oh, ball sack, which slot was it? I think it's the bottom one that I used. Okay. Ah! Why is this keyboard powered up? Oh, these cards have power even though they don't have a data signal. Did that just shut down and reboot? Okay. That's the pain train. It's leaving the station. So the configuration's a complete cluster at this point. The two that I'm really worried about are the Titan X in VM number two and the Titan V in number six. So why don't we do those first? All right, so they're both up, but let's find out if they're gonna stay up. This has a blinking cursor, so that'll tell us if this freezes. Mind you, if it behaves like before, it'll probably just go away. Did that just go away? Oh, whoa, whoa, whoa, whoa, whoa, whoa, whoa, whoa. This one's up. I'm gonna shut it down and reboot it. Is it doing a graceful shutdown? Okay, so this is still working. I can tell, because the system clock is still right. This one finished its update while I wasn't looking. We're gonna rerun our command here. Did we just crash? Oh, yeah, that VM's gone. We're not going Hail Mary anymore. We're not going for glory. So we're gonna go back to basics and we're just gonna start with one, then we're gonna put in two and three and four. We're gonna see where we crap out and see if we can find a pattern here. So after sleeping on it, I thought, maybe we need to get a little bit back to basics here. Now what we can do is start to slowly re-add hardware until we reach a problem. The fuck is that? This is honestly starting to feel a little bit like a journal that I'm keeping on a desert island where I'm the only survivor or some garbage like that. So, while the camera was off last night, we went back to unraid 6.5.3, just in case the newer Linux kernel with all the Spectre and mount down nonsense that's going on was causing some problems. That was a suggestion from the unraid guys. And I went down to one card and it was working. Then I added a second card and boom, already flaky behavior. I don't even know if that was one of the slots I was having trouble with before. So this is like a big problem. And now that I had all the mapping information, I was definitely assigning GPUs to the correct CPU cores. So we weren't having to cross between the CPUs in order to get to the PCI express lanes. So that got me thinking, what if the problem is hardware? Now Ivan is a competent PC technician, but he doesn't have experience with LGA 3647 and it occurred to me that he was the one who installed my CPUs. So that got me thinking, what if we just have a bent pin in the socket? So, there was a little piece of debris in the socket, but it seems to be clear now. All right, so there's one. Let's try the other one. No obvious issues. I should put them in the opposite sockets just to see if maybe I start having problems with different slots or something. Well, so much for that theory. I mean, you never know when re-seating can just magically solve something though. So still I'm not very hopeful, but you never know. So this is interesting. We just got a kernel panic again. This time it's CPU one, processor ID one. It is still possible that one of my $10,000 CPUs is defective, but I'm really leaning towards motherboard BIOS optimizations that are wonky at this point. We're not getting away from these hard resets. So my idea just now was do a fresh install to that VM and see if we can avoid whatever devices it's trying to install drivers for. So that all is behaving completely normally. Wouldn't that be a weird thing? If it's spazzed out over the copied VDISCs, but then getting ready. I don't know about you computer, but I've been ready for a while here. Hey! What? What's going on? Hi. We have a Windows desktop on machine number two. Machine number two. How did this happen? So this is how it's gonna be for now. We finally got machine two working, we think, but Windows updates are not having any of it. Storyblocks video gets you studio quality video clips at a fraction of the cost. You can download all the stock video that your heart desires from their member library, including HD and 4K footage, After Effects templates, motion backgrounds, and more. And we actually use it quite a bit here on our Tech Quickie channel. Plus you can get exclusive discounts on millions of additional marketplace clips. You'll save 40% on the purchase compared to non-members. And what's really cool is the original artist will take a commission on each sale. All the content is royalty free, so you can use it even for commercial use to your heart's content and for personal projects like YouTube videos, for example. And new clips are added regularly, so there's always something new to check out. So check it out today at the link below. So thanks for watching, guys. If you disliked this video, then quite frankly, you and I are in the same boat because I did not have a good time over the last few days working on this, and I'm not looking forward to continuing to work on it. But if you liked it, hit like, get subscribed, and maybe consider checking out where to buy the stuff we featured at the link in the video description, but I wouldn't recommend buying it yet because we haven't quite determined if any of this works. Also down there is our merch store, which has cool shirts like this one and our community forum, which you should definitely join.