 I'm here today at Peacemaker Foamworks, who have worked on small projects you might have heard of, like Deadpool 2 and the new Ghostbusters. They're known for their crazy camera cars, but today their space is being used for possibly the craziest 3D capture setup in the world. Created by Volumetric Camera Systems, we've got over 200 CPU cores, 100 terabytes of storage, and if you follow me in here, 239 cameras capturing a staggering 920K image 30 times a second, which begs the question, who needs to capture that much detail? Oh yeah, it's Neil freaking Blomkamp, the director of District 9, Elysium, and the film we're looking at today, Demonic. It dropped yesterday, and the way they used 3D capture in it is just mind-bendingly cool. And terrifying. And you know what's also terrifying? This segue to our sponsor. Thanks to Origin PC for sponsoring this video. Origin PCs can be customized with 11th Gen Intel Core processors and Nvidia GeForce RTX 30 series graphics with Max-P design. Backed by a 24x7 support team, check out Origin PC at the link below. Intro This whole setup starts with the Yi camera. A 4K action camera that's, well, inexpensive. Volumetric Camera Systems, or VCS, would have obviously preferred to have cinema cameras all around here, but when you have to buy 200 of something plus spares, it gets expensive fast. And it's not like these are bad little cameras. Each one has the Sony IMX377 sensor, the same one as in the Google Pixel. And when you have over 200 of them pointed at a subject, you can get quite a bit of detail out of it. Each camera has custom firmware and an extra pin installed in the USB cable that allows for sync across all of the cameras. This allows for the clocks to be synced between every single camera. So if I hit play on the master camera right here, which is conveniently marked by not having a front plate, all 239 cameras start recording and then click it again and they all turn off. The special thing about this rig though is how all of that is turned into 3D data. This is accomplished using feature detection. So all of the tape in the background creates these sharp edges so AI can go in and figure out where each camera is compared to all of the others in the scene. Also every two cameras, so like these two right here, are used to create a stereo pair and using some complicated trigonometry, they're able to calculate the depth of anything in front of them. This is one of the keys to why the VCS camera system is so sick. By using this depth data in post, they can just tell the software to discard everything behind the subject, removing the need for green screens and allowing for complete 360 capture of the subject. Again, using a bunch of complicated math and AI, and Vidya is actually a sponsor of VCS and contributed at GV100 to help crunch those huge equations, they're able to create a point cloud of the subject with all of the depth data. On its own, this is really cool. It removes the need for a bunch of post-processing time because the animators don't have to worry about things like cloth or hair physics, but the huge advantage over motion capture is the ability to shoot full color textures of the actors in full makeup and costume. This does have its own set of problems though. The large five meter ring of cameras surrounding me is great for filming larger action sequences, but when you consider each one of these cameras has a wide angle 155 degree lens, if you're standing even a meter or two away, there won't be nearly as much texture data, which is where the high resolution array comes in. This was originally made using a trampoline that was cut in half and held together with zip ties, but they upgraded to this steel frame for shooting with Neil. Anyway, sequences will be planned out so that the actor can do their larger movements in the large array and then land in the high resolution array for parts that require more detail. Obviously the camera that is just a foot away from my face is going to be getting a lot more detail than the ones further out. Even that though isn't quite enough detail for VCS, like they are going up against cinema cameras, so they're upscaling all of the footage from 4k to 8k for every single camera. They custom made an algorithm by capturing 4k footage on these cameras, downscaled it to 1080p, and then used the 4k footage to train the AI for their best chance of upscaling it to 8k. And with all that in place, it still isn't quite the level of detail that is needed for a full Blonsie's Hollywood film. Unfortunately there isn't really any way around the detail at this point, but that isn't really a problem. For demonic they did a lot less post-processing than I expected. I left in a lot of the artifacts that happened naturally and it really adds to the horror of the film. It's super cool how Neil and the crew even use stuff like the point cloud data to act as a transition between real life and the simulation parts of a movie. As you've probably guessed by now though, running over 200 cameras at once uses a crap load of storage. Like this thing is creating 9,500 gigapixels per second and 7 to 14 million polygons per frame, which translates into about 10 terabytes of data for every 20 minutes of shooting. The first day of shooting on Neil's movie, poor Tobias and Scott of VCS were here until 6am grabbing the data with the two computers that they had, so clearly an upgrade was needed. Surrounding the camera away we have 24 computers from Oats Studio. Each one is equipped with a 10 core i7-6950k, nice, 64 gigabytes of ram and a GTX 1080 Ti. They don't actually need the GPUs but they certainly need the CPU, ram and IO to copy the data from each stand of cameras. Sorting all 200 plus video files was a huge problem for VCS, so they created their own custom program called Clippy Copy, named after everyone's favorite paperclip. That is based off of the RoboCopy API that automatically pulls and sorts the insane number of files that are made every time that they press play. Once it gets to the post-processing stage, they create a global alignment of all the cameras. This is basically looking at the features on the wall to figure out where each camera is in 3D space. After that they create a depth map of everything in the room using the stereo pairs and from there it's very simple to create a point cloud of the entire scene. Every point in the cloud will give you not just the position in 3D space but also the RGB data of every point. Every three points in the point cloud is then poisson mapped to create polygons and using an extra AI goodness using the camera data the textures are then improved again. After all of that we get what basically looks like the best video game character model ever. So like right here we have a capture of Tobias' nan just chilling in Unreal Engine and I'm able to just sort of float around her in oh geez Unreal Engine using WASD. It's just so cool and she's just playing back in real time in 3D. This is crazy. This processing obviously requires some balling computers that were provided by ManeGear and HP so down here we actually have two tiny backpack rigs rocking an RTX 2080. With all of that the director has an incredible amount of flexibility in post so they're able to pick camera angles, import the character into any environment they want and relight the scene to their liking. In order to relight the character though there can't be any shadows which means they need a lot of light. They've got a lot of light. In our office we actually use these same RE Sky panels we've got three of them and each one will run you about 20 grand so naturally they're using 24 of them here but even that was not enough light. While we were here today they actually installed an additional 11 REM40s which are an additional 40 grand each so unsurprisingly they rented the lights for this shoot. It's also insanely hot in here right now. You probably can't tell because my shirt's black. Ltdstore.com but I'm drenched. You don't necessarily need a balling studio setup like this though to do volumetric capture. Since they're effectively just action cameras on a stand VCS can record anywhere and just you know footage goes locally on the cameras and they have batteries built in. This way they can set up and say like the jungle at a farm or at a trade show and capture Jensen Wong and give us the data allowing us to have a little 3D Jensen wherever we want. For an extra portable setup they created this little box that has an image sensor in each one of these holes and you can set up say five of them around the subject and boom you're good to go. There are additional applications outside of cinema 3D capture. One of their biggest funders is actually Epic Games who hope to use this technology to do things like the Travis Scott concert but without all the time and money required to 3D model the whole thing. Like he could just do a set surrounded by these cameras and bippity bop he's straight into Fortnite. They're also looking into medical and military applications so say there's a world-renowned surgeon who knows how to do a really obscure knee surgery. That surgery can then be recorded in 3D and played back from any angle for training purposes. Future improvements are planned for both hardware and software so hopefully one day cinema grade 3D models can be captured volumetrically. Obviously getting better higher resolution cameras would allow for a better result but as more ground truth data is captured and imported they can train AI algorithms to better interpret the footage so hopefully you'll be able to get a similar level of fidelity to these 200 cameras with just 70 or so. So huge shout out to Tobias and Scott for showing us their awesome tech and Neil Blomkamp and Chris Harvey for sharing their incredible expertise. Demonic is available now so check it out at the link below and as you'd expect from one of Neil's films it's a fantastic technological showcase and proper frightening. And also huge thanks to Segways for bringing us to our sponsor Ting. Ting Mobile has new rates that make it easier to see how much you can save by switching. You can get unlimited talk and text for $10, data plans to start at $15 and their new Set 12 plan with 12 gigs of data for $35 and unlimited data for $45 a month. If you like their previous pay as you use plans it's still there with Ting Mobile's flex plans starting at just $5 per gigabyte. Data can also be shared if you have a family plan, connect more phones and save more. You'll still get national and award winning coverage and pretty much any phone will work with Ting Mobile. Ting Mobile now does have the perfect plan for everyone no matter what your needs are so check them out at linus.ting.com and receive a $25 credit. If you're looking for something else to watch well it isn't really similar to this video but the Porsche Tachcan review was loads of fun to shoot so go watch that.