How to best configure storage for pro level video editing? This is a general question about configuring a machine for video editing and hopefully can be useful over a longer amount of time since the discussion is about general architecture and not necessarily model specific.

I understand there are variables influencing performance of a video editing machine  - processor speed, the application's reliance on video board vs CPU for processing, amount of memory available, and storage drive throughput speeds (vs burst speeds.)   I am getting set to put together an order for a new iMac (27") and need help with drive configuration because that is the most costly portion - the additional money for a larger processor, memory and video board pales in comparison to the storage piece. 

If it matters to the discussion I'll be doing hobby video editing - nothing longer than an hour or so. Typically lots of short clips and stills and transitions. I haven't settled on the software yet but it will need to be more complex than iMovie.

Just to be clear the components I'm considering when thinking about storage configuration are OS/application, source media, and export media. (if there is another variable I'm missing - such as temporary media -please educate me on that as well.

Here are my performance-related drive configuration questions:


Is it advantageous to split the application off onto a separate drive? Or can I have the application and one form of media on the same drive?
If I had to put the App and some media on the same drive, should that media be the source media or the export media?
Would a firewire 800 drive be fast enough as a drive to export to?
Would a firewire 800 drive be fast enough to use for my source media?
I'm fairly certain I don't want to have my source media and my export media on the same drive - is that correct thinking?
I don't think the iMac has an eSATA port for that type of external drive. Can one be added easily?
Is video editing application intensive enough to warrant an expensive electronic drive?


Thanks for the help!

Craig
Without more details about what camera you're shooting with, and what type of color space and compression you're working with, it's hard to give definitive answers for some of these questions. But...

1) Yes. In general you'd want to keep your app separate from the media. Usually you'd have OS X and your editing software of choice installed on the system's main internal drive. It would be kind of silly to have them installed on an external drive anyway.

2) The only situation where you should have the app and any media on the same drive is where the only drive you have is your internal drive and everything's getting stored there. This is fine for home use, but in a professional situation this would limit disk throughput prohibitively.

3) Depends on what format you're compressing to. It's all about bit-rate. FW800 is roughly 800 Mbits/second or 100 MB/s.

It's important to realize that, depending on your drive, this can be further limited by the drive's speed. Even though the FW800 connection can carry 100MB/s the drive may not be able to read or write fast enough to saturate it.

You really need to think about what format(s) you're exporting to, how fast the machine you're building can encode to those formats, and what kind of data rates that produces. That'll give you a number to shoot for.

If exporting HD in H.264 codec on a 2-core machine, even a USB 2.0 drive (max ~30MB/s) can usually keep up, so it really depends on what you're doing.

4) This is where you really need as much speed as possible. If your footage is coming mainly from compressed sources, like an HDSLR, or AVCHD or HDV camcorder this will reduce the amount of bandwidth you need. If you have uncompressed footage, it'll be exponentially greater.

Also, you need to know how many streams you want to be able to read from the disk at the same time when editing. Crunch the bit-rate numbers for the formats you're shooting in and you should get a total that you can try to equate to the speed of a source drive.

5) Because of the differences in speed requirements I just outlined, and the fact that during encoding, you'd be reading from and writing to the same drive, I'd say your instinct is right here. For consumer situations having everything on one drive is fine, it's just slower.

6) You could probably add eSATA to only the MacPro, but have a look at Thunderbolt instead. The drives are still expensive right now, but if you need something VERY high end, you can get insane speeds beyond eSATA or USB 3.

7) By electronic drive I'm assuming you mean an SSD. Having an SSD as your Mac is a great thing. It does so much for the user experience, that if you can afford it, just get it.

What I think you're asking though, is if you should use it as a scratch/temp disk? There's been some debate about the longevity of SSDs when it comes to high amounts of reads/writes (which is what a scratch drive does all day), so right now I'd still probably say it's not cost effective as a scratch drive.

If you're getting the iMac and want to trick it out, get the configuration with the SSD + internal HDD. Use the SSD for the OS and apps, and dedicate the HDD to be used as a scratch disk.

From there, for your externals, you have lots of choices. The iMac has one FW800 and two Thunderbolt ports. Something you didn't ask about, but is very important, is ingesting footage. When importing footage into the machine I assume you'd want that to be as fast as possible as well.

If your drives are on the FW800 bus, you'd want your card reader on the USB bus to maximize the bandwidth available. i.e. Read over USB, write over FW800, rather than trying to do both over either, essentially halving your bandwidth.

If you end up with thunderbolt drives, get a FW800 card reader, which will dramatically speed up your ingest (once your cards can keep up). The Thunderbolt bus has more than enough headroom at this stage (10Gb/s) to handle reading and writing at the same time, so no worries there.

Another thing you should think about is backup. If you can afford a pair of Thunderbolt RAID 5 arrays (one for source, one for export), do it. You'll thank yourself later.

I hope I provided some general advice, but there are lots of configurations you could put together. It really depends on your needs. You need to look at what you're shooting on and exporting to, and how you edit, then do some calculations.