The Elephant was a Trojan Horse: On the Death of Map-Reduce at Google
<i>"and the real contribution from Google in this area was arguably GFS, not Map-Reduce"</i><p>This, a million times over.  Map-reduce is not difficult to implement.  Implementing a <i>distributed, petabyte-scale filesystem</i> to hold the data being accessed by thousands of workers is what's difficult.<p>It's just a shame that Hadoop (and HDFS) are what we're saddled with in the outside world.  They're a total disaster area, from configuration difficulty to memory usage to speed to monitoring. But since HDFS is the only commonly used distributed FS, you're pretty much bound to Hadoop (and the rest of the horrible Apache ecosystem).<p>The world needs a good, stable, well-written distributed filesystem (ideally, one not written in a bloated language designed for remote controls and set-top boxes).