Command-line tools can be faster than a Hadoop cluster (2014)
With all the Hipster tech being released recently, the headline statement holds true for a lot of things, unfortunately.We recently discussed new logging tools at work. It was either a redundant Amazon EC2 cluster with ElasticSearch for $50K monthly, or two large bare metal servers with rsyslog and grep for $400 monthly. The log ingestion and search performance was roughly the same...EDIT: To give everyone a sense of scale, those $200 each bare metal servers are 2x Intel Xeon 6-core + 256GB RAM + 15x 10TB 7200 rpm. We retain logs for 30 days and handle 4-5TB per day.