Command-line tools can be faster than a Hadoop cluster (2014)
We had a poorly performing service which reads from a number of rest endpoints and writes to s3 in date prefixed format. Offshore wrote 3,600 lines of codes targeting kinesis firehose. By just piping the url endpoints to a named pipe and cycling the s3 file in python, my code was 55 lines of code and did the same thing without kinesis. Wrapping things in GNU parallel and using bash flags, it handles any failure cases super gracefully, which is something the offshore code did not do. The India offshore code had a global exception catch-all, and would print the error and return exit success return code... but I guess someone got to put Kinesis on their resume.