Big Data, Spark, Memsql, Realtime, Datapipeline.
built log readers process these events in real-time. One of these consumers is a log persistence service called Secor that reliably writes these events to Amazon S3. Secor was originally built with the purpose of saving logs produced by our monetization pipeline where 0-data loss was extremely