First of all, ingest your data as .json.gz -- line delimited json that&#x27;s gzipped -- chunk this by time range, perhaps hourly, on each box.  Periodically upload these files to the cloud -- S3 or Google CloudStorage, or both for a backup.  You can run this per-node, so it scales perfectly horizontally.  And .json.gz is easy to work with -- looking for a particular event in the last hour?  gunzip -c *.json.gz | grep &#x27;&lt;id&gt;&#x27;  ...<p>Most of the big data tools out there will work with data in this format -- BigQuery, Redshift, EMR.  EMR can do batch processing against this data directly from s3 -- but may not be suitable for anything other than batch processing.  BigQuery and&#x2F;or Redshift are more targeted towards analytics workloads, but you could use them to saw the data into another system that you use for OLAP -- MySQL or Postgres probably.<p>BigQuery has a nice interface and it&#x27;s a better hosted service than Redshift IMO.  If you like that product, you can do streaming inserts in parallel to your gcs&#x2F;s3 uploading process for more real-time access to the data.  The web interface is not bad for casual exploration of terabytes of raw data.  And the price isn&#x27;t terrible either.<p>I&#x27;ve done some consulting in this space -- feel free to reach out if you&#x27;d like some free advice.